[Update] 修改冲突

README.md

 ## Jumpserver 多云环境下更好用的堡垒机
 
+![Total visitor](https://visitor-count-badge.herokuapp.com/total.svg?repo_id=jumpserver)
+![Visitors in today](https://visitor-count-badge.herokuapp.com/today.svg?repo_id=jumpserver)
 [![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
 [![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
 [![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
 [![Paramiko](https://img.shields.io/badge/paramiko-2.4.1-green.svg?style=plastic)](http://www.paramiko.org/)
 
-
 ----
 
-Jumpserver 是全球首款完全开源的堡垒机，使用 GNU GPL v2.0 开源协议，是符合 4A 的专业运维审计系统。
+Jumpserver 是全球首款完全开源的堡垒机，使用 GNU GPL v2.0 开源协议，是符合 4A 的运维安全审计系统。
 
 Jumpserver 使用 Python / Django 进行开发，遵循 Web 2.0 规范，配备了业界领先的 Web Terminal 解决方案，交互界面美观、用户体验好。
 
-Jumpserver 采纳分布式架构，支持多机房跨区域部署，中心节点提供 API，各机房部署登录节点，可横向扩展、无并发限制。
+Jumpserver 采纳分布式架构，支持多机房跨区域部署，支持横向扩展，无资产数量及并发限制。
 
 改变世界，从一点点开始。
 
-- [English Version](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
-
-
-### 功能
+### 核心功能列表
 ----
 
 <table class="subscription-level-table">
-    <tr class="subscription-level-tr-border">
-        <th  style="background-color: #1ab394;color: #ffffff;" colspan="3">Jumpserver提供的堡垒机必备功能</th>
-    </tr>
     <tr class="subscription-level-tr-border">
         <td class="features-first-td-background-style" rowspan="4">身份验证 Authentication</td>
         <td class="features-second-td-background-style" rowspan="3" >登录认证
@@ -34,11 +29,11 @@ Jumpserver 采纳分布式架构，支持多机房跨区域部署，中心节点
         </td>
     </tr>
     <tr class="subscription-level-tr-border">
-        <td class="features-third-td-background-style">LDAP认证
+        <td class="features-third-td-background-style">LDAP 认证
         </td>
     </tr>
     <tr class="subscription-level-tr-border">
-        <td class="features-third-td-background-style">支持OpenID，实现单点登录
+        <td class="features-third-td-background-style">支持 OpenID，实现单点登录
         </td>
     </tr>
     <tr class="subscription-level-tr-border">
@@ -178,35 +173,32 @@ Jumpserver 采纳分布式架构，支持多机房跨区域部署，中心节点
     </tr>
 </table>
 
-
-### 开始使用
+### 安装及使用文档
 ----
 
-- 快速开始文档  [Docker 安装](http://docs.jumpserver.org/zh/docs/dockerinstall.html)
+-  [Docker 快速安装文档](http://docs.jumpserver.org/zh/docs/dockerinstall.html)
+-  [Step by Step 安装文档](http://docs.jumpserver.org/zh/docs/step_by_step.html)
+-  [完整文档](http://docs.jumpserver.org)
 
-- Step by Step 安装文档 [详细部署](http://docs.jumpserver.org/zh/docs/step_by_step.html)
-
-- 也可以查看我们完整文档 [文档](http://docs.jumpserver.org)
-
-### Demo、视频 和 截图
+### 演示视频和系统截图
 ----
 
-我们提供了 Demo 、演示视频和截图可以让你快速了解 Jumpserver
+我们提供了演示视频和系统截图可以让你快速了解 Jumpserver。
 
-- [Demo](https://demo.jumpserver.org/auth/login/?next=/)
-- [视频](https://fit2cloud2-offline-installer.oss-cn-beijing.aliyuncs.com/tools/Jumpserver%20%E4%BB%8B%E7%BB%8Dv1.4.mp4)
-- [截图](http://docs.jumpserver.org/zh/docs/snapshot.html)
+- [演示视频](https://jumpserver.oss-cn-hangzhou.aliyuncs.com/jms-media/%E3%80%90%E6%BC%94%E7%A4%BA%E8%A7%86%E9%A2%91%E3%80%91Jumpserver%20%E5%A0%A1%E5%9E%92%E6%9C%BA%20V1.5.0%20%E6%BC%94%E7%A4%BA%E8%A7%86%E9%A2%91%20-%20final.mp4)
+- [系统截图](http://docs.jumpserver.org/zh/docs/snapshot.html)
 
 ### SDK
 ----
 
-我们还编写了一些SDK，供你的其它系统快速和 Jumpserver API 交互
-
-- [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver其它组件使用这个SDK完成交互
-- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK
+我们编写了一些SDK，供你的其它系统快速和 Jumpserver API 交互。
 
+- [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver 其它组件使用这个 SDK 完成交互
+- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的 Java 版本的 SDK
 
 ### License & Copyright
+----
+
 Copyright (c) 2014-2019 飞致云 FIT2CLOUD, All rights reserved.
 
 Licensed under The GNU General Public License version 2 (GPLv2)  (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

README_EN.md

 ## Jumpserver 
 
+![Total visitor](https://visitor-count-badge.herokuapp.com/total.svg?repo_id=jumpserver)
+![Visitors in today](https://visitor-count-badge.herokuapp.com/today.svg?repo_id=jumpserver)
 [![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
 [![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
 [![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)

apps/assets/models/base.py

@@ -164,9 +164,6 @@ class AssetUser(OrgModelMixin):
     def set_asset_connectivity(self, asset, c):
         key = self.get_asset_connectivity_key(asset)
         Connectivity.set(key, c)
-        # 当为某个系统用户或管理用户设置的的时候，失效掉他们的连接数量
-        amount_key = self.CONNECTIVITY_AMOUNT_CACHE_KEY.format(self.username, '*')
-        cache.delete_pattern(amount_key)
 
     def get_asset_user(self, asset):
         from ..backends import AssetUserManager

apps/assets/models/node.py

@@ -211,7 +211,8 @@ class AssetsAmountMixin:
         cached = cache.get(cache_key)
         if cached is not None:
             return cached
-        assets_amount = self.get_all_assets().count()
+        assets_amount = self.get_all_assets().only('id').count()
+        self.assets_amount = assets_amount
         return assets_amount
 
     @assets_amount.setter

apps/assets/serializers/system_user.py

@@ -72,7 +72,9 @@ class SystemUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
         super().validate_password(password)
         auto_gen_key = self.initial_data.get("auto_generate_key", False)
         private_key = self.initial_data.get("private_key")
-        if not self.instance and not auto_gen_key and not password and not private_key:
+        login_mode = self.initial_data.get("login_mode")
+        if not self.instance and not auto_gen_key and not password and \
+                not private_key and login_mode == SystemUser.LOGIN_AUTO:
             raise serializers.ValidationError(_("Password or private key required"))
         return password
 

apps/assets/tasks.py

@@ -94,7 +94,7 @@ def set_assets_hardware_info(assets, result, **kwargs):
                 break
         else:
             ___cpu_model = 'Unknown'
-        ___cpu_model = ___cpu_model[:64]
+        ___cpu_model = ___cpu_model[:48]
         ___cpu_count = info.get('ansible_processor_count', 0)
         ___cpu_cores = info.get('ansible_processor_cores', None) or \
                        len(info.get('ansible_processor', []))

apps/authentication/utils.py

@@ -7,7 +7,7 @@ from common.utils import get_ip_city, validate_ip
 def write_login_log(*args, **kwargs):
     from audits.models import UserLoginLog
     default_city = _("Unknown")
-    ip = kwargs.get('ip', '')
+    ip = kwargs.get('ip') or ''
     if not (ip and validate_ip(ip)):
         ip = ip[:15]
         city = default_city

apps/common/mixins/serializers.py

 # -*- coding: utf-8 -*-
 #
 
+from django.core.exceptions import ObjectDoesNotExist
 from rest_framework.utils import html
 from rest_framework.settings import api_settings
 from rest_framework.exceptions import ValidationError
@@ -74,16 +75,21 @@ class BulkListSerializerMixin(object):
         for item in data:
             try:
                 # prepare child serializer to only handle one instance
-                if 'id' in item.keys():
-                    self.child.instance = self.instance.get(id=item['id']) if self.instance else None
-                if 'pk' in item.keys():
-                    self.child.instance = self.instance.get(id=item['pk']) if self.instance else None
-
+                if 'id' in item:
+                    pk = item["id"]
+                elif 'pk' in item:
+                    pk = item["pk"]
+                else:
+                    raise ValidationError("id or pk not in data")
+                child = self.instance.get(id=pk)
+                self.child.instance = child
                 self.child.initial_data = item
                 # raw
                 validated = self.child.run_validation(item)
             except ValidationError as exc:
                 errors.append(exc.detail)
+            except ObjectDoesNotExist as e:
+                errors.append(e)
             else:
                 ret.append(validated)
                 errors.append({})

apps/common/parsers/csv.py

@@ -40,7 +40,7 @@ class JMSCSVParser(BaseParser):
     @staticmethod
     def _get_fields_map(serializer):
         fields_map = {}
-        fields = serializer.get_fields()
+        fields = serializer.fields
         fields_map.update({v.label: k for k, v in fields.items()})
         fields_map.update({k: k for k, _ in fields.items()})
         return fields_map
@@ -91,7 +91,7 @@ class JMSCSVParser(BaseParser):
 
             header = next(rows)
             fields_map = self._get_fields_map(serializer)
-            header = [fields_map.get(name, '') for name in header]
+            header = [fields_map.get(name.strip('*'), '') for name in header]
 
             data = []
             for row in rows:

apps/common/renders/csv.py

@@ -20,26 +20,18 @@ class JMSCSVRender(BaseRenderer):
     format = 'csv'
 
     @staticmethod
-    def _get_header(fields, template):
-        if template == 'import':
-            header = [
-                k for k, v in fields.items()
-                if not v.read_only and k != 'org_id'
-            ]
-        elif template == 'update':
-            header = [k for k, v in fields.items() if not v.read_only]
+    def _get_show_fields(fields, template):
+        if template in ('import', 'update'):
+            return [v for k, v in fields.items() if not v.read_only and k != "org_id"]
         else:
-            # template in ['export']
-            header = [k for k, v in fields.items() if not v.write_only]
-        return header
+            return [v for k, v in fields.items() if not v.write_only and k != "org_id"]
 
     @staticmethod
-    def _gen_table(data, header, labels=None):
-        labels = labels or {}
-        yield [labels.get(k, k) for k in header]
+    def _gen_table(data, fields):
+        yield ['*{}'.format(f.label) if f.required else f.label for f in fields]
 
         for item in data:
-            row = [item.get(key) for key in header]
+            row = [item.get(f.field_name) for f in fields]
             yield row
 
     def set_response_disposition(self, serializer, context):
@@ -73,10 +65,9 @@ class JMSCSVRender(BaseRenderer):
             logger.debug(e, exc_info=True)
             value = 'The resource not support export!'.encode('utf-8')
         else:
-            fields = serializer.get_fields()
-            header = self._get_header(fields, template)
-            labels = {k: v.label for k, v in fields.items() if v.label}
-            table = self._gen_table(data, header, labels)
+            fields = serializer.fields
+            show_fields = self._get_show_fields(fields, template)
+            table = self._gen_table(data, show_fields)
 
             csv_buffer = BytesIO()
             csv_buffer.write(codecs.BOM_UTF8)

apps/common/utils/common.py

@@ -106,7 +106,7 @@ def capacity_convert(size, expect='auto', rate=1000):
 
     if expect == 'auto':
         for unit, rate_ in rate_mapping.items():
-            if rate > std_size/rate_ > 1:
+            if rate > std_size/rate_ >= 1 or unit == "T":
                 expect = unit
                 break
 

apps/jumpserver/conf.py

@@ -379,6 +379,7 @@ defaults = {
     'ASSETS_PERM_CACHE_TIME': 3600*24,
     'SECURITY_MFA_VERIFY_TTL': 3600,
     'ASSETS_PERM_CACHE_ENABLE': False,
+    'PERM_SINGLE_ASSET_TO_UNGROUP_NODE': False,
 }
 
 

apps/jumpserver/middleware.py

@@ -57,7 +57,8 @@ class RequestMiddleware:
     def __call__(self, request):
         set_current_request(request)
         response = self.get_response(request)
-        if not settings.SESSION_EXPIRE_AT_BROWSER_CLOSE:
+        is_request_api = request.path.startswith('/api')
+        if not settings.SESSION_EXPIRE_AT_BROWSER_CLOSE and not is_request_api:
             age = request.session.get_expiry_age()
             request.session.set_expiry(age)
         return response

apps/jumpserver/settings.py

@@ -54,6 +54,9 @@ LOG_LEVEL = CONFIG.LOG_LEVEL
 
 ALLOWED_HOSTS = ['*']
 
+# Max post update field num
+DATA_UPLOAD_MAX_NUMBER_FIELDS = 10000
+
 # Application definition
 
 INSTALLED_APPS = [
@@ -615,3 +618,5 @@ ASSETS_PERM_CACHE_TIME = CONFIG.ASSETS_PERM_CACHE_TIME
 
 # Asset user auth external backend, default AuthBook backend
 BACKEND_ASSET_USER_AUTH_VAULT = False
+
+PERM_SINGLE_ASSET_TO_UNGROUP_NODE = CONFIG.PERM_SINGLE_ASSET_TO_UNGROUP_NODE

apps/perms/api/asset_permission.py

@@ -115,6 +115,7 @@ class AssetPermissionViewSet(viewsets.ModelViewSet):
     def filter_user(self, queryset):
         user_id = self.request.query_params.get('user_id')
         username = self.request.query_params.get('username')
+        query_group = self.request.query_params.get('all')
         if user_id:
             user = get_object_or_none(User, pk=user_id)
         elif username:
@@ -123,6 +124,14 @@ class AssetPermissionViewSet(viewsets.ModelViewSet):
             return queryset
         if not user:
             return queryset.none()
+        kwargs = {}
+        args = []
+        if query_group:
+            groups = user.groups.all()
+            args.append(Q(users=user) | Q(user_groups__in=groups))
+        else:
+            kwargs["users"] = user
+        return queryset.filter(*args, **kwargs).distinct()
 
     def filter_user_group(self, queryset):
         user_group_id = self.request.query_params.get('user_group_id')
@@ -148,6 +157,7 @@ class AssetPermissionViewSet(viewsets.ModelViewSet):
     def filter_queryset(self, queryset):
         queryset = super().filter_queryset(queryset)
         queryset = self.filter_valid(queryset)
+        queryset = self.filter_user(queryset)
         queryset = self.filter_keyword(queryset)
         queryset = self.filter_asset(queryset)
         queryset = self.filter_node(queryset)

apps/perms/utils/asset_permission.py

@@ -180,6 +180,19 @@ class GenerateTree:
             assets.append({"id": asset_id, "system_users": system_users})
         return assets
 
+    def set_ungrouped_assets_nodes_if_need(self):
+        if settings.PERM_SINGLE_ASSET_TO_UNGROUP_NODE:
+            return
+        ungrouped_assets_ids = self.nodes[self.ungrouped_key]["assets"]
+        for asset_id in ungrouped_assets_ids:
+            in_nodes = self.all_assets_nodes_keys.get(asset_id, [])
+            for node_key in in_nodes:
+                parents_keys = self.node_util.get_nodes_parents_keys_by_key(node_key, with_self=False)
+                for parent_key in parents_keys:
+                    n = self.nodes[parent_key]
+                self.nodes[node_key]["assets"].add(asset_id)
+        self.nodes.pop(self.ungrouped_key, None)
+
     @timeit
     def get_nodes_with_assets(self):
         """
@@ -198,6 +211,7 @@ class GenerateTree:
         """
         if self._nodes_with_assets:
             return self._nodes_with_assets
+        self.set_ungrouped_assets_nodes_if_need()
         util = PermAssetsAmountUtil()
         nodes_with_assets_amount = util.compute_nodes_assets_amount(self.nodes)
         nodes = []
@@ -219,6 +233,7 @@ class GenerateTree:
         return nodes
 
     def get_nodes(self):
+        self.set_ungrouped_assets_nodes_if_need()
         nodes = list(self.nodes.keys())
         if not nodes:
             nodes.append(const.EMPTY_NODE_KEY)

apps/static/plugins/elfinder/css/theme-gray.css

@@ -1280,7 +1280,8 @@ div.elfinder-cwd-wrapper-list .ui-icon-grip-dotted-vertical {
 }
 .elfinder-cwd-file.ui-selected {
   background: #455158;
-  color: #555;
+  /*color: #555;*/
+  color: #ddd;
   width: 120px !important;
 }
 .elfinder-cwd-filename input,
@@ -1309,10 +1310,10 @@ div.elfinder-cwd-wrapper-list .ui-icon-grip-dotted-vertical {
   padding: 0;
 }
 .elfinder-cwd table tr:nth-child(odd) {
-  background-color: transparent;
+  /*background-color: transparent;*/
 }
 .elfinder-cwd table tr:nth-child(odd).ui-state-hover {
-  background-color: #4c5961;
+  /*background-color: #4c5961;*/
 }
 #elfinder-elfinder-cwd-thead td {
   background: #353b42;

apps/static/plugins/elfinder/i18n/elfinder.zh_CN.js

@@ -422,7 +422,7 @@
 			'minsLeft'        : '剩余 $1 分钟', // from v2.1.17 added 13.11.2016
 			'openAsEncoding'  : '使用所选编码重新打开', // from v2.1.19 added 2.12.2016
 			'saveAsEncoding'  : '使用所选编码保存', // from v2.1.19 added 2.12.2016
-			'selectFolder'    : '选择目录(暂不支持)', // from v2.1.20 added 13.12.2016
+			'selectFolder'    : '选择目录', // from v2.1.20 added 13.12.2016
 			'firstLetterSearch': '首字母搜索', // from v2.1.23 added 24.3.2017
 			'presets'         : '预置', // from v2.1.25 added 26.5.2017
 			'tooManyToTrash'  : '项目太多，不能移动到回收站.', // from v2.1.25 added 9.6.2017

apps/users/api/user.py

@@ -5,10 +5,10 @@ from django.core.cache import cache
 from django.contrib.auth import logout
 from django.utils.translation import ugettext as _
 
-from rest_framework import status
 from rest_framework import generics
 from rest_framework.response import Response
 from rest_framework.permissions import IsAuthenticated
+from rest_framework.serializers import ValidationError
 from rest_framework_bulk import BulkModelViewSet
 from rest_framework.pagination import LimitOffsetPagination
 
@@ -73,10 +73,7 @@ class UserViewSet(IDInCacheFilterMixin, BulkModelViewSet):
         check current user has permission to handle instance
         (update, destroy, bulk_update, bulk destroy)
         """
-        if not self.request.user.is_superuser \
-                and (instance.is_superuser or instance.is_auditor):
-            return True
-        if self.request.user == instance:
+        if (instance.is_superuser or instance.is_auditor) and not self.request.user.is_superuser:
             return True
         return False
 
@@ -92,16 +89,14 @@ class UserViewSet(IDInCacheFilterMixin, BulkModelViewSet):
             return False
         return qs.count() != filtered.count()
 
-    def bulk_update(self, request, *args, **kwargs):
-        """
-        rewrite because limit org_admin update superuser
-        """
-        # restrict the update to the filtered queryset
-        queryset = self.filter_queryset(self.get_queryset())
-        if self._bulk_deny_permission(queryset):
-            data = {'msg': _("You do not have permission.")}
-            return Response(data=data, status=status.HTTP_403_FORBIDDEN)
-        return super().bulk_update(request, *args, **kwargs)
+    def perform_bulk_update(self, serializer):
+        users_ids = [d.get("id") or d.get("pk") for d in serializer.validated_data]
+        users = User.objects.filter(id__in=users_ids)
+        deny_instances = [str(i.id) for i in users if self._deny_permission(i)]
+        if deny_instances:
+            msg = "{} can't be update".format(deny_instances)
+            raise ValidationError({"id": msg})
+        return super().perform_bulk_update(serializer)
 
 
 class UserChangePasswordApi(generics.RetrieveUpdateAPIView):
@@ -184,6 +179,11 @@ class UserProfileApi(generics.RetrieveAPIView):
     def get_object(self):
         return self.request.user
 
+    def retrieve(self, request, *args, **kwargs):
+        age = request.session.get_expiry_age()
+        request.session.set_expiry(age)
+        return super().retrieve(request, *args, **kwargs)
+
 
 class UserResetOTPApi(generics.RetrieveAPIView):
     queryset = User.objects.all()

config_example.yml

@@ -76,3 +76,7 @@ REDIS_PORT: 6379
 # OTP/MFA 配置
 # OTP_VALID_WINDOW: 0
 # OTP_ISSUER_NAME: Jumpserver
+
+# Perm show single asset to ungrouped node
+# 是否把未授权节点资产放入到 未分组 节点中
+# PERM_SINGLE_ASSET_TO_UNGROUP_NODE: false