CVE-2020-7698 (published 2020-07-29T13:15Z)

This affects the package Gerapy from 0 and before 0.9.3. The input being passed to Popen, via the project_configure endpoint, isn’t being sanitized.

Severity: HIGH
CVSS v2 base score: 7.5 (AV:N/AC:L/Au:N/C:P/I:P/A:P)
CVSS v3 base score: 9.8, CRITICAL (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H)

Weakness information:
cwe_id: CWE-78
cwe_name: Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection')
description: The product constructs all or part of an OS command using externally-influenced input from an upstream component, but it does not neutralize or incorrectly neutralizes special elements that could modify the intended OS command when it is sent to a downstream component.
extended_description: 
		   This weakness can lead to a vulnerability in environments in which the attacker does not have direct access to the operating system, such as in web applications. Alternately, if the weakness occurs in a privileged program, it could allow the attacker to specify commands that normally would not be accessible, or to call alternate commands with privileges that the attacker does not have. The problem is exacerbated if the compromised process does not follow the principle of least privilege, because the attacker-controlled commands may run with special system privileges that increases the amount of damage.
            There are at least two subtypes of OS command injection:
               
				 The application intends to execute a single, fixed program that is under its own control. It intends to use externally-supplied inputs as arguments to that program. For example, the program might use system("nslookup [HOSTNAME]") to run nslookup and allow the user to supply a HOSTNAME, which is used as an argument. Attackers cannot prevent nslookup from executing. However, if the program does not remove command separators from the HOSTNAME argument, attackers could place the separators into the arguments, which allows them to execute their own program after nslookup has finished executing.
				 The application accepts an input that it uses to fully select which program to run, as well as which commands to use. The application simply redirects this entire command to the operating system. For example, the program might use "exec([COMMAND])" to execute the [COMMAND] that was supplied by the user. If the COMMAND is under attacker control, then the attacker can execute arbitrary commands or programs. If the command is being executed using functions like exec() and CreateProcess(), the attacker might not be able to combine multiple commands together in the same line.
               
            From a weakness standpoint, these variants represent distinct programmer errors. In the first variant, the programmer clearly intends that input from untrusted parties will be part of the arguments in the command to be executed. In the second variant, the programmer does not intend for the command to be accessible to any untrusted party, but the programmer probably has not accounted for alternate ways in which malicious attackers can provide input.
         
         

Fix information:
repo_url: https://github.com/Gerapy/Gerapy
commit_details: repo_url: https://github.com/Gerapy/Gerapy
author: Germey
author_date: 2020-07-06 22:57:10+08:00
msg: to b1
repository: repo_url: https://github.com/Gerapy/Gerapy
repo_name: Gerapy/Gerapy
description: Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
repo_language: Python
owner: Gerapy
file_changes: filename: README.md
change_type: ModificationType.MODIFY
diff: @@ -8,9 +8,6 @@
 ![Docker Pulls](https://img.shields.io/docker/pulls/germey/gerapy)
 ![PyPI - License](https://img.shields.io/pypi/l/gerapy)
 
-
-> 注：从 Gerapy 2.x 开始，其定位发生改变，不再支持 Scrapyd，转而支持 Docker、Kubernetes 的部署，另外开发还会迁移到 Scrapy 可视化配置和智能解析方面，敬请期待。
-
 Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js.
 
 ## Documentation

programming_language: Markdown
filename: __version__.py
change_type: ModificationType.MODIFY
diff: @@ -1,4 +1,4 @@
-VERSION = (0, 9, '3a3')
+VERSION = (0, 9, '3b1')
 
 __version__ = '.'.join(map(str, VERSION))
 

programming_language: Python
filename: views.py
change_type: ModificationType.MODIFY
diff: @@ -1,3 +1,4 @@
+import re
 from pathlib import Path
 from urllib.parse import unquote
 import base64
@@ -251,6 +252,8 @@ def project_configure(request, project_name):
         configuration = json.dumps(data.get('configuration'), ensure_ascii=False)
         project.update(**{'configuration': configuration})
         
+        # for safe protection
+        project_name = re.sub('[\!\@\#\$\;\&\*\~\"\'\{\}\]\[\-\+\%\^]+', '', project_name)
         # execute generate cmd
         cmd = ' '.join(['gerapy', 'generate', project_name])
         p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
@@ -634,17 +637,15 @@ def job_list(request, client_id, project_name):
     if request.method == 'GET':
         client = Client.objects.get(id=client_id)
         scrapyd = get_scrapyd(client)
-        try:
-            result = scrapyd.list_jobs(project_name)
-            jobs = []
-            statuses = ['pending', 'running', 'finished']
-            for status in statuses:
-                for job in result.get(status):
-                    job['status'] = status
-                    jobs.append(job)
-            return JsonResponse(jobs)
-        except ConnectionError:
-            return JsonResponse({'message': 'Connect Error'}, status=500)
+        result = scrapyd.list_jobs(project_name)
+        jobs = []
+        statuses = ['pending', 'running', 'finished']
+        for status in statuses:
+            for job in result.get(status):
+                job['status'] = status
+                jobs.append(job)
+        return JsonResponse(jobs)
+    
 
 
 @api_view(['GET'])
@@ -663,21 +664,18 @@ def job_log(request, client_id, project_name, spider_name, job_id):
         client = Client.objects.get(id=client_id)
         # get log url
         url = log_url(client.ip, client.port, project_name, spider_name, job_id)
-        try:
-            # get last 1000 bytes of log
-            response = requests.get(url, timeout=5, headers={
-                'Range': 'bytes=-1000'
-            }, auth=(client.username, client.password) if client.auth else None)
-            # Get encoding
-            encoding = response.apparent_encoding
-            # log not found
-            if response.status_code == 404:
-                return JsonResponse({'message': 'Log Not Found'}, status=404)
-            # bytes to string
-            text = response.content.decode(encoding, errors='replace')
-            return HttpResponse(text)
-        except requests.ConnectionError:
-            return JsonResponse({'message': 'Load Log Error'}, status=500)
+        # get last 1000 bytes of log
+        response = requests.get(url, timeout=5, headers={
+            'Range': 'bytes=-1000'
+        }, auth=(client.username, client.password) if client.auth else None)
+        # Get encoding
+        encoding = response.apparent_encoding
+        # log not found
+        if response.status_code == 404:
+            return JsonResponse({'message': 'Log Not Found'}, status=404)
+        # bytes to string
+        text = response.content.decode(encoding, errors='replace')
+        return HttpResponse(text)
 
 
 @api_view(['GET'])
@@ -693,12 +691,9 @@ def job_cancel(request, client_id, project_name, job_id):
     """
     if request.method == 'GET':
         client = Client.objects.get(id=client_id)
-        try:
-            scrapyd = get_scrapyd(client)
-            result = scrapyd.cancel(project_name, job_id)
-            return JsonResponse(result)
-        except ConnectionError:
-            return JsonResponse({'message': 'Connect Error'})
+        scrapyd = get_scrapyd(client)
+        result = scrapyd.cancel(project_name, job_id)
+        return JsonResponse(result)
 
 
 @api_view(['GET'])
@@ -706,12 +701,9 @@ def job_cancel(request, client_id, project_name, job_id):
 def del_version(request, client_id, project, version):
     if request.method == 'GET':
         client = Client.objects.get(id=client_id)
-        try:
-            scrapyd = get_scrapyd(client)
-            result = scrapyd.delete_version(project=project, version=version)
-            return JsonResponse(result)
-        except ConnectionError:
-            return JsonResponse({'message': 'Connect Error'})
+        scrapyd = get_scrapyd(client)
+        result = scrapyd.delete_version(project=project, version=version)
+        return JsonResponse(result)
 
 
 @api_view(['GET'])
@@ -719,12 +711,9 @@ def del_version(request, client_id, project, version):
 def del_project(request, client_id, project):
     if request.method == 'GET':
         client = Client.objects.get(id=client_id)
-        try:
-            scrapyd = get_scrapyd(client)
-            result = scrapyd.delete_project(project=project)
-            return JsonResponse(result)
-        except ConnectionError:
-            return JsonResponse({'message': 'Connect Error'})
+        scrapyd = get_scrapyd(client)
+        result = scrapyd.delete_project(project=project)
+        return JsonResponse(result)
 
 
 @api_view(['POST'])
@@ -829,18 +818,16 @@ def task_remove(request, task_id):
     :return:
     """
     if request.method == 'POST':
-        try:
-            # delete job from DjangoJob
-            task = Task.objects.get(id=task_id)
-            clients = clients_of_task(task)
-            for client in clients:
-                job_id = get_job_id(client, task)
-                DjangoJob.objects.filter(name=job_id).delete()
-            # delete task
-            Task.objects.filter(id=task_id).delete()
-            return JsonResponse({'result': '1'})
-        except:
-            return JsonResponse({'result': '0'})
+        # delete job from DjangoJob
+        task = Task.objects.get(id=task_id)
+        clients = clients_of_task(task)
+        for client in clients:
+            job_id = get_job_id(client, task)
+            DjangoJob.objects.filter(name=job_id).delete()
+        # delete task
+        Task.objects.filter(id=task_id).delete()
+        return JsonResponse({'result': '1'})
+    
 
 
 @api_view(['GET'])
@@ -915,10 +902,7 @@ def render_html(request):
         url = unquote(base64.b64decode(url).decode('utf-8'))
         js = request.GET.get('js', 0)
         script = request.GET.get('script')
-        try:
-            response = requests.get(url, timeout=5)
-            response.encoding = response.apparent_encoding
-            html = process_html(response.text)
-            return HttpResponse(html)
-        except Exception as e:
-            return JsonResponse({'message': e.args}, status=500)
+        response = requests.get(url, timeout=5)
+        response.encoding = response.apparent_encoding
+        html = process_html(response.text)
+        return HttpResponse(html)

programming_language: Python
filename: requirements.txt
change_type: ModificationType.MODIFY
diff: @@ -5,12 +5,12 @@ django-cors-headers==3.2.0
 django-apscheduler==0.3.0
 furl==2.1.0
 jinja2==2.10.1
-scrapy>=1.4.0
+scrapy==1.5.0
 scrapy-redis==0.6.8
 scrapy-splash==0.7.2
 python-scrapyd-api==2.1.2
 redis==2.10.5
-requests>=2.20.0
+requests==2.20.0
 pymongo==3.9.0
 pymysql==0.7.10
 pyquery==1.2.17

programming_language: Batchfile