CVE-2023-49795 (published 2023-12-11T19:15Z)

MindsDB connects artificial intelligence models to real time data. Versions prior to 23.11.4.1 contain a server-side request forgery vulnerability in `file.py`. This can lead to limited information disclosure. Users should use MindsDB's `staging` branch or v23.11.4.1, which contain a fix for the issue.\n

CVSS v3 base score: 5.3, MEDIUM (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N)

Weakness information:
cwe_id: CWE-918
cwe_name: Server-Side Request Forgery (SSRF)
description: The web server receives a URL or similar request from an upstream component and retrieves the contents of this URL, but it does not sufficiently ensure that the request is being sent to the expected destination.
extended_description: By providing URLs to unexpected hosts or ports, attackers can make it appear that the server is sending the request, possibly bypassing access controls such as firewalls that prevent the attackers from accessing the URLs directly. The server can be used as a proxy to conduct port scanning of hosts in internal networks, use other URLs such as that can access documents on the system (using file://), or use other protocols such as gopher:// or tftp://, which may provide greater control over the contents of requests.
         

Fix information:
repo_url: https://github.com/mindsdb/mindsdb
commit_details: repo_url: https://github.com/mindsdb/mindsdb
author: andrew
author_date: 2023-11-29 16:02:18+03:00
msg: fixes:
- SSRF
- arbitrary file write
https://github.com/mindsdb/mindsdb_private/issues/341
repository: repo_url: https://github.com/mindsdb/mindsdb
repo_name: mindsdb/mindsdb
description: The platform for building AI from enterprise data
repo_language: Python
owner: mindsdb
file_changes: filename: file.py
change_type: ModificationType.MODIFY
diff: @@ -14,6 +14,7 @@ from mindsdb.api.http.utils import http_error, safe_extract
 from mindsdb.utilities.config import Config
 from mindsdb.utilities.context import context as ctx
 from mindsdb.utilities import log
+from mindsdb.utilities.security import is_private_url, clear_filename
 
 logger = log.getLogger(__name__)
 
@@ -51,7 +52,7 @@ class File(Resource):
 
         def on_file(file):
             nonlocal file_object
-            data["file"] = file.file_name.decode()
+            data["file"] = clear_filename(file.file_name.decode())
             file_object = file.file_object
 
         temp_dir_path = tempfile.mkdtemp(prefix="mindsdb_file_")
@@ -91,10 +92,15 @@ class File(Resource):
 
         if data.get("source_type") == "url":
             url = data["source"]
-            data["file"] = data["name"]
+            data["file"] = clear_filename(data["name"])
 
             config = Config()
             is_cloud = config.get("cloud", False)
+            if is_cloud and is_private_url(url):
+                return http_error(
+                    400, f'URL is private: {url}'
+                )
+
             if is_cloud is True and ctx.user_class != 1:
                 info = requests.head(url)
                 file_size = info.headers.get("Content-Length")
@@ -123,7 +129,7 @@ class File(Resource):
                     for chunk in r.iter_content(chunk_size=8192):
                         f.write(chunk)
 
-        original_file_name = data.get("original_file_name")
+        original_file_name = clear_filename(data.get("original_file_name"))
 
         file_path = os.path.join(temp_dir_path, data["file"])
         lp = file_path.lower()

programming_language: Python
filename: llama_index_handler.py
change_type: ModificationType.MODIFY
diff: @@ -13,6 +13,7 @@ from llama_index.indices.vector_store.base import VectorStore
 
 from mindsdb.integrations.libs.base import BaseMLEngine
 from mindsdb.utilities.config import Config
+from mindsdb.utilities.security import is_private_url
 
 
 def _validate_prompt_template(prompt_template: str):
@@ -69,7 +70,13 @@ class LlamaIndexHandler(BaseMLEngine):
             if 'source_url_link' not in args['using']:
                 raise Exception("SimpleWebPageReader requires a `source_url_link` parameter. Refer to LlamaIndex documentation for more details.")  # noqa
 
-            reader = SimpleWebPageReader(html_to_text=True).load_data([args['using']['source_url_link']])
+            url = args['using']['source_url_link']
+            config = Config()
+            is_cloud = config.get("cloud", False)
+            if is_cloud and is_private_url(url):
+                raise Exception(f'URL is private: {url}')
+
+            reader = SimpleWebPageReader(html_to_text=True).load_data([url])
 
         else:
             raise Exception(f"Invalid operation mode. Please use one of {self.supported_reader}.")

programming_language: Python
filename: web_handler.py
change_type: ModificationType.MODIFY
diff: @@ -10,6 +10,8 @@ from mindsdb.integrations.libs.response import (
     HandlerResponse as Response,
     RESPONSE_TYPE
 )
+from mindsdb.utilities.security import is_private_url
+from mindsdb.utilities.config import Config
 
 from .urlcrawl_helpers import get_df_from_query_str, get_all_websites
 
@@ -37,7 +39,9 @@ class CrawlerTable(APITable):
                         urls = url
                 else:
                     raise NotImplementedError(
-                        f'url can be url = "someurl", you can also crawl multiple sites, as follows: url IN ("url1", "url2", ..)')
+                        f'url can be url = "someurl", you can also crawl multiple sites, as follows:'
+                        f' url IN ("url1", "url2", ..)'
+                    )
 
             else:
                 pass
@@ -52,7 +56,16 @@ class CrawlerTable(APITable):
 
         if limit < 0:
             limit = 0
-            
+
+        config = Config()
+        is_cloud = config.get("cloud", False)
+        if is_cloud:
+            urls = [
+                url
+                for url in urls
+                if not is_private_url(url)
+            ]
+
         result = get_all_websites(urls, limit, html=False)
         if len(result) > limit:
             result = result[:limit]

programming_language: Python
filename: security.py
change_type: ModificationType.ADD
diff: @@ -0,0 +1,33 @@
+from urllib.parse import urlparse
+import socket
+import ipaddress
+
+
+def is_private_url(url: str):
+    """
+    Raises exception if url is private
+
+    :param url: url to check
+    """
+
+    hostname = urlparse(url).hostname
+    if not hostname:
+        # Unable find hostname in url
+        return True
+    ip = socket.gethostbyname(hostname)
+    return ipaddress.ip_address(ip).is_private
+
+
+def clear_filename(filename: str):
+    """
+    Removes path symbols from filename which could be used for path injection
+    :param s:
+    :return:
+    """
+
+    if not filename:
+        return filename
+    badchars = '\\/:*?\"<>|'
+    for c in badchars:
+        filename = filename.replace(c, '')
+    return filename

programming_language: Python