CVE-2023-46229 (published 2023-10-19T05:15Z)

LangChain before 0.0.317 allows SSRF via document_loaders/recursive_url_loader.py because crawling can proceed from an external server to an internal server.

CVSS v3 base score: 8.8, HIGH (CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H)

Weakness information:
cwe_id: CWE-918
cwe_name: Server-Side Request Forgery (SSRF)
description: The web server receives a URL or similar request from an upstream component and retrieves the contents of this URL, but it does not sufficiently ensure that the request is being sent to the expected destination.
extended_description: By providing URLs to unexpected hosts or ports, attackers can make it appear that the server is sending the request, possibly bypassing access controls such as firewalls that prevent the attackers from accessing the URLs directly. The server can be used as a proxy to conduct port scanning of hosts in internal networks, use other URLs such as that can access documents on the system (using file://), or use other protocols such as gopher:// or tftp://, which may provide greater control over the contents of requests.
         

Fix information:
repo_url: https://github.com/langchain-ai/langchain
commit_details: repo_url: https://github.com/langchain-ai/langchain
author: Eugene Yurtsev
author_date: 2023-10-17 13:41:43-04:00
msg: Add security note to recursive url loader (#11934)

Add security note to recursive loader
repository: repo_url: https://github.com/langchain-ai/langchain
repo_name: langchain-ai/langchain
description: 🦜🔗 Build context-aware reasoning applications
repo_language: Jupyter Notebook
owner: langchain-ai
file_changes: filename: recursive_url_loader.py
change_type: ModificationType.MODIFY
diff: @@ -49,7 +49,36 @@ def _metadata_extractor(raw_html: str, url: str) -> dict:
 
 
 class RecursiveUrlLoader(BaseLoader):
-    """Load all child links from a URL page."""
+    """Load all child links from a URL page.
+
+    **Security Note**: This loader is a crawler that will start crawling
+        at a given URL and then expand to crawl child links recursively.
+
+        Web crawlers should generally NOT be deployed with network access
+        to any internal servers.
+
+        Control access to who can submit crawling requests and what network access
+        the crawler has.
+
+        While crawling, the crawler may encounter malicious URLs that would lead to a
+        server-side request forgery (SSRF) attack.
+
+        To mitigate risks, the crawler by default will only load URLs from the same
+        domain as the start URL (controlled via prevent_outside named argument).
+
+        This will mitigate the risk of SSRF attacks, but will not eliminate it.
+
+        For example, if crawling a host which hosts several sites:
+
+        https://some_host/alice_site/
+        https://some_host/bob_site/
+
+        A malicious URL on Alice's site could cause the crawler to make a malicious
+        GET request to an endpoint on Bob's site. Both sites are hosted on the
+        same host, so such a request would not be prevented by default.
+
+        See https://python.langchain.com/docs/security
+    """
 
     def __init__(
         self,
@@ -60,12 +89,13 @@ class RecursiveUrlLoader(BaseLoader):
         metadata_extractor: Optional[Callable[[str, str], str]] = None,
         exclude_dirs: Optional[Sequence[str]] = (),
         timeout: Optional[int] = 10,
-        prevent_outside: Optional[bool] = True,
+        prevent_outside: bool = True,
         link_regex: Union[str, re.Pattern, None] = None,
         headers: Optional[dict] = None,
         check_response_status: bool = False,
     ) -> None:
         """Initialize with URL to crawl and any subdirectories to exclude.
+
         Args:
             url: The URL to crawl.
             max_depth: The max depth of the recursive loading.

programming_language: Python