CVE-2021-43818 (published 2021-12-13T18:15Z)

lxml is a library for processing XML and HTML in the Python language. Prior to version 4.6.5, the HTML Cleaner in lxml.html lets certain crafted script content pass through, as well as script content in SVG files embedded using data URIs. Users that employ the HTML cleaner in a security relevant context should upgrade to lxml 4.6.5 to receive a patch. There are no known workarounds available.

Severity: MEDIUM
CVSS v2 base score: 6.8 (AV:N/AC:M/Au:N/C:P/I:P/A:P)
CVSS v3 base score: 7.1, HIGH (CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:L/I:L/A:L)

Weakness information:
cwe_id: CWE-79
cwe_name: Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting')
description: The product does not neutralize or incorrectly neutralizes user-controllable input before it is placed in output that is used as a web page that is served to other users.
extended_description: 
            Cross-site scripting (XSS) vulnerabilities occur when:
            
               Untrusted data enters a web application, typically from a web request.
               The web application dynamically generates a web page that contains this untrusted data.
               During page generation, the application does not prevent the data from containing content that is executable by a web browser, such as JavaScript, HTML tags, HTML attributes, mouse events, Flash, ActiveX, etc.
               A victim visits the generated web page through a web browser, which contains malicious script that was injected using the untrusted data.
               Since the script comes from a web page that was sent by the web server, the victim's web browser executes the malicious script in the context of the web server's domain.
               This effectively violates the intention of the web browser's same-origin policy, which states that scripts in one domain should not be able to access resources or run code in a different domain.
            
            There are three main kinds of XSS:
            
               
                  Type 1: Reflected XSS (or Non-Persistent) - 
         			The server reads data directly from the HTTP request and reflects it back in the HTTP response. Reflected XSS exploits occur when an attacker causes a victim to supply dangerous content to a vulnerable web application, which is then reflected back to the victim and executed by the web browser. The most common mechanism for delivering malicious content is to include it as a parameter in a URL that is posted publicly or e-mailed directly to the victim. URLs constructed in this manner constitute the core of many phishing schemes, whereby an attacker convinces a victim to visit a URL that refers to a vulnerable site. After the site reflects the attacker's content back to the victim, the content is executed by the victim's browser.
               
                  Type 2: Stored XSS (or Persistent) - 
               The application stores dangerous data in a database, message forum, visitor log, or other trusted data store. At a later time, the dangerous data is subsequently read back into the application and included in dynamic content. From an attacker's perspective, the optimal place to inject malicious content is in an area that is displayed to either many users or particularly interesting users. Interesting users typically have elevated privileges in the application or interact with sensitive data that is valuable to the attacker. If one of these users executes malicious content, the attacker may be able to perform privileged operations on behalf of the user or gain access to sensitive data belonging to the user. For example, the attacker might inject XSS into a log message, which might not be handled properly when an administrator views the logs.
            
               
                  Type 0: DOM-Based XSS - 
               In DOM-based XSS, the client performs the injection of XSS into the page; in the other types, the server performs the injection. DOM-based XSS generally involves server-controlled, trusted script that is sent to the client, such as Javascript that performs sanity checks on a form before the user submits it. If the server-supplied script processes user-supplied data and then injects it back into the web page (such as with dynamic HTML), then DOM-based XSS is possible.
            
            
            Once the malicious script is injected, the attacker can perform a variety of malicious activities. The attacker could transfer private information, such as cookies that may include session information, from the victim's machine to the attacker. The attacker could send malicious requests to a web site on behalf of the victim, which could be especially dangerous to the site if the victim has administrator privileges to manage that site. Phishing attacks could be used to emulate trusted web sites and trick the victim into entering a password, allowing the attacker to compromise the victim's account on that web site. Finally, the script could exploit a vulnerability in the web browser itself possibly taking over the victim's machine, sometimes referred to as "drive-by hacking."
            In many cases, the attack can be launched without the victim even being aware of it. Even with careful users, attackers frequently use a variety of methods to encode the malicious portion of the attack, such as URL encoding or Unicode, so the request looks less suspicious.
         
         
cwe_id: CWE-74
cwe_name: Improper Neutralization of Special Elements in Output Used by a Downstream Component ('Injection')
description: The product constructs all or part of a command, data structure, or record using externally-influenced input from an upstream component, but it does not neutralize or incorrectly neutralizes special elements that could modify how it is parsed or interpreted when it is sent to a downstream component.
extended_description: Software or other automated logic has certain assumptions about what constitutes data and control respectively. It is the lack of verification of these assumptions for user-controlled input that leads to injection problems. Injection problems encompass a wide variety of issues -- all mitigated in very different ways and usually attempted in order to alter the control flow of the process. For this reason, the most effective way to discuss these weaknesses is to note the distinct features that classify them as injection weaknesses. The most important issue to note is that all injection problems share one thing in common -- i.e., they allow for the injection of control plane data into the user-controlled data plane. This means that the execution of the process may be altered by sending code in through legitimate data channels, using no other mechanism. While buffer overflows, and many other flaws, involve the use of some further issue to gain execution, injection problems need only for the data to be parsed.
         

Fix information:
repo_url: https://github.com/lxml/lxml
commit_details: repo_url: https://github.com/lxml/lxml
author: Stefan Behnel
author_date: 2021-11-11 13:21:08+01:00
msg: Cleaner: Remove SVG image data URLs since they can embed script content.

Reported as GHSL-2021-1038
repository: repo_url: https://github.com/lxml/lxml
repo_name: lxml/lxml
description: The lxml XML toolkit for Python
repo_language: Python
owner: lxml
file_changes: filename: clean.py
change_type: ModificationType.MODIFY
diff: @@ -75,18 +75,25 @@ _looks_like_tag_content = re.compile(
 
 # All kinds of schemes besides just javascript: that can cause
 # execution:
-_is_image_dataurl = re.compile(
-    r'^data:image/.+;base64', re.I).search
+_find_image_dataurls = re.compile(
+    r'^data:image/(.+);base64,', re.I).findall
 _is_possibly_malicious_scheme = re.compile(
-    r'(?:javascript|jscript|livescript|vbscript|data|about|mocha):',
-    re.I).search
+    r'(javascript|jscript|livescript|vbscript|data|about|mocha):',
+    re.I).findall
+# SVG images can contain script content
+_is_unsafe_image_type = re.compile(r"(xml|svg)", re.I).findall
+
 def _is_javascript_scheme(s):
-    if _is_image_dataurl(s):
-        return None
-    return _is_possibly_malicious_scheme(s)
+    is_image_url = False
+    for image_type in _find_image_dataurls(s):
+        is_image_url = True
+        if _is_unsafe_image_type(image_type):
+            return True
+    if is_image_url:
+        return False
+    return bool(_is_possibly_malicious_scheme(s))
 
 _substitute_whitespace = re.compile(r'[\s\x00-\x08\x0B\x0C\x0E-\x19]+').sub
-# FIXME: should data: be blocked?
 
 # FIXME: check against: http://msdn2.microsoft.com/en-us/library/ms537512.aspx
 _conditional_comment_re = re.compile(

programming_language: Python
filename: test_clean.py
change_type: ModificationType.MODIFY
diff: @@ -1,3 +1,5 @@
+import base64
+import gzip
 import unittest
 from lxml.tests.common_imports import make_doctest
 
@@ -143,6 +145,49 @@ class CleanerTest(unittest.TestCase):
                 cleaned,
                 "%s  ->  %s" % (style_code, cleaned))
 
+    def test_svg_data_links(self):
+        # Remove SVG images with potentially insecure content.
+        svg = b'<svg onload="alert(123)" />'
+        svgz = gzip.compress(svg)
+        svg_b64 = base64.b64encode(svg).decode('ASCII')
+        svgz_b64 = base64.b64encode(svgz).decode('ASCII')
+        urls = [
+            "data:image/svg+xml;base64," + svg_b64,
+            "data:image/svg+xml-compressed;base64," + svgz_b64,
+        ]
+        for url in urls:
+            html = '<img src="%s">' % url
+            s = lxml.html.fragment_fromstring(html)
+
+            cleaned = lxml.html.tostring(clean_html(s))
+            self.assertEqual(
+                b'<img src="">',
+                cleaned,
+                "%s  ->  %s" % (url, cleaned))
+
+    def test_image_data_links(self):
+        data = b'123'
+        data_b64 = base64.b64encode(data).decode('ASCII')
+        urls = [
+            "data:image/jpeg;base64," + data_b64,
+            "data:image/apng;base64," + data_b64,
+            "data:image/png;base64," + data_b64,
+            "data:image/gif;base64," + data_b64,
+            "data:image/webp;base64," + data_b64,
+            "data:image/bmp;base64," + data_b64,
+            "data:image/tiff;base64," + data_b64,
+            "data:image/x-icon;base64," + data_b64,
+        ]
+        for url in urls:
+            html = '<img src="%s">' % url
+            s = lxml.html.fragment_fromstring(html)
+
+            cleaned = lxml.html.tostring(clean_html(s))
+            self.assertEqual(
+                html.encode("UTF-8"),
+                cleaned,
+                "%s  ->  %s" % (url, cleaned))
+
     def test_formaction_attribute_in_button_input(self):
         # The formaction attribute overrides the form's action and should be
         # treated as a malicious link attribute

programming_language: Python
repo_url: https://github.com/lxml/lxml
commit_details: repo_url: https://github.com/lxml/lxml
author: Stefan Behnel
author_date: 2021-12-12 15:10:58+01:00
msg: Prepare release of 4.6.5.
repository: repo_url: https://github.com/lxml/lxml
repo_name: lxml/lxml
description: The lxml XML toolkit for Python
repo_language: Python
owner: lxml
file_changes: filename: CHANGES.txt
change_type: ModificationType.MODIFY
diff: @@ -2,7 +2,7 @@
 lxml changelog
 ==============
 
-4.6.5 (2021-12-??)
+4.6.5 (2021-12-12)
 ==================
 
 Bugs fixed

programming_language: Markdown
filename: main.txt
change_type: ModificationType.MODIFY
diff: @@ -159,8 +159,8 @@ Index <http://pypi.python.org/pypi/lxml/>`_ (PyPI).  It has the source
 that compiles on various platforms.  The source distribution is signed
 with `this key <pubkey.asc>`_.
 
-The latest version is `lxml 4.6.4`_, released 2021-11-01
-(`changes for 4.6.4`_).  `Older versions <#old-versions>`_
+The latest version is `lxml 4.6.5`_, released 2021-12-12
+(`changes for 4.6.5`_).  `Older versions <#old-versions>`_
 are listed below.
 
 Please take a look at the
@@ -256,7 +256,9 @@ See the websites of lxml
 ..
    and the `latest in-development version <http://lxml.de/dev/>`_.
 
-.. _`PDF documentation`: lxmldoc-4.6.4.pdf
+.. _`PDF documentation`: lxmldoc-4.6.5.pdf
+
+* `lxml 4.6.5`_, released 2021-12-12 (`changes for 4.6.5`_)
 
 * `lxml 4.6.4`_, released 2021-11-01 (`changes for 4.6.4`_)
 
@@ -284,6 +286,7 @@ See the websites of lxml
 
 * `older releases <http://lxml.de/4.3/#old-versions>`_
 
+.. _`lxml 4.6.5`: /files/lxml-4.6.5.tgz
 .. _`lxml 4.6.4`: /files/lxml-4.6.4.tgz
 .. _`lxml 4.6.3`: /files/lxml-4.6.3.tgz
 .. _`lxml 4.6.2`: /files/lxml-4.6.2.tgz
@@ -297,6 +300,7 @@ See the websites of lxml
 .. _`lxml 4.4.1`: /files/lxml-4.4.1.tgz
 .. _`lxml 4.4.0`: /files/lxml-4.4.0.tgz
 
+.. _`changes for 4.6.5`: /changes-4.6.5.html
 .. _`changes for 4.6.4`: /changes-4.6.4.html
 .. _`changes for 4.6.3`: /changes-4.6.3.html
 .. _`changes for 4.6.2`: /changes-4.6.2.html

programming_language: Markdown
filename: __init__.py
change_type: ModificationType.MODIFY
diff: @@ -1,6 +1,6 @@
 # this is a package
 
-__version__ = "4.6.4"
+__version__ = "4.6.5"
 
 
 def get_include():

programming_language: Python
repo_url: https://github.com/lxml/lxml
commit_details: repo_url: https://github.com/lxml/lxml
author: Stefan Behnel
author_date: 2021-11-11 12:20:57+01:00
msg: Cleaner: Prevent "@import" from re-occurring in the CSS after replacements, e.g. "@@importimport".

Reported as GHSL-2021-1037
repository: repo_url: https://github.com/lxml/lxml
repo_name: lxml/lxml
description: The lxml XML toolkit for Python
repo_language: Python
owner: lxml
file_changes: filename: clean.py
change_type: ModificationType.MODIFY
diff: @@ -541,6 +541,8 @@ class Cleaner(object):
             return True
         if 'expression(' in style:
             return True
+        if '@import' in style:
+            return True
         if '</noscript' in style:
             # e.g. '<noscript><style><a title="</noscript><img src=x onerror=alert(1)>">'
             return True

programming_language: Python
filename: test_clean.py
change_type: ModificationType.MODIFY
diff: @@ -123,6 +123,26 @@ class CleanerTest(unittest.TestCase):
             b'<math><style>/* deleted */</style></math>',
             lxml.html.tostring(clean_html(s)))
 
+    def test_sneaky_import_in_style(self):
+        # Prevent "@@importimport" -> "@import" replacement.
+        style_codes = [
+            "@@importimport(extstyle.css)",
+            "@ @  import import(extstyle.css)",
+            "@ @ importimport(extstyle.css)",
+            "@@  import import(extstyle.css)",
+            "@ @import import(extstyle.css)",
+            "@@importimport()",
+        ]
+        for style_code in style_codes:
+            html = '<style>%s</style>' % style_code
+            s = lxml.html.fragment_fromstring(html)
+
+            cleaned = lxml.html.tostring(clean_html(s))
+            self.assertEqual(
+                b'<style>/* deleted */</style>',
+                cleaned,
+                "%s  ->  %s" % (style_code, cleaned))
+
     def test_formaction_attribute_in_button_input(self):
         # The formaction attribute overrides the form's action and should be
         # treated as a malicious link attribute

programming_language: Python