CVE-2022-24766 (published 2022-03-21T19:15Z)

mitmproxy is an interactive, SSL/TLS-capable intercepting proxy. In mitmproxy 7.0.4 and below, a malicious client or server is able to perform HTTP request smuggling attacks through mitmproxy. This means that a malicious client/server could smuggle a request/response through mitmproxy as part of another request/response's HTTP message body. While mitmproxy would only see one request, the target server would see multiple requests. A smuggled request is still captured as part of another request's body, but it does not appear in the request list and does not go through the usual mitmproxy event hooks, where users may have implemented custom access control checks or input sanitization. Unless mitmproxy is used to protect an HTTP/1 service, no action is required. The vulnerability has been fixed in mitmproxy 8.0.0 and above. There are currently no known workarounds.

Severity: HIGH
CVSS v2 base score: 7.5 (AV:N/AC:L/Au:N/C:P/I:P/A:P)
CVSS v3 base score: 9.8, CRITICAL (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H)

Weakness information:
cwe_id: CWE-444
cwe_name: Inconsistent Interpretation of HTTP Requests ('HTTP Request/Response Smuggling')
description: The product acts as an intermediary HTTP agent
         (such as a proxy or firewall) in the data flow between two
         entities such as a client and server, but it does not
         interpret malformed HTTP requests or responses in ways that
         are consistent with how the messages will be processed by
         those entities that are at the ultimate destination.
extended_description: 
	   HTTP requests or responses ("messages") can be
	   malformed or unexpected in ways that cause web servers or
	   clients to interpret the messages in different ways than
	   intermediary HTTP agents such as load balancers, reverse
	   proxies, web caching proxies, application firewalls,
	   etc. For example, an adversary may be able to add duplicate
	   or different header fields that a client or server might
	   interpret as one set of messages, whereas the intermediary
	   might interpret the same sequence of bytes as a different
	   set of messages. For example, discrepancies can arise in
	   how to handle duplicate headers like two Transfer-encoding
	   (TE) or two Content-length (CL), or the malicious HTTP
	   message will have different headers for TE and
	   CL.
	   The inconsistent parsing and interpretation of messages
	   can allow the adversary to "smuggle" a message to the
	   client/server without the intermediary being aware of it.
	   This weakness is usually the result of the usage
	   of outdated or incompatible HTTP protocol versions in the
	   HTTP agents.
         
         

Fix information:
repo_url: https://github.com/mitmproxy/mitmproxy
commit_details: repo_url: https://github.com/mitmproxy/mitmproxy
author: Maximilian Hils
author_date: 2022-03-16 09:56:50+01:00
msg: security: reject whitespace in HTTP/1 header names

This commit fixes GHSA-gcx2-gvj7-pxv3 by making mitmproxy
reject header names that contain whitespace characters by default.
A new `validate_inbound_headers` option is provided to turn this behavior
off at the expense of allowing HTTP smuggling vulnerabilities.
repository: repo_url: https://github.com/mitmproxy/mitmproxy
repo_name: mitmproxy/mitmproxy
description: An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
repo_language: Python
owner: mitmproxy
file_changes: filename: proxyserver.py
change_type: ModificationType.MODIFY
diff: @@ -98,6 +98,13 @@ class Proxyserver:
             in custom scripts are lowercased before they are sent.
             """,
         )
+        loader.add_option(
+            "validate_inbound_headers", bool, True,
+            """
+            Make sure that incoming HTTP requests are not malformed.
+            Disabling this option makes mitmproxy vulnerable to HTTP smuggling attacks.
+            """,
+        )
 
     async def running(self):
         self.master = ctx.master

programming_language: Python
filename: __init__.py
change_type: ModificationType.MODIFY
diff: @@ -3,6 +3,7 @@ from .read import (
     read_response_head,
     connection_close,
     expected_http_body_size,
+    validate_headers,
 )
 from .assemble import (
     assemble_request, assemble_request_head,
@@ -16,6 +17,7 @@ __all__ = [
     "read_response_head",
     "connection_close",
     "expected_http_body_size",
+    "validate_headers",
     "assemble_request", "assemble_request_head",
     "assemble_response", "assemble_response_head",
     "assemble_body",

programming_language: SQL
filename: read.py
change_type: ModificationType.MODIFY
diff: @@ -38,6 +38,38 @@ def connection_close(http_version, headers):
     )
 
 
+# https://datatracker.ietf.org/doc/html/rfc7230#section-3.2: Header fields are tokens.
+# "!" / "#" / "$" / "%" / "&" / "'" / "*" / "+" / "-" / "." /  "^" / "_" / "`" / "|" / "~" / DIGIT / ALPHA
+_valid_header_name = re.compile(rb"^[!#$%&'*+\-.^_`|~0-9a-zA-Z]+$")
+
+
+def validate_headers(
+    headers: Headers
+) -> None:
+    """
+    Validate headers to avoid request smuggling attacks. Raises a ValueError if they are malformed.
+    """
+
+    te_found = False
+    cl_found = False
+
+    for (name, value) in headers.fields:
+        if not _valid_header_name.match(name):
+            raise ValueError(f"Received an invalid header name: {name!r}. Invalid header names may introduce "
+                             f"request smuggling vulnerabilities. Disable the validate_inbound_headers option "
+                             f"to skip this security check.")
+
+        name_lower = name.lower()
+        te_found = te_found or name_lower == b"transfer-encoding"
+        cl_found = cl_found or name_lower == b"content-length"
+
+    if te_found and cl_found:
+        raise ValueError("Received both a Transfer-Encoding and a Content-Length header, "
+                         "refusing as recommended in RFC 7230 Section 3.3.3. "
+                         "See https://github.com/mitmproxy/mitmproxy/issues/4799 for details. "
+                         "Disable the validate_inbound_headers option to skip this security check.")
+
+
 def expected_http_body_size(
         request: Request,
         response: Optional[Response] = None
@@ -101,10 +133,8 @@ def expected_http_body_size(
     #        a message downstream.
     #
     if "transfer-encoding" in headers:
-        if "content-length" in headers:
-            raise ValueError("Received both a Transfer-Encoding and a Content-Length header, "
-                             "refusing as recommended in RFC 7230 Section 3.3.3. "
-                             "See https://github.com/mitmproxy/mitmproxy/issues/4799 for details.")
+        # we should make sure that there isn't also a content-length header.
+        # this is already handled in validate_headers.
 
         te: str = headers["transfer-encoding"]
         if not te.isascii():

programming_language: Python
filename: _http1.py
change_type: ModificationType.MODIFY
diff: @@ -234,6 +234,8 @@ class Http1Server(Http1Connection):
                 request_head = [bytes(x) for x in request_head]  # TODO: Make url.parse compatible with bytearrays
                 try:
                     self.request = http1.read_request_head(request_head)
+                    if self.context.options.validate_inbound_headers:
+                        http1.validate_headers(self.request.headers)
                     expected_body_size = http1.expected_http_body_size(self.request)
                 except ValueError as e:
                     yield commands.SendData(self.conn, make_error_response(400, str(e)))
@@ -330,6 +332,8 @@ class Http1Client(Http1Connection):
                 response_head = [bytes(x) for x in response_head]  # TODO: Make url.parse compatible with bytearrays
                 try:
                     self.response = http1.read_response_head(response_head)
+                    if self.context.options.validate_inbound_headers:
+                        http1.validate_headers(self.response.headers)
                     expected_size = http1.expected_http_body_size(self.request, self.response)
                 except ValueError as e:
                     yield commands.CloseConnection(self.conn)

programming_language: Python
filename: _http2.py
change_type: ModificationType.MODIFY
diff: @@ -40,7 +40,7 @@ class Http2Connection(HttpConnection):
     h2_conf_defaults = dict(
         header_encoding=False,
         validate_outbound_headers=False,
-        validate_inbound_headers=True,
+        # validate_inbound_headers is controlled by the validate_inbound_headers option.
         normalize_inbound_headers=False,  # changing this to True is required to pass h2spec
         normalize_outbound_headers=False,
     )
@@ -58,6 +58,7 @@ class Http2Connection(HttpConnection):
         if self.debug:
             self.h2_conf.logger = H2ConnectionLogger(f"{human.format_address(self.context.client.peername)}: "
                                                      f"{self.__class__.__name__}")
+        self.h2_conf.validate_inbound_headers = self.context.options.validate_inbound_headers
         self.h2_conn = BufferedH2Connection(self.h2_conf)
         self.streams = {}
 

programming_language: Python
filename: test_read.py
change_type: ModificationType.MODIFY
diff: @@ -4,7 +4,7 @@ from mitmproxy.http import Headers
 from mitmproxy.net.http.http1.read import (
     read_request_head,
     read_response_head, connection_close, expected_http_body_size,
-    _read_request_line, _read_response_line, _read_headers, get_header_tokens
+    _read_request_line, _read_response_line, _read_headers, get_header_tokens, validate_headers
 )
 from mitmproxy.test.tutils import treq, tresp
 
@@ -59,6 +59,19 @@ def test_read_response_head():
     assert r.content is None
 
 
+def test_validate_headers():
+    # both content-length and chunked (possible request smuggling)
+    with pytest.raises(ValueError, match="Received both a Transfer-Encoding and a Content-Length header"):
+        validate_headers(
+            Headers(transfer_encoding="chunked", content_length="42"),
+        )
+
+    with pytest.raises(ValueError, match="Received an invalid header name"):
+        validate_headers(
+            Headers([(b"content-length ", b"42")]),
+        )
+
+
 def test_expected_http_body_size():
     # Expect: 100-continue
     assert expected_http_body_size(
@@ -91,11 +104,6 @@ def test_expected_http_body_size():
     assert expected_http_body_size(
         treq(headers=Headers(transfer_encoding="gzip,\tchunked")),
     ) is None
-    # both content-length and chunked (possible request smuggling)
-    with pytest.raises(ValueError, match="Received both a Transfer-Encoding and a Content-Length header"):
-        expected_http_body_size(
-            treq(headers=Headers(transfer_encoding="chunked", content_length="42")),
-        )
     with pytest.raises(ValueError, match="Invalid transfer encoding"):
         expected_http_body_size(
             treq(headers=Headers(transfer_encoding="chun\u212Aed")),  # "chunKed".lower() == "chunked"

programming_language: Python
filename: test_http.py
change_type: ModificationType.MODIFY
diff: @@ -1261,6 +1261,37 @@ def test_request_smuggling(tctx):
     assert b"Received both a Transfer-Encoding and a Content-Length header" in err()
 
 
+def test_request_smuggling_whitespace(tctx):
+    """Test that we reject header names with whitespace"""
+    err = Placeholder(bytes)
+    assert (
+        Playbook(http.HttpLayer(tctx, HTTPMode.regular), hooks=False)
+        >> DataReceived(tctx.client, b"GET http://example.com/ HTTP/1.1\r\n"
+                                     b"Host: example.com\r\n"
+                                     b"Content-Length : 42\r\n\r\n")
+        << SendData(tctx.client, err)
+        << CloseConnection(tctx.client)
+    )
+    assert b"Received an invalid header name" in err()
+
+
+def test_request_smuggling_validation_disabled(tctx):
+    """Test that we don't reject request smuggling when validation is disabled."""
+    tctx.options.validate_inbound_headers = False
+    assert (
+        Playbook(http.HttpLayer(tctx, HTTPMode.regular), hooks=False)
+        >> DataReceived(tctx.client, b"GET http://example.com/ HTTP/1.1\r\n"
+                                     b"Host: example.com\r\n"
+                                     b"Content-Length: 4\r\n"
+                                     b"Transfer-Encoding: chunked\r\n\r\n"
+                                     b"4\r\n"
+                                     b"abcd\r\n"
+                                     b"0\r\n"
+                                     b"\r\n")
+        << OpenConnection(Placeholder(Server))
+    )
+
+
 def test_request_smuggling_te_te(tctx):
     """Test that we reject transfer-encoding headers that are weird in some way"""
     err = Placeholder(bytes)

programming_language: Python
filename: test_http2.py
change_type: ModificationType.MODIFY
diff: @@ -352,11 +352,11 @@ def test_http2_client_aborts(tctx, stream, when, how):
         assert "peer closed connection" in flow().error.msg
 
 
-@pytest.mark.xfail(reason="inbound validation turned on to protect against request smuggling")
 @pytest.mark.parametrize("normalize", [True, False])
 def test_no_normalization(tctx, normalize):
     """Test that we don't normalize headers when we just pass them through."""
     tctx.options.normalize_outbound_headers = normalize
+    tctx.options.validate_inbound_headers = False
 
     server = Placeholder(Server)
     flow = Placeholder(HTTPFlow)

programming_language: Python