Repository: mitmproxy/mitmproxy

Issue:
mitmproxy doesn't merge multiple Cookie headers when proxying HTTP/2 request to HTTP/1.1 server
#### Problem Description

when mitmproxy running as reverse-proxy mode, mitmproxy accepts HTTP/2 connection, even upstream is plain HTTP server (w/ HTTP 1.1).

in HTTP/2, multiple cookie header is allowed, but old HTTP/1.x isn't. so we should merge multiple Cookie headers to one header when proxying to HTTP/1.1 server (see https://www.rfc-editor.org/rfc/rfc7540#section-8.1.2.5 https://www.rfc-editor.org/rfc/rfc6265#section-5.4 ), but mitmproxy isn't.

![image](https://user-images.githubusercontent.com/6533808/167728747-8e540cd4-acf7-40f1-b32f-4940782041b3.png)

and actually some servers doesn't handle multiple Cookie headers on HTTP/1.x; so it causes mysterious bug (in my case, breaks Mastodon's sign-in).

#### Steps to reproduce the behavior:
1. Run mitmproxy as reverse-proxy mode, like `mitmproxy --mode=reverse:http://localhost -p 443 --certs fullchain-and-privkey.pem --set keep_host_header --modify-header '/x-forwarded-proto/https'`
2. Access to mitmproxy port with HTTPS, `https://localhost` and add some cookies
3. Check actual HTTP request by Wireshark or something (note: some servers are automatically handles multiple cookie headers even HTTP/1.1, like Node.js)

#### System Information

```
$ mitmproxy --version
Mitmproxy: 8.0.0
Python:    3.10.4
OpenSSL:   OpenSSL 1.1.1o  3 May 2022
Platform:  macOS-12.3.1-arm64-arm-64bit
```

(installed from Homebrew)
mitmproxy doesn't merge multiple Cookie headers when proxying HTTP/2 request to HTTP/1.1 server
#### Problem Description

when mitmproxy running as reverse-proxy mode, mitmproxy accepts HTTP/2 connection, even upstream is plain HTTP server (w/ HTTP 1.1).

in HTTP/2, multiple cookie header is allowed, but old HTTP/1.x isn't. so we should merge multiple Cookie headers to one header when proxying to HTTP/1.1 server (see https://www.rfc-editor.org/rfc/rfc7540#section-8.1.2.5 https://www.rfc-editor.org/rfc/rfc6265#section-5.4 ), but mitmproxy isn't.

![image](https://user-images.githubusercontent.com/6533808/167728747-8e540cd4-acf7-40f1-b32f-4940782041b3.png)

and actually some servers doesn't handle multiple Cookie headers on HTTP/1.x; so it causes mysterious bug (in my case, breaks Mastodon's sign-in).

#### Steps to reproduce the behavior:
1. Run mitmproxy as reverse-proxy mode, like `mitmproxy --mode=reverse:http://localhost -p 443 --certs fullchain-and-privkey.pem --set keep_host_header --modify-header '/x-forwarded-proto/https'`
2. Access to mitmproxy port with HTTPS, `https://localhost` and add some cookies
3. Check actual HTTP request by Wireshark or something (note: some servers are automatically handles multiple cookie headers even HTTP/1.1, like Node.js)

#### System Information

```
$ mitmproxy --version
Mitmproxy: 8.0.0
Python:    3.10.4
OpenSSL:   OpenSSL 1.1.1o  3 May 2022
Platform:  macOS-12.3.1-arm64-arm-64bit
```

(installed from Homebrew)


Discussion:
Thanks for the clear report! The relevant code that handles h2 -> h1 conversion is here:
https://github.com/mitmproxy/mitmproxy/blob/01a772ed4ee673307ba6b1eebfdeb5df65b13e99/mitmproxy/proxy/layers/http/_http1.py#L342-L350

I'd be happy to merge a PR that fixes this. 😃 
Thanks for the clear report! The relevant code that handles h2 -> h1 conversion is here:
https://github.com/mitmproxy/mitmproxy/blob/01a772ed4ee673307ba6b1eebfdeb5df65b13e99/mitmproxy/proxy/layers/http/_http1.py#L342-L350

I'd be happy to merge a PR that fixes this. 😃 

Resolving patch:
diff --git a/mitmproxy/proxy/layers/http/_http1.py b/mitmproxy/proxy/layers/http/_http1.py
--- a/mitmproxy/proxy/layers/http/_http1.py
+++ b/mitmproxy/proxy/layers/http/_http1.py
@@ -348,6 +348,12 @@ def send(self, event: HttpEvent) -> layer.CommandGenerator[None]:
                 if "Host" not in request.headers and request.authority:
                     request.headers.insert(0, "Host", request.authority)
                 request.authority = ""
+                cookie_headers = request.headers.get_all("Cookie")
+                if len(cookie_headers) > 1:
+                    # Only HTTP/2 supports multiple cookie headers, HTTP/1.x does not.
+                    # see: https://www.rfc-editor.org/rfc/rfc6265#section-5.4
+                    #      https://www.rfc-editor.org/rfc/rfc7540#section-8.1.2.5
+                    request.headers["Cookie"] = "; ".join(cookie_headers)
             raw = http1.assemble_request_head(request)
             yield commands.SendData(self.conn, raw)
         elif isinstance(event, RequestData):


Test patch:
diff --git a/test/mitmproxy/proxy/layers/http/test_http_version_interop.py b/test/mitmproxy/proxy/layers/http/test_http_version_interop.py
--- a/test/mitmproxy/proxy/layers/http/test_http_version_interop.py
+++ b/test/mitmproxy/proxy/layers/http/test_http_version_interop.py
@@ -11,12 +11,20 @@
 from mitmproxy.proxy.layers import http
 from test.mitmproxy.proxy.layers.http.hyper_h2_test_helpers import FrameFactory
 from test.mitmproxy.proxy.layers.http.test_http2 import (
-    example_request_headers,
     example_response_headers,
     make_h2,
 )
 from test.mitmproxy.proxy.tutils import Placeholder, Playbook, reply
 
+example_request_headers = (
+    (b":method", b"GET"),
+    (b":scheme", b"http"),
+    (b":path", b"/"),
+    (b":authority", b"example.com"),
+    (b"cookie", "a=1"),
+    (b"cookie", "b=2"),
+)
+
 h2f = FrameFactory()
 
 
@@ -69,7 +77,7 @@ def test_h2_to_h1(tctx):
         >> reply()
         << OpenConnection(server)
         >> reply(None)
-        << SendData(server, b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\n")
+        << SendData(server, b"GET / HTTP/1.1\r\nHost: example.com\r\ncookie: a=1; b=2\r\n\r\n")
         >> DataReceived(server, b"HTTP/1.1 200 OK\r\nContent-Length: 12\r\n\r\n")
         << http.HttpResponseHeadersHook(flow)
         >> reply()