Repository: mitmproxy/mitmproxy

Issue:
Host header shown in UI but missing in request
#### Problem Description
The Host header shows in the request flow UI but isn't in the raw request export

#### Steps to reproduce the behavior:
1. Capture a request to https://github.com for example
2. Host header appears in the flow view of request
3. Export as `raw_request` or `raw`
4. Host header is missing in the output file

#### System Information
Mitmproxy: 6.0.0.dev (+49, commit c48a2fc)
Python:    3.8.5
OpenSSL:   OpenSSL 1.1.1g  21 Apr 2020
Platform:  Linux-5.7.15-1-MANJARO-x86_64-with-glibc2.2.5



Discussion:
I think in addons/export.py, function raw_request should not return cleanup_request(f)  , it should just return f.request. 
I think you're right, it looks like the relevant lines in `cleanup_request` are only meant for httpie and curl exports.

https://github.com/mitmproxy/mitmproxy/blob/c48a2fcc4fa057d45b630bc779ec7c392e94b3fa/mitmproxy/addons/export.py#L15-L24
Thanks @zlowly! If one of you could send a PR, that'd be fantastic. ✨ 

Resolving patch:
diff --git a/mitmproxy/addons/export.py b/mitmproxy/addons/export.py
--- a/mitmproxy/addons/export.py
+++ b/mitmproxy/addons/export.py
@@ -18,7 +18,11 @@ def cleanup_request(f: flow.Flow) -> http.HTTPRequest:
     assert isinstance(f, http.HTTPFlow)
     request = f.request.copy()
     request.decode(strict=False)
-    # a bit of clean-up - these headers should be automatically set by curl/httpie
+    return request
+
+
+def pop_headers(request: http.HTTPRequest) -> http.HTTPRequest:
+    # Remove some headers that are redundant for curl/httpie export
     request.headers.pop('content-length')
     if request.headers.get("host", "") == request.host:
         request.headers.pop("host")
@@ -53,6 +57,7 @@ def request_content_for_console(request: http.HTTPRequest) -> str:
 
 def curl_command(f: flow.Flow) -> str:
     request = cleanup_request(f)
+    request = pop_headers(request)
     args = ["curl"]
     for k, v in request.headers.items(multi=True):
         if k.lower() == "accept-encoding":
@@ -70,6 +75,7 @@ def curl_command(f: flow.Flow) -> str:
 
 def httpie_command(f: flow.Flow) -> str:
     request = cleanup_request(f)
+    request = pop_headers(request)
     args = ["http", request.method, request.url]
     for k, v in request.headers.items(multi=True):
         args.append(f"{k}: {v}")


Test patch:
diff --git a/test/mitmproxy/addons/test_export.py b/test/mitmproxy/addons/test_export.py
--- a/test/mitmproxy/addons/test_export.py
+++ b/test/mitmproxy/addons/test_export.py
@@ -144,6 +144,7 @@ def test_req_and_resp_present(self, get_flow):
 
     def test_get_request_present(self, get_request):
         assert b"header: qvalue" in export.raw(get_request)
+        assert b"content-length: 0" in export.raw_request(get_request)
 
     def test_get_response_present(self, get_response):
         delattr(get_response, 'request')
@@ -163,6 +164,7 @@ def test_tcp(self, tcp_flow):
 class TestRawRequest:
     def test_get(self, get_request):
         assert b"header: qvalue" in export.raw_request(get_request)
+        assert b"content-length: 0" in export.raw_request(get_request)
 
     def test_no_request(self, get_response):
         delattr(get_response, 'request')