Repository: mitmproxy/mitmproxy

Issue:
map-local not working in 5.3.0
#### Problem Description
We use mitmproxy locally to replay backend queries made from frontend. After updating from 5.2 to 5.3.0 every command that has the --map-local options seems to work otherwise correctly but mitmproxy seems to completely ignore the --map-local part when replaying. On startup mitmproxy does validate that the local resource is present so it doesn't completely ignore the option but it never actually seems to use it.
The replay command that we use also has flows that are replayed and that part does work as intended. But even if we leave the flows out of the command and only use --map-local command the local resource is not served.

#### Steps to reproduce the behavior:
1. Make a local resource file (local.json) and, optionally, prepare a flows file (mitmproxy-flows)
2. Choose a server you are trying to mimick with replay
3. Run command `mitmdump --mode reverse:http://localhost:8910/ -S mitmproxy-flows --server-replay-nopop --server-replay-refresh --listen-port 7777 --map-local "|~m GET|localhost:8910/api/resource|local.json"`
4. Try to go to URL http://localhost:7777/api/resource

#### System Information
WORKING version info is:
```
Mitmproxy: 5.2 binary
Python:    3.7.8
OpenSSL:   OpenSSL 1.1.1g  21 Apr 2020
Platform:  Darwin-19.6.0-x86_64-i386-64bit
```
(note that I used the macOS binaries here when testing with 5.2 but it was also working with 5.2 installed with homebrew)

NOT WORKING version info is:
```
Mitmproxy: 5.3.0
Python:    3.9.0
OpenSSL:   OpenSSL 1.1.1h  22 Sep 2020
Platform:  macOS-10.15.7-x86_64-i386-64bit
```


Discussion:
@mplattner, can you reproduce this?
Any update to this? This is currently forcing the team to stay in the previous version and I guess I will downgrade too.
While I understand that non-reproduced bugs are not main priority to you but for us this is certain (reproduced on separate machines) problem. @mhils could you at least confirm that there has not been changes in map-local syntax between 5.2 and 5.3.0? We do not see any changes in documentation at least.

Of course there might be more pressing bugs to fix too so I will not complain but I just hope that at least in some version in the future this issue is fixed so that we can continue using the latest version of mitmproxy.

Another comment about versions in homebrew. For two people in our team homebrew decided to do housekeeping and updated mitmproxy version number without consent. That would not be a problem if homebrew would have kept the old version locally OR if the old version would still be available to download in the net. Alas neither of these is true. So for those two people the only option is to use the docker images or the precompiled binaries. Seems rather strange but I guess this is one of those things that can not be helped.
@Krakau: Thanks for the very polite bugging. :) I'll try to squeeze this in somewhere tomorrow.

Regarding your questions:

We didn't even touch the map local addon between 5.2 and 5.3, so no syntax changes: https://github.com/mitmproxy/mitmproxy/commits/master/mitmproxy/addons/maplocal.py
So this is a bit of a curious case anyway. :-)

> Another comment about versions in homebrew. [...]

There's not much we can do on the homebrew front (we do very much appreciate how quick they update if we publish releases though). You can get old standalone binaries at https://mitmproxy.org/downloads/, you can use `pipx`, or you can use the Docker images. Doesn't that qualify as "the old version is still available to download in the net"? What else would you be looking for?
Ok, it looks like I can reproduce this now. The first bad commit is 46fbba639dd3581fa427db6d2.

Resolving patch:
diff --git a/mitmproxy/proxy/protocol/http.py b/mitmproxy/proxy/protocol/http.py
--- a/mitmproxy/proxy/protocol/http.py
+++ b/mitmproxy/proxy/protocol/http.py
@@ -8,6 +8,7 @@
 from mitmproxy import exceptions
 from mitmproxy import http
 from mitmproxy import flow
+from mitmproxy.net.http import url
 from mitmproxy.proxy.protocol import base
 from mitmproxy.proxy.protocol.websocket import WebSocketLayer
 from mitmproxy.net import websockets
@@ -324,7 +325,10 @@ def _process_flow(self, f):
 
         # update host header in reverse proxy mode
         if self.config.options.mode.startswith("reverse:") and not self.config.options.keep_host_header:
-            f.request.host_header = self.config.upstream_server.address[0]
+            f.request.host_header = url.hostport(
+                self.config.upstream_server.scheme,
+                *self.config.upstream_server.address
+            )
 
         # Determine .scheme, .host and .port attributes for inline scripts. For
         # absolute-form requests, they are directly given in the request. For


Test patch:
diff --git a/test/mitmproxy/proxy/test_server.py b/test/mitmproxy/proxy/test_server.py
--- a/test/mitmproxy/proxy/test_server.py
+++ b/test/mitmproxy/proxy/test_server.py
@@ -452,7 +452,7 @@ def test_overridden_host_header(self):
         assert resp.status_code == 200
 
         req = self.master.state.flows[0].request
-        assert req.host_header == "127.0.0.1"
+        assert req.host_header.startswith("127.0.0.1:")
 
     @pytest.mark.asyncio
     async def test_selfconnection(self):