Repository: mitmproxy/mitmproxy

Issue:
Misleading/incorrect message when port 8080 is taken (mitmweb)
#### Problem Description
- When port 8080 is already taken, `mitmweb` recommends you switch to port 8081.
- However, the web UI is already running on port 8081. The two collide, but no error message is printed.
- If you try to proxy your connections to 8081 as it's telling you, everything breaks.
- This is mainly a problem because non-experienced users won't know that the web UI runs on port 8081, and because no error is printed when the two collide, the proxy just doesn't work for no apparent reason.

#### Steps to reproduce the behavior:
1. Run any service on port 8080 (anything works as long as it blocks that port).
2. Run `mitmweb` (no arguments). A message is printed stating that port 8080 is taken and that you should switch to 8081 (!)
3. If you switch to port 8081, no errors are printed but a collision with the web UI happens silently. The web UI opens telling you to point your proxy to port 8081, which doesn't work.

#### System Information
```
Mitmproxy: 9.0.1
Python:    3.10.9
OpenSSL:   OpenSSL 1.1.1s  1 Nov 2022
Platform:  macOS-13.2-arm64-arm-64bit
```
Use dynamic port assignment to resolve collision issue between mitmweb and web UI
#### Description

To solve #5891, the fix introduces dynamic port assignment by specifying port 0. When mitmweb is started without specifying a port, it uses
```python
socket.bind(('', 0))
```
which allows the operating system to assign an available port.

#### Checklist

 - [x] I have updated tests where applicable.
 - [ ] I have added an entry to the CHANGELOG.



Discussion:
This seems like an already fixed issue (I'm using Mitmproxy: 10.0.0.dev), I tried this:

```bash
mitmweb --listen-port 8081  # another service is blocking port 8080
```

and an error is raised:

```python
...
  File "/path/to/mitmproxy/mitmproxy/tools/web/master.py", line 107, in running
    raise OSError(e.errno, message, e.filename) from e
OSError: [Errno 98] Web server failed to listen on 127.0.0.1:8081 with [Errno 98] Address already in use
Try specifying a different port by using `--set web_port=8082`.
```
That's odd. This is what happens for me:
`mitmweb` (no port specifier)
```
Error on startup: [Errno 48] HTTP(S) proxy failed to listen on *:8080 with [Errno 48] error while attempting to bind on address ('0.0.0.0', 8080): address already in use
Try specifying a different port by using `--mode regular@8081`.
```

Then using the recommended flag, it starts fine even though there's the collision.
Which OSes are y'all on?

On Fri, Jan 27, 2023, 13:37 Isaac McFadyen ***@***.***> wrote:

> That's odd. This is what happens for me:
> mitmweb (no port specifier)
>
> Error on startup: [Errno 48] HTTP(S) proxy failed to listen on *:8080 with [Errno 48] error while attempting to bind on address ('0.0.0.0', 8080): address already in use
> Try specifying a different port by using `--mode ***@***.***`.
>
> Then using the recommended flag, it starts fine.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/mitmproxy/mitmproxy/issues/5891#issuecomment-1406447848>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHY2PWASNVSJQOHGTYB4PDWUO6P7ANCNFSM6AAAAAAUIBWKBU>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>

I'm running on macOS Ventura 13.2 (ARM64, Apple Silicon)
I'm on Fedora 37
I tried this on macOS Big Sur:

```python
$ mitmweb  # port 8080 is blocked

Web server listening at http://127.0.0.1:8081/
[Errno 48] error while attempting to bind on address ('0.0.0.0', 8080): address already in use
Task exception was never retrieved
future: <Task finished name='Task-4' coro=<ErrorCheck._shutdown_if_errored() done, defined at /path/to/mitmproxy/venv/lib/python3.8/site-packages/mitmproxy/addons/errorcheck.py:18> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/path/to/mitmproxy/venv/lib/python3.8/site-packages/mitmproxy/addons/errorcheck.py", line 20, in _shutdown_if_errored
    sys.exit(1)
SystemExit: 1
```

This does try to open the web page but I can't replicate rest of the behaviour.
Sorry I was testing an older version of mitmproxy, upgraded to mitmproxy 9.0.1 and this issue does exist.
This seemed like an issue in tornado, I am not yet exactly sure why this works but if I comment out the lines in tornado's `netutils.py` setting `socket.SO_REUSEADDR` then it works as expected on macOS too. These are the lines:

- [netutil.py#L135](https://github.com/tornadoweb/tornado/blob/59536016e37704a1c47a3a0d262f46ceb6132c06/tornado/netutil.py#L135)
- [netutil.py#L141](https://github.com/tornadoweb/tornado/blob/59536016e37704a1c47a3a0d262f46ceb6132c06/tornado/netutil.py#L141)


This could be due to the fact that the proxy binds to all local addresses whereas the web page only binds to 127.0.0.1 and through whatever magic SO_REUSEADDR does, this might be allowing the web page to bind to port 8081 even when the proxy is already bound to that port. Running the following command, it fixes the issue (assuming raising an error is the desired behaviour):

```
mitmweb --mode regular@8081 --web-host 0.0.0.0
```


Resolving patch:
diff --git a/mitmproxy/proxy/mode_servers.py b/mitmproxy/proxy/mode_servers.py
--- a/mitmproxy/proxy/mode_servers.py
+++ b/mitmproxy/proxy/mode_servers.py
@@ -279,7 +279,7 @@ async def _start(self) -> None:
                 assert (
                     self.mode.custom_listen_host is None
                 )  # since [@ [listen_addr:]listen_port]
-                message += f"\nTry specifying a different port by using `--mode {self.mode.full_spec}@{port + 1}`."
+                message += f"\nTry specifying a different port by using `--mode {self.mode.full_spec}@{port + 2}`."
             raise OSError(e.errno, message, e.filename) from e
 
     async def _stop(self) -> None:
diff --git a/mitmproxy/tools/web/master.py b/mitmproxy/tools/web/master.py
--- a/mitmproxy/tools/web/master.py
+++ b/mitmproxy/tools/web/master.py
@@ -103,7 +103,7 @@ async def running(self):
         except OSError as e:
             message = f"Web server failed to listen on {self.options.web_host or '*'}:{self.options.web_port} with {e}"
             if e.errno == errno.EADDRINUSE:
-                message += f"\nTry specifying a different port by using `--set web_port={self.options.web_port + 1}`."
+                message += f"\nTry specifying a different port by using `--set web_port={self.options.web_port + 2}`."
             raise OSError(e.errno, message, e.filename) from e
 
         logger.info(


Test patch:
diff --git a/test/mitmproxy/tools/web/test_master.py b/test/mitmproxy/tools/web/test_master.py
--- a/test/mitmproxy/tools/web/test_master.py
+++ b/test/mitmproxy/tools/web/test_master.py
@@ -15,6 +15,6 @@ async def test_reuse():
     master = WebMaster(Options(), with_termlog=False)
     master.options.web_host = "127.0.0.1"
     master.options.web_port = port
-    with pytest.raises(OSError, match=f"--set web_port={port + 1}"):
+    with pytest.raises(OSError, match=f"--set web_port={port + 2}"):
         await master.running()
     server.close()