Repository: mitmproxy/mitmproxy

Issue:
Async `client_connected` hook is broken in 10.2.4
#### Problem Description

I  finally got around upgrading to mitmproxy 10 and things didn't work at all. I was able to track it down to the async `client_connected` hook not working. It does work in 10.0.0 and broken in 10.2.4. Not sure about other hooks, this one starts the entire chain and hence that's where all connections end.

#### Steps to reproduce the behavior:

addon.py

```py
import asyncio


async def client_connected(client):
    print("client_connected...")
    await asyncio.sleep(1)
    print("...client_connected")
```

1. `mitmdump --scripts addon.py`
2. `curl --insecure --proxy localhost:8080 https://example.com/`

Mitmproxy 9.0.1 output:

```
[10:02:00.891] Loading script addon.py
[10:02:00.892] HTTP(S) proxy listening at *:8080.
[10:02:03.639][[::1]:37674] client connect
client_connected...
...client_connected
[10:02:04.746][[::1]:37674] server connect example.com:443 ([2606:2800:220:1:248:1893:25c8:1946]:443)
[::1]:37674: GET https://example.com/ HTTP/2.0
 << HTTP/2.0 200 OK 1.2k
[10:02:05.091][[::1]:37674] client disconnect
[10:02:05.092][[::1]:37674] server disconnect example.com:443 ([2606:2800:220:1:248:1893:25c8:1946]:443)
```

Mitmproxy 10.0.0 output:

```
[10:06:17.770] Loading script addon.py
[10:06:17.771] HTTP(S) proxy listening at *:8080.
[10:06:19.790][[::1]:33394] client connect
client_connected...
...client_connected
[10:06:20.898][[::1]:33394] server connect example.com:443 ([2606:2800:220:1:248:1893:25c8:1946]:443)
[::1]:33394: GET https://example.com/ HTTP/2.0
 << HTTP/2.0 200 OK 1.2k
[10:06:21.241][[::1]:33394] client disconnect
[10:06:21.241][[::1]:33394] server disconnect example.com:443 ([2606:2800:220:1:248:1893:25c8:1946]:443)
```

Mitmproxy 10.2.4 output:

```
[10:03:04.389] Loading script addon.py
[10:03:04.389] HTTP(S) proxy listening at *:8080.
[10:03:07.451][[::1]:35406] client connect
client_connected...
...client_connected
[10:03:08.452][[::1]:35406] mitmproxy has crashed!
Traceback (most recent call last):
  File "mitmproxy/proxy/server.py", line 381, in server_event
  File "mitmproxy/proxy/layer.py", line 148, in handle_event
  File "mitmproxy/proxy/utils.py", line 27, in _check_event_type
AssertionError: Unexpected event type at HttpProxy._handle_event: Expected Start, got DataReceived(client, b'CONNECT example.com:443 HTTP/1.1\r\nHost: example.com:443\r\nUser-Agent: curl/8.2.1\r\nProxy-Connection: Keep-Alive\r\n\r\n').
```

#### System Information

```
Mitmproxy: 10.2.4 binary
Python:    3.12.2
OpenSSL:   OpenSSL 3.2.1 30 Jan 2024
Platform:  Linux-6.5.0-25-generic-x86_64-with-glibc2.38
```


Discussion:
Broke with `>=10.2.0`
Thanks for the super nice report.

Bisected to 6e38a56f4c399551c3fe3d399bddfa8083ccd0f1 - this is very likely caused by eager task execution as well. 
The problem is here:

https://github.com/mitmproxy/mitmproxy/blob/13f42105fd29c0a5dc1889803bac2cc1a460d63d/mitmproxy/proxy/server.py#L147-L153

The handler is now processing events before we reach `server_event(events.Start())`. We probably shouldn't just reorder those statements as this would cause `.handler` to be unset when `server_event` is called initially. We should figure out if we can explicitly spawn tasks lazily here. :)

Resolving patch:
diff --git a/mitmproxy/proxy/server.py b/mitmproxy/proxy/server.py
--- a/mitmproxy/proxy/server.py
+++ b/mitmproxy/proxy/server.py
@@ -144,13 +144,13 @@ async def handle_client(self) -> None:
             assert writer
             writer.close()
         else:
+            self.server_event(events.Start())
             handler = asyncio_utils.create_task(
                 self.handle_connection(self.client),
                 name=f"client connection handler",
                 client=self.client.peername,
             )
             self.transports[self.client].handler = handler
-            self.server_event(events.Start())
             await asyncio.wait([handler])
             if not handler.cancelled() and (e := handler.exception()):
                 self.log(
@@ -249,20 +249,14 @@ async def open_connection(self, command: commands.OpenConnection) -> None:
                 await self.handle_hook(server_hooks.ServerConnectedHook(hook_data))
                 self.server_event(events.OpenConnectionCompleted(command, None))
 
-                # during connection opening, this function is the designated handler that can be cancelled.
-                # once we have a connection, we do want the teardown here to happen in any case, so we
-                # reassign the handler to .handle_connection and then clean up here once that is done.
-                new_handler = asyncio_utils.create_task(
-                    self.handle_connection(command.connection),
-                    name=f"server connection handler for {addr}",
-                    client=self.client.peername,
-                )
-                self.transports[command.connection].handler = new_handler
-                await asyncio.wait([new_handler])
-
-                self.log(f"server disconnect {addr}")
-                command.connection.timestamp_end = time.time()
-                await self.handle_hook(server_hooks.ServerDisconnectedHook(hook_data))
+                try:
+                    await self.handle_connection(command.connection)
+                finally:
+                    self.log(f"server disconnect {addr}")
+                    command.connection.timestamp_end = time.time()
+                    await self.handle_hook(
+                        server_hooks.ServerDisconnectedHook(hook_data)
+                    )
 
     async def wakeup(self, request: commands.RequestWakeup) -> None:
         await asyncio.sleep(request.delay)
@@ -383,7 +377,7 @@ def server_event(self, event: events.Event) -> None:
                     assert command.connection not in self.transports
                     handler = asyncio_utils.create_task(
                         self.open_connection(command),
-                        name=f"server connection manager {command.connection.address}",
+                        name=f"server connection handler {command.connection.address}",
                         client=self.client.peername,
                     )
                     self.transports[command.connection] = ConnectionIO(handler=handler)


Test patch: