Repository: mitmproxy/mitmproxy

Issue:
Wait for connection hooks to finish before calling HTTP hooks
#### Problem Description

This issue is in response to your comment https://github.com/mitmproxy/mitmproxy/issues/4207#issuecomment-907014592

> Ah, you are hitting a special case here: We are not waiting for server_connected to complete before notifying the HTTP layer that the connection has been established (I didn't think this would be useful to anyone and it surely isn't faster). If you have a concrete use case here please open a separate issue, this should be straightforward to adjust. In fact at the moment it's a bit more complicated because we don't block.

To summarize: I'm doing rather heavy work in every hook and communicate with a different service (gRPC calls to Node.js that process the hooks). When using `@concurrent` (and probably same with async hooks) hooks such as `server_connected` and `request` are racing each other (I think that's actually the only case). With synchronous hooks this cannot happen because whatever I do in `server_connected` blocks the entire proxy.

```py
import time
from mitmproxy.script import concurrent


@concurrent
def client_connected(data):
    print('>>> client_connected')
    time.sleep(2)
    print('<<< client_connected')


@concurrent
def server_connect(data):
    print('>>> server_connect')
    time.sleep(2)
    print('<<< server_connect')


@concurrent
def server_connected(data):
    print('>>> server_connected')
    time.sleep(2)
    print('<<< server_connected')


@concurrent
def request(flow):
    print('>>> request')
    time.sleep(2)
    print('<<< request')


@concurrent
def response(flow):
    print('>>> response')
    time.sleep(2)
    print('<<< response')


@concurrent
def client_disconnected(data):
    print('>>> client_disconnected')
    time.sleep(2)
    print('<<< client_disconnected')


@concurrent
def server_disconnected(data):
    print('>>> server_disconnected')
    time.sleep(2)
    print('<<< server_disconnected')
```

As you can see `server_connected` and `request` are interleaved.

```
>>> client_connected
<<< client_connected
>>> server_connect
<<< server_connect
>>> server_connected
>>> request
<<< server_connected
<<< request
>>> response
<<< response
>>> client_disconnected
<<< client_disconnected
>>> server_disconnected
<<< server_disconnected
```

#### Proposal

Wait for the connection hooks to finish before continuing with the HTTP layer.

#### Alternatives

I could leave the connection event synchronous so that the always block and finish. But depending on the number of clients and servers this would mostly diminish the effect of making all other hooks async or concurrent.


Discussion:
If you remove the `blocking: bool = False` lines in https://github.com/mitmproxy/mitmproxy/blob/main/mitmproxy/proxy/server_hooks.py and then change
https://github.com/mitmproxy/mitmproxy/blob/83b43b93a7ac6cf2bfe557964fd63b376e158e25/mitmproxy/proxy/server.py#L179-L185
to 
```python
await self.handle_hook(server_hooks.ServerConnectedHook(hook_data))
```
and remove
https://github.com/mitmproxy/mitmproxy/blob/83b43b93a7ac6cf2bfe557964fd63b376e158e25/mitmproxy/proxy/server.py#L204

you should be done. I don't have the time to test that properly, but if it works for you please go ahead and open a PR. :)
Thanks, that looks easy enough. I'll look into it!

Resolving patch:
diff --git a/mitmproxy/proxy/server.py b/mitmproxy/proxy/server.py
--- a/mitmproxy/proxy/server.py
+++ b/mitmproxy/proxy/server.py
@@ -176,14 +176,7 @@ async def open_connection(self, command: commands.OpenConnection) -> None:
                 else:
                     addr = human.format_address(command.connection.address)
                 self.log(f"server connect {addr}")
-                connected_hook = asyncio_utils.create_task(
-                    self.handle_hook(server_hooks.ServerConnectedHook(hook_data)),
-                    name=f"handle_hook(server_connected) {addr}",
-                    client=self.client.peername,
-                )
-                if not connected_hook:
-                    return  # this should not be needed, see asyncio_utils.create_task
-
+                await self.handle_hook(server_hooks.ServerConnectedHook(hook_data))
                 self.server_event(events.OpenConnectionCompleted(command, None))
 
                 # during connection opening, this function is the designated handler that can be cancelled.
@@ -201,7 +194,6 @@ async def open_connection(self, command: commands.OpenConnection) -> None:
 
                 self.log(f"server disconnect {addr}")
                 command.connection.timestamp_end = time.time()
-                await connected_hook  # wait here for this so that closed always comes after connected.
                 await self.handle_hook(server_hooks.ServerDisconnectedHook(hook_data))
 
     async def handle_connection(self, connection: Connection) -> None:
diff --git a/mitmproxy/proxy/server_hooks.py b/mitmproxy/proxy/server_hooks.py
--- a/mitmproxy/proxy/server_hooks.py
+++ b/mitmproxy/proxy/server_hooks.py
@@ -20,7 +20,6 @@ class ClientDisconnectedHook(commands.StartHook):
     """
     A client connection has been closed (either by us or the client).
     """
-    blocking = False
     client: connection.Client
 
 
@@ -50,7 +49,6 @@ class ServerConnectedHook(commands.StartHook):
     """
     Mitmproxy has connected to a server.
     """
-    blocking = False
     data: ServerConnectionHookData
 
 
@@ -59,5 +57,4 @@ class ServerDisconnectedHook(commands.StartHook):
     """
     A server connection has been closed (either by us or the server).
     """
-    blocking = False
     data: ServerConnectionHookData


Test patch: