Repository: mitmproxy/mitmproxy

Issue:
Reply from proxy causes Mitmproxy crash
#### Problem Description


#### Steps to reproduce the behavior:
```python
from mitmproxy import http

class MrCrash:

    def request(self, flow: http.HTTPFlow) -> None:
        flow.response = http.Response.make(407)


addons = [
    MrCrash()
]
```
Execute `ALL_PROXY="http://0.0.0.0:8080" curl -vvv -k https://ironpeak.be/`

And in the logs:
```
info: Loading script scripts/crash.py
info: Proxy server listening at http://*:8080
info: 172.17.0.1:58468: client connect
error: 172.17.0.1:58468: mitmproxy has crashed!
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/server.py", line
276, in server_event
    for command in layer_commands:
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 114,
in handle_event
    yield from self.__process(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 142,
in __process
    command = next(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 186,
in handle_event
    yield from self._handle(event)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 114,
in handle_event
    yield from self.__process(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 142,
in __process
    command = next(command_generator)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 590, in event_to_child
    for command in child.handle_event(event):
⇩  [1/1]   [scripts:1]                                                     [*:8080]
Error: 172.17.0.1:58468: mitmproxy has crashed!(more in eventlog)
```

#### System Information
```shell
mitmproxy --version
Mitmproxy: 7.0.0.dev
Python:    3.9.2
OpenSSL:   OpenSSL 1.1.1i  8 Dec 2020
Platform:  Linux-4.19.121-linuxkit-aarch64-with-glibc2.28
```

Reply from proxy causes Mitmproxy crash
#### Problem Description


#### Steps to reproduce the behavior:
```python
from mitmproxy import http

class MrCrash:

    def request(self, flow: http.HTTPFlow) -> None:
        flow.response = http.Response.make(407)


addons = [
    MrCrash()
]
```
Execute `ALL_PROXY="http://0.0.0.0:8080" curl -vvv -k https://ironpeak.be/`

And in the logs:
```
info: Loading script scripts/crash.py
info: Proxy server listening at http://*:8080
info: 172.17.0.1:58468: client connect
error: 172.17.0.1:58468: mitmproxy has crashed!
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/server.py", line
276, in server_event
    for command in layer_commands:
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 114,
in handle_event
    yield from self.__process(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 142,
in __process
    command = next(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 186,
in handle_event
    yield from self._handle(event)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 114,
in handle_event
    yield from self.__process(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 142,
in __process
    command = next(command_generator)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 590, in event_to_child
    for command in child.handle_event(event):
⇩  [1/1]   [scripts:1]                                                     [*:8080]
Error: 172.17.0.1:58468: mitmproxy has crashed!(more in eventlog)
```

#### System Information
```shell
mitmproxy --version
Mitmproxy: 7.0.0.dev
Python:    3.9.2
OpenSSL:   OpenSSL 1.1.1i  8 Dec 2020
Platform:  Linux-4.19.121-linuxkit-aarch64-with-glibc2.28
```

Reply from proxy causes Mitmproxy crash
#### Problem Description


#### Steps to reproduce the behavior:
```python
from mitmproxy import http

class MrCrash:

    def request(self, flow: http.HTTPFlow) -> None:
        flow.response = http.Response.make(407)


addons = [
    MrCrash()
]
```
Execute `ALL_PROXY="http://0.0.0.0:8080" curl -vvv -k https://ironpeak.be/`

And in the logs:
```
info: Loading script scripts/crash.py
info: Proxy server listening at http://*:8080
info: 172.17.0.1:58468: client connect
error: 172.17.0.1:58468: mitmproxy has crashed!
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/server.py", line
276, in server_event
    for command in layer_commands:
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 114,
in handle_event
    yield from self.__process(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 142,
in __process
    command = next(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 186,
in handle_event
    yield from self._handle(event)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 114,
in handle_event
    yield from self.__process(command_generator)
  File "/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layer.py", line 142,
in __process
    command = next(command_generator)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File
"/usr/local/lib/python3.9/site-packages/mitmproxy/proxy/layers/http/__init__.py",
line 590, in event_to_child
    for command in child.handle_event(event):
⇩  [1/1]   [scripts:1]                                                     [*:8080]
Error: 172.17.0.1:58468: mitmproxy has crashed!(more in eventlog)
```

#### System Information
```shell
mitmproxy --version
Mitmproxy: 7.0.0.dev
Python:    3.9.2
OpenSSL:   OpenSSL 1.1.1i  8 Dec 2020
Platform:  Linux-4.19.121-linuxkit-aarch64-with-glibc2.28
```



Discussion:
Thanks for the detailed info. I can reproduce this with the latest master and the [official example](https://github.com/mitmproxy/mitmproxy/blob/6d9f00408c489b9f9e8bf7a9dbf6ac02f271faa0/examples/addons/http-reply-from-proxy.py) when updating it to HTTPS

```py
"""Send a reply from the proxy without sending any data to the remote server."""
from mitmproxy import http


def request(flow: http.HTTPFlow) -> None:
    if flow.request.pretty_url == "https://example.com/path":
        flow.response = http.Response.make(
            200,  # (optional) status code
            b"Hello World",  # (optional) content
            {"Content-Type": "text/html"}  # (optional) headers
        )
```

The response however is successfully delivered to curl.

Full stack

```
Traceback (most recent call last):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/server.py", line 276, in server_event
    for command in layer_commands:
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 186, in handle_event
    yield from self._handle(event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 590, in event_to_child
    for command in child.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 492, in passthrough
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 85, in _handle_event
    yield from self.event_to_child(event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 104, in event_to_child
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 72, in _handle_event
    yield from self.receive_data(event.data)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/tls.py", line 246, in receive_data
    yield from self.event_to_child(
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 104, in event_to_child
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 604, in event_to_child
    conn = self.connections[command.connection]
KeyError: Server({'id': '…cd155e', 'address': ('example.com', 443), 'state': <ConnectionState.CLOSED: 0>, 'tls': True})

```
Thanks for the detailed info. I can reproduce this with the latest master and the [official example](https://github.com/mitmproxy/mitmproxy/blob/6d9f00408c489b9f9e8bf7a9dbf6ac02f271faa0/examples/addons/http-reply-from-proxy.py) when updating it to HTTPS

```py
"""Send a reply from the proxy without sending any data to the remote server."""
from mitmproxy import http


def request(flow: http.HTTPFlow) -> None:
    if flow.request.pretty_url == "https://example.com/path":
        flow.response = http.Response.make(
            200,  # (optional) status code
            b"Hello World",  # (optional) content
            {"Content-Type": "text/html"}  # (optional) headers
        )
```

The response however is successfully delivered to curl.

Full stack

```
Traceback (most recent call last):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/server.py", line 276, in server_event
    for command in layer_commands:
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 186, in handle_event
    yield from self._handle(event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 590, in event_to_child
    for command in child.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 492, in passthrough
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 85, in _handle_event
    yield from self.event_to_child(event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 104, in event_to_child
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 72, in _handle_event
    yield from self.receive_data(event.data)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/tls.py", line 246, in receive_data
    yield from self.event_to_child(
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 104, in event_to_child
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 604, in event_to_child
    conn = self.connections[command.connection]
KeyError: Server({'id': '…cd155e', 'address': ('example.com', 443), 'state': <ConnectionState.CLOSED: 0>, 'tls': True})

```
Thanks for the detailed info. I can reproduce this with the latest master and the [official example](https://github.com/mitmproxy/mitmproxy/blob/6d9f00408c489b9f9e8bf7a9dbf6ac02f271faa0/examples/addons/http-reply-from-proxy.py) when updating it to HTTPS

```py
"""Send a reply from the proxy without sending any data to the remote server."""
from mitmproxy import http


def request(flow: http.HTTPFlow) -> None:
    if flow.request.pretty_url == "https://example.com/path":
        flow.response = http.Response.make(
            200,  # (optional) status code
            b"Hello World",  # (optional) content
            {"Content-Type": "text/html"}  # (optional) headers
        )
```

The response however is successfully delivered to curl.

Full stack

```
Traceback (most recent call last):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/server.py", line 276, in server_event
    for command in layer_commands:
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 186, in handle_event
    yield from self._handle(event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 590, in event_to_child
    for command in child.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 492, in passthrough
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 85, in _handle_event
    yield from self.event_to_child(event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 104, in event_to_child
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 72, in _handle_event
    yield from self.receive_data(event.data)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/tls.py", line 246, in receive_data
    yield from self.event_to_child(
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/tunnel.py", line 104, in event_to_child
    for command in self.child_layer.handle_event(event):
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 114, in handle_event
    yield from self.__process(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layer.py", line 142, in __process
    command = next(command_generator)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 581, in _handle_event
    yield from self.event_to_child(handler, event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 602, in event_to_child
    yield from self.event_to_child(stream, command.event)
  File "/home/alex/forks/mitmproxy/mitmproxy/proxy/layers/http/__init__.py", line 604, in event_to_child
    conn = self.connections[command.connection]
KeyError: Server({'id': '…cd155e', 'address': ('example.com', 443), 'state': <ConnectionState.CLOSED: 0>, 'tls': True})

```

Resolving patch:
diff --git a/mitmproxy/proxy/layers/http/__init__.py b/mitmproxy/proxy/layers/http/__init__.py
--- a/mitmproxy/proxy/layers/http/__init__.py
+++ b/mitmproxy/proxy/layers/http/__init__.py
@@ -376,7 +376,7 @@ def handle_protocol_error(
         is_client_error_but_we_already_talk_upstream = (
             isinstance(event, RequestProtocolError)
             and self.client_state in (self.state_stream_request_body, self.state_done)
-            and self.server_state != self.state_errored
+            and self.server_state not in (self.state_done, self.state_errored)
         )
         need_error_hook = not (
             self.client_state in (self.state_wait_for_request_headers, self.state_errored)


Test patch: