Repository: mitmproxy/mitmproxy

Issue:
Small bug in handling of streamed data from callable flow.*.stream
I have finished writing an addon to save streamed requests and responses to files. While getting it to work I stumbled over the problem that some sites fail to load with: 

```
HTTP/2 connection closed: <ConnectionTerminated error_code:ErrorCodes.FRAME_SIZE_ERROR, last_stream_id:1, additional_data:None>
```

The issue is that the callable `flow.request.stream` and `flow.response.stream` at the end of the stream is supposed to return the zero byte string it receives which is then sent as an HTTP/2 chunk. For responses this may be tolerated by the client but for requests it breaks protocol.

I fixed this issue with this patch which I _think_ is probably the best way to fix it given the contract with the callable:

```diff
diff --git a/mitmproxy/proxy/layers/http/__init__.py b/mitmproxy/proxy/layers/http/__init__.py
index 0b134f7bb..9068e9020 100644
--- a/mitmproxy/proxy/layers/http/__init__.py
+++ b/mitmproxy/proxy/layers/http/__init__.py
@@ -244,10 +244,14 @@ class HttpStream(layer.Layer):
         elif isinstance(event, RequestEndOfMessage):
             if callable(self.flow.request.stream):
                 chunks = self.flow.request.stream(b"")
+                send = True
                 if isinstance(chunks, bytes):
+                    if len(chunks) == 0:
+                        send = False
                     chunks = [chunks]
-                for chunk in chunks:
-                    yield SendHttp(RequestData(self.stream_id, chunk), self.context.server)
+                if send:
+                    for chunk in chunks:
+                        yield SendHttp(RequestData(self.stream_id, chunk), self.context.server)
 
             self.flow.request.timestamp_end = time.time()
             yield HttpRequestHook(self.flow)
@@ -336,10 +340,14 @@ class HttpStream(layer.Layer):
         elif isinstance(event, ResponseEndOfMessage):
             if callable(self.flow.response.stream):
                 chunks = self.flow.response.stream(b"")
+                send = True
                 if isinstance(chunks, bytes):
+                    if len(chunks) == 0:
+                        send = False
                     chunks = [chunks]
-                for chunk in chunks:
-                    yield SendHttp(ResponseData(self.stream_id, chunk), self.context.client)
+                if send:
+                    for chunk in chunks:
+                        yield SendHttp(ResponseData(self.stream_id, chunk), self.context.client)
             yield from self.send_response(already_streamed=True)
 
     @expect(ResponseData, ResponseTrailers, ResponseEndOfMessage)
```


Discussion:

Resolving patch:
diff --git a/mitmproxy/proxy/layers/http/__init__.py b/mitmproxy/proxy/layers/http/__init__.py
--- a/mitmproxy/proxy/layers/http/__init__.py
+++ b/mitmproxy/proxy/layers/http/__init__.py
@@ -244,7 +244,9 @@ def state_stream_request_body(self, event: Union[RequestData, RequestEndOfMessag
         elif isinstance(event, RequestEndOfMessage):
             if callable(self.flow.request.stream):
                 chunks = self.flow.request.stream(b"")
-                if isinstance(chunks, bytes):
+                if chunks == b"":
+                    chunks = []
+                elif isinstance(chunks, bytes):
                     chunks = [chunks]
                 for chunk in chunks:
                     yield SendHttp(RequestData(self.stream_id, chunk), self.context.server)
@@ -336,7 +338,9 @@ def state_stream_response_body(self, event: events.Event) -> layer.CommandGenera
         elif isinstance(event, ResponseEndOfMessage):
             if callable(self.flow.response.stream):
                 chunks = self.flow.response.stream(b"")
-                if isinstance(chunks, bytes):
+                if chunks == b"":
+                    chunks = []
+                elif isinstance(chunks, bytes):
                     chunks = [chunks]
                 for chunk in chunks:
                     yield SendHttp(ResponseData(self.stream_id, chunk), self.context.client)


Test patch: