Repository: django/asgiref

Issue:
WsgiToAsgi should stop sending response body after content-length bytes
Hi,

I was having issues when serving partial requests (using Range headers) from [whitenoise](https://github.com/evansd/whitenoise) wrapped in WsgiToAsgi. Some HTTP and ASGI servers in front of that complained about extra content in the response body (I've had the issue with traefik in front of uvicorn and with [uvicorn itself when using the httptools protocol handler](https://github.com/encode/uvicorn/blob/master/uvicorn/protocols/http/httptools_impl.py#L517)).

One contributing factor is whitenoise which returns the file handle to the whole file in a [wsgiref.FileWrapper](https://github.com/python/cpython/blob/master/Lib/wsgiref/util.py#L11), seeked to the correct position for the range request. It then expects the server to only read as many bytes as defined in the Content-Length header.

The [WSGI specs allows for this behavior](https://www.python.org/dev/peps/pep-3333/#handling-the-content-length-header) and [gunicorn implements it](https://github.com/benoitc/gunicorn/blob/master/gunicorn/http/wsgi.py#L333) and stops reading after as many bytes as specified in Content-Length.

The [ASGI spec](https://asgi.readthedocs.io/en/latest/specs/www.html#http) does not say much about the consistency between Content-Length header and the "http.response.body" events. The relevant parts could be :
> Protocol servers must flush any data passed to them into the send buffer before returning from a send call.

> Yielding content from the WSGI application maps to sending http.response.body messages.

The first one suggests that any piece of body from an "http.response.body" event must be sent, the other one could mean that all behavior defined for WSGI should apply to ASGI, including not-sending extra body, maybe ?

[uvicorn](https://github.com/encode/uvicorn/blob/master/uvicorn/protocols/http/h11_impl.py#L475), [daphe](https://github.com/django/daphne/blob/master/daphne/http_protocol.py#L255) and [hypercorn](https://gitlab.com/pgjones/hypercorn/-/blob/master/src/hypercorn/protocol/http_stream.py#L147) lean more towards the first interpretation and pass through the whole readable content or simply raise an exception.

If you agree that the WsgiToAsgi adapter should clamp the body I can work on a patch.

Aside from this, the ASGI spec could probably be improved by defining this point. I don't really know if ASGI servers should prefer clamping or forwarding extra body responses but I have the vague feeling that allowing ASGI events to generate incorrectly framed HTTP messages could generate hard to diagnose issues in the future without providing any benefit.

If the sendfile extension from #184 is added, the same point will probably arise, it would be very inconvenient to only be able to use sendfile when serving whole files. WSGI also specifies the clamping behavior [when using sendfile](https://www.python.org/dev/peps/pep-3333/#optional-platform-specific-file-handling) and [gunicorn](https://github.com/benoitc/gunicorn/blob/master/gunicorn/http/wsgi.py#L365) implements it.



Discussion:

Patch:
diff --git a/asgiref/wsgi.py b/asgiref/wsgi.py
index 8811118..40fba20 100644
--- a/asgiref/wsgi.py
+++ b/asgiref/wsgi.py
@@ -29,6 +29,7 @@ class WsgiToAsgiInstance:
     def __init__(self, wsgi_application):
         self.wsgi_application = wsgi_application
         self.response_started = False
+        self.response_content_length = None
 
     async def __call__(self, scope, receive, send):
         if scope["type"] != "http":
@@ -114,6 +115,11 @@ class WsgiToAsgiInstance:
             (name.lower().encode("ascii"), value.encode("ascii"))
             for name, value in response_headers
         ]
+        # Extract content-length
+        self.response_content_length = None
+        for name, value in response_headers:
+            if name.lower() == "content-length":
+                self.response_content_length = int(value)
         # Build and send response start message.
         self.response_start = {
             "type": "http.response.start",
@@ -130,14 +136,25 @@ class WsgiToAsgiInstance:
         # Translate the scope and incoming request body into a WSGI environ
         environ = self.build_environ(self.scope, body)
         # Run the WSGI app
+        bytes_sent = 0
         for output in self.wsgi_application(environ, self.start_response):
             # If this is the first response, include the response headers
             if not self.response_started:
                 self.response_started = True
                 self.sync_send(self.response_start)
+            # If the application supplies a Content-Length header
+            if self.response_content_length is not None:
+                # The server should not transmit more bytes to the client than the header allows
+                bytes_allowed = self.response_content_length - bytes_sent
+                if len(output) > bytes_allowed:
+                    output = output[:bytes_allowed]
             self.sync_send(
                 {"type": "http.response.body", "body": output, "more_body": True}
             )
+            bytes_sent += len(output)
+            # The server should stop iterating over the response when enough data has been sent
+            if bytes_sent == self.response_content_length:
+                break
         # Close connection
         if not self.response_started:
             self.response_started = True


Test patch:
diff --git a/tests/test_wsgi.py b/tests/test_wsgi.py
index 3490573..6dcee1c 100644
--- a/tests/test_wsgi.py
+++ b/tests/test_wsgi.py
@@ -1,3 +1,5 @@
+import sys
+
 import pytest
 
 from asgiref.testing import ApplicationCommunicator
@@ -126,6 +128,130 @@ async def test_wsgi_empty_body():
     assert (await instance.receive_output(1)) == {"type": "http.response.body"}
 
 
+@pytest.mark.asyncio
+async def test_wsgi_clamped_body():
+    """
+    Makes sure WsgiToAsgi clamps a body response longer than Content-Length
+    """
+
+    def wsgi_application(environ, start_response):
+        start_response("200 OK", [("Content-Length", "8")])
+        return [b"0123", b"45", b"6789"]
+
+    application = WsgiToAsgi(wsgi_application)
+    instance = ApplicationCommunicator(
+        application,
+        {
+            "type": "http",
+            "http_version": "1.0",
+            "method": "GET",
+            "path": "/",
+            "query_string": b"",
+            "headers": [],
+        },
+    )
+    await instance.send_input({"type": "http.request"})
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.start",
+        "status": 200,
+        "headers": [(b"content-length", b"8")],
+    }
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.body",
+        "body": b"0123",
+        "more_body": True,
+    }
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.body",
+        "body": b"45",
+        "more_body": True,
+    }
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.body",
+        "body": b"67",
+        "more_body": True,
+    }
+    assert (await instance.receive_output(1)) == {"type": "http.response.body"}
+
+
+@pytest.mark.asyncio
+async def test_wsgi_stops_iterating_after_content_length_bytes():
+    """
+    Makes sure WsgiToAsgi does not iterate after than Content-Length bytes
+    """
+
+    def wsgi_application(environ, start_response):
+        start_response("200 OK", [("Content-Length", "4")])
+        yield b"0123"
+        pytest.fail("WsgiToAsgi should not iterate after Content-Length bytes")
+        yield b"4567"
+
+    application = WsgiToAsgi(wsgi_application)
+    instance = ApplicationCommunicator(
+        application,
+        {
+            "type": "http",
+            "http_version": "1.0",
+            "method": "GET",
+            "path": "/",
+            "query_string": b"",
+            "headers": [],
+        },
+    )
+    await instance.send_input({"type": "http.request"})
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.start",
+        "status": 200,
+        "headers": [(b"content-length", b"4")],
+    }
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.body",
+        "body": b"0123",
+        "more_body": True,
+    }
+    assert (await instance.receive_output(1)) == {"type": "http.response.body"}
+
+
+@pytest.mark.asyncio
+async def test_wsgi_multiple_start_response():
+    """
+    Makes sure WsgiToAsgi only keep Content-Length from the last call to start_response
+    """
+
+    def wsgi_application(environ, start_response):
+        start_response("200 OK", [("Content-Length", "5")])
+        try:
+            raise ValueError("Application Error")
+        except ValueError:
+            start_response("500 Server Error", [], sys.exc_info())
+            return [b"Some long error message"]
+
+    application = WsgiToAsgi(wsgi_application)
+    instance = ApplicationCommunicator(
+        application,
+        {
+            "type": "http",
+            "http_version": "1.0",
+            "method": "GET",
+            "path": "/",
+            "query_string": b"",
+            "headers": [],
+        },
+    )
+    await instance.send_input({"type": "http.request"})
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.start",
+        "status": 500,
+        "headers": [],
+    }
+    assert (await instance.receive_output(1)) == {
+        "type": "http.response.body",
+        "body": b"Some long error message",
+        "more_body": True,
+    }
+    assert (await instance.receive_output(1)) == {"type": "http.response.body"}
+
+
 @pytest.mark.asyncio
 async def test_wsgi_multi_body():
     """