Repository: sanic-org/sanic

Issue:
The Sanic built-in server blocks for exactly 90 seconds on status code 412
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Describe the bug

Sanic, like most web servers, usually responds to requests. However, if the response has status code 412, it is very slow and takes exactly **90 extra seconds** to respond, stalling after the handler finishes.

This behavior does not happen when running Sanic with uvicorn. Only the official Sanic server. It also doesn't happen with FastAPI.


### Code snippet

```python
import sanic

app = sanic.Sanic(__name__)

@app.get("/")
async def root(req: sanic.Request):
    status = int(req.args.get("status", "200"))
    return sanic.json({"message": "Hello World"}, status=status)
```

### Expected Behavior

```bash
sanic main:app --port 8051
```

Then:

```sh-session
$ curl http://localhost:8051
{"message":"Hello World"}
$ curl http://localhost:8051/?status=400  # fine
{"message":"Hello World"}
$ curl http://localhost:8051/?status=411  # fine
{"message":"Hello World"}
$ curl http://localhost:8051/?status=413  # fine
{"message":"Hello World"}
$ curl http://localhost:8051/?status=412
# stalls with no response for 90 seconds
```

### How do you run Sanic?

Sanic CLI

### Operating System

Linux

### Sanic Version

Sanic v23.6.0

### Additional context

I have reproduced this on both Linux and macOS. I have also reproduced this using both the Sanic CLI and the `Sanic.serve()` function programmatically.


Discussion:
Thanks for the report. This affects HTTP status ~~304~~ and 412, which are not supposed to have "entity headers" (https://datatracker.ietf.org/doc/html/rfc2616#section-7.1).

The response gets sent immediately (use `curl -v`) but without `content-length` because that header gets stripped off by Sanic but it still sends the body bytes. Curl sees the body but doesn't know what to do with it and waits for disconnection to signal the end of it.

Triaging:
sanic/helpers.py - stripping off entity headers
sanic/response/types.py - handling of 304, 412 by above
sanic/http/http1.py:343 - gets trimmed headers and then sends body anyway

Notes: `transfer-encoding: chunked` is not being stripped as entity header. Due to this a response that looks like chunked encoding, given that header is received by Curl decoded 🥶

```python
return sanic.text("4\r\nBUGS\r\n0\r\n\r\n", status=412, headers={"transfer-encoding": "chunked"})
```

I believe the 412 handling of Sanic as it is now is broken and handled in wrong location, causing this confusion within HTTP protocol (at least HTTP1, didn't check HTTP3 or ASGI). For those status codes in addition to entity headers also `transfer-encoding` and the body data would need to be stripped, it seems.

Referring to @ahopkins for further review.

Status code 304 is NOT affected because it is correctly handled by `sanic.helpers.has_message_body`.

Checking the HTTP RFC, I cannot find why entity headers or body should be stripped off 412 responses. Removing that stripping would allow body in such responses, which seems correct.

Thanks for finding the issue and responding so quickly — makes sense. I wonder why specifically 412 was treated differently. :)

> The response gets sent immediately (use `curl -v`) but without `content-length` because that header gets stripped off by Sanic but it still sends the body bytes. Curl sees the body but doesn't know what to do with it and waits for disconnection to signal the end of it.

Makes sense then, so the 90 seconds is a TCP timeout.

Resolving patch:
diff --git a/sanic/helpers.py b/sanic/helpers.py
--- a/sanic/helpers.py
+++ b/sanic/helpers.py
@@ -122,25 +122,6 @@ def is_hop_by_hop_header(header):
     return header.lower() in _HOP_BY_HOP_HEADERS
 
 
-def remove_entity_headers(headers, allowed=("content-location", "expires")):
-    """
-    Removes all the entity headers present in the headers given.
-    According to RFC 2616 Section 10.3.5,
-    Content-Location and Expires are allowed as for the
-    "strong cache validator".
-    https://tools.ietf.org/html/rfc2616#section-10.3.5
-
-    returns the headers without the entity headers
-    """
-    allowed = set([h.lower() for h in allowed])
-    headers = {
-        header: value
-        for header, value in headers.items()
-        if not is_entity_header(header) or header.lower() in allowed
-    }
-    return headers
-
-
 def import_string(module_name, package=None):
     """
     import a module or class by string path.
diff --git a/sanic/response/types.py b/sanic/response/types.py
--- a/sanic/response/types.py
+++ b/sanic/response/types.py
@@ -24,7 +24,6 @@
     Default,
     _default,
     has_message_body,
-    remove_entity_headers,
 )
 from sanic.http import Http
 
@@ -104,9 +103,6 @@ def processed_headers(self) -> Iterator[Tuple[bytes, bytes]]:
         Returns:
             Iterator[Tuple[bytes, bytes]]: A list of header tuples encoded in bytes for sending
         """  # noqa: E501
-        # TODO: Make a blacklist set of header names and then filter with that
-        if self.status in (304, 412):  # Not Modified, Precondition Failed
-            self.headers = remove_entity_headers(self.headers)
         if has_message_body(self.status):
             self.headers.setdefault("content-type", self.content_type)
         # Encode headers into bytes


Test patch:
diff --git a/tests/test_helpers.py b/tests/test_helpers.py
--- a/tests/test_helpers.py
+++ b/tests/test_helpers.py
@@ -41,28 +41,6 @@ def test_is_hop_by_hop_header():
         assert helpers.is_hop_by_hop_header(header) is expected
 
 
-def test_remove_entity_headers():
-    tests = (
-        ({}, {}),
-        ({"Allow": "GET, POST, HEAD"}, {}),
-        (
-            {
-                "Content-Type": "application/json",
-                "Expires": "Wed, 21 Oct 2015 07:28:00 GMT",
-                "Foo": "Bar",
-            },
-            {"Expires": "Wed, 21 Oct 2015 07:28:00 GMT", "Foo": "Bar"},
-        ),
-        (
-            {"Allow": "GET, POST, HEAD", "Content-Location": "/test"},
-            {"Content-Location": "/test"},
-        ),
-    )
-
-    for header, expected in tests:
-        assert helpers.remove_entity_headers(header) == expected
-
-
 def test_import_string_class():
     obj = helpers.import_string("sanic.config.Config")
     assert isinstance(obj, Config)
diff --git a/tests/test_response.py b/tests/test_response.py
--- a/tests/test_response.py
+++ b/tests/test_response.py
@@ -178,6 +178,10 @@ async def no_content_unmodified_handler(request: Request):
     async def unmodified_handler(request: Request):
         return json(JSON_DATA, status=304)
 
+    @app.get("/precondition")
+    async def precondition_handler(request: Request):
+        return json(JSON_DATA, status=412)
+
     @app.delete("/")
     async def delete_handler(request: Request):
         return json(None, status=204)
@@ -193,6 +197,10 @@ def test_json_response(json_app):
     assert response.text == json_dumps(JSON_DATA)
     assert response.json == JSON_DATA
 
+    request, response = json_app.test_client.get("/precondition")
+    assert response.status == 412
+    assert response.json == JSON_DATA
+
 
 def test_no_content(json_app):
     request, response = json_app.test_client.get("/no-content")