Repository: aio-libs/aiohttp

Issue:
MUST NOT send Content-Length/Transfer-Encoding for 1xx or 204 
🐞 **Describe the bug**

According to RFC7230 (https://tools.ietf.org/html/rfc7230#section-3.3.2 and https://tools.ietf.org/html/rfc7230#section-3.3.1) a server must not send neither the `Content-Length` nor the `Transfer-Encoding` header field for **1xx** and **204** responses.

Note that in practice clients and proxies might get into trouble when receiving such a response. E.g. with a [nginx](http://nginx.org/) reverse proxy this behaviour leads to the following error: `upstream prematurely closed connection ...`

💡 **To Reproduce**

Exemplary for `web.Response`:

```python
from aiohttp import web

async def reproduce(request):
    return web.Response(status=204)

app = web.Application()
app.add_routes([web.get('/', reproduce)])

web.run_app(app, host='localhost', port=8080)
```

```console

$ curl -v -o - "http://localhost:8080/"
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< Content-Length: 0
< Content-Type: application/octet-stream
< Date: Fri, 07 Aug 2020 16:16:33 GMT
< Server: Python/3.8 aiohttp/4.0.0a1
< 
* Connection #0 to host localhost left intact
```

💡 **Expected behavior**

Do not ship the headers mentioned above.

📋 **Your version of the Python**

```console
$ python --version
Python 3.8.2
```

📋 **Your version of the aiohttp/yarl/multidict distributions**

```console
 pip freeze | grep aiohttp
-e git+https://github.com/aio-libs/aiohttp.git@1d296d549050aa335ef542421b8b7dad788246d5#egg=aiohttp
```
The `aiohttp` version should agree with the current `master`.

```console
$ python -m pip show multidict
Name: multidict
Version: 4.7.6
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/ubuntu/work/projects/aiohttp-reproducer/venv/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp
```
```console
$ python -m pip show yarl
Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/ubuntu/work/projects/aiohttp-reproducer/venv/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp
```
MUST NOT send Content-Length/Transfer-Encoding for 1xx or 204 
🐞 **Describe the bug**

According to RFC7230 (https://tools.ietf.org/html/rfc7230#section-3.3.2 and https://tools.ietf.org/html/rfc7230#section-3.3.1) a server must not send neither the `Content-Length` nor the `Transfer-Encoding` header field for **1xx** and **204** responses.

Note that in practice clients and proxies might get into trouble when receiving such a response. E.g. with a [nginx](http://nginx.org/) reverse proxy this behaviour leads to the following error: `upstream prematurely closed connection ...`

💡 **To Reproduce**

Exemplary for `web.Response`:

```python
from aiohttp import web

async def reproduce(request):
    return web.Response(status=204)

app = web.Application()
app.add_routes([web.get('/', reproduce)])

web.run_app(app, host='localhost', port=8080)
```

```console

$ curl -v -o - "http://localhost:8080/"
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 204 No Content
< Content-Length: 0
< Content-Type: application/octet-stream
< Date: Fri, 07 Aug 2020 16:16:33 GMT
< Server: Python/3.8 aiohttp/4.0.0a1
< 
* Connection #0 to host localhost left intact
```

💡 **Expected behavior**

Do not ship the headers mentioned above.

📋 **Your version of the Python**

```console
$ python --version
Python 3.8.2
```

📋 **Your version of the aiohttp/yarl/multidict distributions**

```console
 pip freeze | grep aiohttp
-e git+https://github.com/aio-libs/aiohttp.git@1d296d549050aa335ef542421b8b7dad788246d5#egg=aiohttp
```
The `aiohttp` version should agree with the current `master`.

```console
$ python -m pip show multidict
Name: multidict
Version: 4.7.6
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/ubuntu/work/projects/aiohttp-reproducer/venv/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp
```
```console
$ python -m pip show yarl
Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/ubuntu/work/projects/aiohttp-reproducer/venv/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp
```


Discussion:
This is partially similar to https://github.com/aio-libs/aiohttp/issues/4612. 
This is partially similar to https://github.com/aio-libs/aiohttp/issues/4612. 

Resolving patch:
diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py
--- a/aiohttp/web_response.py
+++ b/aiohttp/web_response.py
@@ -426,6 +426,10 @@ async def _prepare_headers(self) -> None:
                         del headers[hdrs.CONTENT_LENGTH]
                 else:
                     keep_alive = False
+            # HTTP 1.1: https://tools.ietf.org/html/rfc7230#section-3.3.2
+            # HTTP 1.0: https://tools.ietf.org/html/rfc1945#section-10.4
+            elif version >= HttpVersion11 and self.status in (100, 101, 102, 103, 204):
+                del headers[hdrs.CONTENT_LENGTH]
 
         headers.setdefault(hdrs.CONTENT_TYPE, "application/octet-stream")
         headers.setdefault(hdrs.DATE, rfc822_formatted_time())


Test patch:
diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py
--- a/tests/test_web_functional.py
+++ b/tests/test_web_functional.py
@@ -12,6 +12,7 @@
 
 import aiohttp
 from aiohttp import FormData, HttpVersion10, HttpVersion11, TraceConfig, multipart, web
+from aiohttp.hdrs import CONTENT_LENGTH, TRANSFER_ENCODING
 from aiohttp.test_utils import make_mocked_coro
 
 try:
@@ -1879,3 +1880,20 @@ async def handler(request):
     resp = await client.post("/", data=b"data")
     assert resp.status == 200
     assert await resp.text() == "data (2, 4)"
+
+
+@pytest.mark.parametrize(
+    "status", [101, 204],
+)
+async def test_response_101_204_no_content_length_http11(
+    status, aiohttp_client
+) -> None:
+    async def handler(_):
+        return web.Response(status=status)
+
+    app = web.Application()
+    app.router.add_get("/", handler)
+    client = await aiohttp_client(app, version="1.1")
+    resp = await client.get("/")
+    assert CONTENT_LENGTH not in resp.headers
+    assert TRANSFER_ENCODING not in resp.headers