Repository: aio-libs/aiohttp

Issue:
Server static handler includes body for HEAD response
🐞 **Describe the bug**
When using `web.static` to serve static content, the server responds to HEAD requests as if they were GET requests i.e. it includes a body with the file content. Apart from wasting bandwidth, this completely breaks pipelined connections because the content appears where the client is expecting to see the response headers for the next request. Possibly it may cause security issues too if an attacker can control the content because they could then deliver arbitrary headers to a client.

💡 **To Reproduce**

Put the script below in a file called `ah.py` and run it (it will serve the directory it is in on :8080).

```python
#!/usr/bin/env python3

import os
import aiohttp.web as web

app = web.Application()
app.add_routes([web.static('/', os.path.dirname(os.path.realpath(__file__)))])
web.run_app(app)
```

While it's running, run `echo -e 'HEAD /ah.py HTTP/1.1\nHost: localhost:8080\n' | nc localhost 8080` (assumes bash shell for `echo -e`). It will output something like

```
HTTP/1.1 200 OK
Content-Type: text/x-python
Last-Modified: Wed, 10 Jun 2020 10:02:34 GMT
Content-Length: 182
Accept-Ranges: bytes
Date: Wed, 10 Jun 2020 10:11:13 GMT
Server: Python/3.6 aiohttp/3.6.2

#!/usr/bin/env python3

import os
import aiohttp.web as web


app = web.Application()
app.add_routes([web.static('/', os.path.dirname(os.path.realpath(__file__)))])
web.run_app(app)
```

💡 **Expected behavior**

It should not output any body because it is a HEAD request.

📋 **Your version of the Python**
<!-- Attach your version of the Python. -->
```console
$ python --version
python 3.6.6
```

📋 **Your version of the aiohttp/yarl/multidict distributions**
<!-- Attach your version of the distributions in the code blocks below. -->
```console
$ python -m pip show aiohttp
Version: 3.6.2
```
```console
$ python -m pip show multidict
Version: 4.7.6
```
```console
$ python -m pip show yarl
Version: 1.4.2
```


Discussion:
I should clarify that by "pipelined connections" I just mean keep-alive, not sending the second request before the first reply is received. Whether it breaks subsequent requests seems to depend on both timing and the client implementation, because some clients seem to consume (and ignore) the unwanted content, presumably just due to how they do buffering. With requests as the client I can intermittently get it to fail; I'm currently trying to add a new unit test to aiohttp to demonstrate the problem but haven't made it fail yet.

Resolving patch:
diff --git a/aiohttp/web_fileresponse.py b/aiohttp/web_fileresponse.py
--- a/aiohttp/web_fileresponse.py
+++ b/aiohttp/web_fileresponse.py
@@ -360,6 +360,9 @@ async def prepare(
             self.headers[hdrs.CONTENT_RANGE] = 'bytes {0}-{1}/{2}'.format(
                 real_start, real_start + count - 1, file_size)
 
+        if request.method == hdrs.METH_HEAD or self.status in [204, 304]:
+            return await super().prepare(request)
+
         fobj = await loop.run_in_executor(None, filepath.open, 'rb')
         if start:  # be aware that start could be None or int=0 here.
             offset = start


Test patch:
diff --git a/tests/test_web_urldispatcher.py b/tests/test_web_urldispatcher.py
--- a/tests/test_web_urldispatcher.py
+++ b/tests/test_web_urldispatcher.py
@@ -1,3 +1,4 @@
+import asyncio
 import pathlib
 from unittest import mock
 from unittest.mock import MagicMock
@@ -238,6 +239,33 @@ async def test_access_special_resource(tmp_path, aiohttp_client) -> None:
         assert r.status == 403
 
 
+async def test_static_head(tmp_path, aiohttp_client) -> None:
+    # Test HEAD on static route
+    my_file_path = tmp_path / 'test.txt'
+    with my_file_path.open('wb') as fw:
+        fw.write(b'should_not_see_this\n')
+
+    app = web.Application()
+    app.router.add_static('/', str(tmp_path))
+    client = await aiohttp_client(app)
+
+    r = await client.head('/test.txt')
+    assert r.status == 200
+
+    # Check that there is no content sent (see #4809). This can't easily be
+    # done with aiohttp_client because the buffering can consume the content.
+    reader, writer = await asyncio.open_connection(client.host, client.port)
+    writer.write(b'HEAD /test.txt HTTP/1.1\r\n')
+    writer.write(b'Host: localhost\r\n')
+    writer.write(b'Connection: close\r\n')
+    writer.write(b'\r\n')
+    while await reader.readline() != b'\r\n':
+        pass
+    content = await reader.read()
+    writer.close()
+    assert content == b''
+
+
 def test_system_route() -> None:
     route = SystemRoute(web.HTTPCreated(reason='test'))
     with pytest.raises(RuntimeError):