Repository: aio-libs/aiohttp

Issue:
enable_compression doesn't work with FileResponse in sendfile mode
## Long story short

response.enable_compression doesn't work on Heroku.

## Expected behaviour
enable_compression on a response should work on Heroku.


## Actual behaviour
While this worked locally just fine, Heroku didn't like it when I enabled compression on a FileResponse:
Apr 17 11:58:19 my-app-staging heroku/router:  http_error="Bad chunk" at=error code=H17 desc="Poorly formatted HTTP response" method=GET path="/" host=my-app-staging.herokuapp.com request_id=0bc47133-af19-4999-bc0f-179054b7388a fwd="xx.xx.xx.xx" dyno=web.1 connect=0ms service=19ms status=503 bytes= protocol=https 
Apr 17 11:58:19 my-app-staging heroku/router:  http_error="Bad chunk" at=error code=H17 desc="Poorly formatted HTTP response" method=GET path="/" host=my-app-staging.herokuapp.com request_id=9c1b4583-61d5-4867-bf1a-75de194cf71f fwd="xx.xx.xx.xx" dyno=web.1 connect=0ms service=5ms status=503 bytes= protocol=https 
Apr 17 11:58:23 my-app-staging heroku/router:  http_error="Bad chunk" at=error code=H17 desc="Poorly formatted HTTP response" method=GET path="/" host=my-app-staging.herokuapp.com request_id=a2f80aee-0ba0-4117-9139-123a79fd2f1c fwd="xx.xx.xx.xx" dyno=web.1 connect=0ms service=3ms status=503 bytes= protocol=https 

## Steps to reproduce
```
        resp = web.FileResponse('./dist/index.html', headers={'Cache-Control': 'no-cache'})
        resp.enable_compression()
        return resp
```

## Your environment
aiohttp==2.3.6
Using aiohttp directly on Heroku (no gunicorn/etc).


Discussion:
Try to set `AIOHTTP_NOSENDFILE=1` environment variable.
Works now, thanks. Is there a way to set this programmatically?
Not yet.
The problem is: aiohttp uses `sendfile` syscall even for compressed files.
The fast path should be disabled for non-identity content encoding.

The path for `FileResponse` is welcome (the required change is relatively simple).
The issue is waiting for a champion.
I'll take this one.

Resolving patch:
diff --git a/aiohttp/web_fileresponse.py b/aiohttp/web_fileresponse.py
--- a/aiohttp/web_fileresponse.py
+++ b/aiohttp/web_fileresponse.py
@@ -107,7 +107,8 @@ async def _sendfile_system(self, request, fobj, count):
 
         transport = request.transport
         if (transport.get_extra_info("sslcontext") or
-                transport.get_extra_info("socket") is None):
+                transport.get_extra_info("socket") is None or
+                self.compression):
             writer = await self._sendfile_fallback(request, fobj, count)
         else:
             writer = SendfileStreamWriter(
@@ -131,7 +132,7 @@ async def _sendfile_fallback(self, request, fobj, count):
         # fobj is transferred in chunks controlled by the
         # constructor's chunk_size argument.
 
-        writer = (await super().prepare(request))
+        writer = await super().prepare(request)
 
         chunk_size = self._chunk_size
 
diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py
--- a/aiohttp/web_response.py
+++ b/aiohttp/web_response.py
@@ -43,7 +43,7 @@ def __init__(self, *, status=200, reason=None, headers=None):
         self._keep_alive = None
         self._chunked = False
         self._compression = False
-        self._compression_force = False
+        self._compression_force = None
         self._cookies = SimpleCookie()
 
         self._req = None


Test patch:
diff --git a/tests/test_web_sendfile_functional.py b/tests/test_web_sendfile_functional.py
--- a/tests/test_web_sendfile_functional.py
+++ b/tests/test_web_sendfile_functional.py
@@ -1,6 +1,7 @@
 import asyncio
 import os
 import pathlib
+import zlib
 
 import pytest
 
@@ -729,3 +730,25 @@ async def handler(request):
     resp = await client.get('/', headers={'If-Range': lastmod})
     assert 200 == resp.status
     resp.close()
+
+
+async def test_static_file_compression(aiohttp_client, sender):
+    filepath = pathlib.Path(__file__).parent / 'data.unknown_mime_type'
+
+    async def handler(request):
+        ret = sender(filepath)
+        ret.enable_compression()
+        return ret
+
+    app = web.Application()
+    app.router.add_get('/', handler)
+    client = await aiohttp_client(app, auto_decompress=False)
+
+    resp = await client.get('/')
+    assert resp.status == 200
+    zcomp = zlib.compressobj(wbits=-zlib.MAX_WBITS)
+    expected_body = zcomp.compress(b'file content\n') + zcomp.flush()
+    assert expected_body == await resp.read()
+    assert 'application/octet-stream' == resp.headers['Content-Type']
+    assert resp.headers.get('Content-Encoding') == 'deflate'
+    await resp.release()