Repository: aio-libs/aiohttp

Issue:
FileResponse + enable_compression + .gz will compress twice
### Describe the bug

When using `aiohttp.web.FileResponse` to return a file from disk, it is possible to use compression. As I understand from source code - I could not find documentation for FileResponse. There are two possible ways:

1. calling `response.enable_compression()` will make `FileResponse` automatically compress the file
2. when I call `FileResponse('some_file')` and `some_file.gz` exists, it will use the compressed version

Both ways work as expected.

But when both is appliend at the same time:

```python
response = FileResponse('some_file')
response.enable_compression()
return response
```

and _also_ `some_file.gz` exists, the response will be compressed twice - contents of `some_file.gz` is compressed via `deflate`.

### To Reproduce

```python
async def test_file_response_gz_with_enable_compression(aiohttp_client, tmp_path):

    async def file_handler(request):
        assert request.headers['Accept-Encoding'] == 'gzip, deflate'
        (tmp_path / 'file.gz').write_bytes(gzip.compress(b'Hello, compressed!'))
        response = FileResponse(tmp_path / 'file')
        response.enable_compression()
        return response

    app = Application()
    app.router.add_get('/file', file_handler)

    test_client = await aiohttp_client(app)
    resp = await test_client.get('/file')
    assert resp.status == 200
    assert resp.headers['Content-Type'] == 'application/octet-stream'
    assert resp.headers['Content-Encoding'] == 'deflate' # This should be gzip
    assert resp.headers['Transfer-Encoding'] == 'chunked'
    data = await resp.read()

    # This is how it should work:
    # assert data == b'Hello, compressed!'

    # But currently it works this way:
    assert gzip.decompress(data) == b'Hello, compressed!'
```

When using `enable_compression()` with non-compressed `file`, or using compressed `file.gz` without `enable_compression()`, it works as expected:

```python
async def test_file_response_gz_without_enable_compression(aiohttp_client, tmp_path):

    async def file_handler(request):
        assert request.headers['Accept-Encoding'] == 'gzip, deflate'
        (tmp_path / 'file.gz').write_bytes(gzip.compress(b'Hello, compressed!'))
        response = FileResponse(tmp_path / 'file')
        return response

    app = Application()
    app.router.add_get('/file', file_handler)

    test_client = await aiohttp_client(app)
    resp = await test_client.get('/file')
    assert resp.status == 200
    assert resp.headers['Content-Type'] == 'application/octet-stream'
    assert resp.headers['Content-Encoding'] == 'gzip'
    assert 'Transfer-Encoding' not in resp.headers
    data = await resp.read()

    # This is how it should work:
    assert data == b'Hello, compressed!'


async def test_file_response_no_gz_with_enable_compression(aiohttp_client, tmp_path):

    async def file_handler(request):
        assert request.headers['Accept-Encoding'] == 'gzip, deflate'
        (tmp_path / 'file').write_bytes(b'Hello, world!')
        assert not (tmp_path / 'file.gz').exists()
        response = FileResponse(tmp_path / 'file')
        response.enable_compression()
        return response

    app = Application()
    app.router.add_get('/file', file_handler)

    test_client = await aiohttp_client(app)
    resp = await test_client.get('/file')
    assert resp.status == 200
    assert resp.headers['Content-Type'] == 'application/octet-stream'
    assert resp.headers['Content-Encoding'] == 'deflate'
    assert resp.headers['Transfer-Encoding'] == 'chunked'
    data = await resp.read()

    # This is how it should work:
    assert data == b'Hello, world!'

```


### Expected behavior

When both `response.enable_compression()` was called and `some_file.gz` is present, I expect `some_file.gz` is returned as it is, without additional compression, with appropriate headers. Which is how it works if `response.enable_compression()` was not called.

### Logs/tracebacks

```python-traceback
No logs.
```


### Python Version

```console
$ ./venv/bin/python3 --version
Python 3.12.0
```


### aiohttp Version

```console
$ ./venv/bin/python3 -m pip show aiohttp
Name: aiohttp
Version: 3.9.1
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache 2
Location: /home/messa/code/misc/2024/20240108_aiohttp_tests/venv/lib/python3.12/site-packages
Requires: aiosignal, attrs, frozenlist, multidict, yarl
Required-by: pytest-aiohttp
```


### multidict Version

```console
$ ./venv/bin/python3 -m pip show multidict
Name: multidict
Version: 6.0.4
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/messa/code/misc/2024/20240108_aiohttp_tests/venv/lib/python3.12/site-packages
Requires:
Required-by: aiohttp, yarl
```


### yarl Version

```console
$ ./venv/bin/python3 -m pip show yarl
Name: yarl
Version: 1.9.4
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /home/messa/code/misc/2024/20240108_aiohttp_tests/venv/lib/python3.12/site-packages
Requires: idna, multidict
Required-by: aiohttp
```


### OS

$ uname -a
Linux dev-messa202301 6.1.0-0.deb11.7-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.20-2~bpo11+1 (2023-04-23) x86_64 GNU/Linux
$ cat /etc/debian_version
11.8

### Related component

Server

### Additional context

_No response_

### Code of Conduct

- [X] I agree to follow the aio-libs Code of Conduct


Discussion:


It looks like there is a stat in the event loop as well https://github.com/aio-libs/aiohttp/blob/6529c77551e2a61d6d9df72653df92180d1fed7c/aiohttp/web_fileresponse.py#L131

Which is surprising since careful attention has been given to make sure not do to that a few lines below
https://github.com/aio-libs/aiohttp/blob/6529c77551e2a61d6d9df72653df92180d1fed7c/aiohttp/web_fileresponse.py#L136
https://github.com/aio-libs/aiohttp/blob/6529c77551e2a61d6d9df72653df92180d1fed7c/aiohttp/web_fileresponse.py#L251

enable_compression doesn't know if the payload is already compressed since HttpWriter doesn't know that FileResponse has sent a compressed payload
https://github.com/aio-libs/aiohttp/blob/6529c77551e2a61d6d9df72653df92180d1fed7c/aiohttp/http_writer.py#L67
I guess we could have https://github.com/aio-libs/aiohttp/blob/6529c77551e2a61d6d9df72653df92180d1fed7c/aiohttp/http_writer.py#L121 check if the content encoding is already gzip and turn off compression

not sure I really like it knowing about the headers though
https://github.com/aio-libs/aiohttp/blob/6529c77551e2a61d6d9df72653df92180d1fed7c/aiohttp/web_fileresponse.py#L82 might be able to handle this case there

maybe as simple as 

```diff
diff --git a/aiohttp/web_fileresponse.py b/aiohttp/web_fileresponse.py
index eb7a6a31..4ee43e50 100644
--- a/aiohttp/web_fileresponse.py
+++ b/aiohttp/web_fileresponse.py
@@ -134,6 +134,8 @@ class FileResponse(StreamResponse):
             if gzip_path.is_file():
                 filepath = gzip_path
                 gzip = True
+                # Disable compression to avoid double compression
+                self._compression = False
 
         loop = asyncio.get_event_loop()
         st: os.stat_result = await loop.run_in_executor(None, filepath.stat)
```
I'm also seeing a lot more `_sendfile_fallback` than expected. 

Need to investigate that as well
So at least 2 problems here, maybe 3

- [x] blocking I/O (stat in the event loop -- solution is to do both stats in a single executor job
- [x] double compression - potential solution above
- [x] unexpected fallback to sendfile --- not a problem, it was because the connection was tls
I haven't had any luck figuring out why the sendfile fallback is happening
sendfile isn't an issue as it was because my test case had tls.  it works as expected

so two problems to solve here
sendfile fallback also creates more executor jobs that needed. It could do the seek and read in a single job to reduce the amount of futures it has to create and executor use

Resolving patch:
diff --git a/aiohttp/web_fileresponse.py b/aiohttp/web_fileresponse.py
--- a/aiohttp/web_fileresponse.py
+++ b/aiohttp/web_fileresponse.py
@@ -263,6 +263,10 @@ async def prepare(self, request: "BaseRequest") -> Optional[AbstractStreamWriter
             self.headers[hdrs.CONTENT_ENCODING] = encoding
         if gzip:
             self.headers[hdrs.VARY] = hdrs.ACCEPT_ENCODING
+            # Disable compression if we are already sending
+            # a compressed file since we don't want to double
+            # compress.
+            self._compression = False
 
         self.etag = etag_value  # type: ignore[assignment]
         self.last_modified = st.st_mtime  # type: ignore[assignment]


Test patch:
diff --git a/tests/test_web_sendfile_functional.py b/tests/test_web_sendfile_functional.py
--- a/tests/test_web_sendfile_functional.py
+++ b/tests/test_web_sendfile_functional.py
@@ -249,6 +249,32 @@ async def handler(request):
     await client.close()
 
 
+async def test_static_file_with_gziped_counter_part_enable_compression(
+    aiohttp_client: Any, sender: Any
+):
+    """Test that enable_compression does not double compress when a .gz file is also present."""
+    filepath = pathlib.Path(__file__).parent / "hello.txt"
+
+    async def handler(request):
+        resp = sender(filepath)
+        resp.enable_compression()
+        return resp
+
+    app = web.Application()
+    app.router.add_get("/", handler)
+    client = await aiohttp_client(app)
+
+    resp = await client.get("/")
+    assert resp.status == 200
+    body = await resp.read()
+    assert body == b"hello aiohttp\n"
+    assert resp.headers["Content-Type"] == "text/plain"
+    assert resp.headers.get("Content-Encoding") == "gzip"
+    resp.close()
+    await resp.release()
+    await client.close()
+
+
 async def test_static_file_with_content_encoding(
     aiohttp_client: Any, sender: Any
 ) -> None: