Repository: aio-libs/aiohttp

Issue:
StreamResponse + HTTP/1.0 = corrupted file 
I'm using StreamResponse to serve files. My code is

```
from aiohttp import web
response = web.StreamResponse()
response.enable_chunked_encoding()
response.content_length = statinfo.st_size
response.content_type = 'application/octet-stream'
response.start(request)
with open(filename, 'rb') as f:
    while True:
        chunk = f.read(CHUNK_SIZE)
        if not chunk:
            return response
        response.write(chunk)
        yield from response.drain()
```

If I'm using wget to download the file then it works beautifully. But when I'm using Python urllib to download the file, my download is corrupted. The difference seems to be that wget can HTTP/1.1, but urllib only does HTTP/1.0. The network trace is

```
Host: ...
User-Agent: Python-urllib/1.17

HTTP/1.0 200 OK
CONTENT-LENGTH: 815206
CONTENT-TYPE: application/octet-stream
TRANSFER-ENCODING: chunked
CONNECTION: close
DATE: Thu, 27 Aug 2015 17:31:25 GMT
SERVER: Python/3.4 aiohttp/0.17.2

10000
... binary data ...
```

and Python saves everything starting at (and including) the 10000 into the file. 

I think this is because `TRANSFER-ENCODING: chunked` is HTTP/1.1 only, and a HTTP/1.0 client is entirely correct in ignoring it. Aiohttp should not send chunk sizes to HTTP/1.0 clients as they almost certainly end up in the data stream. 



Discussion:
The only thing that the library can do is forbidding chunked encoding if HTTP version is not 1.1
In your example just remove `response.enable_chunked_encoding()` line to make it work with HTTP/1.1

I agree. I'm just saying that the library should do that automatically, and not let the user fall into that trap. 

@vbraun I don't follow.
Do you agree with raising error on trying to call `.enable_chunked_encoding()` when http version is HTTP/1.1 or do you propose something else?

No, I thought it should just fall back to not using chunked encoding. I.e. ignore the call to `enable_chunked_encoding()` on HTTP/1.0. Anything else is just going to result in obscure errors in users of the library, making it unnecessarily difficult to use.

No, without chunked encoding you should collect all sent data into buffer before generating headers.
That's because HTTP/1.0 requires `Content-Length` header set.

Buffering whole response body is not appropriate. 
Well, there is ability to use HTTP/0.9 way -- just don't send Content-Length but drop connection after all data sent but I don't encourage this way.

`StreamResponse` is relative low-level API, so user should care about protocol version and chunked encoding. If you want buffer BODY -- just use `web.Response` instead.

OK, I do know the content length, and I do want to use `StreamResponse` since the body can be very large. I already set the `response.content_length` attribute in the snippet. If I just leave out the `response.enable_chunked_encoding()` then is it streaming the response? 

If you know content length and set up `response.content_length` attr you don't need chunked encoding.
Just send your data slice-by-slice without `response.enable_chunked_encoding()` call -- it works pretty well for both HTTP/1.0 and HTTP/1.1


Resolving patch:
diff --git a/aiohttp/web_reqrep.py b/aiohttp/web_reqrep.py
--- a/aiohttp/web_reqrep.py
+++ b/aiohttp/web_reqrep.py
@@ -24,7 +24,7 @@
                         CIMultiDict,
                         MultiDictProxy,
                         MultiDict)
-from .protocol import Response as ResponseImpl, HttpVersion10
+from .protocol import Response as ResponseImpl, HttpVersion10, HttpVersion11
 from .streams import EOF_MARKER
 
 
@@ -646,6 +646,10 @@ def start(self, request):
             self._start_compression(request)
 
         if self._chunked:
+            if request.version != HttpVersion11:
+                raise RuntimeError("Using chunked encoding is forbidden "
+                                   "for HTTP/{0.major}.{0.minor}".format(
+                                       request.version))
             resp_impl.enable_chunked_encoding()
             if self._chunk_size:
                 resp_impl.add_chunking_filter(self._chunk_size)


Test patch:
diff --git a/tests/test_web_response.py b/tests/test_web_response.py
--- a/tests/test_web_response.py
+++ b/tests/test_web_response.py
@@ -18,8 +18,9 @@ def setUp(self):
     def tearDown(self):
         self.loop.close()
 
-    def make_request(self, method, path, headers=CIMultiDict()):
-        message = RawRequestMessage(method, path, HttpVersion11, headers,
+    def make_request(self, method, path, headers=CIMultiDict(),
+                     version=HttpVersion11):
+        message = RawRequestMessage(method, path, version, headers,
                                     False, False)
         return self.request_from_message(message)
 
@@ -183,6 +184,16 @@ def test_chunk_size(self, ResponseImpl):
         msg.add_chunking_filter.assert_called_with(8192)
         self.assertIsNotNone(msg.filter)
 
+    def test_chunked_encoding_forbidden_for_http_10(self):
+        req = self.make_request('GET', '/', version=HttpVersion10)
+        resp = StreamResponse()
+        resp.enable_chunked_encoding()
+
+        with self.assertRaisesRegex(
+                RuntimeError,
+                "Using chunked encoding is forbidden for HTTP/1.0"):
+            resp.start(req)
+
     @mock.patch('aiohttp.web_reqrep.ResponseImpl')
     def test_compression_no_accept(self, ResponseImpl):
         req = self.make_request('GET', '/')