Repository: aio-libs/aiohttp

Issue:
will there be huge file upload support ?
when uploading huge file ,there is MemoryError as follows.
## any plan to fixed this?

ERROR:aiohttp.web:Error handling request
Traceback (most recent call last):
  File "D:\soft\prog\python3\python-3.5.1\lib\site-packages\aiohttp\server.py", line 285, in start
    yield from self.handle_request(message, payload)
  File "D:\soft\prog\python3\python-3.5.1\lib\site-packages\aiohttp\web.py", line 90, in handle_request
    resp = yield from handler(request)
  File "D:\prog\comp.lang.python\廖2\webfx2.py", line 305, in logger
    return await handler(request)
  File "D:\prog\comp.lang.python\廖2\webfx2.py", line 320, in auth
    return (await handler(request))
  File "D:\prog\comp.lang.python\廖2\webfx2.py", line 338, in response
    r = await handler(request)
  File "D:\soft\prog\python3\python-3.5.1\lib\site-packages\aiohttp\web_urldispatcher.py", line 101, in handler_wrapper
    result = yield from result
  File "D:\prog\comp.lang.python\廖2\webfx2.py", line 211, in **call**
    params = await request.post()
  File "D:\soft\prog\python3\python-3.5.1\lib\site-packages\aiohttp\web_reqrep.py", line 347, in post
    body = yield from self.read()
  File "D:\soft\prog\python3\python-3.5.1\lib\site-packages\aiohttp\web_reqrep.py", line 307, in read
    body.extend(chunk)
MemoryError



Discussion:
@kxepal would you implement iterative server api, like you've done it for client.
I pretty sure existing `aiohttp.multipart` may be adopted for this.

@asvetlov Sure, will do. This should be easy.

@asvetlov I fear that would be breaking compatibility change and it'll turn `post` coroutine into something weird. Well, it's already not good one, but at least it's consistent on what it returns.

How about to have `multipart` coroutine that uses multipart module for working with the request body? As like as we have `text` and `json` there. 

@kxepal yes, you are right.
`.post()` behavior mut be unchanged but we need a new coroutine for parsing request body in new way.

Feel free to add new method. Maybe `.multipart()` is good name, I have no strong opinion. 

Ok, thanks.


Resolving patch:
diff --git a/aiohttp/web_reqrep.py b/aiohttp/web_reqrep.py
--- a/aiohttp/web_reqrep.py
+++ b/aiohttp/web_reqrep.py
@@ -16,7 +16,7 @@
 
 from multidict import CIMultiDict, CIMultiDictProxy, MultiDict, MultiDictProxy
 
-from . import hdrs
+from . import hdrs, multipart
 from .helpers import reify, sentinel
 from .protocol import Response as ResponseImpl
 from .protocol import HttpVersion10, HttpVersion11
@@ -320,6 +320,11 @@ def json(self, *, loads=json.loads, loader=None):
         body = yield from self.text()
         return loads(body)
 
+    @asyncio.coroutine
+    def multipart(self, *, reader=multipart.MultipartReader):
+        """Return async iterator to process BODY as multipart."""
+        return reader(self.headers, self.content)
+
     @asyncio.coroutine
     def post(self):
         """Return POST parameters."""
@@ -336,6 +341,10 @@ def post(self):
             self._post = MultiDictProxy(MultiDict())
             return self._post
 
+        if self.content_type.startswith('multipart/'):
+            warnings.warn('To process multipart requests use .multipart'
+                          ' coroutine instead.', DeprecationWarning)
+
         body = yield from self.read()
         content_charset = self.charset or 'utf-8'
 


Test patch:
diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py
--- a/tests/test_web_functional.py
+++ b/tests/test_web_functional.py
@@ -7,7 +7,7 @@
 import pytest
 from multidict import MultiDict
 
-from aiohttp import FormData, web
+from aiohttp import FormData, multipart, web
 from aiohttp.protocol import HttpVersion, HttpVersion10, HttpVersion11
 from aiohttp.streams import EOF_MARKER
 
@@ -142,6 +142,42 @@ def handler(request):
     assert dct == data
 
 
+@asyncio.coroutine
+def test_multipart(loop, test_client):
+    with multipart.MultipartWriter() as writer:
+        writer.append('test')
+        writer.append_json({'passed': True})
+
+    @asyncio.coroutine
+    def handler(request):
+        reader = yield from request.multipart()
+        assert isinstance(reader, multipart.MultipartReader)
+
+        part = yield from reader.next()
+        assert isinstance(part, multipart.BodyPartReader)
+        thing = yield from part.text()
+        assert thing == 'test'
+
+        part = yield from reader.next()
+        assert isinstance(part, multipart.BodyPartReader)
+        assert part.headers['Content-Type'] == 'application/json'
+        thing = yield from part.json()
+        assert thing == {'passed': True}
+
+        resp = web.Response()
+        resp.content_type = 'application/json'
+        resp.body = b''
+        return resp
+
+    app = web.Application(loop=loop)
+    app.router.add_post('/', handler)
+    client = yield from test_client(app)
+
+    resp = yield from client.post('/', data=writer, headers=writer.headers)
+    assert 200 == resp.status
+    yield from resp.release()
+
+
 @asyncio.coroutine
 def test_render_redirect(loop, test_client):
 
@@ -174,7 +210,8 @@ def check_file(fs):
 
     @asyncio.coroutine
     def handler(request):
-        data = yield from request.post()
+        with pytest.warns(DeprecationWarning):
+            data = yield from request.post()
         assert ['sample.crt'] == list(data.keys())
         for fs in data.values():
             check_file(fs)