Repository: aio-libs/aiohttp

Issue:
reading nested multipart messages does not work correctly
## Long story short

Multipart reader breaks after reading a sub multipart end boundary and starting the next part

## Expected behaviour

Nested multipart reader reads a message created with the multipart writer correctly

## Actual behaviour

```python
ValueError: Invalid boundary b'', expected b'--b0b69248b3a345cf8256a8dd25f07874'
```

## Steps to reproduce

Receive the multipart response from #1525 

Server:
```python
from aiohttp.multipart import MultipartWriter
from aiohttp.web import Response

def handle(request):
  with MultipartWriter('mixed') as root:
    with MultipartWriter('mixed') as subwriter1:
      subwriter1.append('first message')
    root.append(subwriter1, headers=subwriter1.headers)

    with MultipartWriter('mixed') as subwriter2:
      subwriter2.append('second message')
    root.append(subwriter2, headers=subwriter2.headers)
  return Response(body=b''.join(root.serialize()), headers=root.headers)

# ... create web app which responds with the handler above ...
```

Client:
```python
import aiohttp
import asyncio
from aiohttp.multipart import BodyPartReader, MultipartReader

@asyncio.coroutine
def read_multipart(reader):
  while True:
    part = yield from reader.next()
    if part is None: break
    if isinstance(part, BodyPartReader):
      body = yield from part.read(decode=True)
      print('body part: %r' % body)
    else:
      print('nested part')
      yield from read_multipart(part)

@asyncio.coroutine
def request(url):
  response = yield from aiohttp.get(url)
  yield from read_multipart(MultipartReader.from_response(response))

# ... drive event loop and call request(handler_url) ...
```

## Issue

Lines [multipart.py:767](https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/multipart.py#L767) and [multipart.py:969](https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/multipart.py#L969) have line endings which result in an empty line after the end boundary of a multipart message (valid). However the reader terminates after the end boundary and the parent reader now expects the next boundary, but what is found is the blank line from [multipart.py:767](https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/multipart.py#L767)

## Possible fix

```diff
diff --git a/aiohttp/multipart.py b/aiohttp/multipart.py
index af7f19b1..82ad2306 100644
--- a/aiohttp/multipart.py
+++ b/aiohttp/multipart.py
@@ -639,6 +639,7 @@ class MultipartReader(object):
             pass
         elif chunk == self._boundary + b'--':
             self._at_eof = True
+            yield from self._readline()
         else:
             raise ValueError('Invalid boundary %r, expected %r'
                              % (chunk, self._boundary))
```


Discussion:
@kxepal could you check this
Hm...it seems like some of our tests lies us since we have covered nested parts case there. Will check it in a moment.

Resolving patch:
diff --git a/aiohttp/multipart.py b/aiohttp/multipart.py
--- a/aiohttp/multipart.py
+++ b/aiohttp/multipart.py
@@ -639,6 +639,7 @@ def _read_boundary(self):
             pass
         elif chunk == self._boundary + b'--':
             self._at_eof = True
+            yield from self._readline()
         else:
             raise ValueError('Invalid boundary %r, expected %r'
                              % (chunk, self._boundary))


Test patch:
diff --git a/tests/test_multipart.py b/tests/test_multipart.py
--- a/tests/test_multipart.py
+++ b/tests/test_multipart.py
@@ -594,6 +594,7 @@ def test_release(self):
                    b'\r\n'
                    b'passed\r\n'
                    b'----:----\r\n'
+                   b'\r\n'
                    b'--:--'))
         yield from reader.release()
         self.assertTrue(reader.at_eof())
diff --git a/tests/test_py35/test_multipart_35.py b/tests/test_py35/test_multipart_35.py
--- a/tests/test_py35/test_multipart_35.py
+++ b/tests/test_py35/test_multipart_35.py
@@ -46,6 +46,7 @@ async def test_async_for_reader(loop):
             b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03K\xcc\xcc\xcf())'
             b'\xe0\x02\x00\xd6\x90\xe2O\x08\x00\x00\x00',
             b'--::--',
+            b'',
             b'--:--',
             b''])))
     idata = iter(data)