Repository: encode/uvicorn

Issue:
Multipart form handling weird behavior 
### Background 
Hey everyone!

I'm writing an app that processes multipart form with file upload. When I serve it with `uvicorn` and run cURL against the endpoint I get an error `curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding`. Data from backend reaches cURL, but it returns an odd error. 

Looking into `ascii-trace` I can see that data chunk from backend mystically comes to cURL without length line which should be at the beginning of chunk.
```
<= Recv data, 20 bytes (0x14)
0000: Hello, world!
000f: 0
0012:
```
However, dtrace and Wireshark showed me that chunk data was sent from `uvicorn` with length as it should. Other interesting fact is that I get no cURL error if serving app with daphne(`daphne -p 8000 app:App`).

What am I missing? 🤔 

### Environment
1. Mac OS X 
2. uvicorn==0.3.21
3. Python 3.7.0 (with pipenv)
4. uvloop / asyncio gives same result

### Steps to reproduce
1. Create snippet:

```python
class App():
    def __init__(self, scope):
        self.scope = scope

    async def __call__(self, receive, send):
        while True:
            message = await receive()
            print('(%s) message from  recv is %r' % (id(self),message))
            if message["type"] == "http.disconnect":
                return
            else:
                if not message.get("more_body"):
                    await send({
                        'type': 'http.response.start',
                        'status': 200,
                        'headers': [[b'content-type', b'text/plain']]
                    })
                    await send({
                        'type': 'http.response.body',
                        'body': b'Hello, world!',
                        'more_body': False
                    })
```

2. Run http server 
```
python uvicorn app:App
```

3. Query with cURL
```
curl  --trace-ascii curl.trace-uvicorn \
            --form file=@foo \
            --form file2=@foo \
            http://localhost:8000/api/upload
curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding
```

4. Check what's in trace

```
$ cat curl.trace-uvicorn
== Info:   Trying ::1...
== Info: TCP_NODELAY set
== Info: Connection failed
== Info: connect to ::1 port 8000 failed: Connection refused
== Info:   Trying fe80::1...
== Info: TCP_NODELAY set
== Info: Connection failed
== Info: connect to fe80::1 port 8000 failed: Connection refused
== Info:   Trying 127.0.0.1...
== Info: TCP_NODELAY set
== Info: Connected to localhost (127.0.0.1) port 8000 (#0)
=> Send header, 218 bytes (0xda)
0000: POST /api/upload HTTP/1.1
001b: Host: localhost:8000
0031: User-Agent: curl/7.54.0
004a: Accept: */*
0057: Content-Length: 361
006c: Expect: 100-continue
0082: Content-Type: multipart/form-data; boundary=--------------------
00c2: ----68de9f32447ad9b5
00d8:
<= Recv header, 23 bytes (0x17)
0000: HTTP/1.1 100 Continue
== Info: Done waiting for 100-continue
=> Send data, 147 bytes (0x93)
0000: --------------------------68de9f32447ad9b5
002c: Content-Disposition: form-data; name="file"; filename="foo"
0069: Content-Type: application/octet-stream
0091:
=> Send data, 8 bytes (0x8)
0000: content.
=> Send data, 150 bytes (0x96)
0000:
0002: --------------------------68de9f32447ad9b5
002e: Content-Disposition: form-data; name="file2"; filename="foo"
006c: Content-Type: application/octet-stream
0094:
=> Send data, 8 bytes (0x8)
0000: content.
=> Send data, 48 bytes (0x30)
0000:
0002: --------------------------68de9f32447ad9b5--
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 17 bytes (0x11)
0000: server: uvicorn
<= Recv header, 37 bytes (0x25)
0000: date: Tue, 20 Nov 2018 04:21:02 GMT
<= Recv header, 26 bytes (0x1a)
0000: content-type: text/plain
<= Recv header, 28 bytes (0x1c)
0000: transfer-encoding: chunked
<= Recv data, 20 bytes (0x14)
0000: Hello, world!
000f: 0
0012:
== Info: Illegal or missing hexadecimal sequence in chunked-encoding
== Info: stopped the pause stream!
== Info: Closing connection 0
```


Discussion:
Does the same issue replicate with `hypercorn` or `daphne`?
It works fine with `daphne`, but snippet doesn't work completely with `hypercorn`:
```
$ pipenv run hypercorn app:App
Running on http://127.0.0.1:5000 (CTRL + C to quit)
(4315911000) message from  recv is {'type': 'lifespan.startup'}
```
Gets stuck in an infinite loop waiting till I return something back in exchange to `lifespan.startup` (which is probably ok, because I don't handle lifecycle events).

So I've modified my code for `hypercorn` looking at [examples with `await receive()`]( https://raw.githubusercontent.com/pgjones/hypercorn/ad6aee11868fa447844cb695ef5ba3a0181419ca/compliance/h2spec/server.py):

```python
class NoLoopApp:
    def __init__(self, scope):
        self.scope = scope

    async def __call__(self, receive, send):
        message = await receive()
        print('(%s) message from  recv is %r' % (id(self),message))
        if message['type'] == 'lifespan.startup':
            await send({'type': 'lifespan.startup.complete'})
            return
        elif message['type'] == 'lifespan.shutdown':
            await send({'type': 'lifespan.shutdown.complete'})
            return

        if message["type"] == "http.disconnect":
            return
        else:
            if not message.get("more_body"):
                await send({
                    'type': 'http.response.start',
                    'status': 200,
                    'headers': [[b'content-type', b'text/plain']]
                })
                await send({
                    'type': 'http.response.body',
                    'body': b'Hello, world!',
                    'more_body': False
                })
```
This snippet works with `hypercorn` and `daphne`. cURL is happy with what it gets. With `uvicorn` it still fails but with a different error message:

```
$ pipenv run uvicorn server:NoLoopApp

INFO: Started server process [5521]
INFO: Waiting for application startup.
(4454399504) message from  recv is {'type': 'lifespan.startup'}
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
(4454544776) message from  recv is {'type': 'http.request', 'body': b'--------------------------5cf093fda4e9e0ae\r\nContent-Disposition: form-data; name="file"; filename="foo"\r\nContent-Type: application/octet-stream\r\n\r\n', 'more_body': True}
ERROR: ASGI callable returned without starting response.
message is: {'type': 'http.response.start', 'status': 500, 'headers': [(b'content-type', b'text/plain; charset=utf-8'), (b'connection', b'close')]}
INFO: ('127.0.0.1', 51559) - "POST /api/upload HTTP/1.1" 500
message is: {'type': 'http.response.body', 'body': b'Internal Server Error'}
- Writing b'15\r\nInternal Server Error\r\n0\r\n\r\n'
```
@superduper you definitely need the loop, you also don't need to return after sending the `startup.complete`. All the servers should be returning a 500 response with the snippet given. I would try with,

```python
class App:
    def __init__(self, scope):
        self.scope = scope

    async def __call__(self, receive, send):
        while True:
            message = await receive()
            print('(%s) message from  recv is %r' % (id(self),message))
            if message['type'] == 'lifespan.startup':
                await send({'type': 'lifespan.startup.complete'})
            elif message['type'] == 'lifespan.shutdown':
                await send({'type': 'lifespan.shutdown.complete'})
                return

            if message["type"] == "http.disconnect":
                return
            else:
                if not message.get("more_body"):
                    await send({
                        'type': 'http.response.start',
                        'status': 200,
                        'headers': [[b'content-type', b'text/plain']]
                    })
                    await send({
                        'type': 'http.response.body',
                        'body': b'Hello, world!',
                        'more_body': False
                    })
```
@pgjones Your snippet works fine with both `hypercorn` and `daphne`, `cURL` is happy. `uvicorn` however is still giving a response `cURL` is not satisfied with.

PS I removed that loop for illustration&experimentation purpose.
Replicating with:

```python
class App():
    def __init__(self, scope):
        assert scope['type'] == 'http'
        self.scope = scope

    async def __call__(self, receive, send):
        message = await receive()
        await send({
            'type': 'http.response.start',
            'status': 200,
            'headers': [[b'content-type', b'text/plain']]
        })
        await send({
            'type': 'http.response.body',
            'body': b'Hello, world!',
            'more_body': False
        })
```

And:

`curl  -v --form data=123 http://localhost:8000/`

Resolving patch:
diff --git a/uvicorn/protocols/http/httptools_impl.py b/uvicorn/protocols/http/httptools_impl.py
--- a/uvicorn/protocols/http/httptools_impl.py
+++ b/uvicorn/protocols/http/httptools_impl.py
@@ -514,7 +514,7 @@ async def send(self, message):
 
     async def receive(self):
         if self.waiting_for_100_continue and not self.transport.is_closing():
-            self.transport.write(b"HTTP/1.1 100 Continue\r\n")
+            self.transport.write(b"HTTP/1.1 100 Continue\r\n\r\n")
             self.waiting_for_100_continue = False
 
         self.flow.resume_reading()


Test patch: