mitsuhiko/werkzeug — Issue #133: 2 second delay parsing multipart/form-data
Author: twooster
Created: 2011-11-03 19:15:11+00:00
State: open

Note: I'm fairly sure this is a Werkzeug issue, and not a Flask or CURL issue, but I could be wrong. I'm guessing it has to do with byte counting and socket timeouts...

I have a very small Flask application that looks like the following:

```python
from flask import Flask, request

app = Flask(__name__)

@app.route('/test', methods=['GET', 'POST'])
def test():
return '

%r

' % (request.form,)

if __name__ == '__main__':
app.run(port=5555)
```

If I run the server and then issue two very similar cURL commands against it, I get wildly different response times:

tony@quaternion ~/Projects/test/test# time curl -d 'foo=bar' http://localhost:5555/test
127.0.0.1 - - [03/Nov/2011 12:06:15] "POST /test HTTP/1.1" 200 -

ImmutableMultiDict([('foo', u'bar')])

real	0m0.009s
user	0m0.003s
sys	0m0.003s
tony@quaternion ~/Projects/test/test# time curl -F 'foo=bar' http://localhost:5555/test
127.0.0.1 - - [03/Nov/2011 12:06:22] "POST /test HTTP/1.1" 200 -

ImmutableMultiDict([('foo', u'bar')])

real	0m2.009s
user	0m0.003s
sys	0m0.003s
tony@quaternion ~/Projects/test/test#

The only difference between the two commands (as far as I know...) is that with -d, the data is sent URL-encoded, whereas with -F it's sent as multipart/form-data. I went ahead and ran a trace on it, and what I'm seeing is this:

--- modulename: formparser, funcname: _find_terminator
13.09 formparser.py(303): for line in iterator:
--- modulename: wsgi, funcname: make_line_iter
13.09 wsgi.py(606): stream = make_limited_stream(stream, limit)
--- modulename: wsgi, funcname: make_limited_stream
13.09 wsgi.py(577): if not isinstance(stream, LimitedStream):
13.09 wsgi.py(581): return stream
13.09 wsgi.py(607): _read = stream.read
13.09 wsgi.py(608): buffer = []
13.09 wsgi.py(609): while 1:
13.09 wsgi.py(610): if len(buffer) > 1:
13.09 wsgi.py(617): chunks = _read(buffer_size).splitlines(True)
--- modulename: wsgi, funcname: read
13.09 wsgi.py(773): if self._pos >= self.limit:
13.09 wsgi.py(775): if size is None or size == -1: # -1 is for consistence with file
13.09 wsgi.py(777): to_read = min(self.limit - self._pos, size)
13.09 wsgi.py(778): try:
13.09 wsgi.py(779): read = self._read(to_read)
--- modulename: socket, funcname: read
13.09 socket.py(340): rbufsize = max(self._rbufsize, self.default_bufsize)
13.09 socket.py(344): buf = self._rbuf
13.09 socket.py(345): buf.seek(0, 2) # seek end
13.09 socket.py(346): if size < 0:
13.09 socket.py(362): buf_len = buf.tell()
13.09 socket.py(363): if buf_len >= size:
13.09 socket.py(371): self._rbuf = StringIO() # reset _rbuf. we consume it via buf.
13.09 socket.py(372): while True:
13.09 socket.py(373): left = size - buf_len
13.09 socket.py(379): try:
13.09 socket.py(380): data = self._sock.recv(left)
14.74 socket.py(385): if not data:
14.74 socket.py(387): n = len(data)
14.74 socket.py(388): if n == size and not buf_len:
14.74 socket.py(394): return data

Notice the ~2 second delay between socket.py lines 380/385. Obviously, the socket is blocking. Althought this is not Werkzeug code (at this point), I'm assuming the problem might have to do with the value of `left` or the initial configuration of the socket.

Thanks for looking, and keep up the great work on Werkzeug/Flask et. al.