unbit/uwsgi — pull request #609 by trawick (2014-04-29 17:15:34+00:00)
Title: Fix uWSGI FastCGI protocol handler to read the FCGI_STDIN "eof" record.

FastCGI "gateways" send a FCGI_STDIN record with len 0 to indicate
end of stream, even if no request body is present. The uWSGI
protocol handler does not ensure that it has been read, so if
the gateway writes the EOF in a separate syscall, it is possible
for uWSGI + application to finish the request using prior data
and close the socket prior to the gateway trying to write the EOF.

httpd 2.4's mod_proxy_fcgi can fail in this manner intermittently.
(Example runs had in the general neighborhood of 100 failures out
of a 100,000 requests, over both TCP and Unix sockets.)

nginx doesn't normally encounter an error because the FCGI_STDIN
record with len 0 is just about always in the same buffer as the
parameters. (If because of configuration the size of the
parameters was much larger and it exactly filled the first I/O
buffer, I expect that the same issue would occur with nginx.)

In the patch I check in the FastCGI protocol close handler
whether or not this eof has been seen, or at least read from
the OS. If not, I try one more read with the minimum possible
timeout (1 second). If that isn't the eof packet, there are
bigger problems, such as it being a request body that the
application didn't try to read, which is also going to cause
an I/O error in the gateway. Thus, after reading one last
time the code doesn't try to see what it read.

This change resolves the issue I had with mod_proxy_fcgi, and
I didn't find any functional or performance regression with either
nginx or httpd, using a simple WSGI application under control
of uWSGI.

A bit more info is at

http://emptyhammock.blogspot.com/2014/04/sysdig-really-is-strace-plus-lsof.html
and eventually at
http://emptyhammock.com/projects/info/pyweb/web-server-improvements.html