benoitc/gunicorn — pull request #654 by avaitla (2013-12-04 12:10:11+00:00)
Title: Reinit Hub when using old version of gevent.

Hi again!

Epoll throws an EEXIST Error, so we cannot have more than one functional worker.
Related to https://github.com/benoitc/gunicorn/commit/f47262c5aef31145467ce4ad7d16b4dde48d88ce
https://github.com/benoitc/gunicorn/issues/478

I have attempted a patch, which I tried to make appropriate for the current code, but let me know
if there is something better or I am mistaken.

Failure noticed and reproducible on Centos 6.4 x86_64. Using gunicorn master
branch + gevent 0.13.8 + libevent 1.4. In general I believe this should effect
any user using epoll based gevent 0.13.8. Indeed, it cannot be reproduced
on OS X 10.8 most likely for this reason.

When we run the basic example app as: 'gunicorn -w 2 -k gevent myapp:app'
where the worker count is 2 or greater we will be thrown errors of the form indefinitly
as the second worker constantly dies and the arbiter attempts to respawn it.

2013-12-02 03:12:11 [5933] [ERROR] Exception in worker process:
Traceback (most recent call last):
File "../ENV/lib/python2.6/site-packages/gunicorn-18.1-py2.6.egg/gunicorn/arbiter.py", line 499, in spawn_worker
worker.init_process()
File "../ENV/lib/python2.6/site-packages/gunicorn-18.1-py2.6.egg/gunicorn/workers/ggevent.py", line 182, in init_process
super(GeventWorker, self).init_process()
File "../ENV/lib/python2.6/site-packages/gunicorn-18.1-py2.6.egg/gunicorn/workers/base.py", line 112, in init_process
self.run()
File "../ENV/lib/python2.6/site-packages/gunicorn-18.1-py2.6.egg/gunicorn/workers/ggevent.py", line 117, in run
server.start()
File "../ENV/lib/python2.6/site-packages/gevent/baseserver.py", line 149, in start
self.start_accepting()
File "../xapi/ENV/lib/python2.6/site-packages/gevent/server.py", line 99, in start_accepting
self._accept_event = core.read_event(self.socket.fileno(), self._do_accept, persist=True)
File "core.pyx", line 308, in gevent.core.read_event.__init__ (gevent/core.c:4499)
File "core.pyx", line 252, in gevent.core.event.add (gevent/core.c:3365)
IOError: [Errno 17] File exists

Indeed I have confirmed locally that the errno 17 is set on line 300 in libevent sources here:

https://github.com/libevent/libevent/blob/patches-1.4/epoll.c

I have also confirmed that it is due to this commit alone (indeed the other ones work fine)

https://github.com/benoitc/gunicorn/commit/f47262c5aef31145467ce4ad7d16b4dde48d88ce

Since we've removed the @classmethod setup, the fork call is no longer patched in time (in the arbiter config),
so we cannot call monkey.patch_all as we did previously, and thus we don't get the fork and hub reinit from:

https://github.com/surfly/gevent/blob/gevent013/gevent/hub.py