Issue: Is Peewee Greenlet-safe?
Author: joeyespo
State: closed
Created: 2016-08-07 15:48:15

My target setup is to use Flask, Peewee, and `LocalProxy` as fixed in #1025 for the development simplicity and power. (Peewee is awesome!) For production, I plan to use Gunicorn with Gevent workers to handle as many simultaneous connections as this stack will allow. (Then scale with more complicated methods _after_ proving the need.)

My question is, does Peewee work well with Greenlets? Adding to my example in #1025:

```
import peewee
from werkzeug.local import LocalProxy


@LocalProxy
def db():
    _db = getattr(g, '_db', None)
    if _db is None:
        _db = g._db = db_url.connect(
            current_app.config['DATABASE_URL'], threadlocals=False)
    return _db


class BaseModel(peewee.Model):
    class Meta:
        database = db


class MyModel(BaseModel):
    ...

def init_app(app):
    @app.teardown_appcontext
    def teardown_db(exception):
        _db = getattr(g, '_db', None)
        if _db and not _db.is_closed():
            _db.close()
        g._db = None
```

From what I understand, the above will lazily create a new connection when a model is accessed. [One per request](http://flask.pocoo.org/docs/0.11/appcontext/#locality-of-the-context). (Or rather, one per application context, so you can access models outside of requests too with [`app.app_context`](http://flask.pocoo.org/docs/0.11/appcontext/#creating-an-application-context). Say, in a migration script.) Since that handles connections [across threads](http://werkzeug.pocoo.org/docs/0.11/local/) too, I turned `threadlocals` off.

Is this correct? Or is something missing (say, explicit transaction handling)?

Thanks!


For questions regarding `werkzeug.LocalProxy`, you'll need to refer to the folks that maintain that project as I am not familiar with that particular class.

I can tell you, however, that peewee works well with gevent (fuck yeah, gevent! asyncio is a heaping pile of turds). If you're using `psycopg2` with `PostgresqlDatabase`, here's a snippet that will green up your connection:

```
# patch_psycopg2.py
from gevent import monkey

from gevent.socket import wait_read, wait_write
from psycopg2 import extensions


def monkeypatch_psycopg2():
    extensions.set_wait_callback(gevent_wait_callback)

def gevent_wait_callback(conn, timeout=None):
    """A wait callback useful to allow gevent to work with Psycopg."""
    while 1:
        state = conn.poll()
        if state == extensions.POLL_OK:
            break
        elif state == extensions.POLL_READ:
            wait_read(conn.fileno(), timeout=timeout)
        elif state == extensions.POLL_WRITE:
            wait_write(conn.fileno(), timeout=timeout)
        else:
            raise psycopg2.OperationalError(
                "Bad result from poll: %r" % state)
```

Then, in your WSGI file or wherever you're doing the rest of the gevent monkey-patching, simply call `monkeypatch_psycopg2()` and it will make `psycopg2` greenlet-friendly.

I'm not sure how familiar you are with gevent, but one particular but extremely important quirk is that you need to do your monkey-patching **before** anything else. Like, it has to be basically the first thing you do in your WSGI file.

If you're using MySQL, God help you. (just kidding: I'm just not familiar with a similar process for MySQL's python drivers).

Hope that helps!


Oh, I should add:

Peewee uses thread-locals to maintain connection state. If you plan on using gevent, just be sure the threading module gets patched as well. That way those thread-locals will become greenlet-locals, which is what you want.