Issue: Document gevent at one place
Author: slavkoja
State: closed
Created: 2018-02-19 10:19:49

I search your docs and found here and there some `gevent` related information. The most one is from retired `flask-peewee` project (i am mostly interested in `peewee` + `flask`). I did heavy search on internet, but i was no able to find any reliable information and most of the found are Django related, where the reliable sources strongly suggests against use of Django ORM with `gevent`. I am not able to decide, if they can/have to be applied to `peewee` too.

Please, can you add separate section in docs, and clarify usage of the `peewee` with `gevent` and all related settings at one place?

regards

I'm not sure I understand what your question is. gevent+flask+peewee is my preferred setup for all my projects...the thing I like about gevent is that you don't really need to change your code.

If you're using gevent and Postgresql, you can register a wait callback. Here is the code from the [psycogreen](https://bitbucket.org/dvarrazzo/psycogreen) project (see project for license and details):

```python
"""A wait callback to allow psycopg2 cooperation with gevent.

Use `patch_psycopg()` to enable gevent support in Psycopg.
"""

# Copyright (C) 2010-2012 Daniele Varrazzo <daniele.varrazzo@gmail.com>
# All rights reserved.  See COPYING file for details.


from __future__ import absolute_import

import psycopg2
from psycopg2 import extensions

from gevent.socket import wait_read, wait_write

def patch_psycopg():
    """Configure Psycopg to be used with gevent in non-blocking way."""
    if not hasattr(extensions, 'set_wait_callback'):
        raise ImportError(
            "support for coroutines not available in this Psycopg version (%s)"
            % psycopg2.__version__)

    extensions.set_wait_callback(gevent_wait_callback)

def gevent_wait_callback(conn, timeout=None):
    """A wait callback useful to allow gevent to work with Psycopg."""
    while 1:
        state = conn.poll()
        if state == extensions.POLL_OK:
            break
        elif state == extensions.POLL_READ:
            wait_read(conn.fileno(), timeout=timeout)
        elif state == extensions.POLL_WRITE:
            wait_write(conn.fileno(), timeout=timeout)
        else:
            raise psycopg2.OperationalError(
                "Bad result from poll: %r" % state)
```

gevent monkey-patches socket (and threading) so pure python database drivers like pymysql should automatically work fine with gevent.

I am sorry, my English is not best and sometime my writings are not clear, i will try to be more verbose.

Consider, that i am new to `gevent` and relative new to `peewee` and i wan to to know, if are peewee's DB adapters safe to use (nonblocking). It is not explicitly declared (at least i not found this) in docs and i have basically two options: go into sources or try it. But i am not sure, if i will able to distinguish it in sources. I did some tests, but again, i am not sure if i choose the right test case for it.

You provided example with Postgresql, and mentioned `pymysql`, but what about SQLite, or APSW, or ...? Somewhere in your docs i read about `SqliteQueueDatabase` and `gevent`, then have i to choose this if i want use `gevent` with SQLIte? Etc, etc.

The `flask-peewee` doc (linked from peewee's doc) has dedicated page for `gevent` usage. But `flask-peewee` is no more developed, then one cannot know, if the information there is still valid and if it is valid for `peewee` itself.

You (the author) know these nuances best, then i ask to collect all related information at one place, where the new users can read it (and do not bother you with questions or waste time on internet search, where is near nothing about this) and then they can choose right tools for it needing.

So `gevent` uses an event loop, meaning when a socket wants to do some blocking operation, it registers with the event loop, which will wake it up when data is ready for it to read/write. Currently gevent does not work with disk i/o, only socket i/o, so you will only need to be concerned about your peewee database when you're talking to a db over a socket. For this reason, sqlite will always be blocking, since it runs in the same address space as your application. If you're talking to a remote postgres database (or mysql), you can use gevent to make the db sockets cooperative. This will happen automatically if the driver is python, but in the case of psycopg2, you are best off registering a wait callback, as shown in the above code.

Regarding SqliteQueueDatabase, this module exists to work around the fact that SQLite locks the database for writes. You can read [more about the problem here](http://charlesleifer.com/blog/multi-threaded-sqlite-without-the-operationalerrors/). If you are writing a multi-threaded (or multi-green-thread) application, you want to avoid locking the db from multiple threads. The SqliteQueueDatabase works by using a single dedicated worker thread for processing all writes, ensuring that there's never more than one connection writing to the db. The SqliteQueueDatabase accepts a parameter, `use_gevent`, which uses gevent threading primitives instead of the standard lib threading primitives, so you can easily use the library in an application that uses gevent. If you're using gevent and monkey-patching everything already, then technically you don't need to specify use_gevent, because the standard lib threading implementation will have been monkeypatched already. The bottom line, however, is that you should only use SqliteQueueDatabase if you thoroughly understand the issue, the implementation and the trade-offs, as it's designed for a rather narrow use-case and carries a significant trade-off -- no transactional support.

To make a long story short, if you're writing a web app with flask, peewee and gevent, and want to use sqlite, you probably don't need to do anything special. Just use `peewee.SqliteDatabase` (or `playhouse.sqlite_ext.SqliteExtDatabase`).

Very thanks, that i exactly what i want to know now, and slightly more than i was asking to exactly mention in dedicated doc's `gevent` section. ;-)