Repository: freedomofpress/securedrop

Issue:
async_genkey runs in a thread that has no flask application
# Bug

## Description

async_genkey tries to access the database in a separate thread and fails with the error like

<pre>
[2018-02-06 00:05:33,620] ERROR in utils: async_genkey for source (filesystem_id=AXT3PQIZMYAMHOZ4MIVX6ZGHCWP3GKWNI5UW2UMMANIGQJ6CUTYH45VTEP63X2DAWPRTS3PUPQBAFCLI4JMOZHVZPOZXGIKWMCFJQ3I=): No application found. Either work inside a view function or push an application context. See http://flask-sqlalchemy.pocoo.org/contexts/.
</pre>

See an example of failure probably due to this race https://gist.github.com/conorsch/b76b0ae5f81b1a78428d59e1b4666e7a#file-app-test-failures-log-L358

## Steps to Reproduce

Not sure.

## Expected Behavior

* pytest tests/test_integration.py runs with no failure.

## Actual Behavior

* pytest tests/test_integration.py runs sometime fails with
<pre>
No application found. Either work inside a view function or push an application context. See http://flask-sqlalchemy.pocoo.org/contexts/.
</pre>



Discussion:
@heartsucker do you have a suggestion on how to fix **async_genkey** ?
This class of issue is only going to be exacerbated by #2969 and #2877. I wonder if it would make sense to replace use of `db` from the `from db import db` with a manually generated connection using `sqlalchemy.create_engine`. This would ensure that the DB connection's life cycle is managed independently of the application.
For the record: http://docs.sqlalchemy.org/en/latest/orm/contextual.html
Refs https://github.com/freedomofpress/securedrop/issues/2609
Refs https://github.com/freedomofpress/securedrop/issues/2614

So, what about we create a session in async_genkey and following http://docs.sqlalchemy.org/en/latest/orm/session_basics.html instead of using the query property ?

<pre>
session = scoped_session(sessionmaker(autocommit=False,
                                         autoflush=False,
                                         bind=engine))
source = session.query(Source).filter(Source.filesystem_id == filesystem_id).one()
source.last_updated = datetime.utcnow()
</pre>

This will effectively write to the same SQlite in parallel and rely on proper locking to avoid races. I'm not sure how good SQlite as used by sqlalchemy is when presented with this use case though (http://www.sqlite.org/wal.html is a good read).

sqlachemy sessiosn are not thread safe http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#is-the-session-thread-safe

Resolving patch:
diff --git a/securedrop/source_app/main.py b/securedrop/source_app/main.py
--- a/securedrop/source_app/main.py
+++ b/securedrop/source_app/main.py
@@ -90,7 +90,8 @@ def lookup():
         # Only do this if the journalist has flagged the source as one
         # that they would like to reply to. (Issue #140.)
         if not crypto_util.getkey(g.filesystem_id) and g.source.flagged:
-            async_genkey(g.filesystem_id, g.codename)
+            db_uri = current_app.config['SQLALCHEMY_DATABASE_URI']
+            async_genkey(db_uri, g.filesystem_id, g.codename)
 
         return render_template(
             'lookup.html',
@@ -164,7 +165,8 @@ def submit():
             # (gpg reads 300 bytes from /dev/random)
             entropy_avail = get_entropy_estimate()
             if entropy_avail >= 2400:
-                async_genkey(g.filesystem_id, g.codename)
+                db_uri = current_app.config['SQLALCHEMY_DATABASE_URI']
+                async_genkey(db_uri, g.filesystem_id, g.codename)
                 current_app.logger.info("generating key, entropy: {}".format(
                     entropy_avail))
             else:
diff --git a/securedrop/source_app/utils.py b/securedrop/source_app/utils.py
--- a/securedrop/source_app/utils.py
+++ b/securedrop/source_app/utils.py
@@ -3,13 +3,14 @@
 
 from datetime import datetime
 from flask import session, current_app, abort, g
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
 from threading import Thread
 
 import crypto_util
 import i18n
 import store
 
-from db import db
 from models import Source
 
 
@@ -68,21 +69,23 @@ def wrapper(*args, **kwargs):
 
 
 @async
-def async_genkey(filesystem_id, codename):
+def async_genkey(db_uri, filesystem_id, codename):
     crypto_util.genkeypair(filesystem_id, codename)
 
     # Register key generation as update to the source, so sources will
     # filter to the top of the list in the journalist interface if a
     # flagged source logs in and has a key generated for them. #789
+    session = sessionmaker(bind=create_engine(db_uri))()
     try:
-        source = Source.query.filter(Source.filesystem_id == filesystem_id) \
-                       .one()
+        source = session.query(Source).filter(
+            Source.filesystem_id == filesystem_id).one()
         source.last_updated = datetime.utcnow()
-        db.session.commit()
+        session.commit()
     except Exception as e:
         logging.getLogger(__name__).error(
                 "async_genkey for source (filesystem_id={}): {}"
                 .format(filesystem_id, e))
+    session.close()
 
 
 def normalize_timestamps(filesystem_id):


Test patch: