Repository: freedomofpress/securedrop

Issue:
Source Interface requests fail with 500 error, due to session issue
## Description

In some situations, requests to the source interface may start to fail, returning the 500 error page. Once the 500 errors start, they continue until the Tor Browser cache is cleared, either explicitly or by starting a new browser session. With source error logging enabled, the following errors are seen on failing requests:

```
[Thu Apr 18 09:46:09.516056 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169] [2019-04-18 09:46:09,510] ERROR in app: Exception on / [GET]
[Thu Apr 18 09:46:09.516238 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169] Traceback (most recent call last):
[Thu Apr 18 09:46:09.516279 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2292, in wsgi_app
[Thu Apr 18 09:46:09.516317 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     response = self.full_dispatch_request()
[Thu Apr 18 09:46:09.516363 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1815, in full_dispatch_request
[Thu Apr 18 09:46:09.516442 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     rv = self.handle_user_exception(e)
[Thu Apr 18 09:46:09.516479 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1718, in handle_user_exception
[Thu Apr 18 09:46:09.516514 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     reraise(exc_type, exc_value, tb)
[Thu Apr 18 09:46:09.516549 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1811, in full_dispatch_request
[Thu Apr 18 09:46:09.516584 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     rv = self.preprocess_request()
[Thu Apr 18 09:46:09.516619 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2087, in preprocess_request
[Thu Apr 18 09:46:09.516654 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     rv = func()
[Thu Apr 18 09:46:09.516688 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/var/www/securedrop/source_app/decorators.py", line 23, in decorated_function
[Thu Apr 18 09:46:09.516724 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     return f(*args, **kwargs)
[Thu Apr 18 09:46:09.516758 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/var/www/securedrop/source_app/__init__.py", line 159, in setup_g
[Thu Apr 18 09:46:09.516793 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     g.codename = session['codename']
[Thu Apr 18 09:46:09.516828 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 377, in <lambda>
[Thu Apr 18 09:46:09.516864 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     __getitem__ = lambda x, i: x._get_current_object()[i]
[Thu Apr 18 09:46:09.516899 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]   File "/usr/local/lib/python2.7/dist-packages/flask/sessions.py", line 83, in __getitem__
[Thu Apr 18 09:46:09.516933 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169]     return super(SecureCookieSession, self).__getitem__(key)
[Thu Apr 18 09:46:09.516968 2019] [wsgi:error] [pid 7324:tid 3457186817792] [remote 127.0.0.1:29169] KeyError: 'codename'
```

## Steps to Reproduce
This error was initially hit by:
1) starting a source interface session on a 0.12.1 Xenial install
2) updating the 0.12.1 Xenial install to 0.12.2~rc1 via cron-apt
3) running a 0.12.1 db restore against the 0.12.2 database
4) attempting to continue the source session.

It's also been reproduced during a test session by creating multiple sources and logging in and out repeatedly (h/t @eloquence ), but is not reliably reproducible.

## Expected Behavior
Source Interface requests for valid URLs return the correct results.

## Actual Behavior
SI requests all return 500 errors.

## Comments




Discussion:
I will try to reproduce it now.
Followed the steps


-    starting a source interface session on a 0.12.1 Xenial install
-    updating the 0.12.1 Xenial install to 0.12.2~rc1 via cron-apt
-    running a 0.12.1 db restore against the 0.12.2 database
-    attempting to continue the source session.

But, could not reproduce the error :(
I should clarify that the testing that I did which resulted in the error was on a production instance running 0.12.1. The sequence was basically: login to source interface, submit a file, log out; rinse and repeat as new source (*without* resetting my Tor identity). I did this 10 times or so.

I don't believe I took any unusual steps during the process; at one point, I was just effectively locked out in the manner Kevin describes, until I reset my identity. If what I experienced is the same issue, it is not a regression since 0.12.1.
^^ tried the above submission as new source around 30 times on `0.12.2` rc, could not reproduce it :( Need to find an easier way to reproduce this.
I've removed the release QA label as we have no evidence that this is a regression introduced after 0.12.1. So far reproduction has been extremely difficult. As time allows, let's aim to understand the cause better, but per discussion yesterday, this is not a 0.12.2 blocker.

Resolving patch:
diff --git a/securedrop/source_app/main.py b/securedrop/source_app/main.py
--- a/securedrop/source_app/main.py
+++ b/securedrop/source_app/main.py
@@ -67,6 +67,13 @@ def create():
 
             # Issue 2386: don't log in on duplicates
             del session['codename']
+
+            # Issue 4361: Delete 'logged_in' if it's in the session
+            try:
+                del session['logged_in']
+            except KeyError:
+                pass
+
             abort(500)
         else:
             os.mkdir(current_app.storage.path(filesystem_id))


Test patch:
diff --git a/securedrop/tests/test_source.py b/securedrop/tests/test_source.py
--- a/securedrop/tests/test_source.py
+++ b/securedrop/tests/test_source.py
@@ -146,7 +146,31 @@ def test_generate_too_long_codename(source_app):
     )
 
 
-def test_create_duplicate_codename(source_app):
+def test_create_duplicate_codename_logged_in_not_in_session(source_app):
+    with patch.object(source.app.logger, 'error') as logger:
+        with source_app.test_client() as app:
+            resp = app.get(url_for('main.generate'))
+            assert resp.status_code == 200
+
+            # Create a source the first time
+            resp = app.post(url_for('main.create'), follow_redirects=True)
+            assert resp.status_code == 200
+            codename = session['codename']
+
+        with source_app.test_client() as app:
+            # Attempt to add the same source
+            with app.session_transaction() as sess:
+                sess['codename'] = codename
+            resp = app.post(url_for('main.create'), follow_redirects=True)
+            logger.assert_called_once()
+            assert ("Attempt to create a source with duplicate codename"
+                    in logger.call_args[0][0])
+            assert resp.status_code == 500
+            assert 'codename' not in session
+            assert 'logged_in' not in session
+
+
+def test_create_duplicate_codename_logged_in_in_session(source_app):
     with patch.object(source.app.logger, 'error') as logger:
         with source_app.test_client() as app:
             resp = app.get(url_for('main.generate'))
@@ -157,12 +181,17 @@ def test_create_duplicate_codename(source_app):
             assert resp.status_code == 200
 
             # Attempt to add the same source
-            app.post(url_for('main.create'), follow_redirects=True)
+            resp = app.post(url_for('main.create'), follow_redirects=True)
             logger.assert_called_once()
             assert ("Attempt to create a source with duplicate codename"
                     in logger.call_args[0][0])
+            assert resp.status_code == 500
             assert 'codename' not in session
 
+            # Reproducer for bug #4361
+            resp = app.post(url_for('main.index'), follow_redirects=True)
+            assert 'logged_in' not in session
+
 
 def test_lookup(source_app):
     """Test various elements on the /lookup page."""