Repository: freedomofpress/securedrop

Issue:
Sources with a null value for last_updated will break the web UI
## Description

If a source is created without a `last_updated` filed, it will cause the following error on the journalist index page.

```
172.17.0.1 - - [10/Oct/2018 18:49:40] "GET / HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/heartsucker/code/freedomofpress/securedrop/securedrop/journalist_app/main.py", line 79, in index
    starred=starred)
  File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 135, in render_template
    context, ctx.app)
  File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 117, in _render
    rv = template.render(context)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/heartsucker/code/freedomofpress/securedrop/securedrop/journalist_templates/index.html", line 1, in top-level template code
    {% extends "base.html" %}
  File "/home/heartsucker/code/freedomofpress/securedrop/securedrop/journalist_templates/base.html", line 50, in top-level template code
    {% block body %}{% endblock %}
  File "/home/heartsucker/code/freedomofpress/securedrop/securedrop/journalist_templates/index.html", line 25, in block "body"
    {% include '_source_row.html' %}
  File "/home/heartsucker/code/freedomofpress/securedrop/securedrop/journalist_templates/_source_row.html", line 4, in top-level template code
    <time class="date" title="{{ source.last_updated|rel_datetime_format }}" datetime="{{ source.last_updated|rel_datetime_format(fmt="%Y-%m-%d %H:%M:%S%Z") }}">{{ source.last_updated|rel_datetime_format(relative=True) }}</time>
  File "/home/heartsucker/code/freedomofpress/securedrop/securedrop/template_filters.py", line 12, in rel_datetime_format
    time = dates.format_timedelta(datetime.utcnow() - dt,
TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'
```

The root cause of this is the schema in `sqlite` permits null values in the `last_updated` columns.

```sql
CREATE TABLE sources (
	id INTEGER NOT NULL, 
	uuid VARCHAR(36) NOT NULL, 
	filesystem_id VARCHAR(96), 
	journalist_designation VARCHAR(255) NOT NULL, 
	flagged BOOLEAN, 
	last_updated DATETIME,
        -- ^ allows null :(
	pending BOOLEAN, 
	interaction_count INTEGER NOT NULL, 
	PRIMARY KEY (id), 
	UNIQUE (uuid), 
	UNIQUE (filesystem_id), 
	CHECK (flagged IN (0, 1)), 
	CHECK (pending IN (0, 1))
);
```

## Steps to Reproduce

Manually create a source with `last_updated = null`.

## Expected Behavior

The app should recover gracefully or (better) disallow this entirely.

## Actual Behavior

The app blows up.


Discussion:

Resolving patch:
diff --git a/securedrop/journalist_app/main.py b/securedrop/journalist_app/main.py
--- a/securedrop/journalist_app/main.py
+++ b/securedrop/journalist_app/main.py
@@ -62,6 +62,7 @@ def index():
         # the Pocoo style guide, IMHO:
         # http://www.pocoo.org/internal/styleguide/
         sources = Source.query.filter_by(pending=False) \
+                              .filter(Source.last_updated.isnot(None)) \
                               .order_by(Source.last_updated.desc()) \
                               .all()
         for source in sources:


Test patch:
diff --git a/securedrop/tests/test_journalist.py b/securedrop/tests/test_journalist.py
--- a/securedrop/tests/test_journalist.py
+++ b/securedrop/tests/test_journalist.py
@@ -1942,3 +1942,20 @@ def test_col_process_successfully_unstars_sources(journalist_app,
 
     source = Source.query.get(test_source['id'])
     assert not source.star.starred
+
+
+def test_source_with_null_last_updated(journalist_app,
+                                       test_journo,
+                                       test_files):
+    '''Regression test for issues #3862'''
+
+    source = test_files['source']
+    source.last_updated = None
+    db.session.add(source)
+    db.session.commit()
+
+    with journalist_app.test_client() as app:
+        _login_user(app, test_journo['username'], test_journo['password'],
+                    test_journo['otp_secret'])
+        resp = app.get(url_for('main.index'))
+        assert resp.status_code == 200