Repository: freedomofpress/securedrop

Issue:
Ensure latest app code runs cleanly in staging environment
Due to churn on the develop branch, the layout of the application docroot has changed quite a bit, and the current AppArmor profile is not permitting the application to run when profiles are set to enforce. We'll need to update the AppArmor profile for Apache iteratively, to make sure we whitelist only what's required. There may be slight changes necessary to the app-code build role, as well, such as creating directories, but we'll cross that bridge when we come to it. 

There have also been changes to the application config values (in `config.py`) that aren't mindful of upgrade strategy: we'll need to make sure all these app code adjustments work well when existing 0.3.12 hosts upgrade.

Will keep a running list of required changes going here:

* [x] `util.py[c]` file not permitted for reading
* [x] jquery version has changed 2.1.1 -> 2.1.4
* [x] `config.CUSTOM_NOTIFICATION` may not be defined
* [x] `webassets` directory may need to be created (via deb package include; see also #1643)
* [x] stop setting complain mode on AppArmor profiles

Also considering dropping the `aa-complain` strategy in the Staging environment, since having AppArmor errors in staging is actually quite useful. 


Discussion:
Given the history on the util.py file, the most straightforward solution may be to move the custom `PathException` back into `store.py`, where it used to live. See:

* 7ae0fd9c884e4334b1b668e3af04c3ea63f5b258
* 956b2137255a39fb7091a82b669af6537ba0548d
> Also considering dropping the `aa-complain` strategy in the Staging environment, since having AppArmor errors in staging is actually quite useful. 

I feel strongly we should stop setting `aa-complain` in staging. The `development` VM already provides devs with a platform for rapid prototyping without worrying about AppArmor profiles—the staging environment, as another step toward imitating production instances, should indeed enable AppArmor. That's the rationale on the strategy front.

The rationale on the technical front is that since merging #1464, the staging environment actually has AppArmor profiles enabled via the deb package `postinst` script. We _could_ refactor the package build logic to disable that check intelligently if we're building for staging, but I don't think that's worth the effort, as it introduces complexity that may permit mistakes to slip in on the build process.

Using the staging environment to rebuild packages, install them, then validate via manual testing (or automated, via the application tests) that code changes play well with the Apache vhosts and the corresponding AppArmor rules is a sane and intuitive method for ensuring we're delivering stable code.

Resolving patch:
diff --git a/securedrop/source.py b/securedrop/source.py
--- a/securedrop/source.py
+++ b/securedrop/source.py
@@ -20,7 +20,6 @@
 import crypto_util
 import store
 import template_filters
-import util
 from db import db_session, Source, Submission, Reply, get_one_or_else
 from request_that_secures_file_uploads import RequestThatSecuresFileUploads
 from jinja2 import evalcontextfilter
@@ -125,7 +124,8 @@ def check_tor2web():
 @app.route('/')
 def index():
     return render_template('index.html',
-                           custom_notification=config.CUSTOM_NOTIFICATION)
+                           custom_notification=getattr(config,
+                           'CUSTOM_NOTIFICATION', ''))
 
 
 def generate_unique_codename(num_words=7):
diff --git a/securedrop/store.py b/securedrop/store.py
--- a/securedrop/store.py
+++ b/securedrop/store.py
@@ -12,7 +12,6 @@
 from werkzeug import secure_filename
 
 from secure_tempfile import SecureTemporaryFile
-from util import PathException
 
 import logging
 log = logging.getLogger(__name__)
@@ -21,6 +20,14 @@
     "^(?P<index>\d+)\-[a-z0-9-_]*(?P<file_type>msg|doc\.(gz|zip)|reply)\.gpg$").match
 
 
+class PathException(Exception):
+
+    """An exception raised by `util.verify` when it encounters a bad path. A path
+    can be bad when it is not absolute or not normalized.
+    """
+    pass
+
+
 def verify(p):
     """Assert that the path is absolute, normalized, inside `config.STORE_DIR`, and
     matches the filename format.
diff --git a/securedrop/util.py b/securedrop/util.py
deleted file mode 100644
--- a/securedrop/util.py
+++ /dev/null
@@ -1,11 +0,0 @@
-# -*- coding: utf-8 -*-
-
-from os import path
-
-
-class PathException(Exception):
-
-    """An exception raised by `util.verify` when it encounters a bad path. A path
-    can be bad when it is not absolute or not normalized.
-    """
-    pass


Test patch:
diff --git a/securedrop/tests/test_source.py b/securedrop/tests/test_source.py
--- a/securedrop/tests/test_source.py
+++ b/securedrop/tests/test_source.py
@@ -14,6 +14,7 @@
 import version
 import utils
 import json
+import config
 
 
 class TestSourceApp(TestCase):
@@ -244,6 +245,28 @@ def test_submit_sanitizes_filename(self, gzipfile):
                                     mode=ANY,
                                     fileobj=ANY)
 
+    def test_custom_notification(self):
+        """Test that `CUSTOM_NOTIFICATION` string in config file
+        is rendered on the Source Interface page. We cannot assume
+        it will be present in production instances, since it is added
+        via the Ansible config, not the Debian package scripts."""
+        custom_msg = config.CUSTOM_NOTIFICATION
+
+        dev_msg = ("This is an insecure SecureDrop Development server "
+                   "for testing ONLY. Do NOT submit documents here.")
+        staging_msg = "This is a SecureDrop Staging VM for testing ONLY"
+
+        self.assertTrue(custom_msg in (dev_msg, staging_msg))
+        resp = self.client.get('/')
+        self.assertEqual(resp.status_code, 200)
+        # The app-tests aren't host-aware, so we can't accurately predict
+        # which custom notification message we want. Let's check for both,
+        # and fail only if both are not found.
+        try:
+            self.assertIn(dev_msg, resp.data)
+        except AssertionError:
+            self.assertIn(staging_msg, resp.data)
+
     def test_tor2web_warning_headers(self):
         resp = self.client.get('/', headers=[('X-tor2web', 'encrypted')])
         self.assertEqual(resp.status_code, 200)
diff --git a/testinfra/app/test_apparmor.py b/testinfra/app/test_apparmor.py
--- a/testinfra/app/test_apparmor.py
+++ b/testinfra/app/test_apparmor.py
@@ -58,8 +58,7 @@ def test_apparmor_ensure_not_disabled(File, Sudo, profile):
     with Sudo():
         assert not f.exists
 
-@pytest.mark.skipif(os.environ['SECUREDROP_TESTINFRA_TARGET_HOST'] != 'app-staging',
-                    reason='only to be run on app-staging')
+
 @pytest.mark.parametrize('complain_pkg', sdvars.apparmor_complain)
 def test_app_apparmor_complain(Command, Sudo, complain_pkg):
     """ Ensure app-armor profiles are in complain mode for staging """
@@ -68,8 +67,7 @@ def test_app_apparmor_complain(Command, Sudo, complain_pkg):
         c = Command.check_output("aa-status | {}".format(awk))
         assert complain_pkg in c
 
-@pytest.mark.skipif(os.environ['SECUREDROP_TESTINFRA_TARGET_HOST'] != 'app-staging',
-                    reason='only to be run on app-staging')
+
 def test_app_apparmor_complain_count(Command, Sudo):
     """ Ensure right number of app-armor profiles are in complain mode """
     with Sudo():
diff --git a/testinfra/build/test_securedrop_deb_package.py b/testinfra/build/test_securedrop_deb_package.py
--- a/testinfra/build/test_securedrop_deb_package.py
+++ b/testinfra/build/test_securedrop_deb_package.py
@@ -141,3 +141,27 @@ def test_deb_package_contains_no_config_file(File, Command, deb):
     # would be cleaner. Will defer to adding lintian tests later.
     c = Command("dpkg-deb --contents {}".format(deb_package.path))
     assert not re.search("^.*config\.py$", c.stdout, re.M)
+
+
+@pytest.mark.parametrize("deb", deb_packages)
+def test_deb_package_contains_no_generated_assets(File, Command, deb):
+    """
+    Ensures the `securedrop-app-code` package does not ship a minified
+    static assets, which are built automatically via Flask-Assets, and may be
+    present in the source directory used to build from.
+    """
+    deb_package = File(deb.format(
+        securedrop_test_vars.securedrop_version))
+
+    # Only relevant for the securedrop-app-code package:
+    if "securedrop-app-code" in deb_package.path:
+        c = Command("dpkg-deb --contents {}".format(deb_package.path))
+        # static/gen/ directory should exist
+        assert re.search("^.*\./var/www/securedrop/static/gen/$", c.stdout, re.M)
+        # static/gen/ directory should be empty
+        assert not re.search("^.*\./var/www/securedrop/static/gen/.+$", c.stdout, re.M)
+
+        # static/.webassets-cache/ directory should exist
+        assert re.search("^.*\./var/www/securedrop/static/.webassets-cache/$", c.stdout, re.M)
+        # static/.webassets-cache/ directory should be empty
+        assert not re.search("^.*\./var/www/securedrop/static/.webassets-cache/.+$", c.stdout, re.M)