internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: freedomofpress/securedrop

Issue:
Logging out a source does not display a localized "Thank you for exiting your session!" message
# Bug

## Description

When a source clears their session via the logout button, their locale is popped as well. Thus, the following message is not displayed in the correct language.

```
Important!
Thank you for exiting your session! Please select "New Identity" from the green Onion
button in the Tor browser to clear all history of your SecureDrop usage from this device.
```

## Steps to Reproduce

1. Open source interface
2. Create new user
3. Submit doc
4. Log out

## Expected Behavior

The logout page displays the (quite important) log out message in the correct locale (including HTML with `dir="rtl"` set if needed).

## Actual Behavior

The page is displayed in the default locale.


Discussion:

Resolving patch:
diff --git a/securedrop/source_app/main.py b/securedrop/source_app/main.py
--- a/securedrop/source_app/main.py
+++ b/securedrop/source_app/main.py
@@ -215,8 +215,11 @@ def login():
     @view.route('/logout')
     def logout():
         if logged_in():
-            session.clear()
             msg = render_template('logout_flashed_message.html')
+
+            # clear the session after we render the message so it's localized
+            session.clear()
+
             flash(Markup(msg), "important hide-if-not-tor-browser")
         return redirect(url_for('.index'))
 


Test patch:
Repository: freedomofpress/securedrop

Issue:
Strip unnecessary whitespace from templates
# Feature request

## Description

In relation to #1152, we can marginally improve the load speed on pages by using [whitespace control](http://jinja.pocoo.org/docs/dev/templates/#whitespace-control) in our jinja templates. This will only provide a small decrease in pages sizes, but every byte counts and it's relatively easy to implement.

## User Stories

As a source, I want my page to load 3 milliseconds faster.


Discussion:

Resolving patch:
diff --git a/securedrop/journalist.py b/securedrop/journalist.py
--- a/securedrop/journalist.py
+++ b/securedrop/journalist.py
@@ -37,6 +37,8 @@
 
 assets = Environment(app)
 
+app.jinja_env.trim_blocks = True
+app.jinja_env.lstrip_blocks = True
 app.jinja_env.globals['version'] = version.__version__
 if getattr(config, 'CUSTOM_HEADER_IMAGE', None):
     app.jinja_env.globals['header_image'] = config.CUSTOM_HEADER_IMAGE
diff --git a/securedrop/source_app/__init__.py b/securedrop/source_app/__init__.py
--- a/securedrop/source_app/__init__.py
+++ b/securedrop/source_app/__init__.py
@@ -38,6 +38,8 @@ def create_app(config):
 
     i18n.setup_app(app)
 
+    app.jinja_env.trim_blocks = True
+    app.jinja_env.lstrip_blocks = True
     app.jinja_env.globals['version'] = version.__version__
     if getattr(config, 'CUSTOM_HEADER_IMAGE', None):
         app.jinja_env.globals['header_image'] = config.CUSTOM_HEADER_IMAGE


Test patch:
Repository: freedomofpress/securedrop

Issue:
Abort with 500 if duplicate codename is ever generated
## Description

For defense in depth, we should bail (`abort(500)`) if a duplicate codename is ever generated instead of logging an error and continuing on. [See discussion here](https://github.com/freedomofpress/securedrop/pull/2377#discussion_r141571021).


Discussion:

Resolving patch:
diff --git a/securedrop/source_app/main.py b/securedrop/source_app/main.py
--- a/securedrop/source_app/main.py
+++ b/securedrop/source_app/main.py
@@ -54,6 +54,10 @@ def create():
             current_app.logger.error(
                 "Attempt to create a source with duplicate codename: %s" %
                 (e,))
+
+            # Issue 2386: don't log in on duplicates
+            del session['codename']
+            abort(500)
         else:
             os.mkdir(store.path(filesystem_id))
 


Test patch:
diff --git a/securedrop/tests/test_source.py b/securedrop/tests/test_source.py
--- a/securedrop/tests/test_source.py
+++ b/securedrop/tests/test_source.py
@@ -149,6 +149,7 @@ def test_create_duplicate_codename(self, logger):
             logger.assert_called_once()
             self.assertIn("Attempt to create a source with duplicate codename",
                           logger.call_args[0][0])
+            assert 'codename' not in session
 
     def test_lookup(self):
         """Test various elements on the /lookup page."""
Repository: freedomofpress/securedrop

Issue:
Session expiring do not display a localized logout message.
# Bug

## Description

Like #2391, if a source has their session expire, they will not be shown a localized message when they log out.

## Steps to Reproduce

Set session expire to 30 seconds. Log in. Set locale to not-english. Wait 30 seconds. Refresh. See no-localized flashed message.

## Expected Behavior

The logout message is localized.

## Actual Behavior

It is not.


Discussion:

Resolving patch:
diff --git a/securedrop/source_app/__init__.py b/securedrop/source_app/__init__.py
--- a/securedrop/source_app/__init__.py
+++ b/securedrop/source_app/__init__.py
@@ -79,8 +79,11 @@ def setup_g():
         g.locales = i18n.get_locale2name()
 
         if 'expires' in session and datetime.utcnow() >= session['expires']:
-            session.clear()
             msg = render_template('session_timeout.html')
+
+            # clear the session after we render the message so it's localized
+            session.clear()
+
             flash(Markup(msg), "important")
 
         session['expires'] = datetime.utcnow() + \


Test patch:
Repository: freedomofpress/securedrop

Issue:
add-admin returns confusing strings
# Bug

## Description

When adding the first admin user on a new SecureDrop install, it returns a confusing string
`./manage py add-admin`
Username: 
Note: _Journalist_ passwords are now autogenerated

## Steps to Reproduce

Add a new admin user on a fresh SecureDrop install

## Expected Behavior

Our script outputs administrator added to the user 

## Actual Behavior

Our script says we created a new journalist user not an administrator.


Discussion:
I am ready to count this as an UX bug. We should update the text to administrator.

Resolving patch:
diff --git a/securedrop/manage.py b/securedrop/manage.py
--- a/securedrop/manage.py
+++ b/securedrop/manage.py
@@ -148,16 +148,16 @@ def _make_password():
 def _add_user(is_admin=False):
     username = _get_username()
 
-    print("Note: Journalist passwords are now autogenerated.")
+    print("Note: Passwords are now autogenerated.")
     password = _make_password()
-    print("This journalist's password is: {}".format(password))
+    print("This user's password is: {}".format(password))
 
     is_hotp = _get_yubikey_usage()
     otp_secret = None
     if is_hotp:
         while True:
             otp_secret = raw_input(
-                'Please configure your YubiKey and enter the secret: ')
+                "Please configure this user's YubiKey and enter the secret: ")
             if otp_secret:
                 break
 


Test patch:
Repository: freedomofpress/securedrop

Issue:
Add Brazil flag for nice display in the language picker
Since the `pt_BR` locale is ready to go (see #2630), we should add a Brazil flag to the language picker.


Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -59,9 +59,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '0.4.4'
+version = '0.5-rc2'
 # The full version, including alpha/beta/rc tags.
-release = '0.4.4'
+release = '0.5-rc2'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
diff --git a/securedrop/version.py b/securedrop/version.py
--- a/securedrop/version.py
+++ b/securedrop/version.py
@@ -1 +1 @@
-__version__ = '0.4.4'
+__version__ = '0.5-rc2'


Test patch:
diff --git a/molecule/builder/tests/vars.yml b/molecule/builder/tests/vars.yml
--- a/molecule/builder/tests/vars.yml
+++ b/molecule/builder/tests/vars.yml
@@ -1,5 +1,5 @@
 ---
-securedrop_version: "0.4.4"
+securedrop_version: "0.5-rc2"
 ossec_version: "2.8.2"
 keyring_version: "0.1.1"
Repository: freedomofpress/securedrop

Issue:
CSS breakage for source in de_DE
# Bug

## Description

The source interface when someone tries to submit a document using `de_DE` locale, the css breaks for the buttons below.

## Steps to Reproduce

Using `de_DE` locale, try to submit a document as source.



## Expected Behavior

The buttons should come up horizontally. 

## Actual Behavior

![de_css_break](https://user-images.githubusercontent.com/272303/33506010-d2037fc2-d713-11e7-849e-da9c5702a755.png)




Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -59,9 +59,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '0.5-rc2'
+version = '0.5-rc4'
 # The full version, including alpha/beta/rc tags.
-release = '0.5-rc2'
+release = '0.5-rc4'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
diff --git a/securedrop/source_app/main.py b/securedrop/source_app/main.py
--- a/securedrop/source_app/main.py
+++ b/securedrop/source_app/main.py
@@ -227,8 +227,11 @@ def logout():
         if logged_in():
             msg = render_template('logout_flashed_message.html')
 
-            # clear the session after we render the message so it's localized
+            # Clear the session after we render the message so it's localized
+            # If a user specified a locale, save it and restore it
+            user_locale = g.locale
             session.clear()
+            session['locale'] = user_locale
 
             flash(Markup(msg), "important hide-if-not-tor-browser")
         return redirect(url_for('.index'))
diff --git a/securedrop/version.py b/securedrop/version.py
--- a/securedrop/version.py
+++ b/securedrop/version.py
@@ -1 +1 @@
-__version__ = '0.5-rc2'
+__version__ = '0.5-rc4'


Test patch:
diff --git a/molecule/builder/tests/vars.yml b/molecule/builder/tests/vars.yml
--- a/molecule/builder/tests/vars.yml
+++ b/molecule/builder/tests/vars.yml
@@ -1,5 +1,5 @@
 ---
-securedrop_version: "0.5-rc2"
+securedrop_version: "0.5-rc4"
 ossec_version: "2.8.2"
 keyring_version: "0.1.1"
 
diff --git a/securedrop/tests/test_source.py b/securedrop/tests/test_source.py
--- a/securedrop/tests/test_source.py
+++ b/securedrop/tests/test_source.py
@@ -194,7 +194,9 @@ def test_login_and_logout(self):
 
             # sessions always have 'expires', so pop it for the next check
             session.pop('expires', None)
-            self.assertTrue(not session)
+
+            self.assertNotIn('logged_in', session)
+            self.assertNotIn('codename', session)
 
             self.assertIn('Thank you for exiting your session!', resp.data)
Repository: freedomofpress/securedrop

Issue:
Tor2Web warning does not display well
# Bug

## Description

The &lt;strong> tags are incorrectly set.

## Steps to Reproduce

firefox https://zdf4nikyuswdzbt6.onion.to/

## Expected Behavior

Spaces between words

## Actual Behavior

Missing spaces and weird bold / regular changes.

![a](https://user-images.githubusercontent.com/433594/33742345-3c1b4d20-dba8-11e7-8e37-1489cc69cffb.png)




Discussion:
The non-displaying image is apparently missing for you, it's the black exclamation triangle /static/i/font-awesome/exclamation-triangle-black.png. Easy to test/see if you send an extra header: X-tor2web, which also makes it painful to see other proxies that don't give you that header and where this warning isn't displayed. 

Resolving patch:
diff --git a/securedrop/source_app/__init__.py b/securedrop/source_app/__init__.py
--- a/securedrop/source_app/__init__.py
+++ b/securedrop/source_app/__init__.py
@@ -71,8 +71,10 @@ def check_tor2web():
         # about Tor2Web, corresponding to the initial page load.
         if 'X-tor2web' in request.headers:
             flash(Markup(gettext(
-                '<strong>WARNING:</strong> You appear to be using Tor2Web. '
-                'This <strong>does not</strong> provide anonymity. '
+                '<strong>WARNING:&nbsp;</strong> '
+                'You appear to be using Tor2Web. '
+                'This <strong>&nbsp;does not&nbsp;</strong> '
+                'provide anonymity. '
                 '<a href="{url}">Why is this dangerous?</a>')
                 .format(url=url_for('info.tor2web_warning'))),
                   "banner-warning")


Test patch:
Repository: freedomofpress/securedrop

Issue:
async_genkey: thread safety issue in async_genkey by calling current_app in separate thread.
# Bug
async_genkey calls current_app in some conditions and can raise `RuntimeError: Working outside of application context.`

## Description

async_genkey calls current_app in some conditions and can raise `RuntimeError: Working outside of application context.` Example: https://gist.github.com/tmc/095d490608c7cc5e2aa5f4825cd7a4a3#file-gistfile1-txt-L1122

## Steps to Reproduce

This is a flake I have running tests inside of a container in a qubes 3.2 machine.

## Expected Behavior
To not break in case of an underlying error.

## Actual Behavior
Breaks when trying to call current_app.

## Comments
There's clearly another underlying issue here but it shouldn't blow up in this manner.



Discussion:

Resolving patch:
diff --git a/securedrop/source_app/utils.py b/securedrop/source_app/utils.py
--- a/securedrop/source_app/utils.py
+++ b/securedrop/source_app/utils.py
@@ -1,3 +1,4 @@
+import logging
 import subprocess
 
 from datetime import datetime
@@ -78,7 +79,7 @@ def async_genkey(filesystem_id, codename):
         source.last_updated = datetime.utcnow()
         db_session.commit()
     except Exception as e:
-        current_app.logger.error(
+        logging.getLogger(__name__).error(
                 "async_genkey for source (filesystem_id={}): {}"
                 .format(filesystem_id, e))
 


Test patch:
Repository: freedomofpress/securedrop

Issue:
Prevent admin journalists from deleting themselves
To prevent lockouts and other forms of usability degradation, I propose one of two things:
1) a full ban on any admin from deleting themselves
2) an admin may not delete themselves if they are the last remaining admin


Discussion:
There's still SSH though? Just not convinced this is really a problem that needs solving.
Agreed with @fowlslegs. I've actually used the GUI to delete the last remaining Admin intentionally during on-sites, due to staffing changes, then recreated via the CLI. 
Fair enough. It just seemed like a silly thing to be able to do.
Closing as per Conor and I's comments.

Resolving patch:
diff --git a/securedrop/journalist_app/admin.py b/securedrop/journalist_app/admin.py
--- a/securedrop/journalist_app/admin.py
+++ b/securedrop/journalist_app/admin.py
@@ -208,7 +208,13 @@ def set_password(user_id):
     @admin_required
     def delete_user(user_id):
         user = Journalist.query.get(user_id)
-        if user:
+        if user_id == g.user.id:
+            # Do not flash because the interface already has safe guards.
+            # It can only happen by manually crafting a POST request
+            current_app.logger.error(
+                "Admin {} tried to delete itself".format(g.user.username))
+            abort(403)
+        elif user:
             db_session.delete(user)
             db_session.commit()
             flash(gettext("Deleted user '{user}'").format(


Test patch:
diff --git a/securedrop/tests/test_journalist.py b/securedrop/tests/test_journalist.py
--- a/securedrop/tests/test_journalist.py
+++ b/securedrop/tests/test_journalist.py
@@ -270,6 +270,34 @@ def test_admin_delete_user(self):
         # Verify journalist is no longer in the database
         self.assertEqual(Journalist.query.get(self.user.id), None)
 
+    def test_admin_cannot_delete_self(self):
+        # Verify journalist is in the database
+        self.assertNotEqual(Journalist.query.get(self.user.id), None)
+
+        self._login_admin()
+        resp = self.client.post(url_for('admin.delete_user',
+                                        user_id=self.admin.id),
+                                follow_redirects=True)
+
+        # Assert correct interface behavior
+        self.assert403(resp)
+
+        resp = self.client.get(url_for('admin.index'))
+        self.assert200(resp)
+        self.assertIn("Admin Interface", resp.data)
+        # The user can be edited and deleted
+        self.assertIn(escape("Edit user {}".format(self.user.username)),
+                      resp.data)
+        self.assertIn(
+            escape("Delete user {}".format(self.user.username)),
+            resp.data)
+        # The admin can be edited but cannot deleted
+        self.assertIn(escape("Edit user {}".format(self.admin.username)),
+                      resp.data)
+        self.assertNotIn(
+            escape("Delete user {}".format(self.admin.username)),
+            resp.data)
+
     def test_admin_deletes_invalid_user_404(self):
         self._login_admin()
         invalid_user_pk = max([user.id for user in Journalist.query.all()]) + 1
Repository: freedomofpress/securedrop

Issue:
Enable vacuum and secure delete on sqlite
It would be valuable to enable secure delete and auto vacuum on the database when available.

Even if the database of securedrop holds very minimal metadata information this would reduce leakage of some metadata about the names of files and the number of submissios or past users enabled on the platform.

For sqlite this could be achieved by issuing the following queries:
```sql
PRAGMA secure_delete = ON
PRAGMA auto_vacuum = FULL
```

the two can be issued at the time of database creation and would garantee that entries are securely deleted from the database and not just marked as free.

In SQLAlchemy this can be achieved with:
```python
    engine.execute('PRAGMA secure_delete = ON')
    engine.execute('PRAGMA auto_vacuum = FULL')
```

a similar result could be probably achieved on other databases

references:
- https://www.sqlite.org/pragma.html#pragma_secure_delete
- https://www.sqlite.org/pragma.html#pragma_auto_vacuum


Discussion:

Resolving patch:
diff --git a/securedrop/db.py b/securedrop/db.py
--- a/securedrop/db.py
+++ b/securedrop/db.py
@@ -41,6 +41,9 @@
         config.DATABASE_ENGINE + ":///" +
         config.DATABASE_FILE
     )
+
+    engine.execute('PRAGMA secure_delete = ON')
+    engine.execute('PRAGMA auto_vacuum = FULL')
 else:  # pragma: no cover
     engine = create_engine(
         config.DATABASE_ENGINE + '://' +


Test patch:
Repository: freedomofpress/securedrop

Issue:
PyCrypto is no longer being actively maintained
As first noted in https://github.com/freedomofpress/securedrop/issues/204#issuecomment-272338348:

> So it's come to our attention that we need to replace PyCrypto, the progress of which is pretty much stalled (e.g., [this year-old vulnerability](https://github.com/dlitz/pycrypto/issues/173) has still not been fixed in a release, even though it's patched in master--note this one does not effect SD).... Ideally, we'd use https://github.com/pyca/pynacl, which bundles `libsodium`, as a replacement...

While as noted there are no known PyCrypto vulnerabilities that affect SD, we still should still have it in our timeline for the coming months to look into replacements.


Discussion:
We might want to add https://github.com/pyca/cryptography as well, depending on the functionality that's being replaced.
Both would work, but libsodium offers a lot more functionality that we might not use at first, but will be available should we choose to use it later.
Hmm, `pycryptodome` is an actively maintained drop-in replacement for the aging `PyCrypto` library, this is probably the best candidate for a quick replacement: https://github.com/Legrandin/pycryptodome

Resolving patch:
diff --git a/securedrop/crypto_util.py b/securedrop/crypto_util.py
--- a/securedrop/crypto_util.py
+++ b/securedrop/crypto_util.py
@@ -5,7 +5,7 @@
 import os
 import subprocess
 
-from Crypto.Random import random
+from Cryptodome.Random import random
 import gnupg
 from gnupg._util import _is_stream, _make_binary_stream
 import scrypt
diff --git a/securedrop/secure_tempfile.py b/securedrop/secure_tempfile.py
--- a/securedrop/secure_tempfile.py
+++ b/securedrop/secure_tempfile.py
@@ -4,9 +4,9 @@
 from tempfile import _TemporaryFileWrapper
 
 from gnupg._util import _STREAMLIKE_TYPES
-from Crypto.Cipher import AES
-from Crypto.Random import random
-from Crypto.Util import Counter
+from Cryptodome.Cipher import AES
+from Cryptodome.Random import random
+from Cryptodome.Util import Counter
 
 
 class SecureTemporaryFile(_TemporaryFileWrapper, object):


Test patch:
diff --git a/securedrop/tests/functional/functional_test.py b/securedrop/tests/functional/functional_test.py
--- a/securedrop/tests/functional/functional_test.py
+++ b/securedrop/tests/functional/functional_test.py
@@ -12,7 +12,7 @@
 import traceback
 import requests
 
-from Crypto import Random
+from Cryptodome import Random
 from selenium import webdriver
 from selenium.common.exceptions import (WebDriverException,
                                         NoAlertPresentException)
Repository: freedomofpress/securedrop

Issue:
Bump Ansible to 2.4
## Description

The current version of Ansible in the admin workstation uses PyCrypto==2.6.1 as a dependency, which is causing CI safety failures when checking the admin pip requirements due to the fact that there is a CVE in PyCrypto 2.6.1. See upstream discussion in https://github.com/ansible/ansible/issues/23179. 

We should bump to a more recent version of Ansible in the admin workstations that does not have PyCrypto as a dependency

## User Stories

As a SecureDrop administrator, I don't want to run software relying on unmaintained dependencies.
Temporarily disable safety check
## Description

We'll need to temporarily disable safety in order to merge until #2926 is resolved (and we'll need to cherry pick the disabling of safety into the 0.5.2 release branch). 

## User Stories

As a SecureDrop maintainer, I don't want to merge with failing CI. 


Discussion:

Resolving patch:
diff --git a/install_files/ansible-base/callback_plugins/ansible_version_check.py b/install_files/ansible-base/callback_plugins/ansible_version_check.py
--- a/install_files/ansible-base/callback_plugins/ansible_version_check.py
+++ b/install_files/ansible-base/callback_plugins/ansible_version_check.py
@@ -19,7 +19,7 @@ def print_red_bold(text):
 class CallbackModule(CallbackBase):
     def __init__(self):
         # Can't use `on_X` because this isn't forwards compatible with Ansible 2.0+
-        required_version = '2.3.2'  # Keep synchronized with group_vars/all/main.yml
+        required_version = '2.4.2'  # Keep synchronized with requirements files
         if not ansible.__version__.startswith(required_version):
             print_red_bold(
                 "SecureDrop restriction: only Ansible {version}.* is supported. "


Test patch: