Repository: freedomofpress/securedrop

Issue:
Use upstream version of python-gnupg
See https://github.com/freedomofpress/securedrop/issues/250#issuecomment-86237456 for context. We have worked with the upstream maintainer to fix the issues that were preventing us from using the upstream version with SecureDrop. Now that they are fixed, we should start using the upstream and discontinue our use of the outdated `gnupg-securedrop` [fork](https://pypi.python.org/pypi/gnupg-securedrop/1.2.5-9-g6f9d63a-dirty).



Discussion:
Please let me know if there's anything else I can do to make this easier on you guys.

Also consider removing GnuPG, moving to a pure python PGP implementation, also as a way to improve the sandboxing profiles (removing the "/bin/sh" required by gnupg in apparmor) https://github.com/globaleaks/GlobaLeaks/issues/860 .
We are slowly considering OpenPGP-Python https://github.com/singpolyma/OpenPGP-Python and PGPy  https://github.com/SecurityInnovation/PGPy

yep, i think in the end it would be useful to agree on a common solution in order to jointly support the develpement/completion of one of the existing pure python PGP libraries and ask shared funder to fund the audit of the library.

What's the status on this @garrettr? I think for 0.3.7 it would be good to move to the latest upstream stable (2.0.2) from Isis. I like @fpietrosanti's suggestion of using https://github.com/SecurityInnovation/PGPy, but it has many problems still (e.g., https://github.com/SecurityInnovation/PGPy/issues/139), and I don't believe it is stable enough or has been given the audit that we should require for such an important component of the system. Maybe open a separate ticket "move to pure python openpgp" ticket for the matter, but for now just go back to tracking stable python-gnupg? X-ref #1301.

@fowlslegs I think it's fine to target this for 0.3.7 as a subtask of #1301. You will need to thoroughly test the new library to make sure it is working as expected. The automated test suite (`./manage.py test`) will do a lot for you, but I would do some manual testing as well just to make sure there are no surprises. 

> Maybe open a separate ticket "move to pure python openpgp" ticket for the matter, but for now just go back to tracking stable python-gnupg?

I wouldn't bother. At the inception of the project, the motivations to use GPG were:
1. It was the gold standard for open source encryption software, and using it was preferable to "rolling our own" crypto.
2. It was the best built-in mechanism for "double-click decrypting" (decrypting via an easy-to-use GUI) files on a Tails-based Secure Viewing Station.

My, how time flies! Since then, there's been an explosion of work in open source crypto libraries and frameworks. Today, I would probably use an authenticated encryption (AE) primitive based on a widely used and well-exercised library such as NaCL. By comparison, GPG uses outdated cryptographic primitives, is slow, more prone to vulnerabilities (to take just one recent example, side channels that lead to key recovery in libgcrypt), full of cruft, and has a complex deployment story. Thus, I consider motivation 1 to be obsolete.

We've been internally discussing reworking the journalist workflow, including developing a client for journalists to work with submissions. This makes motivation 2 for using GPG obsolete: if we control the journalist's decryption client, we can do whatever we want for encryption and decryption, and aren't forced to use GPG just because it's built-in on Tails.

Finally, GPG's inconsistent interfaces and changes in behavior across versions make it a nightmare to work with programmatically. For context, see any of the gnupg-related issues on SecureDrop, python-gnupg's issue tracker, python-gnupg's source code, or [Mailpile's blog post](https://www.mailpile.is/blog/2014-10-07_Some_Thoughts_on_GnuPG.html) and [follow-up post](https://www.mailpile.is/blog/2015-02-26_Revisiting_the_GnuPG_discussion.html). We have already wasted a ton of time on this project with annoying issues caused by GPG. Now that the reasons for using it are obsolete, I seriously believe we should consider replacing it entirely.

Woops.

I agree regarding ditching PGP before switching to a pure Python implementation. For the immediate future 0.3.7 (and future releases until a re-architecture), we should track upstream stable versions. I'll get on testing that.


Resolving patch:
diff --git a/securedrop/crypto_util.py b/securedrop/crypto_util.py
--- a/securedrop/crypto_util.py
+++ b/securedrop/crypto_util.py
@@ -154,15 +154,11 @@ def encrypt(plaintext, fingerprints, output=None):
         fingerprints = [fingerprints, ]
     fingerprints = [fpr.replace(' ', '') for fpr in fingerprints]
 
-    if isinstance(plaintext, unicode):
-        plaintext = plaintext.encode('utf8')
-
-    encrypt_fn = gpg.encrypt if isinstance(plaintext, str) else gpg.encrypt_file
-    out = encrypt_fn(plaintext,
-                     *fingerprints,
-                     output=output,
-                     always_trust=True,
-                     armor=False)
+    out = gpg.encrypt(plaintext,
+                      *fingerprints,
+                      output=output,
+                      always_trust=True,
+                      armor=False)
     if out.ok:
         return out.data
     else:
diff --git a/securedrop/secure_tempfile.py b/securedrop/secure_tempfile.py
--- a/securedrop/secure_tempfile.py
+++ b/securedrop/secure_tempfile.py
@@ -6,6 +6,7 @@
 from Crypto.Random import random
 from Crypto.Util import Counter
 
+from gnupg._util import _STREAMLIKE_TYPES
 
 class SecureTemporaryFile(_TemporaryFileWrapper):
     """Temporary file that is ephemerally encrypted on the fly.
@@ -77,3 +78,8 @@ def read(self, count=None):
 
     def close(self):
         return _TemporaryFileWrapper.close(self)
+
+# python-gnupg will not recognize our SecureTemporaryFile as a stream-like type
+# and will attempt to call encode on it, thinking it's a string-like type. To
+# avoid this we add it the list of stream-like types.
+_STREAMLIKE_TYPES.append(_TemporaryFileWrapper)


Test patch: