Repository: freedomofpress/securedrop

Issue:
Consider using scrypt instead of bcrypt
scrypt is another adaptive key stretching algorithm in the vein of bcrypt. Unlike bcrypt, however, it was specifically designed to thwart attacks by powerful adversaries capable of building large clusters of custom hardware for password cracking. Given securedrop's threat model, scrypt might a better choice than bcrypt; however, scrypt is significantly newer and comparatively less tested.



Discussion:
At globaleaks we're using too scrypt by following this specific approach, described in our application security design and details: https://docs.google.com/a/apps.globaleaks.org/document/d/1SMSiAry7x5XY9nY8GAejJD75NWg7bp7M1PwXSiwy62U/pub#h.ibk1v235g7wb

The reason I think scrypt should be eventually implemented is because of our threat model. When bcrypt was designed back in 1999 its threat was application specific hardware bruteforcing arrays. However we need to be thinking about FPGA arrays and cost to the adversary. scrypt has the potential to pretty much force an adversary to have to use FPGAs, and via the memory hardening settings you can also increase the cost factor ( for example of ram ) from a few million dollars to crack a codename in say one year, to the potential of billions of dollars.

@taipo, do you have evidence for your claims regarding the use of FPGA arrays for cracking bcrypt?

I derived my opinion from reading this 'STRONGER KEY DERIVATION VIA SEQUENTIAL
MEMORY-HARD FUNCTIONS' 
From: https://www.tarsnap.com/scrypt/scrypt.pdf

Also see "Table 1. Estimated cost of hardware to crack a password in 1 year."

I am not saying that there is anything specific about scrypt that means an adversary has to use FPGAs, or that FPGA's could increase the effectiveness of determined attacks even further than ASIC's, I am looking at which method gives us the potential to make bruteforcing attempts the most expensive.

Also:
- http://www.openwall.com/presentations/Passwords12-The-Future-Of-Hashing/

Future dev of 'supercomputers':
- "James Bamford's book The Shadow Factory reported that NSA told the Pentagon it would need an exaflop computer by 2018." - https://en.wikipedia.org/wiki/Shadow_Factory p339
- http://www.infoworld.com/t/networking/ibm-breaks-petaflop-barrier-263
- http://www.defencenews.in/defence-news-internal.asp?get=new&id=500
- http://www.h-online.com/newsticker/news/item/IDF-Intel-says-Moore-s-Law-holds-until-2029-734779.html

One of the difficulties is finding working benchmarks for attacking scrypt key hashes. For now I can only go by information mostly authored by Colin Percival.

Summary of points where scrypt is said to get one over bcrypt ( plus some extras ):
- scrypt, via its memory cost and parallelization settings, can be configured to extend the cost to the attacker, considerably past the highest practical bcrypt cost setting without incurring memory and cpu costs on the host webserver.
- although its hard to find info on this, it appears that there is a limited password byte length for bcrypt. "Finally, the key argument is a secret encryption key, which can be a user-chosen password of up to 56 bytes (including a terminating zero byte when the key is an ASCII string)." 
  from: https://www.usenix.org/legacy/events/usenix99/provos/provos_html/node4.html
  see also: http://security.stackexchange.com/questions/39849/does-bcrypt-have-a-maximum-password-length
- Opinion: these organisations: NSA, GCHQ, CSE of Canada and the Chinese Governments, do not publish the amount of money they have invested into building custom password-cracking units, so while 'more' might be insane in some thinking, more is better than not enough because we are largely left making guesses by comparing what is the best of the best being developed and in play currently. So it looks like we will be contending with exaflop rigs ( see links above ) in about 5 years time.
- according to Percival, scrypt is 2⁵ times more expensive to attack than bcrypt
  from: https://www.tarsnap.com/scrypt/scrypt-slides.pdf slide 19
  So in my thinking, if those other two cost settings cannot be bypassed, then that makes scrypt the number 1 contender method for some future version of SecureDrop.

These are all great points and resources, thanks @Taipo! It should be pretty easy for someone to implement this using [this Python library](https://pypi.python.org/pypi/scrypt/).


Resolving patch:
diff --git a/securedrop/crypto_util.py b/securedrop/crypto_util.py
--- a/securedrop/crypto_util.py
+++ b/securedrop/crypto_util.py
@@ -1,15 +1,16 @@
 # -*- coding: utf-8 -*-
 import os
-import bcrypt
 import subprocess
-import threading
+from base64 import b32encode
+import re
+
 from Crypto.Random import random
 import random as badrandom
 import gnupg
+import scrypt
+
 import config
 import store
-from base64 import b32encode
-import re
 
 # to fix gpg error #78 on production
 os.environ['USERNAME'] = 'www-data'
@@ -19,23 +20,20 @@
     # Optiimize crypto to speed up tests (at the expense of security - DO NOT
     # use these settings in production)
     GPG_KEY_LENGTH = "1024"
-    BCRYPT_ID_SALT = bcrypt.gensalt(log_rounds=0)
-    BCRYPT_GPG_SALT = bcrypt.gensalt(log_rounds=0)
+    SCRYPT_PARAMS = dict(N=2**1, r=1, p=1)
 else:
     GPG_KEY_LENGTH = "4096"
-    BCRYPT_ID_SALT = config.BCRYPT_ID_SALT
-    BCRYPT_GPG_SALT = config.BCRYPT_GPG_SALT
-
-DEFAULT_WORDS_IN_RANDOM_ID = 8
+    SCRYPT_PARAMS = config.SCRYPT_PARAMS
 
+SCRYPT_ID_PEPPER = config.SCRYPT_ID_PEPPER
+SCRYPT_GPG_PEPPER = config.SCRYPT_GPG_PEPPER
 
-class CryptoException(Exception):
-    pass
+DEFAULT_WORDS_IN_RANDOM_ID = 8
 
 # Make sure these pass before the app can run
 # TODO: Add more tests
 def do_runtime_tests():
-    assert(config.BCRYPT_ID_SALT != config.BCRYPT_GPG_SALT)
+    assert(config.SCRYPT_ID_PEPPER != config.SCRYPT_GPG_PEPPER)
     # crash if we don't have srm:
     try:
         subprocess.check_call(['srm'], stdout=subprocess.PIPE)
@@ -44,6 +42,25 @@ def do_runtime_tests():
 
 do_runtime_tests()
 
+GPG_BINARY = 'gpg2'
+try:
+    p = subprocess.Popen([GPG_BINARY, '--version'], stdout=subprocess.PIPE)
+except OSError:
+    GPG_BINARY = 'gpg'
+    p = subprocess.Popen([GPG_BINARY, '--version'], stdout=subprocess.PIPE)
+
+assert p.stdout.readline().split()[
+    -1].split('.')[0] == '2', "upgrade GPG to 2.0"
+gpg = gnupg.GPG(gpgbinary=GPG_BINARY, gnupghome=config.GPG_KEY_DIR)
+
+words = file(config.WORD_LIST).read().split('\n')
+nouns = file(config.NOUNS).read().split('\n')
+adjectives = file(config.ADJECTIVES).read().split('\n')
+
+
+class CryptoException(Exception):
+    pass
+
 
 def clean(s, also=''):
     """
@@ -54,18 +71,15 @@ def clean(s, also=''):
     >>> clean("Helloworld")
     'Helloworld'
     """
-    # safe characters for every possible word in the wordlist
-    # includes capital letters because bcrypt hashes are base32-encoded with
-    # capital letters
-    ok = '!#%$&)(+*-1032547698;:=?@acbedgfihkjmlonqpsrutwvyxzABCDEFGHIJKLMNOPQRSTUVWXYZ'
+    # safe characters for every possible word in the wordlist includes capital
+    # letters because codename hashes are base32-encoded with capital letters
+    ok = ' !#%$&)(+*-1032547698;:=?@acbedgfihkjmlonqpsrutwvyxzABCDEFGHIJKLMNOPQRSTUVWXYZ'
     for c in s:
         if c not in ok and c not in also:
             raise CryptoException("invalid input: %s" % s)
-    return s
-
-words = file(config.WORD_LIST).read().split('\n')
-nouns = file(config.NOUNS).read().split('\n')
-adjectives = file(config.ADJECTIVES).read().split('\n')
+    # scrypt.hash requires input of type str. Since the wordlist is all ASCII
+    # characters, this conversion is not problematic
+    return str(s)
 
 
 def genrandomid(words_in_random_id=DEFAULT_WORDS_IN_RANDOM_ID):
@@ -78,35 +92,24 @@ def displayid(n):
     return badrandom_value.choice(adjectives) + " " + badrandom_value.choice(nouns)
 
 
-
-def shash(s, salt=BCRYPT_ID_SALT):
+def hash_codename(codename, salt=SCRYPT_ID_PEPPER):
     """
-    >>> shash('Hello, world!')
+    >>> hash_codename('Hello, world!')
     'EQZGCJBRGISGOTC2NZVWG6LILJBHEV3CINNEWSCLLFTUWZLFHBTS6WLCHFHTOLRSGQXUQLRQHFMXKOKKOQ4WQ6SXGZXDAS3Z'
     """
-    return b32encode(bcrypt.hashpw(s, salt))
-
-GPG_BINARY = 'gpg2'
-try:
-    p = subprocess.Popen([GPG_BINARY, '--version'], stdout=subprocess.PIPE)
-except OSError:
-    GPG_BINARY = 'gpg'
-    p = subprocess.Popen([GPG_BINARY, '--version'], stdout=subprocess.PIPE)
-
-assert p.stdout.readline().split()[
-    -1].split('.')[0] == '2', "upgrade GPG to 2.0"
-gpg = gnupg.GPG(gpgbinary=GPG_BINARY, gnupghome=config.GPG_KEY_DIR)
+    return b32encode(scrypt.hash(clean(codename), salt, **SCRYPT_PARAMS))
 
 
 def genkeypair(name, secret):
     """
-    >>> if not gpg.list_keys(shash('randomid')):
-    ...     genkeypair(shash('randomid'), 'randomid').type
+    >>> if not gpg.list_keys(hash_codename('randomid')):
+    ...     genkeypair(hash_codename('randomid'), 'randomid').type
     ... else:
     ...     u'P'
     u'P'
     """
-    name, secret = clean(name), shash(clean(secret, ' '), salt=BCRYPT_GPG_SALT)
+    name = clean(name)
+    secret = hash_codename(secret, salt=SCRYPT_GPG_PEPPER)
     return gpg.gen_key(gpg.gen_key_input(
         key_type=GPG_KEY_TYPE, key_length=GPG_KEY_LENGTH,
         passphrase=secret,
@@ -138,10 +141,6 @@ def get_key_by_fingerprint(fingerprint):
     return matches[0] if matches else None
 
 
-def _shquote(s):
-    return "\\'".join("'" + p + "'" for p in s.split("'"))
-
-
 def encrypt(fp, s, output=None):
     r"""
     >>> key = genkeypair('randomid', 'randomid')
@@ -171,7 +170,7 @@ def decrypt(name, secret, s):
     ... )
     'Goodbye, cruel world!'
     """
-    secret = shash(secret, salt=BCRYPT_GPG_SALT)
+    secret = hash_codename(secret, salt=SCRYPT_GPG_PEPPER)
     return gpg.decrypt(s, passphrase=secret).data
 
 
diff --git a/securedrop/example_config.py b/securedrop/example_config.py
--- a/securedrop/example_config.py
+++ b/securedrop/example_config.py
@@ -8,8 +8,9 @@
 NOUNS='./dictionaries/nouns.txt'
 ADJECTIVES='./dictionaries/adjectives.txt'
 JOURNALIST_KEY='' # fingerprint of the public key for encrypting submissions
-BCRYPT_ID_SALT=''    # bcrypt.gensalt(); for constructing public ID from source codename
-BCRYPT_GPG_SALT=''   # bcrypt.gensalt(); for stretching source codename into GPG passphrase
+SCRYPT_ID_PEPPER=''    # os.urandom(32); for constructing public ID from source codename
+SCRYPT_GPG_PEPPER=''   # os.urandom(32); for stretching source codename into GPG passphrase
+SCRYPT_PARAMS = dict(N=2**14, r=8, p=1)
 
 ### Theming Options
 
diff --git a/securedrop/source.py b/securedrop/source.py
--- a/securedrop/source.py
+++ b/securedrop/source.py
@@ -62,15 +62,15 @@ def decorated_function(*args, **kwargs):
 @ignore_static
 def setup_g():
     """Store commonly used values in Flask's special g object"""
-    # ignore_static here because `crypto_util.shash` is bcrypt (very time consuming),
-    # and we don't need to waste time running if we're just serving a static
-    # resource that won't need to access these common values.
+    # ignore_static here because `crypto_util.hash_codename` is scrypt (very
+    # time consuming), and we don't need to waste time running if we're just
+    # serving a static resource that won't need to access these common values.
     if logged_in():
         # We use session.get (which defaults to None if 'flagged' is not in the
         # session) to avoid a KeyError on the redirect from login/ to lookup/
         g.flagged = session.get('flagged')
         g.codename = session['codename']
-        g.sid = crypto_util.shash(g.codename)
+        g.sid = crypto_util.hash_codename(g.codename)
         g.loc = store.path(g.sid)
 
 
@@ -118,7 +118,7 @@ def generate():
 
 @app.route('/create', methods=['POST'])
 def create():
-    sid = crypto_util.shash(session['codename'])
+    sid = crypto_util.hash_codename(session['codename'])
     if os.path.exists(store.path(sid)):
         # if this happens, we're not using very secure crypto
         log.warning("Got a duplicate ID '%s'" % sid)
@@ -203,7 +203,7 @@ def delete():
 
 
 def valid_codename(codename):
-    return os.path.exists(store.path(crypto_util.shash(codename)))
+    return os.path.exists(store.path(crypto_util.hash_codename(codename)))
 
 
 @app.route('/login', methods=('GET', 'POST'))


Test patch:
diff --git a/securedrop/test.py b/securedrop/test.py
--- a/securedrop/test.py
+++ b/securedrop/test.py
@@ -26,7 +26,7 @@
 
 
 def _block_on_reply_keypair_gen(codename):
-    sid = crypto_util.shash(codename)
+    sid = crypto_util.hash_codename(codename)
     while not crypto_util.getkey(sid):
         sleep(0.1)