Repository: freedomofpress/securedrop

Issue:
Can we get rid of "flag this source for reply"?
I can't seem to find the original issue now, but my understanding is that the reason gpg keys don't get automatically generated for sources is to prevent an entropy depletion attack that works by attackers creating tons of sources, and therefore forcing the server to generate tons of gpg keys.

In order to prevent that attack, sources now don't get gpg keys generated until the journalist logs in and flags the source for reply. The next time the source logs in the gpg key gets generated.

I think that the fact that the source needs to login twice before they can get a response is pretty bad. It's also not even clear what's going on. When the source logs in it just says that the journalist wants to reply, but doesn't include a reply yet.

What about this instead?

SecureDrop shouldn't try to generate a gpg key for a source until they have submitted something. If they just create a codename and don't do anything else, it shouldn't try to create a key for them.

When a source does submit something, instead of generating a key immediately it should put their codename into a gpg key generation queue, perhaps that's always running in a separate thread. That thread can constantly be doing something like this:
- Pop a codename off the queue
- Is the entropy pool full enough to generate a key? If not, wait for it to fill.
- Generate a key for that codename, and erase the codename from the queue.
- Wait 15 seconds or so.
- Loop.

This way keys will automatically get generated in the background without ever depleting the entropy pool, but there's no extra annoying step that might get in the way of communicating with sources.

Additionally, the journalist interface should show the current status of the gpg generation queue somewhere -- maybe where the "flag this source for reply" button currently is, it can say how many gpg keys are in front of it in the queue for generation.

Can we get rid of "flag this source for reply"?
I can't seem to find the original issue now, but my understanding is that the reason gpg keys don't get automatically generated for sources is to prevent an entropy depletion attack that works by attackers creating tons of sources, and therefore forcing the server to generate tons of gpg keys.

In order to prevent that attack, sources now don't get gpg keys generated until the journalist logs in and flags the source for reply. The next time the source logs in the gpg key gets generated.

I think that the fact that the source needs to login twice before they can get a response is pretty bad. It's also not even clear what's going on. When the source logs in it just says that the journalist wants to reply, but doesn't include a reply yet.

What about this instead?

SecureDrop shouldn't try to generate a gpg key for a source until they have submitted something. If they just create a codename and don't do anything else, it shouldn't try to create a key for them.

When a source does submit something, instead of generating a key immediately it should put their codename into a gpg key generation queue, perhaps that's always running in a separate thread. That thread can constantly be doing something like this:
- Pop a codename off the queue
- Is the entropy pool full enough to generate a key? If not, wait for it to fill.
- Generate a key for that codename, and erase the codename from the queue.
- Wait 15 seconds or so.
- Loop.

This way keys will automatically get generated in the background without ever depleting the entropy pool, but there's no extra annoying step that might get in the way of communicating with sources.

Additionally, the journalist interface should show the current status of the gpg generation queue somewhere -- maybe where the "flag this source for reply" button currently is, it can say how many gpg keys are in front of it in the queue for generation.



Discussion:
How does the queue idea solve the original DoS vector? The same scenario basically happens naturally, since GPG key generation blocks on entropy. The only difference is you'd been blocking one key at a time, which is not better from the perspective of avoiding DoS because legitimate sources still have to wait an uncertain and potentially ridiculous amount of time (`(len(keygen)+15)*(# spam DoS accounts)`) for reply keys to be generated.

Without a hardware RNG, I also think an attacker could easily keep us in a global minima for entropy, which in turn would affect the performance of the whole system because it uses so much crypto.

Finally, I'm not sure how to determine when the pool is "full enough" to generate a GPG key. I don't think it is simple to compute, but would have to read the GPG source to know.

Do you know much about hardware RNG, or USB entropy creating devices? I think it's worth investing in them to make source communication more usable.

Even without better methods of collecting entropy, I think that the flag option should be configurable. As far as I know no one has actually got entropy-DOSed before -- this is just theoretical.

How about this. We build a test for an entropy DOS -- even something as simple as 4 new PGP keys created in the same minute. If that happens, ossec can send an email alert and it can switch to forcing journalists to flag for reply for the next 24 hours before the restriction gets lifted again.

It seems unreasable to make SD way less usable to defend against attacks that aren't happening, when it only needs to be unusable during an attack.

Would a journalist be able to buy a trustworthy hardware RNG? For many newsrooms it should be enough, but the NSA has targeted random number algorithms and online-ordered products in the past. It sounds paranoid, but the NY Times bought one of their laptops in a store to assure that it wouldn't be specially targeted for them. It's not as easy to find specialized hardware without custom-ordering.

SparkFun has an interesting tutorial where someone [made a random number generator](https://www.sparkfun.com/tutorials/132) by themself.

> Do you know much about hardware RNG, or USB entropy creating devices? I think it's worth investing in them to make source communication more usable.

I know that they are expensive. It is also hard to make specific recommendations of trustworthy hardware, especially post-Snowden.

> How about this. We build a test for an entropy DOS -- even something as simple as 4 new PGP keys created in the same minute. If that happens, ossec can send an email alert and it can switch to forcing journalists to flag for reply for the next 24 hours before the restriction gets lifted again.

Sure, that seems like it would be a natural thing to include as part of our general plan for DoS mitigation. Some good work was done on this at the last hackathon, but no pull requests. I'll follow up with the people who were working on it.

@msjoinder As you point out, it is hard for us to know whether a hardware RNG is trustworthy. I don't know how we would determine that with a high enough degree of confidence to make an official recommendation. @dolanjs any thoughts on this?

How about this. We keep track of the timestamp when the last key was generated (probably in the database, but we'd need to make a new table), called last_keygen.

When a source that doesn't already have a key submits a message or file:
- If last_keygen < 20 minutes ago:
  - Update last_keygen to now
  - Generate a new key for the source

This way we can have up to one new key generated every 20 minutes without a problem. If new sources submit more frequently than this (which could be a DOS attack), then it reverts to the annoying "flag to reply" behavior, but otherwise keys get generated immediately.

What do you think? If this looks like a good solution I can start working on it.

And as far as where to store last_keygen, we could just store a timestamp in the file $securedrop_root/last_keygen and update that file every time a new key is generated. But it might make sense to store this in the db instead. We could make a new options table that has columns "key" and "value", to store settings like this.

> If last_keygen < 20 minutes ago: 

Should be `> 20`?

Since this can be DOS'ed by someone who submits a bogus file every 20 minutes, why not just generate keys as fast as possible and revert to 'flag this source' when entropy is low?

Oops, timestamp math is confusing.

Well as @garrettr says above:

> Finally, I'm not sure how to determine when the pool is "full enough" to generate a GPG key. I don't think it is simple to compute, but would have to read the GPG source to know.

Either way could work though.

@diracdeltas you can look at /proc/sys/kernel/random/entropy_avail to see how much entropy is available. What would be a good number? This seems to have some good info, but doesn't actually list a number: http://crypto.stackexchange.com/questions/9412/what-to-watch-for-with-openssl-generating-weak-keys-low-entropy -- it seems like needs to be at the very least 128, but probably something closer to 500.

Alternatively, instead of limiting keygens to once every 20 minutes, we could do something like limit it to 5 every hour. Though once every x minutes is simpler.

According to [this](http://unix.stackexchange.com/questions/119967/rsa-2048-keypair-generation-via-openssl-0-5s-via-gpg-30s-why-the-difference) gpg reads 300 bytes of entropy from /dev/random to generate a 2048-bit key. I don't know if it's different for 4096-bit keys or not.

But 300 bytes is 2400 bits. I propose that we revert back to "flag for reply' behaviour if /proc/sys/kernel/random/entropy_avail is less than 2400, and otherwise we generate the key right away.

How does the queue idea solve the original DoS vector? The same scenario basically happens naturally, since GPG key generation blocks on entropy. The only difference is you'd been blocking one key at a time, which is not better from the perspective of avoiding DoS because legitimate sources still have to wait an uncertain and potentially ridiculous amount of time (`(len(keygen)+15)*(# spam DoS accounts)`) for reply keys to be generated.

Without a hardware RNG, I also think an attacker could easily keep us in a global minima for entropy, which in turn would affect the performance of the whole system because it uses so much crypto.

Finally, I'm not sure how to determine when the pool is "full enough" to generate a GPG key. I don't think it is simple to compute, but would have to read the GPG source to know.

Do you know much about hardware RNG, or USB entropy creating devices? I think it's worth investing in them to make source communication more usable.

Even without better methods of collecting entropy, I think that the flag option should be configurable. As far as I know no one has actually got entropy-DOSed before -- this is just theoretical.

How about this. We build a test for an entropy DOS -- even something as simple as 4 new PGP keys created in the same minute. If that happens, ossec can send an email alert and it can switch to forcing journalists to flag for reply for the next 24 hours before the restriction gets lifted again.

It seems unreasable to make SD way less usable to defend against attacks that aren't happening, when it only needs to be unusable during an attack.

Would a journalist be able to buy a trustworthy hardware RNG? For many newsrooms it should be enough, but the NSA has targeted random number algorithms and online-ordered products in the past. It sounds paranoid, but the NY Times bought one of their laptops in a store to assure that it wouldn't be specially targeted for them. It's not as easy to find specialized hardware without custom-ordering.

SparkFun has an interesting tutorial where someone [made a random number generator](https://www.sparkfun.com/tutorials/132) by themself.

> Do you know much about hardware RNG, or USB entropy creating devices? I think it's worth investing in them to make source communication more usable.

I know that they are expensive. It is also hard to make specific recommendations of trustworthy hardware, especially post-Snowden.

> How about this. We build a test for an entropy DOS -- even something as simple as 4 new PGP keys created in the same minute. If that happens, ossec can send an email alert and it can switch to forcing journalists to flag for reply for the next 24 hours before the restriction gets lifted again.

Sure, that seems like it would be a natural thing to include as part of our general plan for DoS mitigation. Some good work was done on this at the last hackathon, but no pull requests. I'll follow up with the people who were working on it.

@msjoinder As you point out, it is hard for us to know whether a hardware RNG is trustworthy. I don't know how we would determine that with a high enough degree of confidence to make an official recommendation. @dolanjs any thoughts on this?

How about this. We keep track of the timestamp when the last key was generated (probably in the database, but we'd need to make a new table), called last_keygen.

When a source that doesn't already have a key submits a message or file:
- If last_keygen < 20 minutes ago:
  - Update last_keygen to now
  - Generate a new key for the source

This way we can have up to one new key generated every 20 minutes without a problem. If new sources submit more frequently than this (which could be a DOS attack), then it reverts to the annoying "flag to reply" behavior, but otherwise keys get generated immediately.

What do you think? If this looks like a good solution I can start working on it.

And as far as where to store last_keygen, we could just store a timestamp in the file $securedrop_root/last_keygen and update that file every time a new key is generated. But it might make sense to store this in the db instead. We could make a new options table that has columns "key" and "value", to store settings like this.

> If last_keygen < 20 minutes ago: 

Should be `> 20`?

Since this can be DOS'ed by someone who submits a bogus file every 20 minutes, why not just generate keys as fast as possible and revert to 'flag this source' when entropy is low?

Oops, timestamp math is confusing.

Well as @garrettr says above:

> Finally, I'm not sure how to determine when the pool is "full enough" to generate a GPG key. I don't think it is simple to compute, but would have to read the GPG source to know.

Either way could work though.

@diracdeltas you can look at /proc/sys/kernel/random/entropy_avail to see how much entropy is available. What would be a good number? This seems to have some good info, but doesn't actually list a number: http://crypto.stackexchange.com/questions/9412/what-to-watch-for-with-openssl-generating-weak-keys-low-entropy -- it seems like needs to be at the very least 128, but probably something closer to 500.

Alternatively, instead of limiting keygens to once every 20 minutes, we could do something like limit it to 5 every hour. Though once every x minutes is simpler.

According to [this](http://unix.stackexchange.com/questions/119967/rsa-2048-keypair-generation-via-openssl-0-5s-via-gpg-30s-why-the-difference) gpg reads 300 bytes of entropy from /dev/random to generate a 2048-bit key. I don't know if it's different for 4096-bit keys or not.

But 300 bytes is 2400 bits. I propose that we revert back to "flag for reply' behaviour if /proc/sys/kernel/random/entropy_avail is less than 2400, and otherwise we generate the key right away.


Resolving patch:
diff --git a/securedrop/source.py b/securedrop/source.py
--- a/securedrop/source.py
+++ b/securedrop/source.py
@@ -205,7 +205,14 @@ def submit():
         submission = Submission(g.source, fname)
         db_session.add(submission)
 
-    g.source.pending = False
+    if g.source.pending:
+        g.source.pending = False
+
+        # Generate a keypair now, if there's enough entropy (issue #303)
+        entropy_avail = int(open('/proc/sys/kernel/random/entropy_avail').read())
+        if entropy_avail >= 2400:
+            crypto_util.genkeypair(g.sid, g.codename)
+
     g.source.last_updated = datetime.now()
     db_session.commit()
     normalize_timestamps(g.sid)


Test patch: