ministryofjustice/atet — pull request #394 by threedaymonk (2014-11-21 15:19:37+00:00)
Title: Random application references

We are concerned about the current method of application reference generation: the application reference is a function of the ID and a secret, but

* It's reversible.
* An attacker can obtain a set of IDs and valid application references from the application, from which they can potentially generate more valid application references.
* Having the application reference gives access to all the information that's been entered.

The proposed solution is to generate and store random application references by:

* Generating a large random value in the range [0, 4294967295] with Ruby's SecureRandom library (backed by OpenSSL).
* Render this number in base 32 encoding.
* Take the last m digits, where m is a large enough number to give us a comfortable trade-off of security vs usability; effectively, this means the random number modulo 32m
* If there's a collision (probability of 0.00018 with 8 digits and 20,000 live applications, so low but high enough to consider) we repeat the process until we have generated a unique value.

With eight digits, this gives about 10^12 (one trillion) possible application references, so we rely on rate limiting to prevent brute-forcing.

We check against existing applications references and generate another random reference if there is a collision. This does still leave a race condition, in that two simultaneous applications could generate the same reference. The probability of two simultaneous generations colliding is one in a trillion, however.

It would be cleaner to generate and check for uniqueness in the database. As the reference does not appear to be queried on a new record, there wouldn't be any problems with this at an application level. Alternatively, just making the references longer (19 or 20 digits) would make collisions effectively impossible and we could simplify much of this.