bkeepers/swordfish — pull request #15 by josh (2012-08-03 00:09:16+00:00)
Title: Convergent Encryption

Sorry @bkeepers, you got me excited and I've been wanting to mess around with this idea for a long time. I dunno if you actually want to go this direction, but let me try to persuade you this architecture is at least cool even if we don't use it.

*(this might get long)*

## Convergent Encryption

The gist is you use a hash of the plain text to encrypt itself. This allows anyone with the same plain to generate the same encryption keys.

Consider this basic content addressable model (similar to git)

So you have the data `foo`. Its SHA1 is `0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33` so you use its hash as the key.

```
"0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33" => "foo"
```

We can take that hash of the original contents and use it as the encryption key. Then rehash the cipher text and produce another key. This would be the content addressable key.

AES encrypt `"foo"` with `"0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"` as the key gives you something like `"U2FsdGVkX1+T6CfaNzHyfHDxX5OAVh614kGNiNhIIYg="`. Know we rehash that cipher text to give us `73d4ba65d222aebc70e98cd11a86e4cf701671c3`.

So know we have this in our database:

```
"73d4ba65d222aebc70e98cd11a86e4cf701671c3" => "U2FsdGVkX1+T6CfaNzHyfHDxX5OAVh614kGNiNhIIYg="
```

which requires `0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33` to decrypt.

So note, to retrieve any piece of plain text we need a content addressable id and a AES key (2 SHA1 hashes).

To keep track of this data, the server can store a reference the content addressable id. It just points to cipher text so thats okay. From here, the user can encrypt that key with their own personal password and store that encrypted key on the server. This would work for either symmetric or asymmetric encryption.

But storing one piece of information is kinda boring. But consider how git objects work, we have blobs and trees.

```
{ "afd00be567bbf5ebe35497f323d8220c4d8b2a63" =>
[ ["foo.txt", 0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33]
["bar.txt", 62cdb7020ff920e5aa642c3d4066950dd1f01f4d]
["baz.txt", bbe960a25ea311d21d40669e93df2003ba9b90a2] ] }
```

We can have a directory listing by pointing filenames at blob ids. This directory index can be treated as content on its own and hashed itself.

The encrypted version just needs one extra piece of info for each entry, the key.

```
{ "f9b96f332f7763ef84cd76973c4c0aae12868df4" =>
[ ["foo.txt", 0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33, 73d4ba65d222aebc70e98cd11a86e4cf701671c3]
["bar.txt", 62cdb7020ff920e5aa642c3d4066950dd1f01f4d, f85ec68d87a16612a527e4926839f3370aa8a44d]
["baz.txt", bbe960a25ea311d21d40669e93df2003ba9b90a2, 722ab4fb5ef08360a413992f29fdfa1736c71ac8] ] }
```

If you we're given this index, you'd have everything you would need to retrieve and decrypt all those files. Now lets encrypt that too. Same method, use the content addressable hash as the key.

```
"U2FsdGVkX19Q4Uut6dYKKXicFkABLPfHpSYH14KneVkFx9OzdEg8971AWEJnLQ03
HE+Ey9h5oXPvsHzbCQeSLl/ObUGY9CKIvtjEjn7D1VvWhXPamJaH7XYGsCjU59Ud
CqjwOZKFznzFaM673xF/DB5d4NDYDbp3DHyj9qyPtW7ch2zFAqXgk9Aya9iglc48
dnG2x9wCIANq2gba6ay0R+a0RyFAwFHL4CeJO8nEonfVZ/BQZsR4s3MrSZ8Z01aV
W9pbi91azOtaCKCBoI4s/n9e3y4/rZ+gdWoUw7V/cds="
```

Then hash it again to get `871eab8408b011d9d2b5aac9fd7fa97f0cf8bcb3`.

So now, given the id for the index `871eab8408b011d9d2b5aac9fd7fa97f0cf8bcb3` and is decryption key `f9b96f332f7763ef84cd76973c4c0aae12868df4` you could retrieve all the files.

All this can be nested many times still leaving you with 2 pieces, the id pointer and a single decryption key. Everything that needs to be in the store db itself, is always fully encrypted. And you get all the advantages of an immutable store.

## Swordfish encrypted db

So I really think we want the whole db encrypted. Not just passwords, all usernames too. As well as the directory list.

### Server Side

The server side would be mostly be a dead simple key value store. Hey, mongodb is already great for this.

```
POST /objects data=U2FsdGVkX1+T6CfaNzHyfHDxX5OAVh614kGNiNhIIYg=
# => { 'id' => '73d4ba65d222aebc70e98cd11a86e4cf701671c3' }

GET /objects/73d4ba65d222aebc70e98cd11a86e4cf701671c3
# => { 'data' => 'U2FsdGVkX1+T6CfaNzHyfHDxX5OAVh614kGNiNhIIYg=' }
```

Is pretty much all we need. Both items and the vault indexes can be stored the same.

```
# item name, content id, decryption key
[ [ "GitHub", "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33", "73d4ba65d222aebc70e98cd11a86e4cf701671c3" ]
[ "Google", "bbe960a25ea311d21d40669e93df2003ba9b90a2", "f85ec68d87a16612a527e4926839f3370aa8a44d" ]
[ "Twitter", "bbe960a25ea311d21d40669e93df2003ba9b90a2", "722ab4fb5ef08360a413992f29fdfa1736c71ac8" ] ]
```

We only need to manage mutable state that the Vault level. Each vault needs a pointer to its encrypted item index. The decryption key could be stored in a few different ways. Easiest would be to use a symmetric key for now. Each user's account could store their encrypted version.

But we may want to consider doing something like git's commit object for history. Every change could have a pointer to the items listing object and a pointer back to the previous. Which this we get password revision history for frees. Also a revision/commit object would be a prefect place to reference audit information. Anytime theres a change we can store user/ip/location metadata about it.

### Client Side

So the server side is just a dumb content store like git. We only need a few methods for interacting with the store and doing all the encryption transparently. And since its all immutable, caching is simple. It all can go directly into localStorage with no issues (storing the encrypted response, never plain text).

See `app/assets/javascripts/collections/store.coffee` in the pull diff.

### Implementation

This code is kinda crapy. I couldn't quite get all the existing UI to work. Some backbone events weren't triggering.

Anyway, let me know if you think its a good storage direction.