pouchdb/pouchdb — pull request #2893 by nolanlawson (2014-10-18 18:30:25+00:00)
Title: (#2818) - compaction removes orphaned attachments

Migrated idb/websql to have a new many-to-many table
of seqs to digests, fixed compaction of attachments
in all three adapters, added many new tests.

This is a new PR, born of the ashes of https://github.com/pouchdb/pouchdb/pull/2846, because I didn't want to bore y'all with the details of how I got to this commit.

What changed:

**WebSQL and IndexedDB**

Migration required. I created a new many-to-many table/objectStore called `ATTACH_AND_SEQ` which contains a mapping of the `seq` to `digest`. I removed the old `refs` system (where we had a big JSON object of docId/revs stored along with the digest/binary) for a few reasons:

1. This way we're more explicit about the data relations, and can move logic into the database, e.g. when checking for orphans, "which digests have no associated seqs anymore, i.e. are orphaned?" is just a SQL query.
2. Keeping it in a separate objectStore means that IndexedDB doesn't have to read in the whole binary just to update the `refs`.

Besides the extra table, a migration is also required because we were doing the refs incorrectly. When a document was modified but its attachment wasn't, we weren't correctly updating the `refs` associated with the `digest`. We are now, and there's a migration test for that.

**LevelDB**

I kept the status quo, i.e. `refs` stored in the `attachmentStore`, with a separate `binaryStore` mapping digests to binaries. Reason being, I couldn't imagine any perf benefits from the many-to-many store, plus it would be tricky to write a many-to-many store given LevelDB's structure. We can always migrate in the future if we think it would bring perf benefits.

Also, by keeping the `binaryStore` separate from the `attachmentStore`, we were already correctly updating the `refs`, so there's no migration necessary. I added a LevelDB migration test to make sure, though, and I also moved all the migration tests into `tests/leveldb` to keep them corralled.