share/livedb — pull request #33 by devongovett (2014-10-18 19:25:03+00:00)
Title: Support expiring ops from oplog

This goes along with https://github.com/share/livedb-mongo/pull/12. It handles emitting `Missing operations` errors when operations requested from the oplog are not there. This happens in `submit` and `getOps` (and by extension `fetch`, since it calls `getOps`).

This error should occur if a client has been offline longer than the ttl and tries to submit/fetch some ops. It should only occur if other edits have taken place while the user was offline, which have also expired. Specifically:

1. user 1 goes offline
2. user 1 edits
3. user 2 edits
4. ops are dropped (user 2’s ops expire)
5. user 1 comes back online and submits its ops
6. ERROR

If 3 and 4 are swapped (i.e. user 2 doesn’t edit anything before the ops are dropped), the error shouldn't occur. Basically, it should be fairly rare.

When the error happens, ShareJS sends it back to the client, which has an opportunity to notify the user that their changes could not be merged. Not ideal, but necessary to keep the data size in mongo from growing forever.

One case I thought of but I'm not sure about is whether we need to get the snapshot version in `getOps` when `to` is null to determine whether ops are expected from the oplog. Right now, the check for missing operations only occurs when there are ops returned (check `ops[0].v === from`). If `to` is null, we may also need to check if ops were expected if `from < snapshot.v`, which means fetching the snapshot there. I don't really like that, but it may be necessary. In `submit` this is handled already. What do you think?