ukhas/habitat — Issue #171: The parser parses some documents several times unnecessarily
Author: danielrichman
Created: 2012-02-26 13:41:48+00:00
State: open
Labels: couchdb, parser, python

I presume it's when the continuous _changes says that the doc has changed and hasn't been parsed (filter) several times while the parser is parsing that doc, and they get queued up. I.e., when there are quite a few quick receivers.

It's not really a problem, though it does load down couch & the parser a little more than necessary.

The easiest solution would be to switch off _changes' include_docs and grab the latest doc from couch every time, but that's a bit meh, adding another round trip to the database.

Could keep the _id and _rev of the last document we saved, and compare it to the one we're about to parse, and if the _id is the same but the _rev we just saved is newer, ignore? Provided it's "okay" to rev.split("-")[0] and compare that integer... also a bit meh.