basho/bitcask — pull request #144 by engelsanchez (2014-03-05 02:06:12+00:00)
Title: Add sibling->regular entry conversion sweeper

This fix is just a translated version of Scott's change here:
1563feb3e64c85a5498e992090e99fb4d1825555

To allow Bitcask to handle multiple iterators, the keydir was modified
to allow multiple versions of an entry. That way multiple iterators
could see different snapshots of the data as it was modified.
Removing versions of entries that were no longer needed only happened on
access to those entries, so that potentially a lot of unneeded memory
could stay around for a long time. This adds a mechanism to reclaim that
memory little by little. On each get/put/delete operation, the sweeper
spends a bounded amount of time removing stale siblings.
The amortization mechanism attempts to limit itself to less than
1 msec of additional latency for any get/put/delete call, but
it doesn't always stay strictly under that limit when
you're freeing hundreds of megabytes of data.