rackerlabs/blueflood — pull request #155 by fourk (2013-11-26 23:22:13+00:00)
Title: WIP -- Batch Rollup Writes

This will cause us to batch rollup writes.

The idea is that we have one threadpool doing the actual writes. We have a RollupBatchWriter per LocatorFetchRunnable (i.e, one per shard+gran we are rolling). RollupRunnable will feed the RollupBatchWriter rollups to write. These are enqueued, and if the queue has 50 or more pending rollups, it creates a RollupBatchWriteRunnable and puts it in the threadpool.

RollupExecutionContext was modified to have both a readCounter and a writeCounter, where it previously had just one counter representing both. LocatorFetchRunnable now waits for both read completion and write completion in the same way it previously waited for the single counter.

There are still a few things I'd like to do, but will do in separate pull request(s), to keep the diff easily reviewable.
* Handle exceptions during read and during write. (i.e: https://github.com/rackerlabs/blueflood/issues/115)
* Clean up the service package, it's got something like 30 classes in there now.
* Make rollup write and read threads separately configurable, make batch size configurable
* Get rid of the below method in AstyanaxWriter, replacing usage using the newly added insertBasicRollups method.

```writer.java
// todo: this method should be made private. outside of this class, it is only used by tests.
public void insertRollups(Locator locator, Map rollups,
ColumnFamily destCF) throws ConnectionException {
```