This checkin does batching of TimelineAggregator chunk creates and
invalidates/deletes.

collector/src/main/java/com/ning/arecibo/collector/guice/CollectorConfig.java:

    Add int parameter maxChunkIdsToInvalidateOrDelete, defaulting to
    1000.  This means that by default, we'll batch up chunk inserts
    and deletes or invalidates whenever the count of chunks aggregated
    exceeds 1000.

collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregator.java:

    Add some counters to distinguish between queued write operations
    and the actual writes.

    Add a pair of lists, one to hold new chunks produced by
    aggregation, and the other to hold the chunkIds of aggregated
    chunks to be deleted or marked not valid.  In
    aggregateHostSampleChunks(), add the new chunk and the ids of the
    chunks aggregated to these lists, and if the list of ids exceeds
    the threshold, call performWrites().

    When we're done with an aggregation level, of there are any writes
    queued, call performWrites().

    Add new method performWrites(), which in a single transaction,
    inserts the new chunks and deletes or marks not valid the old
    chunks.

    Fix bug: Remove redundant timelineChunksCreated.inc().

collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregatorDAO.java:

    In makeTimelineChunkValid(), change the chunkId to be a long.

util/src/main/resources/com/ning/arecibo/util/timeline/persistent/TimelineDAOQueries.sql.stg:

    In insertTimelineChunk, explicitly initialize aggregation_level
    and not_valid, rather than letting the table default be used.
This checkin adds timeline and sample coding support sufficient to
allow the TimelineAggregator to combine the compressed encodings
intelligently, rather than just concating them.  This greatly
compresses aggregations of default-valued timelines.  Since this is
fairly tricket code, this checkin also adds a ton of unit tests for
combining.

It also optimizes the basic search query for chunks to aggregate.
This should reduce the number of chunks scanned in prod by a factor of
6 or more.

collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregator.java:

    Use the new coder combine() methods to compress aggregated
    time and sample byte stream.

    Pass chunksToAggregate to the dao method that gets candidate
    chunks.

collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregatorDAO.java:

    Remove now unused insertNewInvalidTimelineChunk.

    In getTimelineAggregationCandidates, bind chunksToAggregate.

collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineHostEventAccumulator.java:

    Treat opcodes as ints, since byte is signed and we have opcodes
    above 127.

    The max repeat count is now MAX_SHORT_REPEAT_COUNT.

collector/src/main/resources/com/ning/arecibo/collector/persistent/TimelineAggregatorDAO.sql.stg:

    Remove never-used -getHostIdsAndSampleIdsOfAggregationCandidates.

    Update getTimelineAggregationCandidates to only return chunks for
    which aggregation is possible.

util/src/main/java/com/ning/arecibo/util/timeline/DefaultTimelineDAO.java:

    Remove unused imports.

util/src/main/java/com/ning/arecibo/util/timeline/RepeatSample.java:

    There are two sample repeat codes, one allowing up to 255 repeats,
    and the other allowing up to 65535 repeats.

    Add incrementRepeatCount(int addend), because combining timelines
    means combining repeats.

    Overload getOpcode() to return either the BYTE or SHORT opcode,
    depending on the repeat count.

util/src/main/java/com/ning/arecibo/util/timeline/SampleCoder.java:

    Make this instantiable, holding the state and methods necessary to
    do sample coding, moved here from TimelineChunkAccumulator.

    Add compress() and decompress() methods for sample lists.

    Add coder and scanner support for both BYTE and SHORT sample
    repeat opcodes.

    Add new method combineSampleBytes(), which combines and compresses
    a list of timelines.

    Use DataInputStream.read() when we read a byte; it has a cleaner
    API.

util/src/main/java/com/ning/arecibo/util/timeline/SampleOpcode.java:

    Make opcode index values be ints, not bytes, to side-step signed
    byte problems with opcodes > 127.

    Add BYTE and SHORT versions of the repeat opcodes.

util/src/main/java/com/ning/arecibo/util/timeline/ScalarSample.java:

    Add an equals() overloading so it can be used in tests.

util/src/main/java/com/ning/arecibo/util/timeline/TimeCursor.java:

    Add support for scanning both BYTE and SHORT repeats.

    Fix bug: for unrepeated delta times, it read one ahead.

util/src/main/java/com/ning/arecibo/util/timeline/TimelineChunkAccumulator.java:

    Move all the coding machinery to base type SampleCoder.

util/src/main/java/com/ning/arecibo/util/timeline/TimelineCoder.java:

    Move MAX_DELTA_TIME to TimelineOpcode.

    Add support for both BYTE and SHORT delta repeaters.

    Add major new method: combineTimelines(), called by the
    aggregator.

util/src/main/java/com/ning/arecibo/util/timeline/TimelineOpcode.java:

    Add support for both BYTE and SHORT delta repeaters opcodes.

    Make the non-delta-valued opcodes  run from 0xFF down.

    Change the max delta time to 240, which leaves 13 unused opcodes
    above it.

util/src/test/java/com/ning/arecibo/util/timeline/TestSampleCoder.java:

    Add comprehensive tests of combining sample byte sequences and use
    of the SHORT repeat facility.

util/src/test/java/com/ning/arecibo/util/timeline/TestTimelineChunkAccumulator.java:

    Add tests of BYTE and SHORT repeaters.

util/src/test/java/com/ning/arecibo/util/timeline/TestTimelineCoder.java:

    Add comprehensive tests of combining time byte sequences and use
    of the SHORT repeat facility.
collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregator.java:

    If we didn't make new aggregates at the current level, there is no
    point in trying later levels, so exit the loop.

util/src/main/java/com/ning/arecibo/util/timeline/SampleCoder.java:

    The aggregator was getting errors because the combiner was using
    sample value equals(), which will return true for objects of
    different types.  Add a new static method sameSampleValues() which
    only returns true if the objects have the same class.

util/src/main/java/com/ning/arecibo/util/timeline/ScalarSample.java:

    Use new static method SampleCoder.sameSampleValues()
collector/src/main/java/com/ning/arecibo/collector/persistent/TimelineAggregator.java:

    Demote an info message to debug.

util/src/main/java/com/ning/arecibo/util/timeline/SampleCoder.java:

    Fix a thinko in the sample timeline combining code: It used to
    call decodeScalarValue() to read the opcode args.  But that method
    returns Double values for opcodes like HALF_FOR_DOUBLE, because
    that's what the scanner wants.  So for the combining code, add new
    method decodeOpcodeArg() to return what was read from the stream
    without casting to double.

util/src/main/java/com/ning/arecibo/util/timeline/TimelineCoder.java:

    Remoe a TODO about adding timeline combining, since that's now
    done.

util/src/test/java/com/ning/arecibo/util/timeline/TestSampleCoder.java:

    Add a test for the failing case that requires decodeOpcodeArg().