metamx/druid — pull request #848 by drcrallen (2014-11-12 22:46:10+00:00)
Title: TopN performance improvements

Re-factor scanAndAggregate in PooledTopN

* Loops are now a little bit tighter when looping over aggregates. This will hopefully assist in loop execution optimization.
* Pre-calculated the aggregate offsets instead of shifting them during runtime.
* Cursor loop could use some TLC, but would require a massive refactoring on how TopN queries are executed.
* Any potential modifications to query workflow need to account for Stream vs Batch data, and that not all data will be array backed that comes in.
* Changed looping to attempt to do 8 at a time for both aggregates and dim values
* Any dim value collection or aggregate collection with less than 8 at a time, or the remainder of values which are not evenly divisible by 8, has values serviced by a SWITCH statement rather than looping anything.

Change data storage type in TopNNumericResultBuilder.

* Use PriorityQueue to store
* Checks to see if should even bother adding to Queue before adding.
* Re-orders Queue on build() call.
* Ideally the order would be directly preserved on build(), but this is a close second.

General performance improvements:

TopN multivalue multi aggregate: just shy of 30% faster

TopN multivalue single aggregate: just shy of 5% faster