Make all aggregators reader-context-aware.

This removes the overhead of polling a Bytes/Double/Long-Values instance in
every call to collect.

Additionally, the AggregationsCollector has been changed to wrap a simple array
instead of an ArrayList.

Close #4841