Only iterate over all ChunkAggregators once.

There was no reason to iterate over them twice and this results in a bit
of a speedup.
Signed-off-by: Chris Sanders <zidane@outlook.com>