Added better metrics.  Also added functionality to adjust the batch size.
Refactored code to merge right before the global group by, not sure how that effects things in the long run but we shall see.