Adding pluggablility to work allocation, along with a new work allocation strategy to group small topics partitions within the same task.

The purpose of the new work allocation is to reduce the overall output file count.  Also included in this request is some optimizations for the daily sweeper code.
How do you get the avgMsgSize