Queue up partition jobs in parallel to speed up generation of 100s of segments
Fix multi-segment generator so it doesn't go over our host limit (bad comparator).