added untested logic for detecting maps and reduces which died unexpectedly so we can count restarts for those
reworked intermediate merging to use a spill file containing all shards