HPCC-9572 - Fix hash dedup row leak after spill event

When hash dedup spills, it continues to use the bucket hash tables
for dedupping purposes. The memory manager callback to flush
the HT was not protected by a mutex, which meant the reset could
overlap with another thread adding new rows, causing the flush to
in effect lose those newly added rows.

Also, close all bucket spill streams at end of phase (instead of
waiting for each new level to start). Minor significance, but the
output streams will have some buffered rows, and it is better to
close and flush those out upfront, rather than leave them in memory
whilst the other phases are being handled.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4568 from jakesmith/hpcc-9572

HPCC-9572 - Fix hash dedup row leak after spill event

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>