HPCC-9399 Add stepped dedup to hthor and clean up roxie code

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>