jakesmith/HPCC-Platform — pull request #5 by jakesmith (2013-05-01 12:38:33+00:00)
Title: Lookup optimization

This is primarily for the MANY LOOKUP case, where it is expected that there will be many clashes in the lookup table.

When there's a HT clash, the previous algorithm performed a linear scan to find the next available empty slot.
If there few duplicates and assuming the hashing distributes reasonably evenly, this is not so much of an issue, but if there are many duplicates of a given value, then the scan becomes expensive and the linear space they fill causes other non-duplicate clashes and more scanning.

This change, tracks the consumed HT range when there's clash.