HPCC-8245 - Smart Lookup Join

Implementation of Lookup Join that on OOM fails over to a
hash distributed lookup join, then if that OOM's, fails over to
a regular fully distributed(both sides) local join.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Clear up a few things

1) bug, race between 1st/2nd broadcasters on different slaves
2) bug, some denormalizelookup cases missing from joinhelper
changes
3) Ensure all members initialized in constructor
(move some things from init() to constructor in the process)
4) Clear up tabs vs spaces
5) Unify on one createLookupJoinSlave() method and use TAK type
6) Add #TEST_FAILOVER_ defines
7) Comment in JFsmart usage

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Changes following review

1) simplify CThorExpandingRowArray::compact
2) make failover options runtime options for testing purposes
3) coupleo of minor simplifications

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Switch to TAKsmart

Plus a few minor changes after review

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - more minor review changes
HPC-8245 - Avoid deadlock potential whilst flushing

Need to avoid holding a shared lock whilst appending to row
arrays. Otherwise, roxiemem's spilling callback could stall if
the append causes a row array resize

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Ensure flushedRowMarkers is clear at start

Need to clear at start, to avoid issues with reruns in child
queries

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Split row-owning vs non-owning into 2 implementations

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Be careful handling stop case

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8245 - Ensure smartjoin test order consistent

If smartjoin spills the output order is indeterminate

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4868 from jakesmith/hpcc-8245

HPCC-8245 - Smart Lookup Join

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>