HPCC-11928 Smart Join failing over to full join too early.

When smart join ran out of memory, it attempted to 1st failover
to a local lookup join, however, in doing so it consumed too
much memory too early, causing the local lookup to failover to
a standard join too early.
The reasons it used too much memory too early were:

1) A distributor on the LHS was started asynchronously, meaning
the receiver side started buffering up rows contributing to an
overall shortage of memory.

2) The local row tables from each node that are built up as
part of the 1st phase (std. lookup) are cleared of non-local rows
by the first OOM (failover to local lookup), these partial row
arrays can be quite big and sparse and had were not spillable.
a) Transfer the biggest chunk into the loader for the rest, to
allow as much as possible to free up from these row arrays
b) TBD: realloc the sprase array ptr to smaller required size
(See HPCC-12511)
c) Make them spillable at a lower priority than gathering to
local join stage. If the gather OOM's then we have to failover
to std. join, spilling the row arrays may free up enough memory
for local lookup to fit.

3) When the broadcast detects an OOM and clears non-locals, it
must still deal with rows in transit, these can cause another OOM
particularly if they involve a ptr array expansion. Handle by
spilling to a separate single overflow (there won't be that many)

4) Needed a separate final broadcast to check if anyone had
failed over from local lookup to full lookup

5) When failing over to local lookup, transfer the largest slave
array into the loader, to free up it's memory (ptr table) asap

6) Ensure ptr table arrays are destroyed by stream spillers asap,
i.e. when all rows consumed, or spilt and remove MM buffer
callbacks asap also, to avoid pointless callbacks to something
that can no longer spill

7) Rename a few test member variables for clarity,
e.g. isLocalLookup()

8) Add some more comments to explain tactics

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-11928 Fix a couple of issues picked up in review

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-11928 rhsAlreadyChanges

1) Take account of local and global already sorted meta data
2)Spotted that marker may have been sorting when didn't need to
3) Avoided rhs rows moving in and out of collector if already
collated.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #6710 from jakesmith/hpcc-11928

HPCC-11928 Smart Join failing over to full join too early.

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>