HPCC-3326 Thor Lookup Join improvements

Lookup join gathers the RHS into an expanding row array, then exapands that
row array to fit a proportianally sized hash table in at the end.
Change so that:

1) The hash table is a separate allocate, this a) means it's not requiring
and even bigger contiguous single array and b) the initial row array can be
released after hashing
2) If the input row count is known, pre-allocate the hash table
3) The broadcasting/collection of RHS was very inefficient. It involved
compressing blocks on all nodes, node 1 requesting them serially,
uncompressing, collecting into HT, recompressing and broadcasting back to
others.
Change scheme, so streams and broadcasts from all and keeps blocks
compressed until arrive at final destination

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-633 Add support for loopagain support

Add support for LOOP, where the condition is on a dataset

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-2960 Configmgr - Navigator multiselect can cause display issues

- When using multi-select no error message will be displayed
- Prevent (skip in multiselect) error message when copying 'Directories'
  because only 1 instance of directories can exist

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
Minor changes following review
HPCC-2960 Configmgr - Navigator multiselect can cause display issues

- Minor reformat based on code review

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-7939 Confusing code in checkLogicalName

This fix adds some commentary to make clear that looping should continue
through each file in a multi file

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
HPCC-3034 Make publishing and copying of queries aware of packages

Publish and "queries copy" will no longer clone file information
if that file or superfile is defined in a package mapped to the
given process/target/query.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
Add some loop in child query tests

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
fix whitespace issues
HPCC-7954 Configmgr - improper popup when changing ESP service bindings

- Ensure that user is modifying the service field before triggering
  confirmation pop-up

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-7884 Detect superfile error on reload subs

This check throws an error on invalid ranges, which will trigger an error
message on both listing a superfile and trying to add new subfiles to it.
Also relevant to HPCC-7938.

Signed-off-by: Renato Golin <rengolin@hpccsystems.com>
HPCC-7963 Configmgr - Missing && breaking build

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-2346 two log directories for dafilesrv created on startup

Dafilesrv was unable to create an instance-specific logfile folder since
it has no config file, and therefore didn't know the instance name. This
code passes the instance name to dafilesrv on the command line, which
dafilesrv then uses to build up the logfile location

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
HPCC-2884 he NOSORT option of JOIN not complete

Signed-off-by: Richard Taylor <richard.taylor@lexisnexis.com>
Merge pull request #3459 from garonsky/fix_build_break

HPCC-7963 Configmgr - Missing && breaking build

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge remote-tracking branch 'rtaylor/b2884' into candidate-3.10.x

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3444 from garonsky/issue-2960_Multiselect_Issues

HPCC-2960 Configmgr - Navigator multiselect can cause display issues

Reviewed-By: Kevin Wang <kevin.wang@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3458 from garonsky/issue-7954_resourceBasedn_in_ESP_Asks_to_update_filebasedn

HPCC-7954 Configmgr - improper popup when changing ESP service bindings

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3448 from RussWhitehead/dafLog

HPCC-2346 two log directories for dafilesrv created on startup

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3441 from RussWhitehead/daliPerf

HPCC-7939 Confusing code in checkLogicalName

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>