HPCC-3326 Thor Lookup Join improvements

Lookup join gathers the RHS into an expanding row array, then exapands that
row array to fit a proportianally sized hash table in at the end.
Change so that:

1) The hash table is a separate allocate, this a) means it's not requiring
and even bigger contiguous single array and b) the initial row array can be
released after hashing
2) If the input row count is known, pre-allocate the hash table
3) The broadcasting/collection of RHS was very inefficient. It involved
compressing blocks on all nodes, node 1 requesting them serially,
uncompressing, collecting into HT, recompressing and broadcasting back to
others.
Change scheme, so streams and broadcasts from all and keeps blocks
compressed until arrive at final destination

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-633 Add support for loopagain support

Add support for LOOP, where the condition is on a dataset

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-2960 Configmgr - Navigator multiselect can cause display issues

- When using multi-select no error message will be displayed
- Prevent (skip in multiselect) error message when copying 'Directories'
  because only 1 instance of directories can exist

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
Minor changes following review
HPCC-2960 Configmgr - Navigator multiselect can cause display issues

- Minor reformat based on code review

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-7939 Confusing code in checkLogicalName

This fix adds some commentary to make clear that looping should continue
through each file in a multi file

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
HPCC-3034 Make publishing and copying of queries aware of packages

Publish and "queries copy" will no longer clone file information
if that file or superfile is defined in a package mapped to the
given process/target/query.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
Add some loop in child query tests

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-7955 Configmgr - Disassociating ldap from esp causes seg fault

- If selecting no ldap (empty) from dropdown, the filesBasedn will
  be set to empty in the esp service upon confirmation.

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
fix whitespace issues
HPCC-7954 Configmgr - improper popup when changing ESP service bindings

- Ensure that user is modifying the service field before triggering
  confirmation pop-up

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
Merge pull request #3452 from garonsky/issue-7955_Disassociateing_an_ldap_server_from_esp_causes_seg_fault

HPCC-7955 Configmgr - Disassociating ldap from esp causes seg fault

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7955 Configmgr - Disassociating ldap from esp causes seg fault

- If selecting no ldap (empty) from dropdown in the ESP, dali, or ESP
  service bindings (with no associated LDAP), the filesBasedn will  be set to
  empty in the esp service(s) upon confirmation.
- Fix seg fault.

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
Merge pull request #3457 from garonsky/issue-7955_Disassociateing_an_ldap_server_from_esp_causes_seg_fault

HPCC-7955 Configmgr - Disassociating ldap from esp causes seg fault

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7884 Detect superfile error on reload subs

This check throws an error on invalid ranges, which will trigger an error
message on both listing a superfile and trying to add new subfiles to it.
Also relevant to HPCC-7938.

Signed-off-by: Renato Golin <rengolin@hpccsystems.com>
HPCC-7963 Configmgr - Missing && breaking build

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-2346 two log directories for dafilesrv created on startup

Dafilesrv was unable to create an instance-specific logfile folder since
it has no config file, and therefore didn't know the instance name. This
code passes the instance name to dafilesrv on the command line, which
dafilesrv then uses to build up the logfile location

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
HPCC-2884 he NOSORT option of JOIN not complete

Signed-off-by: Richard Taylor <richard.taylor@lexisnexis.com>
Merge pull request #3459 from garonsky/fix_build_break

HPCC-7963 Configmgr - Missing && breaking build

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge remote-tracking branch 'rtaylor/b2884' into candidate-3.10.x

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>