HPCC-3326 Thor Lookup Join improvements

Lookup join gathers the RHS into an expanding row array, then exapands that
row array to fit a proportianally sized hash table in at the end.
Change so that:

1) The hash table is a separate allocate, this a) means it's not requiring
and even bigger contiguous single array and b) the initial row array can be
released after hashing
2) If the input row count is known, pre-allocate the hash table
3) The broadcasting/collection of RHS was very inefficient. It involved
compressing blocks on all nodes, node 1 requesting them serially,
uncompressing, collecting into HT, recompressing and broadcasting back to
others.
Change scheme, so streams and broadcasts from all and keeps blocks
compressed until arrive at final destination

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-633 Add support for loopagain support

Add support for LOOP, where the condition is on a dataset

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-7931 In ecl CLI, package action renamed to packagemap

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
HPCC-2960 Configmgr - Navigator multiselect can cause display issues

- When using multi-select no error message will be displayed
- Prevent (skip in multiselect) error message when copying 'Directories'
  because only 1 instance of directories can exist

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
Minor changes following review
HPCC-2960 Configmgr - Navigator multiselect can cause display issues

- Minor reformat based on code review

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-7939 Confusing code in checkLogicalName

This fix adds some commentary to make clear that looping should continue
through each file in a multi file

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
HPCC-3034 Make publishing and copying of queries aware of packages

Publish and "queries copy" will no longer clone file information
if that file or superfile is defined in a package mapped to the
given process/target/query.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
HPCC-7941 Update ConfigMgr Images and Docs

Update docs and images depicting configmgr
to reflect or capture changes

Signed-off-by: G Panagiotatos <greg.panagiotatos@lexisnexis.com>
HPCC-7931 Edits from code review

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
HPCC-7941 Update ConfigMgr Images and Docs

Update docs and images depicting configmgr
to reflect or capture changes

Signed-off-by: G Panagiotatos <greg.panagiotatos@lexisnexis.com>
HPCC-7917 Add CLI support for a --help option

Support --help in addition to existing "ecl help xxx" command because
many users find it more intuitive, and expect it to work.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
Merge pull request #3440 from JamesDeFabia/PackageToPackageMapRename2

HPCC-7931 In ecl CLI, package action renamed to packagemap

Reviewed-By: Anthony Fishbeck <anthony.fishbeck@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>

Merge pull request #3449 from afishbeck/ecl_roxie_help7917

HPCC-7917 Add CLI support for a --help option

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-2885 not all LIMIT options are documented on JOIN

Signed-off-by: Richard Taylor <richard.taylor@lexisnexis.com>
Add some loop in child query tests

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-3327 Row arrays limited to 500 million rows

In places (in thor) where an expanding array of rows is allocated from the
roxie/thor memory manager (i.e.. most activities that need to gather large
numbers of rows), we will hit issues (either incorrect behaviour or failures)
when the array size exceeds 4Gb.

With 64-bit pointers, that effectively creates a limit on the number of rows
at 500 million.

Change roxiemem layer to support allocations of > 4gb. This should push the
limit up to 4 billion rows.

We will need to address that limit too, but in a separate issue.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Community Edition 3.8.6 Release Candidate 1

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3446 from richardkchapman/roxiemem-3327

HPCC-3327 Row arrays limited to 500 million rows

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-7955 Configmgr - Disassociating ldap from esp causes seg fault

- If selecting no ldap (empty) from dropdown, the filesBasedn will
  be set to empty in the esp service upon confirmation.

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>