HPCC-3326 Thor Lookup Join improvements

Lookup join gathers the RHS into an expanding row array, then exapands that
row array to fit a proportianally sized hash table in at the end.
Change so that:

1) The hash table is a separate allocate, this a) means it's not requiring
and even bigger contiguous single array and b) the initial row array can be
released after hashing
2) If the input row count is known, pre-allocate the hash table
3) The broadcasting/collection of RHS was very inefficient. It involved
compressing blocks on all nodes, node 1 requesting them serially,
uncompressing, collecting into HT, recompressing and broadcasting back to
others.
Change scheme, so streams and broadcasts from all and keeps blocks
compressed until arrive at final destination

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-2197 Ecl package command line parameters need to be more consistent

The same parameters for different actions should be spelled
the same and should be consistent with ecl command line parameters
when applicable.  Required parameters should not be listed as optional
(i.e. target has replaced queryset and is required)

Signed-off-by: Stuart Ort <stuart.ort@lexisnexis.com>
HPCC-3365 Add params for per-query memory and time limits from CLI

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
HPCC-3153 Filter WU names before showing on Browse Workunits page

On some environments, EclWatch Browse Workunits page shows a few items
which are not workunits, such as 'globel', 'Query', etc. Those items
are retrieved from the '/WorkUnits' tree using querySDS().connect().
Now, I filter out those names before showing workunits on Browse
Workunits page.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
HPCC-7916 Add ecl roxie commands

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
HPCC-7918 Configmgr - Save As from context menu locks up browser

- Change menu name to 'Save Environment As' from 'Save Environment
  As...' to properly capture context menu click

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-2309 Configmgr - Entering LDAP ou info is error-prone

- When modifying the 'ldapServer' field under the 'LDAP' tab for
  the 'Dali Server' component, pop-up will ask to update the 'filesBasedn'
  field to match that of the selected LDAP server.

- When modifying the 'ldapServer' field under the 'Authentication' tab for
  the 'Esp' component, a pop-up will ask to update the 'filesBasedn' field
  to match that of the selected LDAP server for all bound esp services
  that contain a filesBasedn field.

- When modifying the 'serverice' field under the 'ESP Service Bindings' tab
  for the 'Esp' component, a pop-up will ask to update the 'filesBasedn'
  field for all bound esp services that contain a filesBasedn field.

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-7919 Report errors that prevented loading a Roxie query

In order to present better feedback to users when queries don't deploy
successfully, improve the output of control:queries to return any
associated error message if a query is suspended. Also allow the output
to be limited to a specified set of queries

The incoming message is of the form:

<control:queries>
 <Query id='queryname'/>
 ... more queries here if you want ...
</control:getQueryInfo>

Code has been refactored to common up the functionality with the just-added
control:getQueryXrefInfo. As a result the output from that query now uses
<Queries> rather than <QueryInfo> as the surrounding element name.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7888 Ensure query dlls in separate dir

In multi slave setups, the query dll must not clash.
Ensure it's in it's own directory.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-7888 Reinstate lost dllsToSlaves=false changes

Previous commit to address this issue, inadvertently lost made in gh-2592

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #3416 from richardkchapman/roxie-query-info

HPCC-7919 Report errors that prevented loading a Roxie query

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-2058 Add a ,UNORDERED option to an index read

Allow ,UNORDERED to be specified on an INDEX as a hint to allow the engines
to potentially optimize the way rows are returned.

I also renamed the flag from unsorted because it has a slightly stronger
meaning.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #3421 from ghalliday/issue2058

HPCC-2058 Add a ,UNORDERED option to an index read

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Split off closedown branch for 3.8.6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Split off closedown branch for 3.8.6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7920 Core in dafilesrv if sent a VER command in authenticated mode

When dafilesrv is running in authenticated mode (started with -A) , it will
core if sent a VER command by dafscontrol.

VER is the only command that is NOT authenticated, and requires some special
handling. It wasn't quite special enough, unfortunately.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3418 from richardkchapman/dafilesrv-core

HPCC-7920 Core in dafilesrv if sent a VER command in authenticated mode

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-7856 Include macros with parameters-with-defaults in dependencies

This addresses several issues
* You can now sumbit a macro that has default values for all its parameters as
  a query.
* The same macros are now supported in the legacy dependency generation
* Common up some of the macro processing code between the two.
* Sort the module and symbol lists so dependencies are generated in a
  deterministic order.
* Fix potential caching problems if no error handler provided to the dependency
  gathering code.  (It didn't count the number of errors, so the code couldn't
  tell whether an error had occurred processing a definition.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-7882 Update docs to reflect changes to ecl CLI

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
HPCC-2197 Ecl package command line parameters need to be more consistent

Code changes based on code review comments

Signed-off-by: Stuart Ort <stuart.ort@lexisnexis.com>