HPCC-716 Include superfile into addSuper transactions

When files are added into new superfile and a failure occurs when adding
one of the files, the empty superfile is not deleted. That may mislead
us. This fix moves the superfile creation into the same transaction of
adding the files into the superfile. If there is a failure when adding
the files, the whole transaction will be rolled back so that the empty
superfile will be deleted.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
HPCC-7869 Roxie cores in regression suite

Code introduced while fixing gh-2704 (commit 2a0d31bf) introduced
a core while resolving a dynamic file.

This is a regression and is not in any shipping version

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7870 Convert 'Slave Number' input to integer before comparison

On Ecl Workunit Details page (a system with 100 slave nodes), a user
specifies 39 as a Slave Number and clicks the 'get slave log' button.
An error message shows: 'Slave Number cannot be greater than 100'.
The problem was because javascript identifies the user input using
string and do string comparison. This fix converts 'Slave Number' input
(and total number of slaves) to integer before a comparison.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
HPCC-3198 Remove 'Run ECL' link from EclWatch

My previous fix removed the 'Run ECL' link from the ECL menu. But, an
'ECLDirect' menu group is added since default ESP getNavigationData()
is called. This fix adds an empty getNavigationData() into EclDirect
service to avoid the default getNavigationData().

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
HPCC-674 Add new word remove functions to Std.Str

Add functions to remove the first, last or nth words from a string.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-7880 - Additional thor memory tracing

Trace # maxCores setting per job
Trace roxiemem and heap usage at end of each job.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #3390 from jakesmith/tracing

HPCC-7880 - Additional thor memory tracing

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3385 from wangkx/h7870

HPCC-7870 Convert 'Slave Number' input to integer before comparison

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3387 from wangkx/h3198b

HPCC-3198 Remove 'Run ECL' link from EclWatch

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-3326 Thor Lookup Join improvements

Lookup join gathers the RHS into an expanding row array, then exapands that
row array to fit a proportianally sized hash table in at the end.
Change so that:

1) The hash table is a separate allocate, this a) means it's not requiring
and even bigger contiguous single array and b) the initial row array can be
released after hashing
2) If the input row count is known, pre-allocate the hash table
3) The broadcasting/collection of RHS was very inefficient. It involved
compressing blocks on all nodes, node 1 requesting them serially,
uncompressing, collecting into HT, recompressing and broadcasting back to
others.
Change scheme, so streams and broadcasts from all and keeps blocks
compressed until arrive at final destination

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #3384 from richardkchapman/roxie-core-dynamic-file-7869

HPCC-7869 Roxie cores in regression suite

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #3382 from wangkx/h716a

HPCC-716 Include superfile into addSuper transactions

Reviewed-By: Renato Golin <rengolin@hpccsystems.com>
Reviewed-By: Jake Smith <jake.smith@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7828 Replacement Images for ECLWatch

Replaces older images of ECLWatch with new images
 that depict current menu options.
No Run Ecl link.

Signed-off-by: G Panagiotatos <greg.panagiotatos@lexisnexis.com>
HPCC-3263 Flag libraries when published to QuerySets

WsEcl needs an efficient way of being able to recognize library
entries in querysets.

When libraries are published add an "@isLibrary" attribute to the
Query element in the QuerySet.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
HPCC-7877 Fix ECL CLI errors referring to Cluster rather than Target

Error text references to "Cluster" changed to "Target".

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
Merge pull request #3393 from afishbeck/publish_library

HPCC-3263 Flag libraries when published to QuerySets

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3394 from afishbeck/cli_cluster_error7877

HPCC-7877 Fix ECL CLI errors referring to Cluster rather than Target

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7868 PackageSet needs a unique id

In order to be able to refer to a particular PackageSet in ecl-package,
we need to give the PackageSet a unique id. Move the name of the roxies
to apply to onto a new process attribute.

Note that this area has already changed since 3.8.x, so this does not
create any (new) incompatibility.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7886 Issue warnings and retry on thread start timeout

If a new thread startup fails to wake within the timeout,
issue warning logs and retry instead of failing after 1st
timeout.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-7887 rowarray.ensure(0) debug assert

If a ensure(0) on a CThorExpandingRowArray used for a stable sort is called, a
debug build will hit an dbgassertex in allocateStableTable, complaining that
the co-array 'rows' hasn't been instantiated.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>