HPCC-9572 - Fix hash dedup row leak after spill event

When hash dedup spills, it continues to use the bucket hash tables
for dedupping purposes. The memory manager callback to flush
the HT was not protected by a mutex, which meant the reset could
overlap with another thread adding new rows, causing the flush to
in effect lose those newly added rows.

Also, close all bucket spill streams at end of phase (instead of
waiting for each new level to start). Minor significance, but the
output streams will have some buffered rows, and it is better to
close and flush those out upfront, rather than leave them in memory
whilst the other phases are being handled.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9609 - Tidy up dataLinkStart calls

Unecessary calls passing in activity string/id, when CThorDataLink
already had activity with same info.

Also resolve a uninitialized warning picked up by Valgrind.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4568 from jakesmith/hpcc-9572

HPCC-9572 - Fix hash dedup row leak after spill event

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-9597 Combine multiple ds[1].<field> into a single row

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-9627 Remove unused resourcing link type

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-9628 Eclcc hangs at end if libarchive dynamically unloaded

In my tests setup, I have the libs directory on my plugins path, meaning that
eclcc will try to load libarchive as a plugin, then unload it when it finds it
is not.

This means that MODEXIT gets called before removeFileHook (due to bugs in the
modexit logic - it should really only be called on the last unload, not on any
unload).

Protect against that event by setting lock ptr to NULL.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9602 Avoid temporary variables when returing simple expressions

The "very temporary" fix has been in there for many many years, and has
been redundant for most of them.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-9600 Use library classes for exists and count

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #4598 from ghalliday/issue9600

HPCC-9600 Use library classes for exists and count

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4597 from ghalliday/issue9602

HPCC-9602 Avoid temporary variables when returing simple expressions

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4595 from ghalliday/issue9627

HPCC-9627 Remove unused resourcing link type

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4594 from ghalliday/issue9597

HPCC-9597 Combine multiple ds[1].<field> into a single row

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9593 Added documentation pertaining to cmake structure

Cmake structure outline and brief documentation 1st draft.

Signed-off-by: Philip Schwartz <philip.schwartz@lexisnexis.com>

Reworked a bit prior to pulling.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9593 Added documentation pertaining to cmake structure

Reworked into standard filenames and add rst formatting.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4599 from richardkchapman/cmake_docs

Cmake docs

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9626 - Clear up various issues spotted by valgrind

A few minor leaks and new/free mismatches

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9633 - Watchdog crash if collects after shutdown

If the watchdog happens to be about to collect progress/stats
when thor is shutting down, it could crash. Check if stopping
after sleep and quit before collecting

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9634 Uninitialized members of ThorDataLinkMetaInfo

If a splitter has a null input (because pruned) the null input
implementation had uninitialized ThorDataLinkMetaInfo members.
I don't think any were of consequence, the critical members
totalRowsMin/Max were initialized.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9635 - Ensure LIMIT rowLimit member is initialized

rowLimit was not initialized until start()
getMetaInfo can be called before start, which would have
led to the wrong totalRowsMax being set

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9636 OUTPUT(module) not outputting DICTIONARY attributes

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>