HPCC-9572 - Fix hash dedup row leak after spill event

When hash dedup spills, it continues to use the bucket hash tables
for dedupping purposes. The memory manager callback to flush
the HT was not protected by a mutex, which meant the reset could
overlap with another thread adding new rows, causing the flush to
in effect lose those newly added rows.

Also, close all bucket spill streams at end of phase (instead of
waiting for each new level to start). Minor significance, but the
output streams will have some buffered rows, and it is better to
close and flush those out upfront, rather than leave them in memory
whilst the other phases are being handled.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4568 from jakesmith/hpcc-9572

HPCC-9572 - Fix hash dedup row leak after spill event

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-9628 Eclcc hangs at end if libarchive dynamically unloaded

In my tests setup, I have the libs directory on my plugins path, meaning that
eclcc will try to load libarchive as a plugin, then unload it when it finds it
is not.

This means that MODEXIT gets called before removeFileHook (due to bugs in the
modexit logic - it should really only be called on the last unload, not on any
unload).

Protect against that event by setting lock ptr to NULL.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9636 OUTPUT(module) not outputting DICTIONARY attributes

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4609 from richardkchapman/output-module-dictionary

HPCC-9636 OUTPUT(module) not outputting DICTIONARY attributes

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-9638 Filter "Clear" does not reset some drop down controls

Fixed TargetSelectWidget to detect "null".
Re-factored Select controls into TargetSelectWidget.

Fixes HPCC-9638

Signed-off-by: Gordon Smith <gordon.smith@lexisnexis.com>
HPCC-9621 IE8 + XP ECL Watch Issues

New ECL Watch does not display in IE 8 + XP.

Fixes HPCC-9621

Signed-off-by: Gordon Smith <gordon.smith@lexisnexis.com>
Merge pull request #4611 from GordonSmith/HPCC-9638

HPCC-9638 Filter "Clear" does not reset some drop down controls

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9631 - KeyedJoin may miss cached IKeyIndex's

Due to an uninitialized 'crc' variable, that the key index cache
uses to form a key for caching IKeyIndex's. KeyedJoin would
probably have been missing cache hits and consequently, creating
extra IKeyIndex's unecessarily

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge branch 'hpcc-9631' into candidate-4.0.0

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4592 from GordonSmith/HPCC-9621

HPCC-9621 IE8 + XP ECL Watch Issues

Reviewed-By: Miguel Vazquez <miguel.vazquez@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #4596 from richardkchapman/hook-lockup

HPCC-9628 Eclcc hangs at end if libarchive dynamically unloaded

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-8053 - Timeouts can cause superfile update inconsistencies

Locking timeouts in transactions can cause add/remove super file
inconsistencies, resulting in corrupt superfiles (wrong # of files
in meta info and/or subfiles with duplicate @num's)

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4613 from jakesmith/hpcc-8053b

HPCC-8053 - Timeouts can cause superfile update inconsistencies

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9545 ecl-packagemap-validate option to check subfiles exist in dfs

Adds a new --check-dfs option to ecl-packagemap-validate which will
verify that all subfiles in the packagemap are defined in the local
dfs.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
Merge branch 'pmValidateCheckDfs9545' into candidate-4.0.0

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9502 Option when spraying wildcards to fail if no matches

Add new boolean parameter called 'failIfNoSourceFile' with false default value
to
 - SprayFixed()
 - SprayVariable()
 - SprayDelimited()
 - SprayXml()

functions.

Add new command line parameter with same name (failIfNoSourceFile) to Dfuplus
too.

Extend DFUWorkUnit, SprayFixed and SprayVariable template structures with
a new field (same name as the new parameter) into ws_fs.ecm to handle
parameter passing from client side to server sides.

Signed-off-by: Attila Vamos <attila.vamos@gmail.com>
Merge branch 'HPCC-9502-feature' into candidate-4.0.0

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9660 Fix optimization of a DENORMALIZE with an empty rhs

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>