HPCC-9783 - Fix issue with prefix self join, HINT(parallel_match)

The self join helper, doesn't handle prefix joining, so the
regular join helper is used if a prefix join is detected.
However, when HINT(parallel_match) is used and the multi threaded
helper is overlayed, it incorrectly switched some code based on
selfjoin. As a consequence it mismatched and lost records.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9817 Fix issues comparing against a NULL unicode string

Strange as it may seem, the ICU compare functions check for NULL
pointers as parameters, and sometimes abort early giving the wrong result.
See http://sourceforge.net/mailarchive/message.php?msg_id=29406824 for
a similar issue.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-9819 Fix build break in windows caused by HPCC-9789

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #4722 from ghalliday/issue9819

HPCC-9819 Fix build break in windows caused by HPCC-9789

Merged following review by Jake
HPCC-9823 Docs: Fix VM Requirement

Fix HPCC-9823 Docs: Fix VM Requirement.
Change requirement for VMware from 3.0
or later to 4.0 or later.

Signed-off-by: g-pan <greg.panagiotatos@lexisnexis.com>
HPCC-9822 Docs:Add linebreak logic

Fix HPCC-9822 Docs:Add linebreak logic
Added linebreak logic to fo(pdf)
html generation not automated yet.

Signed-off-by: g-pan <greg.panagiotatos@lexisnexis.com>
HPCC-9825 ECL Watch Nested Dataset Result Corruption

Rows with a single item would not be treated as an array.

Fixes HPCC-9825

Signed-off-by: Gordon Smith <gordon.smith@lexisnexis.com>
HPCC-9346 - Slightly improve the 'No active dali' error

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-9828 - Resolve package watcher notification/reload deadlock

Multiple CDaliPackageWatcher's could be notified in parallel
on different threads. Each invoked a reload of all packages in
the manager, which could result in deadlock, if two threads
both tried to unregister a watcher that was already in
it's notify method.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8979 - daliadmin dalilocks misreporting superfiles

Parsing the lock info, did not take account of SuperFiles.
As a cosequence, superfiles were listed as locks on their
parent scope.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4689 from jakesmith/hpcc-9783

HPCC-9783 - Fix issue with prefix self join, HINT(parallel_match)

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-9785 - Optimize the parallel and unsorted join helpers

The multi threaded join helpers were very inefficient in
processing the match/transform of groups and delivery the
results to the output.

The main 2 changes are:
1) Ensure each thread has it's own allocator, to avoid heavy
contention in the memory manager
2) Implement an efficient interface for multiple threads to
write to and a reader read from (a pipe with multiple writers).
This avoids heavy contetion and delays on mutexs and semaphores
that was being hit inside SimpleInterThreadQueueOf

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Changes folowing review and further testing

+ Add joinHelper_threads option
+ Add granularity, num threads and limit options
+ Correct a comment
+ Change when CThorSpillableRowArray::flush shuffles ptr array,
with behaviour here, it could cause the row array to keep growing
leaving lots of empty space behind it.
+ Fixed a couple of issue if aborting
+ add array appendRows method for efficiency and clarity
+ add a regression test joinmulti.ecl

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #4732 from jakesmith/hpcc-9346

HPCC-9346 - Slightly improve the 'No active dali' error

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #4726 from g-pan/H9823-VMReq

HPCC-9823 Docs: Fix VM Requirement

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #4734 from jakesmith/hpcc-9828

HPCC-9828 - Resolve package watcher notification/reload deadlock

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #4727 from g-pan/H9822-br

HPCC-9822 Docs:Add linebreak logic

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #4735 from jakesmith/hpcc-8979

HPCC-8979 - daliadmin dalilocks misreporting superfiles

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-9826 Update to dojo 1.9.1

Also add D3/TopoJson library.

Fixes HPCC-9826

Signed-off-by: Gordon Smith <gordon.smith@lexisnexis.com>
HPCC-9802 Delete logical file using removeEntry(), not detach()

Existing ECLWatch code deletes a logical file using detach(). When
the file is associated with a superfile, the superfile may not be
able to be deleted. This fix calls removeEntry() to delete a file.
The removeEntry() should check whether the file can be deleted or
not if it is a subfile. This fix also cleans the code.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>