Merge remote-tracking branch 'origin/candidate-3.8.x' into candidate-3.8.6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Community Edition 3.8.6-1 Gold

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge remote-tracking branch 'origin/candidate-3.8.x' into candidate-3.8.6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Community Edition 3.8.6-2 Gold

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge remote-tracking branch 'origin/candidate-3.8.x' into candidate-3.8.6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Community Edition 3.8.6-3 Gold

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8129 Fix bug generating an optimized hash function on a child dataset

For DEDUP(ds, x,y,z), if the list of fields contained a child dataset the
hash function generated when targetting Thor was incorrect, and could
cause thor to crash.  It could possibly be triggered in other situations
as well.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge remote-tracking branch 'origin/candidate-3.8.x' into candidate-3.8.6

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Community Edition 3.8.6-4 Gold

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
HPCC-8073 Edit documentation about Roxie packages, fix typo

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
HPCC-8120 Firstn deadlock

FirstN in combination with certain upstream activities can caused
a deadlock. The problem occurs, when the firstN lookahead hits
it's limit on some nodes and attempts to a) signal the next node
and b) wait for the reader to finish processing or signal stop
the rows it has buffered.
Either one can block, which will prevent the lookahead signalling
'stop' to input it has been reading on. In some circumstances
those inputs will be blocked themselves, e.g. because their
buffers are full, waiting for either read or stop signal.

Spin off the signal to next node and local row cache flush to
a separate thread, and allow the input stop to be called.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8151 Group dedup all, was stripping end of group markers.

The non-hashing variety of group dedup all, was not returning the
end of group markers to the output, i.e. it returned the results
effectively as one large group.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #3565 from ghalliday/issue8129

HPCC-8129 Fix bug generating an optimized hash function on a child dataset

Reviewed-By: Renato Golin <rengolin@hpccsystems.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3569 from jakesmith/firstn-deadlock

HPCC-8120 Firstn deadlock

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8073 Additional edits of Packages chapter from code review

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
Merge pull request #3575 from JamesDeFabia/ImprovePackagesChapter

HPCC-8073 Improve documentation about Roxie packages

Reviewed-By: Stuart Ort <stuart.ort@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3588 from jakesmith/dedupgroup

HPCC-8151 Group dedup all, was stripping end of group markers.

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com> 
Revert "HPCC-8048 Add Remove Super File to transactions"

This reverts commit dd759d337bf45c1f9449ed9914f4ae86812c4abf.

Delay the superfile transaction changes until 4.0, as they are not
really appropriate for a point release (3.10.2) and will not all be
ready for 3.10.0

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge branch 'candidate-3.10.x'

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8048 Add Remove Super File to transactions

Adding removal of super files inside transactions, with full
roll-back capabilities. Tests included.

Signed-off-by: Renato Golin <rengolin@hpccsystems.com>