HPCC-8120 Firstn deadlock

FirstN in combination with certain upstream activities can caused
a deadlock. The problem occurs, when the firstN lookahead hits
it's limit on some nodes and attempts to a) signal the next node
and b) wait for the reader to finish processing or signal stop
the rows it has buffered.
Either one can block, which will prevent the lookahead signalling
'stop' to input it has been reading on. In some circumstances
those inputs will be blocked themselves, e.g. because their
buffers are full, waiting for either read or stop signal.

Spin off the signal to next node and local row cache flush to
a separate thread, and allow the input stop to be called.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8213 Disable sub-file checking on internal build

Signed-off-by: Renato Golin <rengolin@hpccsystems.com>
Merge pull request #3630 from rengolin/dfs-superbug

HPCC-8213 Disable sub-file checking on internal build

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3631 from jakesmith/firstn-deadlock-3.8.6

HPCC-8120 Firstn deadlock

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge remote-tracking branch 'origin/candidate-3.8.x' into candidate-3.8.6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Community Edition Release 3.8.6-6

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8216 Csvread,opt heading crash

If a csv read has OPT and HEADING attributes and the logical file
is missing, a crash will occur inside the master serialization
code for the activity

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #3636 from jakesmith/csvheadingopt-bug

HPCC-8216 Csvread,opt heading crash

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Community Edition Release 3.8.6-7

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8221 ESP wasn't correctly loading plugin definitions

Due to a change in the way scopes are loaded on demand, esp wasn't correctly
picking up symbols defined in plugins because they were still waiting to be
loaded.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #3639 from ghalliday/issue8221

HPCC-8221 ESP wasn't correctly loading plugin definitions

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-7944 Add setting, to allow master memory to be specified

Also fixes a bug, that if globalMemorySize was left blank
and slavesPerNode was >1, it would limit master memory to the
same calculated memory as each slave
(which is 75% physical / slavesPerNode)

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #3640 from jakesmith/mastermem-3.8.6

HPCC-7944 Add setting, to allow master memory to be specified

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8225 Fix global ENTH deadlock issue

Global Enth can deadlock if the input count is pre-known, and
the activity starts but no records are ever requested (nextRow).
This fix ensures that the get/send calls are made in stop()
if has not occured in nextRow()

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8220 Configmgr - Pluging path is not populated

- Populate pluginPath from attributes in
  Configmgr espsmc service

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
Merge pull request #3644 from jakesmith/enth-deadlock

HPCC-8225 Fix global ENTH deadlock issue

Reviewed-By: Gavin Halliday <gavin.halliday@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Community Edition Release 3.8.6-8

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #3647 from garonsky/issue-8220_PluginPath_in_esp_xml_not_populated

HPCC-8220 Configmgr - Pluging path is not populated

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-8234 Avoid deadlock on balanced splitter

If some of the outputs on a splitter are conditional and therefore
never started, a balanced splitter will deadlock when it reaches
it's buffering limit

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-8235 Ensure end-of-groups stripped for sorted join input

A grouped input to a global join, which was presorted by the join
keys, was not stripping the end of group markers off, as a
consequence it would stop reading the input at the first end of group.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>