Merge pull request #2188 from richardkchapman/precise-workaround-2187

gh-2187 hpcc-init fails to stop thor on Ubuntu 12.04

Reviewed-By: Jake Smith <jake.smith@lexisnexis.com>
#2242 Removed all "stream" references

The feature is now called hdfs/hpcc connector, and this name is reflected in
all user visible files. Internal build processes also reflect this name shift.

To build this feature, turn this flag on: USE_HDFSCONNECTOR

Signed-off-by: Rodrigo Pastrana <Rodrigo.Pastrana@lexisnexis.com>
Merge pull request #2249 from rpastrana/replace_streaming_labels

Replace streaming labels

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
#2243 H2H logs to standard hpccsystems log location

The H2H feature now logs to the standard hpccsystems log location.
HPCC environment.conf variables are used to determine appropriate location.
New component "mydataconnector" dir is created and logs written there in
the following format: mydataconnectors/HDFSCONNECTOR.$nodeid.$PID.log.

Signed-off-by: Rodrigo Pastrana <Rodrigo.Pastrana@lexisnexis.com>
The H2H feature now logs to the standard hpccsystems log location.
HPCC environment.conf variables are used to determine appropriate location.
New component "mydataconnector" dir is created and logs written there in
the following format: mydataconnectors/HDFSCONNECTOR.$nodeid.$PID.log.

Removed ability to overwrite h2h log location, per Phil's comment.

Signed-off-by: Rodrigo Pastrana <Rodrigo.Pastrana@lexisnexis.com>
Using sizeof(record, max) to determine records' max length, which is used
to determine termination threshold in case of data format error.
This replaces the use of sizeof(record), which didn't allow the use of var
len record definitions.

Signed-off-by: Rodrigo Pastrana <Rodrigo.Pastrana@lexisnexis.com>
Merge pull request #2265 from rpastrana/h2h_allow_varlen_CSV_pipein

H2h allow varlen csv pipein

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge branch 'move_h2h_logs_to_standard_loc' of https://github.com/rpastrana/HPCC-Platform into rpastrana-move_h2h_logs_to_standard_loc

Conflicts:
	plugins/dataconnectors/hdfsconnector/hdfspipe.in

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
GH#2276 Created guide for HDFS connector

Signed-off-by: JamesDeFabia <James.Defabia@LexisNexis.com>
Prevent FROMXML being forced to execute inside a graph

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #2277 from JamesDeFabia/HDFSConnectorGuideRevisions

Hdfs connector guide revisions

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com> 
Add progress of loop (counter)

See mix/max/skew details of loop progress
Add ability to turn on logging for child queries, which are normally
supressed

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Remove blocking progress crit

a legacy critical block was blocking graph progress updates (and watchdog),
whilst child queries were being processed

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Remove misleading CThorExpandingRowArray constructor

More trouble than it was worth, remove constructor which assumes row
interfaces is same as activities. There were a couple of cases there were
wrong. Probably had no consequence, as array row interface only used if rows are
serialized/allocated within array. In cases where they are not spilling
derivities that doesn't happen.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Allow graph results to have spill priority

This was mainly to address a problem with an attempt to spill the counter
result, which had no serializer and hence crashed. But don't want it to try
to spill the counter result anyway (as always 1 row).

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge branch 'candidate-3.6.x' into candidate-3.8.x

Recent changes on 3.6.x branch (H2H-related) merged to 3.8

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2172 from g-pan/master

Revised Version XML files (cherry picked from master)

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Fix issue with childif and graph progress

The time stat. collection for a child if activity was causing an exception,
if connected to anything other than the 1st input

Also fix potential race in progress collection in join and diskread

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Added CMakeLists.txt files to allow for system build of HDFS_to_HPCC_Connector doc
Merge pull request #2286 from g-pan/gpan-36x

Added CMakeLists.txt files to allow for system build of HDFS_to_HPCC

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>