Split branch for 3.8.x closedown

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2154 from richardkchapman/gitinclude

FEATURE - hook-based git file reading

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
FEATURE - hook-based git file reading

Another attempt to fix compile errors/warnings that showed up on vc++ compiler,
due to rather strange c++ handling of a trailing \ in a comment.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2162 from richardkchapman/gitinclude

FEATURE - hook-based git file reading

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #2163 from richardkchapman/gitinclude

FEATURE - hook-based git file reading

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Dataset count range starts at 1
Fix problem using unsigned numbers in #line

Some compilers appear to have a problem using unsigned constiants
(e.g., 5U) in a #line directive.  This fix ensures the unsigned
extension isn't added to the numbers.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Revert superfile regression, hint what to do to fix later
Merge pull request #2171 from ghalliday/unsignedline

Fix problem using unsigned numbers in #line

Reviewed-By: Renato Golin <rengolin@hpccsystems.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2165 from rengolin/dataset

Dataset count range starts at 1

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
FEATURE - support for reading directly from zip and other archives

Similar to the way that we support reading from git repositories, and
using the same hooks and filename format.

Uses libarchive.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2167 from rengolin/dfs-daregress

Revert superfile regression, hint what to do to fix later

Reviewed-By: Jake Smith <jake.smith@lexisnexis.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge recent changes from branch candidate-3.8.x to master

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
gh-2166 Add the ability to properly activate / deactive package sets

Signed-off-by: Stuart Ort <stuart.ort@lexisnexis.com>
Fix internal error on complex nested field selection

In some circumstances it was possible to get the following error:
Attempting to lookup field <name> in a dataset which has no active element.
This is preventing another option being enabled that would solve some
problems with conditional evaluation of expressions.

The root cause is the handling of no_select.  A no_select should have a
newAtom attribute associated with it if it is selecting a field from a
new row.  So a[1].b.c is represented as
  select(select(a[1],b,new),c)

Some places were incorrectly checking if the current select had an
associated new attribute instead of checking if the root select had
one.

no_select(no_selectnth) now always has a newAttribute associated with it.
This simplifies and optimizes the processing of unnormalized trees.

This also contains a fix for hqliproj projecting a datarow, and better
expansion of (a.b.c.d within transforms)

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
FEATURE - support for reading directly from zip and other archives

Refactored zip and git file hook functionality to use dyaically loaded plugin
library structure.
Default USE_LIBARCHIVE of fon Windows, to avoid breaking everyone's builds
Update lastPos properly for more accurate detection of unsupported seek
requests.
Removed some unnecessary tracing.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2179 from ghalliday/newselector

Fix internal error on complex nested field selection

Reviewed-By: Renato Golin <rengolin@hpccsystems.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
gh-2182 Pkg file delete should not require a local copy of the pkg file

Signed-off-by: Stuart Ort <stuart.ort@lexisnexis.com>
Merge pull request #2183 from stuartort/pkg_delete_nofileneeded

gh-2182 Pkg file delete should not require a local copy of the pkg file

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
SelfJoin object leak, cause error if spilled

SelfJoin was leaking the lhs input stream, as a result, if a local selfjoin
had spilt whilst sorting, the spill files remained opened and an error
ensued.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>