FEATURE - support for reading directly from zip and other archives

Similar to the way that we support reading from git repositories, and
using the same hooks and filename format.

Uses libarchive.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Fix internal error on complex nested field selection

In some circumstances it was possible to get the following error:
Attempting to lookup field <name> in a dataset which has no active element.
This is preventing another option being enabled that would solve some
problems with conditional evaluation of expressions.

The root cause is the handling of no_select.  A no_select should have a
newAtom attribute associated with it if it is selecting a field from a
new row.  So a[1].b.c is represented as
  select(select(a[1],b,new),c)

Some places were incorrectly checking if the current select had an
associated new attribute instead of checking if the root select had
one.

no_select(no_selectnth) now always has a newAttribute associated with it.
This simplifies and optimizes the processing of unnormalized trees.

This also contains a fix for hqliproj projecting a datarow, and better
expansion of (a.b.c.d within transforms)

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
FEATURE - support for reading directly from zip and other archives

Refactored zip and git file hook functionality to use dyaically loaded plugin
library structure.
Default USE_LIBARCHIVE of fon Windows, to avoid breaking everyone's builds
Update lastPos properly for more accurate detection of unsupported seek
requests.
Removed some unnecessary tracing.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2179 from ghalliday/newselector

Fix internal error on complex nested field selection

Reviewed-By: Renato Golin <rengolin@hpccsystems.com>
Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
gh-2182 Pkg file delete should not require a local copy of the pkg file

Signed-off-by: Stuart Ort <stuart.ort@lexisnexis.com>
Merge pull request #2183 from stuartort/pkg_delete_nofileneeded

gh-2182 Pkg file delete should not require a local copy of the pkg file

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
SelfJoin object leak, cause error if spilled

SelfJoin was leaking the lhs input stream, as a result, if a local selfjoin
had spilt whilst sorting, the spill files remained opened and an error
ensued.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Release extra tests for nested selects

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #2186 from ghalliday/eg4

Release extra tests for nested selects

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2185 from jakesmith/selfjoin-leak

SelfJoin object leak, cause error if spilled

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge latest changes from candidate-3.8.x

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2159 from richardkchapman/zipinclude

FEATURE - support for reading directly from zip and other archives

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge latest changes from candidate-3.8.x to master

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>