HPCC-10186 Better Java Support for JAR Files and the ClassPath

Support a :classpath('xx') on the IMPORT command to add additional class paths
per imported function.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-9595 Implement generalized paging/cache for DFU files

This fix replaces the existing paging/no cache function for
browsing logical files with new paging/cache function. The
MDFS_ITERATE_FILTEREDFILES flag and processing code are added to
support more filters on dali server side. In ESP side, a new method
getLogicalFilesSorted() and supporting code are added to set/send
the MDFS_ITERATE_FILTEREDFILES request with filters to dali server,
read the response into IPropertyTreeIterator for sorting, and sort/
cache the logical files. The old paging code is moved into a new
method which will be called for legacy dali.

3 minor changes: (1) Inside the response, replace file attribute
IsKeyFile to ContentType. (2) In dfu.xslt, rename the sort field
'Size' to 'FileSize'. (3) In dfu_search.xslt, merge the Sort Filter
FirstNType with FileSize.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
HPCC-10699 Configuration: Add support for esp service plugins

  - add esp service plugin support to
     ./opt/HPCCSystems/componentfiles/plugins/<service_name>/<config files>

  - service plugins may require configuration files including:

      - buildset.xml : explicit services configured through ConfigMgr
         Installed into
         ./opt/HPCCSystems/componentfiles/configxml/plugins/<service_name>/

      - genenvrules.conf : explicit services that should not be
         generated by default. Installed into
         ./opt/HPCCSystems/componentfiles/configxml/plugins/<service_name>/

      - cgencomplist.xml : implicit services share an explicit service
         component name with a custom xsl, explict services define their
         own component name. Installed into
         ./opt/HPCCSystems/componentfiles/configxml/plugins/<service_name>/

      - esp_service_<service_name>.xsd : explicit services configured
         through ConfigMgr. Installed into
         ./opt/HPCCSystems/componentfiles/configxml/

      - esp_service_<service_name>.xsl : explicit and implicit services
         requires at a minimum binding info.  Refer to WS-SMC section in
         esp_service_WsSMC.xsl as an example of what is required.
         Installed into
         ./opt/HPCCSystems/componentfiles/configxml/atmark_temp/

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-10775 Set job priority for multiple jobs in one ESP call

The existing code sets one job priority per ESP call. In the new
ECLWatch UI, multiple jobs may be selected to change job priority.
Multiple ESP calls have to be sent at about the same time. After
several successful calls, ESP does not respond. That is because
multiple threads try to access the same job queue at the same time.
One thread locks the job queue before previous thread updates the
priority and releases the lock. This fix moves the code of lock()
and updatePriority() into a critical section. A thread cannot
access the code before another thread leaves the code.

In this fix, I also add the code to support batch function which
allows to set job priority for multiple jobs in one ESP call.

This fix also depreciates two variables from SetJobPriority
request since they are not used.
HPCC-10186 Better Java Support for JAR Files and the ClassPath

Fixes following code review

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10699 Configuration: Add support for esp service plugins

   - Fix dead-lock bug
   - Improve logging
   - Fix include issue in cmake files
   - Fix based on review
	- Use Macros for in cmake files
	- minor refactor

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
HPCC-10871 ECL helper version incompatible

Earlier changes in this file SHOULD have increased this version.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10775 Fix typo
HPCC-10874 Incorrect logic in deleteLRUPersists

Picked up by coverity scan - a continue that was intended to redo an outer
loop is in fact working on an inner loop. In practice, the difference is
likely to be undetectable.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-0000 WIP

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10875 Uninitialized member variables in ccdserver.cpp

All were in fact initialized before use, but this helps prevent warnings from
Eclipse and Coverity (and possible future errors).

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10876 Fix various issues highlighted by coverity scan

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #5417 from afishbeck/publishNoForeign

HPCC-10864: Command option to fail publish if query has foreign files

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
HPCC-10298 CSV auto discovery should check the validity and uniqueness of the
discovered field name.

Add code for validate the filed name and check its uniqueness.
    All invalid character change to '_'.

    If fieldname isn't unique then add '_'+<field index>+'_'+<timestamp>
    postfix.

Signed-off-by: Attila Vamos <attila.vamos@gmail.com>
HPCC-10896 - Lookup Join leaking rows in hash table.

The overloaded reset() of the std. lookup was resetting the
hash table size, before the constructor had a chance to free
the rows within the HT, therefore they leaked.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
HPCC-10891 Create /var/log/HPCCSystems if it doesnot exist
HPCC-10829 Removing files from roxie may not remove files from hpcc-data2

Something of a hack, but will be better 99% of the time, and no worse the
remaining 1%.

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #5435 from richardkchapman/roxie-data-delete-copy

HPCC-10829 Removing files from roxie may not remove files from hpcc-data

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
Merge pull request #5433 from jakesmith/hpcc-10896

HPCC-10896 - Lookup Join leaking rows in hash table.

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
HPCC-10846 Allow #option ('saveCppTempFiles') to preserve C++ files.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>