Merge pull request #1982 from pschwartz/dafilesrv_fixes

Added temporary ulimit settings for dafilesrv.

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #1980 from richardkchapman/git-tag-multiple

Git tag multiple

Reviewed-by: Gavin Halliday <ghalliday@hpccsystems.com>
gh1967 Don't assume roxie response case matches queryset query name

Fixes gh-1967.

Signed-off-by: Anthony Fishbeck <Anthony.Fishbeck@lexisnexis.com>
ChooseN issues in child queries

1) Choosen within a global child query, e.g. a global loop query, would
deadlock on 2nd iteration. Due to a member variable not being reset on restart, choosen was only sending counts to the master on the iteration.

2) helper getLimit / numToSkip, weren't being tested in start(), so if they
were computed values, and being called repeatedly (i.e. in a child query),
then the 1st value was being returned.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Fix problems with hthor not caching allocators

Various pieces of code involved in loops were recreating an allocator
each time which lead to vast numbers of allocators being created.
With the new heap code it resulted in an O(n^2) clean up operation.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #1985 from jakesmith/childfirstn

ChooseN issues in child queries

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #1973 from afishbeck/json_response

gh1967 Don't assume roxie response case matches queryset query name

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #1986 from ghalliday/hthorallocator

Fix problems with hthor not caching allocators

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Replacing TAKtemptable for TAKinlinetable

Code generation now creating inline tables (rather than
temp tables), including temp rows, since numRows() is
being used by the engine to limit size.

Added some deprecated comments, so we can easily find
and remove all deprecated code in the next major release.
gh-1495 Thor Master is now assigned to one of the support nodes

Thor Master is assigned to one of the support nodes and will no longer overlap with Thor Slaves.


Signed-off-by: Sridhar Meda <sridhar.meda@lexisnexis.com>
Use correct architecture dependent hadoop lib path
USE ARCH64BIT cmake var to generate correct hadoop lib path, based on
uris used by hadoop: Linux-amd64-64 or Linux-i386-32.

Signed-off-by: Rodrigo Pastrana <Rodrigo.Pastrana@lexisnexis.com>
Failure in User Group Delete

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
gh-1956 Rename instance directory when component name is changed

As the instance directory is based on the name of the component, change the instance directory when the name of the component is changed via ConfigMgr.

Signed-off-by: Sridhar Meda <sridhar.meda@lexisnexis.com>
Always add a U suffix to an unsigned constant

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #1991 from rpastrana/build_with_32bit_hdfs

Build with 32bit hdfs

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Increase ECL and Activity versions

Due to the new construct DATASET(COUNT) and the recent changes
in InlineTable (respectively), both the ECL and the Activity
version numbers are being increased.

Increase the MIN_ACTIVITY_INTERFACE_VERSION to match once the
old TempTableArg interface is removed for good.
Various index count and aggregate fixes

Remove projects from count(keyed-limit(project(index)))
  A missing case from a switch statement meant projects weren't
  stripped when there was a keyed limit on the index read.

Reduce the generated code size for an index count/exists
  The numValid(size, ptr) member was never called for an index, so default
  it to throwing an exception in the base class, and clean up numValid(ptr)
  code.

CHOOSEN(LIMIT(index-read))
  The choosen was not being combined within the compound indexread if there
  was a limit.  It is however valid to do so since the choosen is done last.
  (A limit cannot be combined when there is an existing choosen.)
  This was unfortunate because it was frustrating  another optimization.
  count(index-read) > n being converted to count(choosen(index-read,n+1)) > n
  (This is temporarily retained, but will be removed in a subsequent commit).

IF(count(x) > 0, x, y) optimization
  Currently the intention is to perform a count-index on x before reading x.
  However if x contains a limit,skip it is going to need to read the entire
  result, so don't perform the count separately.

Don't generate count(index) if there is a skip limit

Optimize count(x) > 0 to exists(x)
  Instead of (count(x) > 0) becoming count(choosen(x, 1)) > 0 it is now
  transformed to exists(x) - which is either as efficient, or more
  efficient as the count representation.

Remove no_notexists since never created

Terminate hthor exists aggregate activity early

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge latest changes from candidate-3.6.x into candidate-3.6.2

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
Merge recent changes from candidate-3.6.2 into candidate-3.6.x

Manually excluded CMakeLists.txt version number change

Signed-off-by: Richard Chapman <rchapman@hpccsystems.com>
gh1993 EclWatch Crash when searching WUs by Ecl text

Multiple problems are fixed here: when no WU matches
the search, the calculation based on page size causes
the crash; if number of matching WUs is less than page
size, the calculation based on page size also causes a
crash; we should use the WU id of matching WUs to get
WU information; Owned IEspECLWorkunit without
getClear() also causes a crash.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>