Various index count and aggregate fixes

Remove projects from count(keyed-limit(project(index)))
  A missing case from a switch statement meant projects weren't
  stripped when there was a keyed limit on the index read.

Reduce the generated code size for an index count/exists
  The numValid(size, ptr) member was never called for an index, so default
  it to throwing an exception in the base class, and clean up numValid(ptr)
  code.

CHOOSEN(LIMIT(index-read))
  The choosen was not being combined within the compound indexread if there
  was a limit.  It is however valid to do so since the choosen is done last.
  (A limit cannot be combined when there is an existing choosen.)
  This was unfortunate because it was frustrating  another optimization.
  count(index-read) > n being converted to count(choosen(index-read,n+1)) > n
  (This is temporarily retained, but will be removed in a subsequent commit).

IF(count(x) > 0, x, y) optimization
  Currently the intention is to perform a count-index on x before reading x.
  However if x contains a limit,skip it is going to need to read the entire
  result, so don't perform the count separately.

Don't generate count(index) if there is a skip limit

Optimize count(x) > 0 to exists(x)
  Instead of (count(x) > 0) becoming count(choosen(x, 1)) > 0 it is now
  transformed to exists(x) - which is either as efficient, or more
  efficient as the count representation.

Remove no_notexists since never created

Terminate hthor exists aggregate activity early

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
ISSUE:#1714 Restrict special chars from comp names

configgen will indicate and error when encountering
special characters in component names.

configmgr will pop-up an error when users attempt
to enter special characters for component names

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
gh-1997 Fix usability issues for User Permission view

Existing user permission page has two usability issues.
(1) multiple permission tables are displayed in one page
which makes the page difficult to read. (2) for each
permission table (permission group), there is an Add
button which allows to add more permission to that group.
The page shows the Add button and the reource list even
though no more permission can be added for some resources.
This fix only shows the first permission table and the
expend links for other tables. This fix also filters out
the resources from the resource list if no more permission
can be added. If the resource list is empty, the Add
button will not be displayed for that permission table.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
ISSUE:1190 envgen error checking for integer params

Add integer validation for command line
parameters.

Add meaningful error messages when integer
validation fails.

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
cannotRemove refactoring

Renamed cannotModify to canModify, same with remove,
and moving the logic inside the file, since it didn't
belong to directories anyway. Instead of using flags,
now using what the methods mean and one call the other
when need.

There is a kludge in Dir::cannotRemove() that will be
cleaned in a subsequent patch, as soon as doRemoveEntry
gets a file as argument, not a filename.

If I manage to finish this second step before this pull
request is pulled, I'll merge them two.
Add superfile name to error
This commit only reformats the code.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
gh-1776 EclWatch pauses/clears queues for Roxie/HThor

A drop down menu is added for Roxie/HThor cluster to
pauses/clears queues. I also improved the index.xslt:
better confirmation messages; removed old code lines
which are no longer used; changed some function names
and variable names for better read.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
Revised based on review

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
Fix #1515 invalid c++ if csvread contians a child query

If a compound csv read (e.g., containing a project) contained
a colocal child query, the data from the parent row wasn't correctly
passed to the child activities.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Adjust LDAP password expiration from UTC to local time

Currently password expiration is store in UTC while the code that computes
time until expiration expects it to be local.

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
LDAP check for non-expiring passwords

Even when the domain policy specifies that account passwords expire after
a certain time period, individual accounts can still be specified as never
expiring (admin accounts, for instance).  Add an addition test for this
case when a user logs in.

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
Ensure subgraphs of a global loop are global

If the loop is global and executing subgraphs synchronously, then ensure all
subgraphs of the loop are marked global. Otherwise, the locally executed
subgraphs, will call back to the master and can cause race conditions, when
creating the graph on the master.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Guard against potential race condition

For child graphs, that are initiating creation on master, there's a possible
race condition if multiple slaves initialize the graph and/or request graph
initialization data at the same time

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Merge pull request #2112 from ghalliday/issue1515

Fix #1515 invalid c++ if csvread contains a child query

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2104 from RussWhitehead/ldap

Adjust LDAP password expiration from UTC to local time

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Ensure jobtemp spill files are replicated

There can be a large period of time between thor executing different
graphs in a workunit, e.g., if other high priority jobs take precedence.
This means it is more likely that a machine may have been swapped in
the meantime.  Therefore any file which has lifetime beyond a single
graph should be replicated - otherwise the entire job may need to be
rerun.

Fixes #1944.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Common up null datasets with minor differences

A minor improvement to the generated code for a few queries.
The type of a dataset may contain an annotation indicating the name of
the original record.  If one null dataset expression contains an
annotation and another doesn't they should be treated as identical.

(Had a minor effect on a couple of queries).

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Merge pull request #2107 from ghalliday/nullconstant

Common up null datasets with minor differences

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Merge pull request #2118 from ghalliday/issue1944

Ensure jobtemp spill files are replicated

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>