Various index count and aggregate fixes

Remove projects from count(keyed-limit(project(index)))
  A missing case from a switch statement meant projects weren't
  stripped when there was a keyed limit on the index read.

Reduce the generated code size for an index count/exists
  The numValid(size, ptr) member was never called for an index, so default
  it to throwing an exception in the base class, and clean up numValid(ptr)
  code.

CHOOSEN(LIMIT(index-read))
  The choosen was not being combined within the compound indexread if there
  was a limit.  It is however valid to do so since the choosen is done last.
  (A limit cannot be combined when there is an existing choosen.)
  This was unfortunate because it was frustrating  another optimization.
  count(index-read) > n being converted to count(choosen(index-read,n+1)) > n
  (This is temporarily retained, but will be removed in a subsequent commit).

IF(count(x) > 0, x, y) optimization
  Currently the intention is to perform a count-index on x before reading x.
  However if x contains a limit,skip it is going to need to read the entire
  result, so don't perform the count separately.

Don't generate count(index) if there is a skip limit

Optimize count(x) > 0 to exists(x)
  Instead of (count(x) > 0) becoming count(choosen(x, 1)) > 0 it is now
  transformed to exists(x) - which is either as efficient, or more
  efficient as the count representation.

Remove no_notexists since never created

Terminate hthor exists aggregate activity early

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
ISSUE:#1714 Restrict special chars from comp names

configgen will indicate and error when encountering
special characters in component names.

configmgr will pop-up an error when users attempt
to enter special characters for component names

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
gh-1997 Fix usability issues for User Permission view

Existing user permission page has two usability issues.
(1) multiple permission tables are displayed in one page
which makes the page difficult to read. (2) for each
permission table (permission group), there is an Add
button which allows to add more permission to that group.
The page shows the Add button and the reource list even
though no more permission can be added for some resources.
This fix only shows the first permission table and the
expend links for other tables. This fix also filters out
the resources from the resource list if no more permission
can be added. If the resource list is empty, the Add
button will not be displayed for that permission table.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
ISSUE:1190 envgen error checking for integer params

Add integer validation for command line
parameters.

Add meaningful error messages when integer
validation fails.

Signed-off-by: Gleb Aronsky <gleb.aronsky@lexisnexis.com>
cannotRemove refactoring

Renamed cannotModify to canModify, same with remove,
and moving the logic inside the file, since it didn't
belong to directories anyway. Instead of using flags,
now using what the methods mean and one call the other
when need.

There is a kludge in Dir::cannotRemove() that will be
cleaned in a subsequent patch, as soon as doRemoveEntry
gets a file as argument, not a filename.

If I manage to finish this second step before this pull
request is pulled, I'll merge them two.
Add superfile name to error
This commit only reformats the code.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
gh-1776 EclWatch pauses/clears queues for Roxie/HThor

A drop down menu is added for Roxie/HThor cluster to
pauses/clears queues. I also improved the index.xslt:
better confirmation messages; removed old code lines
which are no longer used; changed some function names
and variable names for better read.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
Revised based on review

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
LDAP check for non-expiring passwords

Even when the domain policy specifies that account passwords expire after
a certain time period, individual accounts can still be specified as never
expiring (admin accounts, for instance).  Add an addition test for this
case when a user logs in.

Signed-off-by: William Whitehead <william.whitehead@lexisnexis.com>
Ensure subgraphs of a global loop are global

If the loop is global and executing subgraphs synchronously, then ensure all
subgraphs of the loop are marked global. Otherwise, the locally executed
subgraphs, will call back to the master and can cause race conditions, when
creating the graph on the master.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Guard against potential race condition

For child graphs, that are initiating creation on master, there's a possible
race condition if multiple slaves initialize the graph and/or request graph
initialization data at the same time

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Stop LIMIT SKIP|ONFAIL attributes being ignored

On very rare occasions the SKIP/ONFAIL attribute of a limit could
be ignored.  It occurred when a limit was incorrectly evaluated
using inline code, rather than executed in a child query.  This fix
ensures it is executed in the correct place.

Example was SET(LIMIT(ds, ONFAIL), field).

There were no other examples in any archived queries.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Add ConfigMgr support for ws_ecl VIPS

Added ability to configure VIPS in ws_ecl esp service via ConfigMgr.
Updated XSD files to read in the new fields. Updated the XSL files to generate the VIPS into the esp configuration.
Also, updated the handling of the xpathType attribute to allow resolution against multiple values.
This was needed to handle the requirement that each Roxie only appear in atmost one VIP.

Fixes gh-2077

Signed-off-by: Sridhar Meda <sridhar.meda@lexisnexis.com>
A couple of mptags not deallocated by acts

Global sort and join, allocated an mptag but neglected to deallocate.
This was noticed, after Thor had been running for a good while and the leaks
had built up, eventually triggering a 'Out of message tags' error.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Potential failure in overflow buffer code

Some thread unsafeness in the overflow buffering code, could lead multiple
readers to cause the readersInUse count to go astray, and result in a
spurious assert later and/or other potential issues

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>
Fix gh-#2071 DFS exception when View Data File

When ldap is turned on and a user trys to view a data file,
the DFS exception '3 Lookup access denied for scope' shows.
The exception is thrown by the ViewFileWeb object when it
does a file look-up. Since the data file has been resolved
by WsDFU service, WsDFU service should pass the solved file
point to ViewFileWeb object and the ViewFileWeb may use the
point without doing the file loop-up. This fix adds a
method to the ViewFileWeb class. The method allows the
solved file point being given to the ViewFileWeb.

I also revise the code in WsDFU service for better
functionalities.

Signed-off-by: Kevin Wang <kevin.wang@lexisnexis.com>
Merge pull request #2102 from smeda/issue2077

gh-2077 Add support in ConfigMgr for configuring VIPS in ws_ecl esp service

Reviewed-By: Richard Chapman <rchapman@hpccsystems.com>
Delay reading files in the include directories

Improves the startup time for eclcc in non-legacy mode.
The call to contents->length() had the side-effect of loading the file
into memory (since it may need to convert formats etc.).  This fix
delays the call to length() until it is needed.

Signed-off-by: Gavin Halliday <gavin.halliday@lexisnexis.com>
Splitter in child query race

Small window, where a downstream window might ask for meta info from
splitter output, before inputs have been reinitialized in reruns of a child
query.
Clear out old splitter input instances at end of query execute.

Signed-off-by: Jake Smith <jake.smith@lexisnexis.com>