New version of some SQL used with quota enforcement that performs much
better at scale.

This new SQL and supporting mods in the ICAT function
(generateSpecialQuery) were provided by Giacomo Mariani and colleagues
at SuperComputing Applications and Innovation Department CINECA,
Bologna, Italy, after investigating SQL that was running very slowly.

I also checked the new SQL by hand to convince myself that the old and
new SQL statements are logically equivalent, although both of them are
a bit complicated.  It helps to work with the statements with a text
editor, breaking them up into components (for the new, by the UNIONs).
The new SQL is doing a UNION of 3 selects, the first of which is
handling group quotas on specific-resources, the second is group
quotas on total use, and the third is for both specific-resources and
total-usage for the particular user.  It seems to be logically
equivalent to the old version where the old had two OR conditions in
some nested conditions (the new form there's one OR, and it's in the
last select of the three).  One minor change from that provided is
removing a trailing ';' on the SQL; this is not an issue with Postgres
but may be with others.

The two queries in this ICAT function are used when quotas are being
enforced (i.e. by enabling quotas via the acRescQuotaPolicy rule in
the core.re file).  There's a setRescQuota function that is called
from the rsDataObjCreate API function, that calls chlGenQuery with
this special option (QUOTA_QUERY).

There are two SQL forms, but it's the second one, quotaQuery2, that is
used in practice (as called by the rsDataObjCreate API function).

I also made some minor changes to logging in the function, changing
some rodsLog calls that were LOG_ERROR to LOG_DEBUG and I added some
(also only at LOG_DEBUG (i.e. normally not logged)).

As part of this commit, I've also updated the quotaTest.pl script to
test this SQL.  See the instructions at the start of the script.  One
needs to enable the enforcement via core.re first, and then run
'quotaTest.pl enabled'.  But this test confirms that that SQL runs OK
and that that quotas are enforced in the various modes (for users or
groups, on particular resources or for total storage).  It's still
worthwhile to run 'quotaTest.pl' without the 'enabled' option (with it
off in core.re) to perform other tests.

Thanks to Giacomo Mariani and colleagues for this improvement.



git-svn-id: svn://irodssvn.ucsd.edu/trunk@5465 ea137bd6-aa86-47b6-b90d-b101d9f19a90
A couple minor changes for building with a C compiler (gcc).
We use a C compiler on AIX and whenever HPSS is included.

These C++'isms crept into code recently.

There are other issues C++/C issues we need to fix before the release.


I was able to easily test this on linux by just editing
config/plaform.mk, replacing g++ with gcc near the top:
CC=/usr/bin/gcc
CC_IS_GCC=1
#CCFLAGS=
LDR=/usr/bin/gcc




git-svn-id: svn://irodssvn.ucsd.edu/trunk@5466 ea137bd6-aa86-47b6-b90d-b101d9f19a90
databook rules

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5467 ea137bd6-aa86-47b6-b90d-b101d9f19a90
fix a gcc compiler warning

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5468 ea137bd6-aa86-47b6-b90d-b101d9f19a90
prevent redefinition of macros

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5469 ea137bd6-aa86-47b6-b90d-b101d9f19a90
For HDFS (Hadoop File System" Support, the following files



git-svn-id: svn://irodssvn.ucsd.edu/trunk@5470 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Also need to modify for HDFS


git-svn-id: svn://irodssvn.ucsd.edu/trunk@5471 ea137bd6-aa86-47b6-b90d-b101d9f19a90
struct definition parser; xml struct representation converter; code generation tool (requires libxslt and libxml4); generated code for session variables from struct definition; dynamic type checking for session variables; future coercion kind for session variable type backward compatibility; typechecker code refactoring; re debugger refactoring: new rule engine debugger data structures; rule engine events; AVU based attribute which turns on and off rule engine logging and microservices for setting and getting this attribute




git-svn-id: svn://irodssvn.ucsd.edu/trunk@5472 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Add a ms param type for data obj info struct

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5473 ea137bd6-aa86-47b6-b90d-b101d9f19a90
fix an imcoll error that Jean-Yves found

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5474 ea137bd6-aa86-47b6-b90d-b101d9f19a90
New i-command scripts, iTicket, iGet, and iDel for working with
tickets and directory trees, provided by Giacomo.

These work-around some ticket limitations when working with directory
trees.  The idea is to generate a ticket for each subdirectory and use
them instead of the top-level collection ticket when needed.  See the
scripts and their help text for more.  Also see the irod-chat thread
'Problem with iticket for iget subdirectories.'  Thanks to Giacomo
Mariani of the SuperComputing Applications and Innovation Department
CINECA, Bologna, Italy, for these.

See the README.txt and its references for more.



git-svn-id: svn://irodssvn.ucsd.edu/trunk@5475 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Fix gcc compiler error and other errors

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5476 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Update the code generation tool makefile

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5477 ea137bd6-aa86-47b6-b90d-b101d9f19a90
One more template for code generation tool

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5478 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Add compatibility with OpenSSL on Centos5 (openssl pre version 1.0 in
Enterprise Linux 5) as a build-time option. Thanks to by Andreas
Lindqvist (Swedish NSC-SNIC) for this fix.

For those distributions that include an older version of OpenSSL,
previous to 1.0, such as Centos5, there are some compatibility issues
that we believe are safely resolved with these changes.  This is
ifdef-controlled, and is not enabled by default.

To enable this, uncomment the following line in config/config.mk:
# EL5_SSL = 1
This is below the # USE_SSL = 1 line which you uncomment to enable SSL.

Thanks to Andreas Lindqvist of the National Supercomputer Centre
(NSC), Swedish National Infrastructure for Computing (SNIC) for this
improvement.


git-svn-id: svn://irodssvn.ucsd.edu/trunk@5479 ea137bd6-aa86-47b6-b90d-b101d9f19a90
macro based annotation support for code generation tool; clean up code generation template; reduce repeated template code in generation templates;

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5480 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Add a test to build iRODS with gcc instead of g++.  This is run once a
day via the Tinderbox system (other modes run at different hours of
the day).  This is a resync with the script in use (on pivo.ucsd.edu
under /tbox).

Also, changed some times for some of the tests and fixed a minor bug.

To avoid certain problems, the C complier is used on AIX, Solaris, and
whenever HPSS is used.  With this change in this tinderbox test
script, we can also test building with C on other platforms, such as
our Ubuntu test host, catching C++isms (such as // comments) if/when
they are added.



git-svn-id: svn://irodssvn.ucsd.edu/trunk@5481 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Update a recent change to handle systems where awk is not gawk (such
as Ubuntu 13.04 where awk is mawk by default).  Sigh...  Check on the
g++ version by using cut and expr instead of awk.



git-svn-id: svn://irodssvn.ucsd.edu/trunk@5482 ea137bd6-aa86-47b6-b90d-b101d9f19a90
Fix an non-IES compiler error Wayne found

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5483 ea137bd6-aa86-47b6-b90d-b101d9f19a90
remove old PRODS and web contents (which were the PHP libraries) and inserted README files that point to the GForge/git versions

git-svn-id: svn://irodssvn.ucsd.edu/trunk@5484 ea137bd6-aa86-47b6-b90d-b101d9f19a90