Add alarms and transfermanager database to system-test setup

HSQLDB conection parameters are defined too, although the alarms schema
is limited to postgresql at the moment. For that reason and because
HSQLDB is a single process database, the alarms service uses the xml
store.

The httpd.alarms.* properties are marked immutable, as modifying many of
them will have no effect as they are not used as part of the definitions
of derived properties (e.g. changing httpd.alarms.db.type had no effect).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/6534/
Drop JDBC driver properties

JDBC 4 drivers auto-register through the Java ServiceLoader facility. It
is no longer recommended to load the driver through Class.forName. This
allows us to get rid of all the db.driver properties (which where not DRY
anyway, as the driver identification was duplicated between the driver
property and the url property).

DataNucleus unfortunately insists on loading something, so for the time
being I gave it a dummy class to load (java.lang.String). An upcomming
patch will solve this while moving everything to HikariCP.

The patch marks all *.db.driver properties as obsolete. Use of JDBC 4
compliant drivers is required.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Albert Rossi <arossi@fnal.gov>
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6535/
Use Spring's Environment and PropertySource feature to inject configuration

Spring 3.1 introduced the Spring Environment, which provides a flexible way
to add additional property sources. This patch updates UniversalSpringCell
to inject our configuration into the context as property sources. This allows
us to get rid of dCache specific boilerplate in our Spring XML files.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/6536/
webadmin: Inject service configuration into Spring environment

This allows the properties to be used more widely, such as for activating
profiles.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/6537/
dcache-core, alarms: remove postgresql precondition on schema

There is nothing in the liquibase schema for alarms which specifically requires postgresql.
Removing this precondition allows the rdbms option to be tested in system-test using hsqldb.

Target: master
Request: 2.8
Request: 2.7
Request: 2.6
Patch: http://rb.dcache.org/r/6548/
Require-book: no
Require-notes: no
Acked-by: Gerd
Use HikariCP connection pool in most services

After having tested HikariCP as part of space manager, the next step is to try
it for other services. In the interest of maximizing internal testing before
public release, this patch switches most services to HikariCP. A few services
require a bit more work and will be updated in a separate patch.

Since DataNucleus doesn't have direct support for HikariCP, the patch changes
DataNucleus invocation to inject a pre-configured data source.

A custom DriverManagerDataSource had to be added to chimera as HikariCP only
works with DataSources, not DriverManager directly.

Most *.db.connections.partition-count properties are gone, and
*.db.connections.{min,max}-per-partition have been replaced by
*.db.connections.{min,max}.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6538/
billing,webadmin,alarms: Use HikariCP

I had to refactor the code somewhat to make it easy to inject other data
sources that what DataNucleus can create itself. As far as possible I have
moved instantiation and wiring into Spring, thus removing lots of glue code.

With this patch, all *.db.connections.partition-count properties are obsolete
and *.db.connection.{min,max}-per-partition have all been renamed to
*.db.connection.{min,max}.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/6541/
libs: update to bugfix nfs4j-0.7.3

Changelog for nfs4j-0.7.2..nfs4j-0.7.3
    * [f2b0d2b] vfs: fix regression introduced in 1c916c5

Acked-by: Gerd Behrmann
Target: master, 2.8, 2.7
Require-book: no
Require-notes: no
core, billing: remove unit test

The unit test for billing persistence uses timeouts/sleep to check the presence and absence of entries in the database.  This timing, however, is inherently unstable because it depends on the OS and the thread scheduler in particular.  Thus the test will sometimes fail during the build process.

However, the usefulness of this test is very limited, so I propose simply to eliminate it.

Target: master
Request: 2.8
Request: 2.7
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/6556/
Require-book: no
Require-notes: no
Acked-by: Gerd
Acked-by: Pau
chimera: Fix option parsing in FsFactory

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6558/
chimera: Add help command in shell

The help command was lost in recent refactorings of cells.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6559/
chimera: Add time options to ls command

Adds options to select atime, ctime and creation time
for ls output in the Chimera shell.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6560/
xrootd: Upgrade to xrootd4j v1.2.4

Contains the following changes over v1.2.3:

42101c6 xrootd4j: Make errors null terminated
5ae3a44 Fix read beyond end of file

These changes improve compatibility with the "new xrootd client".

Target: trunk
Request: 2.8
Request: 2.7
Request: 2.6
Ticket: http://rt.dcache.org/Ticket/Display.html?id=8156
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6557/
pool: Fix access time update

The bug causes access time update to happen on pool to pool and not on regular
downloads - the reverse is supposed to happen.

This affects sweeper garbage collection order, space cost calculation, and
access time update in the name space.

Target: trunk
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Ticket: http://rt.dcache.org/Ticket/Display.html?id=8234
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6564/
packages, fhs: fix saxon includes/excludes expression

The current includes/excludes expression in the fhs package assembly xml fails to move the saxon jars with groupId net.sf.saxon or with artifact "saxon-dom" into the saxon subdirectory.

Added the appropriate globbing to fix this.

Target: master
Request: 2.8
Request: 2.7
Request: 2.6
Patch: http://rb.dcache.org/r/6566/
Require-book: no
Require-notes: no
Acked-by: Gerd
Revert "packages, fhs: fix saxon includes/excludes expression"

This reverts commit 9ddc482591a557ef2b363b9c6738dd2007879ff6.

Taking the saxon jars off of the classpath stops the privilege-xacml
client from finding a class.
dcache-core, alarms: restore xml connector to initialization properties

A recent refactoring (http://rb.dcache.org/r/6541) of the connector initialization for the alarms appender inadvertently eliminated the url property for the xml datanucleus plugin.

This patch restores it.

(Note: This is my fault, as I acked that patch and should have noticed it).

Target: master
Require-book: no
Require-notes: no
Patch: http://rb.dcache.org/r/6571
Acked-by: Paul
dcache-webadmin, alarms: remove required annotations from datastore setters

For the profile "alarms-off", the placeholder datastore object is constructed without the following properties:

persistenceManagerFactory
alarmsCleanerEnabled
alarmsCleanerSleepInterval
alarmsCleanerSleepIntervalUnit
alarmsCleanerDeleteThreshold
alarmsCleanerDeleteThresholdUnit

However, the setters on these properties have the "@Required" annotation. This causes a fatal exception at startup.

Removing the annotations restores proper initialization.

(Once again I should have caught this regression, introduced in 6541).

Target: master
Require-notes: no
Require-book: no
Patch: http://rb.dcache.org/r/6572
Acked-by: Gerd
xrootd: Add support for kXR_locate, kXR_query and kXR_set

kXR_locate always returns the door or pool itself, no matter whether the file
exists or not. This avoids an extra name space lookup. It is unknown whether
the argument to kXR_locate contains the opaque information, thus whether the
pool can rely on the uuid being present.

kXR_set is only implemented for appid.

kXR_query is implement for kXR_Qconfig and kXR_Qcksum. In pools we again have
the issue that the spec doesn't define whether the path contains the opaque
info.  The patch speculatively looks for the uuid and returns the checksum if
found - otherwise the client is redirected to the door.

The behaviour on kXR_open is altered slightly to return clients with missing
uuid or invalid uuid back to the door rather than return an authorization
error. The client likely just uses an outdated uuid or wants to open a second
file.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6565/
gplazma2-xacml: revert version of opensaml

The privilege xacml client is incompatible with the newer opensaml library.
In particular, the client bootstrap method calls a method which is missing
in the newer version.

Restores opensaml to 2.4.1 (last compatible).  Also eliminates unused opensaml1 library.

Target: master
Request: 2.8
Patch: http://rb.dcache.org/r/6581/
Bug: http://rt.dcache.org/Ticket/Display.html?id=8244
Requires-book: no
Requires-notes: no
Acked-by: Paul