srm: Minimize use of AuthorizationRecord

AuthorizationRecord is a legacy class from before gPlazma 2 existed.
Since the introduction of gPlazma 2, we have tried to use Subject
to represent users everywhere in dCache. SRM is the last component
in which this was not the case. This patch addresses this inconsistency.

AuthorizationRecord is not removed completely as the SRM relies
on storing the user identity in the database together with user
requests. To maintain this functionality and preserve backwards
compatibility, AuthorizationRecord is still used for this purpose.

The SRM itself uses the SRMUser interface to abstract from the
exact representation of a user. This patch introduces the DcacheUser
class, which wraps a Subject to expose it as an SRMUser. The
dCache specific SRM backend is updated to extract the Subject
directly from DcacheUser. DcacheUserPersistenceManager acts as a
bridge to persist and load DcacheUser instances to and from the
database. Internally, DcacheUserPersistenceManager however simply
interfaces with the legacy AuthRecordPersistenceManager. This is
the only place the legacy AuthorizationRecord class is used.

The custom authorization caching of the SRM has been replaced by
a CachingLoginStrategy.

The wiring related to the above classes has been moved into the
Spring XML file.

The only admin visible change is that the output format of the SRM
ls admin commands has changed slightly. It now shows a list of
principals as an identification of the request owner. If output
of these commands is used in the book, the book must be updated.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5691/
system-test: Fixed several bashisms in populate script

system-test failed to compile on systems on which /bin/sh does not
point to bash (eg Debian and Ubuntu).

Also reformatted output to fit in 80-character terminal.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5752/
httpd: Parameterize war unpack directory

Addresses the issue that the webapp unpack directory is /var/tmp in
tar and system-test builds. For these builds the unpack directory
should be within the var directory of the installation directory.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5753/
system-test: build authorized-keys file

The admin interface supports authentication via SSH pub/priv keys,
which are often stored in well-known locations.

This patch takes the user's public keys, if found, and uses them to
build an authorized-keys file for the admin interface.

Target: master
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5754/
Acked-by: Gerd Behrmann
fix servlet classpath clash

The servlet API jar files are signed.  Recent versions of java (or
possibly bug http://bugs.sun.com/view_bug.do?bug_id=6805618 ) means
that, on some platforms, all the classes in a package must be signed
by the same certificate.  A failure results in an error message and
dCache failing to startup.

Currently, dCache includes a Jetty-supplied 'javax.servlet' jar file
(v3.0), and a 'javax.servlet'-supplied 'servlet-api' jar file (v2.5).
These are signed by different certificates and, since the v3.0 file
contains additional classes, if v2.5 precedes v3.0 in the classpath
then the classloader will find classes in the same package with
different signatures.

The previous version of Jetty depended on servlet-api jar file and,
while this too had two jar files (a Jetty-supplied and a
'javax.servlet'-supplied) since the two jar files have the same set of
classes, the classloader will take javax.servlet.* classes from one
jar file, independent of classpath order.

This patch fixes this problem by excluding the references to
servlet-api dependency in axis and axisg jar files.

Target: master
Requires-nodes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5757
Acked-by: Gerd Behrmann
Add DB driver to the list of classes to load

The chimera-cli command was overhauled to make it significantly faster.
One technique used was to limit the classes in the classpath to only
those needed.

Unfortunately, this work excluded the database-driver jar. The current
chimera-cli works for system-test, as the plugin mechanism ensures the
database-driver jar is included; however, the PostgreSQL driver is not
included, resulting in chimera-cli no longer working for machines using
PostgreSQL.  The error message is a stack-trace that starts:

[root@bes-srm dcache]# chimera-cli ls /
Exception in thread "main" java.lang.ClassNotFoundException:
org/postgresql/Driver

[reported in user-forum]

Target: master
Request: 2.6
Requires-notes: yes
Requires-book: no
Patch: http://rb.dcache.org/r/5755/
Acked-by: Gerd Behrmann
info: fix concurrent modification

Gerd reported seeing this:

06 jun 2013 21:44:33 (info) [] Uncaught exception in thread Thread-79
java.util.ConcurrentModificationException: null
        at java.util.HashMap$HashIterator.remove(HashMap.java:912) ~[na:1.7.0_11]
        at org.dcache.services.info.gathers.MessageHandlerChain.flushOldMetadata(MessageHandlerChain.java:230) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at org.dcache.services.info.gathers.MessageHandlerChain.getMetricTTL(MessageHandlerChain.java:173) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at org.dcache.services.info.gathers.MessageHandlerChain.getMetricTTL(MessageHandlerChain.java:37) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at org.dcache.services.info.gathers.CellMessageHandlerSkel.answerArrived(CellMessageHandlerSkel.java:150) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at dmg.cells.nucleus.CellNucleus$CallbackTask.innerRun(CellNucleus.java:971) ~[cells-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at dmg.cells.nucleus.CellNucleus$AbstractNucleusTask.run(CellNucleus.java:929) ~[cells-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_11]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_11]
        at dmg.cells.nucleus.CellNucleus$1.run(CellNucleus.java:657) [cells-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]

The problem is triggered by having multiple messenger threads and the
class not being thread-safe.

This patch updates the class to make it thread-safe

Target: trunk
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5756/
Acked-by: Gerd Behrmann
ssh2: Honor admin.colors.enable

Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
Acked-by:
Target: trunk
Require-book: no
Require-notes: no
Remove line that sets colors from environment param and not from within dCache
Change parameter to no type definition and switch line to above cell related properties
ssh2: Honor admin.colors.enable

Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
Acked-by: Gerd
Target: master
Require-book: no
Require-notes: no
Merge color fix branch with current master because not all changes got committed with the last push.

Sorry.

ssh2: Honor admin.colors.enable
Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
Acked-by: Gerd
Target: master
Require-book: no
Require-notes: no
ssh2: Honor admin.colors.enable

    Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

    Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
    Acked-by: Gerd
    Target: master
    Require-book: no
    Require-notes: no

Squashed commit of the following:

commit 8e3db9a3f6cd0c9b8ce6f340f67eb3ecf3651163
Merge: 3887dc4 3e02f94
Author: Christian Bernardt <christian.bernardt@desy.de>
Date:   Mon Jul 15 18:33:45 2013 +0200

    Merge color fix branch with current master because not all changes got committed with the last push.

    Sorry.

    ssh2: Honor admin.colors.enable
    Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

    Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
    Acked-by: Gerd
    Target: master
    Require-book: no
    Require-notes: no

commit 3887dc446e7d388659ffb651e5a0a9e1db67f868
Author: Christian Bernardt <christian.bernardt@desy.de>
Date:   Mon Jul 15 14:15:39 2013 +0200

    ssh2: Honor admin.colors.enable

    Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

    Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
    Acked-by: Gerd
    Target: master
    Require-book: no
    Require-notes: no

commit 3e02f94eebb02bddf81441a9dd7ac41344577e74
Author: Christian Bernardt <christian.bernardt@desy.de>
Date:   Mon Jul 15 14:15:39 2013 +0200

    Change parameter to no type definition and switch line to above cell related properties

commit 89bd221adafba600ce35c5f103f65a6736c408e6
Author: Christian Bernardt <christian.bernardt@desy.de>
Date:   Mon Jul 15 12:22:09 2013 +0200

    Remove line that sets colors from environment param and not from within dCache

commit 0def3e296a3018c173062457b7256a4ac207ef30
Author: Christian Bernardt <christian.bernardt@desy.de>
Date:   Mon Jul 15 11:41:13 2013 +0200

    ssh2: Honor admin.colors.enable

    Ssh2 admin interface did not honor admin.colors.enable, this patch resolve this problem.

    Ticket: http://rt.dcache.org/Ticket/Display.html?id=7820
    Acked-by:
    Target: trunk
    Require-book: no
    Require-notes: no
Merge branch 'master' of github.com:dCache/dcache
loginbroker: Comply with new configuration policy

Drops the service srm-loginbroker. All doors now register with
a single login broker called LoginBroker and SRM and all
monitoring services query this one LoginBroker to locate doors.
SRM is updated to exclude srm as a transfer protocol, thus
making it safe to register all doors in the same login broker.

The dcache.service.loginbroker property is updated to take
a comma separated list of loginbrokers to register with or
to query. httpd now uses this property to configure which
loginbrokers to query.

New properties

loginbroker.cell.name
srm.protocols.disallowed.get
srm.protocols.disallowed.put
srm.protocols.preferred

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5742/
cleaner: Clean up cleaner.properties

Besides cleaning up the comments, this patch turns the previously
obsoleted property cleaner.hsm into a deprecated property.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5744/
spacemanager: Allow previously obsoleted values for deprecated srmSpaceManagerEnabled

This patch adds legacy values for srmSpaceManaerEnabled. Thus one can
once again use true, on, yes, enabled, false, off, no, disabled, but
only when using the legacy property.

Also fixed a bug in srm.batch introduced when previously deprecating
srmSpaceManagerEnabled.

Require-book: no
Require-notes: yes
Target: trunk
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5748/
replica: Restored legacy values of deprecated replicaManager property

When renaming the configuration properties of replica manager, the
replicaManager was marked deprecated and the allowed values changed
to true and false rather than yes and no. This patch restored the
legacy values such that yes and no can be used. The new
dcache.enable.replica property still requires the values true and
false.

The default value of the property has been changed from yes/true to
no/false, as this better reflect most dCache instances.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5749/
alarms: Comply with new configuration policy

RELEASE NOTES:
BOOK:

Deprecated:

alarms.server.config
alarms.store.db.type
alarms.store.path
alarms.store.db.rdbms
alarms.store.db.driver
alarms.store.db.host
alarms.store.db.user
alarms.store.db.pass
alarms.store.db.properties
alarms.store.db.url

New:

alarms.cell.name=alarms
alarms.log.config.path=${alarms.server.config}
(one-of?off|error|warn|info|debug|${alarms.server.log.level})alarms.log.root-level=${alarms.server.log.level}
(one-of?off|xml|rdbms|${alarms.store.db.type})alarms.db.type=off
alarms.db.xml.path=${alarms.store.path}
alarms.db.rdbms.type=${alarms.store.db.rdbms}
alarms.db.driver=${alarms.store.db.driver}
alarms.db.host=${$alarms.store.db.host}
alarms.db.user=${$alarms.store.db.user}
alarms.db.password=${alarms.store.db.pass}
alarms.db.name=alarms
alarms.db.config.path=${alarms.store.db.properties}
alarms.db.url=${alarms.store.db.url}
alarms.db.schema.auto
alarms.db.schema.changelog

Obsolete:
alarms.store.db.changelog

Target: master
Require-notes: yes
Require-book: yes
Patch: http://rb.dcache.org/r/5743
Acked-by: Gerd
Fix CellStub timeout handling

A recent change to CellStub introduced a time unit property. Thus
the interpretation of getTimeout() changed, as the value was no
longer guaranteed to be in milliseconds. This patch updates the
code that uses getTimeout() and either lets it use getTimeoutInMillis()
instead, or updates it to be time unit aware. The patch also fixes
several calls to setTimeout().

Thanks to Dmitry for finding this.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/dashboard/