scripts: don't create pool with deprecated properties

The recent changes to the properties resulted in properties 'name' and
'path' being deprecated in favour of 'pool.name' and 'pool.path'.  Most
references were updated, but a couple were missed.

This patch fixes those missed properties to use the new naming scheme.

Target: master
Patch: http://rb.dcache.org/r/5896/
Acked-by: Tigran Mkrtchyan
Requires-book: no
Requires-notes: no
Fix more deprecated properties

When creating a new pool, a wait-for-files configuration is added. This generated
wait-for-files uses the deprecated property 'path'.  If the admin configures the pool's path
with the new property ('pool.path') then '${pool}' expands to the empty string and the pool
doesn't start.

This patch fixes this by using the new property.

Target: master
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5897/
Acked-by: Tigran Mkrtchyan
srm: Use ISO 8601 format for transition time stamp with millisecond resolution

While debugging some issues at NDGF, I was frustrated by the lack of
precision in the timestamp of SRM transitions. Turns out the timestamps
are stored with millisecond precision, it's just the formatting doesn't
include milliseconds.

This patch switches the format to ISO 8601 compliant timestamps with
milliseconds.

Also turned on transition logging in system-test.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5892/
pool: Add rate limit for pnfsmanager messages

Our admins have pointed out, that using sweeper purge or sweeper free
on a pool can easily cause an outtage in dCache: The huge number of
deregistrations sent to pnfsmanager can create a long queue which causes
timeouts in other services. Indeed, we recently had a surge of FTS
failures due to slow srmLs responses caused by 'sweeper free' calls
on a pool.

This patch addresses this problem by adding a rate limiter to pnfsmanager
requests from a pool. Rather than an adhoc solution in the pool, the patch
extends CellStub with the ability to limit the number of concurrent
requests and the ability to apply a rate limit. Thus any service that makes
use of CellStub can now easily limit how much load it induces on other
services. At the moment this is however only used in pools.

I added support for both limiting the number of concurrent requests and the
rate. According to Little's law, either has the same expressive power in a
stable system, although to convert one to the the other one has to know
the request latency. In an unstable system (consistently higher inflow than
outflow), the two are not equivalent. A rate limiter would keep sending
requests to the remote service at the configured rate, while limiting the
concurrency would lead to the sender being throttled. On the other hand,
a rate limiter can be applied to both RPC style messages and notification
(messages without a reply). Hence both have benefits and drawbacks.

The defaults in CellStub are such that no limit is applied. For pools, a
rate limit of 250 Hz pnfs manager requests is the default.

New property introduced by this patch: pool.service.pnfsmanager.rate.

Target: trunk
Require-book: yes
Require-notes: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5878/
srm: Add failure counters for SRMv2 interface and refine counter formatting

The srm (and many other services) collects stats of requests and failures.
However for the SRMv2 interface failures were not counted. This is rectified
by this patch.

The patch also adjusts the formatting of the request gauges as seen in
the admin shell: The sampling period is printed in seconds rather than
milliseconds (the high resolution is not needed, IMO, and it makes it
hard to read), thousand separators are added to several columns, and a
couple of spelling errors are fixed.

While at it, I removed some dead code in the SRM.

Target: trunk
Require-book: yes
Require-notes: yes
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5895/
doors: fix support for binding to a specific interface

There are a number of problems with how doors handle configuration that describes to which
network interface they should bind, if any.  First, the default value (the empty string) is
not accepted by xrootd and webdav seems to interpret this as "bind to localhost".  Second, the
admin interface (a.k.a. "ssh2") doesn't support binding at all and always listens on all
interfaces.

This patch fixes these problems.  It also changes the place-holder value for wildcard from the
empty string to 'any'.

FOR RELEASE NOTES:

The keyword 'any' is used when specifying a door should listen on any interface (previously it
was the empty string).  The default value has been adjusted accordingly.

Requires-notes: yes
Requires-book: yes
Target: master
Patch: http://rb.dcache.org/r/5898/
Acked-by: Gerd Behrmann
spacemanager,transfermanagers: Don't rely on SRM db properties

Several service have historically piggybacked on the SRM db
settings. Nowadays each service has its own DB properties, however
spacemanager and transfermanagers defines it's properties in
terms of the SRM properties, thus causing an unwanted coupling.

We still want backwards compatibility and thus let spacemanager
and transfermanagers default to the deprecated (legacy) srm
properties, however the new properties should be decoupled. This
structure matches what is already done in pinmanager.

At the same time, both spacemanager and transfermanagers did
not actually respect it's own properties: That's because they
used the JDBC URL defined for the SRM and these didn't make
use of properties like spacemanager.db.host and spacemanager.db.name.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5901/
srm: Drop properties marked obsolete or forbidden in 2.6

We cannot keep them forever and for other services we dropped
old properties in 2.7.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5902/
pnfsmanager: Use pnfsmanager.enable.* for properties that enable features

Changes

pnfsmanager.folding -> pnfsmanager.enable.folding
pnfsmanager.inherit-file-ownership -> pnfsmanager.enable.inherit-file-ownership
pnfsmanager.full-path-permission-check -> pnfsmanager.enable.full-path-permission-check
pnfsmanager.acl -> pnfsmanager.enable.acl

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5903/
SRM: fix ping

SRM includes code that checks if a reponse is successful.  It does
this by invoking the getReturnStatus that Axis creates, via
reflection.

Unfortunately, not all response objects contain this method (the
response from a ping operation is the only example).  The current
code that evaluates whether a response is successful results in a
NoSuchMethodException being thrown for such operations.  This
exception is treated as a failure, resulting in ping no longer
working.

This patch fixes this problem by catching the NoSuchMethodException
and treating such operations as always succeeding.  (Catching the
exception like this is not nice, but seems the least ugly way of
discovering if the response object contains the desired method).

Target: master
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5904/
Acked-by: Gerd Behrmann
pool: update nfs mover to fit into current design

after last mover code restructuring, NFS mover was adopted
not in the optimal way. This change makes NFS mover to naturally
fit into new design.

Acked-by: Paul Millar
Target: master
Require-book: no
Require-notes: no
Patch: http://rb.dcache.org/r/5893/