nfs: mark unused nfs.db.host property as forbidden

The nfs.db.host property exists in the nfs.properties files
but it is not used anywhere.
It was probably introduced during the merge of nfs3 and nfs41.

To avoid confusion about the usage of this property this patch
marks it as forbidden and refers the user to nfs.db.url instead.

Ticket: http://rt.dcache.org/Ticket/Display.html?id=8465
Acked-by:
Target: master
Request: 2.10
Require-book: no
Require-notes: no
mark internal <service>.db.xyz properties as immutable

clean, nfs and pnfsmanager have some db related properties that
must not be changed by the user.

This patch marks them as immutable.

Ticket:
Acked-by:
Target: master
Require-book: yes
Require-notes: yes
cells: Orderly shutdown of multi-threaded cells

Cells used to have a single message delivery thread, but some years ago
this was extended to allows multi-threaded executors to be injected for
message delivery.

During shutdown, cells inject a LastMessageEvent. With a single message
delivery thread, delivery of the LastMessageEvent guarantees that no
further messages are processed. However in a multi-threaded setup, there
could be concurrent requests still being processed.

The consequence of the above is that the cell may proceed shutdown even
though messages are actively being processed. This in turn leads to all
kinds of errors as resources like database connection pools get closed
while still being used.

This patch resolves the issue by removing the LastMessageEvent. Instead
cells rely on shutdown of the message executor to ensure that no further
messages are going to be processed.

Also changed the point at which background tasks are stopped: We now no
longer execute them after the cell has been shut down.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7375/
loginmanager: Associate ScheduledExecutor with the cell's thread group

Would lead to log messages to be associated with the wrong cell. Also
caused issues during shutdown.

Target: trunk
Request: 2.10
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7382/
pnfsmanager: Controlled shutdown of processing threads

PNFS manager's processing threads currently rely on cells automatically killing
threads with an interrupt if the cell fails to shut them down.  This presents
two problems: One is that it makes cell shutdown slower (unfortunate, but not
critical); the other is that it means the processing threads run after the
database connection pool has been closed (worse as it causes partial processing
of requests and quite some noise in the logs).

This patch reworks the PNFS manager shutdown to explicitly shut down the
processing threads. Queued requests are rejected by returning a
NoRouteToCellException to the requestor. The currently being processed requests
are allowed to finish (although they only get half a second to do so).

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7383/
loginbroker: Kill worker threads on shutdown

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7395/
httpd: Kill transfer observer threads on shutdown

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7396/
pool: Kill sweeper thread on shutdown

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7397/
poolmanager: Kill watchdog on shutdown

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7398/
srm-client: Add interactive SRM shell

Incomplete proof of concept of an interactive srm shell. The most
important features, up- and download, are missing.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7265/
check-config: Produce error when using scoped properties

In the confusion of scoped properties, some sites have used the
scoping operator in their local configuration (probably not realizing
it was an operator and treating it as if it was part of the name).

When upgrading to 2.10, those sites will not receive any warnings or
errors about such definitions, as only the unscoped names are marked
obsolete/deprecated/forbidden. dCache does output an info level message
saying that dCache itself doesn't use the property, but sites tend
to ignore that message (based on a sample size of one).

This patch resolves this by flagging scoped properties as errors.
This prevents a slash from being used in a property name, but given
the history of scoped properties, it is probably wise to disallow
that symbol for some years.

Demo:

    $ packages/system-test/target/dcache/bin/dcache check-config
    [ERROR] dcache.conf:2: Property xrootd/xrootdPlugins is a scoped property.
                           Scoped properties are no longer supported.
    [INFO] system-test.conf:1: Property system-test.home is not a standard
                               property
    Found 1 error.

Target: trunk
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Require-notes: yes
Require-book: yes
Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7401/
chimera: Refine IOException to ChimeraFsException

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7404/
srm-client: Report srmPutDone failure

srmcp currently ignores failures to finalize an upload with srmPutDone.
This patch fixes that problem.

Target: trunk
Request: 2.10
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7405/
pnfsmanager: Check file type before overwrite

When preparing a file for upload for the SRM, pnfsmanager fails to check the
file type of any existing file with the same path. The upload eventually fails
during srmPutDone because Chimera refuses to move the file, but this failure
ought to happen during srmPrepareToPut. This patch resolves this problem.

Target: trunk
Request: 2.10
Request: 2.9
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7407/
doors: Base64 session counter

To shorten the session string a bit, this patch base64 encodes the
session counter.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7408/
poolmanager: Warn when spacecostfactor is too big

When space cost factor is large, wass may run into a problem caused by
the limited precision of floating point arithmetic. When that happens
the code reaches a statement that was thought to be unreachable.

This patch adds a check that detects the situation and generates
a more suitable error message.

Target: trunk
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Patch: https://rb.dcache.org/r/7399/
srm-client: add srmfs to RPM manifest

The recently added srmfs client was not included in the RPM manifest.
This patch adds it.

Target: master
Patch: https://rb.dcache.org/r/7409/
Acked-by: Gerd Behrmann
Requires-notes: no
Requires-book: no
poolmanager: Warn when spacecostfactor is too big

Patch: https://rb.dcache.org/r/7399
chimera: Add messages to JdbcFs#move exceptions

Otherwise the exceptions will have a null message, which in other
places may lead to rather meaningless error messages being logged.

Target: trunk
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7403/
srm-client: delegate by default for https 3rd-party transfers

By default, the srm-client srmcp command will delegate only
when it thinks it makes sense.  Unfortunately, it makes the
wrong decision for http and https transfers: mistakenly
delegating for the former and not for the latter.

This patch fixes this.  It also modifies a variable name to
make more sense.

Target: master
Patch: https://rb.dcache.org/r/7393/
Acked-by: Gerd Behrmann
Request: 2.10
Require-notes: yes
Require-book: no