pool: Rewrite of HSM interface

The patch introduces a new SPI for tape drivers. One possible implementation
of this SPI is included for calling out to legacy HSM scripts. Thus backwards
compatibility is maintained, with only minor behavioural changes.

The main class of the new HSM subsystem is NearlineStorageHandler.  It replaces
the old HsmStorageHandler2 class. NearlineStorageHandler handles store, fetch
and remove requests. It passes on those requests to a NearlineStorageProvider
implementation. NearlineStorageProvider is implemented by nearline storage
providers (drivers) and implementations are discovered through Java's
ServiceLoader facility.

NearlineStorageHandler interacts with the provider by submitting request
objects represented by the three interfaces StageRequest, FlushRequest and
RemoveRequest. These interfaces provide interfaces to report completion or
failure, but also for reporting transition from queued to activate state and
for space allocation. NearlineStorageHandler uses those callbacks to implement
the usual pre-flush lookups, checksum policies, allocate space, etc. A provider
thus does not interact with the rest of dCache and is unaware of the details of
such interactions and policies. Queuing on the other hand is fully implemented
by the provider. This is to give the provider as much flexibility to optimize
interaction with the tape system as possible. The provider may have no queueing
at all and submit all requests to the tape system, it may have detailed
knowledge about tape layout and reorder requests itself, or it may have a
simple queueing scheme like we had before. The point is that this is left to
the provider. Similarly, for staging the point of space allocation is left to
the provider, which allows the provider to request more stages from tape than
will currently fit on the pool, allocating space in the order in which files
arrive from tape.

The functionality of the old HsmStorageHandler2 class has thus been split
between NearlineStorageHandler and the provider.

Since queueing is part of the provider, the maximum number of concurrent tasks
is thus also part of the provider and may not exist for some providers. Thus it
is no longer reported to pool manager for flush and stage queues.

The HsmSet class has been refactored to allow the provider to be specified when
definining new nearline storages. The new 'hsm create' command must be called
before 'hsm set' and accepts the provider name. Care has been taken to maintain
backwards compatibility with old pool setup files, both for defining storages
and for setting concurreny limits.

A default provider has been implemented. This provider uses FIFO queuing and
callouts to an external HSM script. It is thus very similar to the old
HsmStorageHandler2. Old tape integration scripts can be reused without
modification.

Three additional providers are provided for purposes of demonstration and testing,
but may also have pratical use:

- The copy provider copies files to another directory (like our standard hsmcp
  scripts).
- The link provider creates hard links in another directory (achieving a
  similar effect to the copy provider, but without the physical copy and
  limited to the same file system).
- The tar provider is incomplete, but serves as a demo of a provider that works
  on several files at once: It bundles multiple files into a tarball and reorders
  staging by tarball. It doesn't support request cancellation or file removal.

Due to the changes in the hsm commands, it is advisable to regenerate the pool
setup file after upgrade (run 'save' in the pool).

Observable changes in behaviour:

- Concurrency of the script provider is configured through options rather than
  the dedicated 'xx set max active' commands.
- Concurrency settings are interpreted per nearline storage instance per pool,
  not as a sum for all nearline storages on a pool.
- Timeouts are interpreted for the entire request period, not just for the
  period during which there is a callout to the script. This is not set in
  stone, but the point being that a client doesn't care whether we are queueing
  or waiting for a script: SRM in particular defines a timeout for the stage and
  after that the client doesn't care about the result anymore. I hope that
  eventually we can include the timeout in the stage request from SRM, thus
  avoiding to configure the timeout in the pool.
- When the timeout is adjusted, this also affects existing requests (although
  the script provider doesn't adjust it's internal timeout when the script was
  already started, but this is subject to change).
- Changed the default of cleaner.ensble.hsm to true, meaning HSM clean is
  enabled by default (defaulting to not rely on some external component seems
  like the more sensible choice).
- Rather than as-hoc return codes, most HSM failures are translated to the
  default CacheException return code. The script provider obviously still uses
  the exit code of the script as the return code.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>