pool: Fix race conditions and counting errors in mover scheduler

SimpleIoScheduler had several bugs:

- The scheduler loop took an element out of the queue before acquiring
  a transfer slot. This means that when all transfer slots are taken, the
  scheduler loop blocks for a slot to become available and at the same
  time the mover was already removed from the queue.

  Thus the mover is no longer counted as queued and also not counted
  as running, meaning we "lost" one mover in the mover stats.

  It also means that if an attempt is made to cancel this mover, the mover
  is considered to be running and thus the mover is killed rather than
  just removed from the queue. However since the mover is not actually
  running, the kill will not have an effect until a transfer slot becomes
  available - at which point the mover is killed right away.

- Post processing happened after the transfer slot was released. This meant
  that registration in the name space and on-write checksum computation
  would not be subject to the regular queue restrictions. They would not
  count towards the mover stats and there was in fact no limit as to how
  many concurrent on-write checksum computations could run.

This patch solves both problems.

Target: trunk
Request: 2.5
Request: 2.2
Request: 1.9.12
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5288/
pool: Kill movers on shutdown

The SimpleIoScheduler has a shutdown method, but it doesn't actually
do anything.

This patch updates the shutdown method to kill all movers. Movers
are given a moment to die before the shutdown method returns. This
prevents that dependent components are shut down before movers
terminate.

An explicit dependency had to be added between the execution services
and the repository. This is to prevent that spring shuts down the
repository component before the movers are terminated.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5300/
pool: Ensure clean thread shutdown

One can use one of two strategies for dealing with the life time
of threads, thread pools, timers, etc.

- Either one binds the lifetime to the cell, meaning that one has
  to shut it down when the cell is killed (at the latest), or

- one declares the threads as daemon threads and shares them
  between cells

Note that one should not use non-shared daemon threads outliving
the cell. The reason is that if one were to start, stop and
start the same cell repeatedly (eg as we do with some doors),
then we would create a new daemon thread every time. Thus such
daemon threads are shared threads.

Shared threads have the downside that they break the isolation
of the cell. A cornerstone of robust software design is to build
bulkheads into the software and the cell boundary is one
of those bulkheads in dCache - shared thread pools punch wholes
in those bulkheads.

That being said, there are situations in which shared threads are
useful - eg because the threads access a shared resource like a
disk, or because explicitly managing the lifetime is not possible
without major architectural changes in the code.

This patch modifies several pool threads to either classify them
as shared daemon threads or to manage their lifetime.

It would probably be better to move the thread pool instantiation
into spring and inject into the proper components. This would
allow us to share thread pools between classes in a pool, and
would make it easier to use the pool name as part of the thread
names. I have chosen not to do so in this patch, concentrating on
the shutdow issue first.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5301
pool: Use ListenableFuture rather than CompletionHandler

The pool queuing and mover wrapping code relies on the
CompletionHandler interface from the NIO 2 async code.
This interface is a poor fit for our needs as it doesn't
provide means of cancelling the result (which means we
had to add a custom Cancellable interface) and has
features like attachables, that we don't need.

The Guava ListenableFuture - an extension of the regular
JDK Future concept - fits our needs much better. This
patch refactors the code to use ListenableFuture and this
allows custom cancellation code to be removed, and also
stream lines the code a bit.

The patch moves some error handling from SimpleIoScheduler
to PoolIORequest. The code was tightly coupled to
PoolIORequest anyway and fits more naturally inside that
class.

The patch is part of a series of pathes to clean up the
pool mover handling.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5308
poolmanager: Clarify use of preallocated space in pool selection

Pool manager can make use of the fie size or preallocated space
during write pool selection. The field is unused for all other
pool selections, yet the field is present in the base class. This
makes the purpose of the field less clear than it should be.

This patch moves the field to the write pool selection message
and the field is changed to precallocated to reflect its use. It
differs from the file size in storage info/file attributes by
being a preallocation, not an exact size of the final file. It is
only ever used when locating a pool with enough free space.

Some DCAP specific code is moved from pool manager to the DCAP
door.

The patch also makes file size and checksum attributes required
for read pool selection. These should be available to a pool
when staging a file or copying it between pools.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5322
pool: Let DoorTransferFinished and PoolFileFlushed use FileAttributes

We are slowly moving away from StorageInfo. This patch moves two
pool notification messages to FileAttributes, although it is still
used to transport StorageInfo.

Care is taken to maintain backwards compatibility with old pools
(two way for DoorTransferFinished as it may be sent between pools).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5323/
login: add commands to dump and clear login cache

Ticket: #7728
Acked-by: Paul Millar
Target: trunk
Require-book: no
Require-notes: yes