ftp: Add LoginCellProvider for FTP

Adding a dedicating FTP provider for LoginCellFactory
allows the shared executor to be bound to the lifetime
of the LoginManager.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6310/
ftp: Fix deadlock introduced in recent refactoring

Recent changes to the FTP door introduced a deadlock:

"pool-116-thread-7":
        at org.dcache.ftp.door.AbstractFtpDoorV1.setTransfer(AbstractFtpDoorV1.java:2380)
        - waiting to lock <0x00000007c5eade98> (a org.dcache.ftp.door.GsiFtpDoorV1)
        at org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer.transferCompleted(AbstractFtpDoorV1.java:1022)
        - locked <0x00000007c5eb3860> (a org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer)
        at org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer.finished(AbstractFtpDoorV1.java:972)
        at org.dcache.util.Transfer.finished(Transfer.java:464)
        - locked <0x00000007c5eb3860> (a org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer)
        at org.dcache.ftp.door.AbstractFtpDoorV1.messageArrived(AbstractFtpDoorV1.java:1444)
        at sun.reflect.GeneratedMethodAccessor132.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.dcache.cells.CellMessageDispatcher$ShortReceiver.deliver(CellMessageDispatcher.java:289)
        at org.dcache.cells.CellMessageDispatcher.call(CellMessageDispatcher.java:202)
        at org.dcache.cells.AbstractCell.messageArrived(AbstractCell.java:859)
        at dmg.cells.nucleus.CellAdapter.messageArrived(CellAdapter.java:981)
        at dmg.cells.nucleus.CellNucleus$DeliverMessageTask.innerRun(CellNucleus.java:1037)
        at dmg.cells.nucleus.CellNucleus$AbstractNucleusTask.run(CellNucleus.java:946)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
"Thread-160":
        at org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer.abort(AbstractFtpDoorV1.java:1060)
        - waiting to lock <0x00000007c5eb3860> (a org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer)
        at org.dcache.ftp.door.AbstractFtpDoorV1$FtpTransfer.abort(AbstractFtpDoorV1.java:1036)
        at org.dcache.ftp.door.AbstractFtpDoorV1.shutdown(AbstractFtpDoorV1.java:1363)
        - locked <0x00000007c5eade98> (a org.dcache.ftp.door.GsiFtpDoorV1)
        at diskCacheV111.doors.LineBasedDoor$CommandQueue.stop(LineBasedDoor.java:343)
        - locked <0x00000007c5ed1290> (a diskCacheV111.doors.LineBasedDoor$CommandQueue)
        at diskCacheV111.doors.LineBasedDoor.run(LineBasedDoor.java:176)
        at java.lang.Thread.run(Thread.java:744)

This deadlock is caused by unnecessary synchronization in the shutdown
method. This synchronization did not exist in the old code.

The patch also adds an accessor for the transfer field, thus solving
potential race conditions.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6318/
ftp: Fix race that lead to erroneous error message in log file

Removes the following error message:

05 dec 2013 14:35:34 (httpd) [door:httpd@dCacheDomain:1386250529077 door:httpd@dCacheDomain:1386250530581 door:httpd@dCacheDomain:1386250530980 door:httpd@dCacheDomain:13
86250531049 door:httpd@dCacheDomain:1386250531677 door:httpd@dCacheDomain:1386250531744 door:httpd@dCacheDomain:1386250533468 door:httpd@dCacheDomain:1386250533660 door:h
ttpd@dCacheDomain:1386250534760] Cannot abort transfer that already completed: 451 Aborting transfer due to session termination

This error is a result of a race in which door shutdown and
the transfer termination sequence conincide. The patch
rearranges the termination sequence to mark the transfer
as completed before providing the final reply to the client.
The patch also removes the error logging.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6320/
ftp: Fix race conditions caused by message reordering

Since DoorTransferFinished and GftpTransferStarted take different
routes, it is possible for these messages to be delivered out of
order. As a result the FTP door produces a 226 reply earlier than
is allowed.

This patch introduces an abstract base class for asynchronous
redirected transfers. That class deals with out of order
notifications. As a side effect, a small pause during transfer
abortion has been removed.

The patch also replaces several 426 replies with 451. 451 is for
internal errors while 426 is for client initiated transfer
abort.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6322/
commons: use long to store updates count of a gauge

to prevent integer overflow

Ticket: #8158
Acked-by: Gerd Behrmann
Target: master
Require-book: no
Require-notes: no
srm: refactor SchedulerFactory

This patch modifies SchedulerFactory so it has a single concern:
creating Scheduler instances.  The current implementation of
SchedulerFactory has two other concerns: creating all necessary
schedulers and maintaining a collection of schedulers. These
additional activities are moved elsewhere.

Creating all necessary schedulers has moved to the SRM class.
Ultimately this should go into the spring XML file, but I didn't want
to do too much in one patch.

Most of the behaviour associated with multiple schedules has moved
into the new SchedulerContainer class and the process of submitting a
Job to a scheduler is achieved via this new class.

One design goal is that SchedulerContainer should not 'leak' Scheduler
instances.  The getScheduler-like methods has been removed: both from
SchedulerContainer/SchedulerFactory and from SRM class.  Activity that
iterates over all the Schedulers has moved to SchedulerContainer.
Activity that targets a specific scheduler is implemented with a
method that accepts a Class<? extends Job> argument to target a
specific Scheduler.

Note that the current code has the Job classes being responsible for
adding themselves to a scheduler.  To try to keep the patch short,
this mode of operation is maintained; however, Job is supplied with a
Scheduler rather than discovering the scheduler itself.

The handler classes must be able to add a Job to a scheduler.  The
patch achieves this via the SRM object passed to the handler's
constructor.  Future patch may refactor this so the object to which a
Job is submitted is supplied on the constructor.

Another future direction is to make Scheduler an interface that the
current Scheduler class implements.  The SchedulerContainer would also
implement this interface and the similarity between SchedulerContainer
and Scheduler becomes clearer.

Another beneficial side-effect from this patch is that
SchedulerFactory is no longer a singleton.

Target: master
Requires-book: no
Requires-notes: no
Patch: http://rb.dcache.org/r/6321/
Acked-by: Gerd Behrmann
ftp: Refactor command queue into sequential executor

The FTP door uses a custom CommandQueue to provide the
ability to sequentially process commands using a shared
thread pool. This patch refactors this code into a more
generic SequntialExecutor class. It implements the
ExecutorService interface and is semantically similar to
a single thread executor. However the thread used for
processing tasks is allocated on demands from a shared
Executor.

The selective enabling of interrupts has been removed
from the FTP door. This was a workaround for bad exception
handling in old versions of the door. The workaround is
not easily ported to SequentialExecutor and is therefore
removed.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6325/
srm: fix Copy scheduling

Recent refactoring resulted in srmCopy operations failing with the message:

    Scheduler for class class org.dcache.srm.request.CopyRequest is not
    supported.

This patch fixes srmCopy and the error message.

Target: master
Requires-book: no
Requires-notes: no
common: simplify gauge logic

removed useless if statement

Acked-by: Gerd Behrmann
Acked-by: Karsten Schwank
Target: master
Require-book: no
Require-notes: no