dCache/dcache — pull request #64 by gbehrmann (2013-05-22 08:21:56+00:00)
Title: pool: Configure timeout for pool to pool transfer

Addresses the issue that a pool to pool transfer does not time out
if the underlying TCP connection "gets stuck".

Back when we used DCAP for pool to pool transfers, we configured TCP
keep alive to ensure that we eventually get a timeout if the TCP stacks
of the two operating systems disagree about the state of the TCP
connection. Ever since we switched to HTTP, this functionality was lost.

At NDGF we observed a situation in which a migration from one pool to
another got stuck because the TCP connection got into an inconsistent
state. The target pool saw the connection as established, while the
source pool no longer had a connection. The transfer forever stayed
in the DoorFinished state, as dCache expected that the TCP connection
would eventually be closed (since dCache knew it had been closed on the
source pool already).

We cannot get Sun's HTTP client to enable TCP keep alive, however we
can configure connect and read timeouts. Since we only ever need these
timeouts for the rare case when the OSes get confused, I don't see a
reason to make these values end user configurable. I have simply hard
coded some conservative timeout values.

Target: trunk
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Paul Millar
Patch: http://rb.dcache.org/r/5556/
(cherry picked from commit 7de28ba592e39c46df4bf430b6d6d1a8151bd901)