Fix dCache FTP and SRM to be standards-compliant by default

The overwriteEnabled property affects FTP and SRM doors.  The default
is currently 'false', which prevents overwriting of data if the file
already exists.

RFC 959 is quite clear:

         STORE (STOR)

            This command causes the server-DTP to accept the data
            transferred via the data connection and to store the data as
            a file at the server site.  If the file specified in the
            pathname exists at the server site, then its contents shall
            be replaced by the data being transferred.  A new file is
            created at the server site if the file specified in the
            pathname does not already exist.

Therefore, by setting overwriteEnabled to 'false', dCache is not
RFC-959 compliant.  This is our default behaviour.

Similarly, SRM protocol is also quite clear how storage elements are
expected to behave.  Setting overwriteEnabled=false breaks that
expected behaviour.

Moreover, there is no additional security from setting
overwriteEnabled=false.  The client can simply delete the data and
upload afresh.

The net result is that dCache is, by default, not standards compliant
for no direct benefit.

For the 2.2 branch, we maintain the existing default value, but
include the warning text.

FOR RELEASE NOTES:

This patch adds an explanation of the overwriteEnabled configuration option.
This controls dCache's behaviour when user attempts to overwrite an existing
file via FTP or SRM.

The default behaviour is not standards-complaint.  This patch does NOT
alter this behaviour.  Sites are invited to review their configuration.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/5544/
Acked-by: Dmitry Litvintsev
Acked-by: Tigran Mkrtchyan
Acked-by: Gerd Behrmann
Requires-notes: yes
Requires-book: no
Merge pull request #59 from paulmillar/fix/2.2/ftp_overwrite

Fix dCache FTP and SRM to be standards-compliant by default
dcache: billing -- fix error in liquibase varchar changeset

version of http://rb.dcache.org/r/5555 for dCache 2.2

module: dcache

The liquibase changeset 4.1.7 was discovered to contain an off-by-one error in the string used to test the condition for changing the varchar length in the billing tables (the actual length of the string was 256, not 257). As a result, 4.1.7. 4.1.8 and 4.1.9 never will run, and all billing installations created by liquibase (which hardcodes the default to 256) remained as such.

The reason for the requested change was that the path and errormessage attributes of certain tables can well exceed 256.

The new changesets correct the error.

Target: 2.2
Patch: http://rb.dcache.org/r/5559
Require-book: no
Require-notes: yes
Acked-by: Dmitry

RELEASE	NOTES: 	Fix string lengths in billing databse tables. Tables created automatically contained character columns limited to 256 which is too restrictive for path and errormessage columns in billinginfo, doorinfo and storageinfo tables. Additionaly, the length of pnfsid string was limited to 36.
A previous patch designed to address this issue failed to so.]
Merge pull request #63 from alrossi/fix/2.2/billing_varchar_lengths

dcache: billing -- fix error in liquibase varchar changeset
pool: Configure timeout for pool to pool transfer

Addresses the issue that a pool to pool transfer does not time out
if the underlying TCP connection "gets stuck".

Back when we used DCAP for pool to pool transfers, we configured TCP
keep alive to ensure that we eventually get a timeout if the TCP stacks
of the two operating systems disagree about the state of the TCP
connection. Ever since we switched to HTTP, this functionality was lost.

At NDGF we observed a situation in which a migration from one pool to
another got stuck because the TCP connection got into an inconsistent
state. The target pool saw the connection as established, while the
source pool no longer had a connection. The transfer forever stayed
in the DoorFinished state, as dCache expected that the TCP connection
would eventually be closed (since dCache knew it had been closed on the
source pool already).

We cannot get Sun's HTTP client to enable TCP keep alive, however we
can configure connect and read timeouts. Since we only ever need these
timeouts for the rare case when the OSes get confused, I don't see a
reason to make these values end user configurable. I have simply hard
coded some conservative timeout values.

Target: trunk
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5556/
(cherry picked from commit 7de28ba592e39c46df4bf430b6d6d1a8151bd901)
Merge pull request #65 from gbehrmann/fix/2.2/rb5556

pool: Configure timeout for pool to pool transfer
GLUE2: add GLUE2EndpointImplementation information

The EGI GLUE2 profile requires us to publish the implementation name
and version for all published endpoints, which we currently don't
do.

This patch adds this information.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/5591/
Acked-by: Gerd Behrmann
Merge pull request #68 from paulmillar/fix/2.2/glue2_add_endpoint_version

GLUE2: add GLUE2EndpointImplementation information
srm: Make login broker of SRM configurable and clean up properties

The patch allows the login brokers used by the SRM to be configurable.
Since the SRM uses two login brokers, one for lookup and one for
registration, I have renamed all related properties such that those
related to door lookup begin with srm.lookup. and those for login
broker registration begin with srm.loginbroker.

When merging into stable branches, the old properties will not be marked
deprecated.

Deprecated by this patch:

srmLoginBrokerUpdatePeriod
srmNumberOfDoorsInRandomSelection

Introduced by this patch:

srm.lookup.loginbroker
srm.lookup.timeout
srm.lookup.population-size
srm.loginbroker
srm.loginbroker.period
srm.loginbroker.threshold
srm.loginbroker.family
srm.loginbroker.version

Target: trunk
Request: 2.6
Request: 2.2
Require-book: yes
Require-notes: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5604/
pool: Improve logging of common errors

Improve log levels and log messages of common errors found in
production log files.

Target: trunk
Require-book: no
Require-notes: yes
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7685
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Dmitry Litvinsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5255/
broadcast: Fix expiration of dynamic subscribers

Addresses the issue that HSM cleaner is broken as it newer
receives pool-up notifications.

BroadcastRegistrationTask supplies a relative timeout, but the
broadcast registration cell expects an absolute timeout.

Target: trunk
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5601/
(cherry picked from commit 6276fca8040315bcf38e9c77a60e3c87258027e1)
Merge pull request #75 from gbehrmann/fix/2.2/rb5255

pool: Improve logging of common errors
Merge pull request #72 from gbehrmann/fix/2.2/rb5604

srm: Make login broker of SRM configurable and clean up properties
Merge pull request #79 from gbehrmann/fix/2.2/rb5601

broadcast: Fix expiration of dynamic subscribers
chimera: update tags mtime on re-write

keep nfs client cache happy

Acked-by: Dmitry Litvintsev
Patch: http://rb.dcache.org/r/5617/
Target: trunk, 2.6, 2.2
Require-book: no
Require-notes: no
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>