Allow retry on transient transfer failures

This patch fixes a minor regression introduced in 2.8 that prevents retry
on a transient error. The patch also removes retry on a permanent error.

Target: trunk
Request: 2.8
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6707/
(cherry picked from commit ef8ef96c690b3f005c4388e404f2fc7faf8f6993)
spacemanager: Fix database sequence initialization

The default range for sequences starts at 1. This caused failures
with the schema changeset when migrating from a database where the
largest id was smaller than 1.

Target: trunk
Request: 2.8
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6711/
(cherry picked from commit e3ac8f69bd2e42b33066d169b6dd7ff95f476a6e)
pool: Fix pf command regression

A recent patch changed a number of variables and fields from String to
FsPath. Inadvertently the return message of the 'pf' command in the pool
was changed to, leading to Serialization errors.

Target: trunk
Request: 2.8
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6713/
(cherry picked from commit baff93afad3518e64ab0075904fedba0b7612d48)
http: fix multipart reply

fixes compatibility with libneon ( davix and ROOT )

after second reading of http://tools.ietf.org/html/rfc2616
each Range block should not contain Content-length, but
complete requests must.

Acked-by: Gerd Behrmann
Target: master, 2.8, 2.7, 2.6
Require-book: no
Require-notes: no
(cherry picked from commit 702c68b36d369876a70a5ef8cf81dded9cf740ec)
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Merge pull request #461 from gbehrmann/fix/2.8/rb6713

pool: Fix pf command regression
Merge pull request #460 from gbehrmann/fix/2.8/rb6711

spacemanager: Fix database sequence initialization
Merge pull request #459 from gbehrmann/fix/2.8/rb6707

Allow retry on transient transfer failures
http: adjust unit test to multirage changes

Acked-by: Paul Millar
Target: master, 2.8, 2.7, 2.6
Require-book: no
Require-notes: no
(cherry picked from commit a000a7301b9ef31bfc35d184d367bf98f3344784)
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
nfs: return NFSERR_IO is we detect broken file on tape

currently we return NFSERR_LAYOUTTRYLATER and client
tries later. Stop infinite loop as we cant get file
anyway.

Acked-by: Gerd Behrmann
Acked-by: Paul Millar
Target: master
Require-book: no
Require-notes: no
(cherry picked from commit ef2091a9130cc0206e2534685b34ea5d6abe5590)
Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Merge pull request #464 from kofemann/fix-http-davix-2.8

http: fix multipart reply
Merge pull request #466 from kofemann/stop-on-broken-tape-2.8

nfs: return NFSERR_IO is we detect broken file on tape
[maven-release-plugin] prepare release 2.8.2
[maven-release-plugin] prepare for next development iteration
srm: fix race condition if stop is called too soon after start

The WorkSupplyService fetches information that is protected by
the Scheduler monitor.  If the stop method is called while the
WorkSupplyService is not resting then the thread calling stop
enters the Scheduler monitor and waits for the WorkSupplyService
service to end, resulting in a deadlock.

This patch fixes this by moving the WorkSupplyService activation
and deactivation so it happens outside the Scheduler monitor.

Target: master
Acked-by: Dmitry Litvintsev
Patch: http://rb.dcache.org/r/6750/
Request: 2.8
Merge pull request #467 from paulmillar/fix/2.8/srm-start-stop-race

srm: fix race condition if stop is called too soon after start
loginbroker: Fix login broker registration retry

Fixes a problem in which doors would log "Failed to send update to LoginBroker"
in a busy retry loop when unable to register with LoginBroker. This typically
happens during shutdown.

The intention of the code was to switch to an eager retry loop with a one
second sleep between retries, however the code switches the update period to
'normal' update before attempting to send an update and then switches back to
'eager' mode. Changing the scheduling time causes the task to be rescheduled,
but rescheduling schedules a new update with an initial delay of zero, thus
causing an immediate retry. Thus failure to deliver an update initiates a
tight retry loop without any delays.

Also fixes a bug in which the 'lb set update' command was not respected.

Target: trunk
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6787/
(cherry picked from commit 6cbd994e7a2e68d34543cc461467e37ac6a2bb6b)
pool: Fix lock problems in xrootd and http movers

When the pool is full, a blocking preallocate call could block
simple 'mover ls' queries and thus cause the entire pool to appear
unresponsive.

Target: trunk
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6795/
(cherry picked from commit a5b01e4a15f473c8c7c02205cbc1d4ee01151e04)
pool: Avoid memory exhaustion problems on write with xrootd and http

The http and xrootd movers use an OrderedMemoryAwayThreadPoolExecutor. This
thread pool limits memory usage per client, however it relies on being able to
estimate the size of the messages in the queue. It knows how to calculate the
size for ChannelBuffer objects, however it doesn't know how to do this for most
other objects. Since we store protocol messages in the queue, the limit doesn't
have the intended effect. The consequence is that a slow disk or full pool can
cause the queue to fill up and trigger OOM.

The patch solves this by moving the ExecutorHandler to the front of the
pipeline such that only ChannelBuffers are stored in the queue.

Target: trunk
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6796/
(cherry picked from commit 7e6df2048c31c62cbd75b0aac0c1817b048f3faa)
billing: Make indexer case insensitive

The billing indexer turns the billing output patterns into regular expressions
and uses those when parsing billing files. Literal text in the pattern are
translated to literal text in the regular expression.

Unfortunately, at some point we altered the capitalization of 'Unknown' in the
output pattern. This causes the billing indexer to fail to match older billing
files.

The patch changes the pattern to case insensitive matching. The generated bloom
filter is still case sensitive.

It is recomended that existing indexes are reindexed using the
'/usr/sbin/dcache-billing-indexer -all' command.

Target: trunk
Require-notes: yes
Require-book: no
Request: 2.8
Request: 2.7
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6798/
(cherry picked from commit ca3f2447a8a4f2658dd37d94efced5e68f691b78)
srm: Fix scheduling period for expiration task

I believe the intention was to schedule the task every 60 seconds, not every
60 milliseconds.

Target: trunk
Request: 2.8
Request: 2.7
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6816/
(cherry picked from commit 8d4b4bd7adf6f85989d21a4b3a7c71f1ea987f3c)