pool: Make xrootd max frame size configurable

Addresses the issue that the xrootdMoverMaxFrameSize property was
ignored by the xrootd mover.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5622/
Squashed commit of the following:

commit f13fd991d99207f321c20642ef8c4d28c29c6703
Author: Dmitry Litvintsev <litvinse@fnal.gov>
Date:   Mon Jun 10 18:57:03 2013 -0500

    do not check gplazmaPolicy variable embedded gPlazma fragment
    module: do not check gplazmaPolicy variable embedded gPlazma fragment

    do not check gplazmaPolicy variable embedded gPlazma fragment

    Ticket:
    Acked-by: Tigran Mkrtchyan
    Patch: http://rb.dcache.org/r/5635/
    Target: trunk
    Require-book: no
    Require-notes: no
Inject configuration properties directly into spring cells

Addresses the issue that configuration properties had to be repeated
as cell arguments to make them available in spring. With this patch,
the cell environment is injected directly into the spring application
context, thus simplifying the batch files.

The patch updates all batch files instantiating UniversalSpringCell.

This change was discussed during the 2013 developers workshop.

A few properties have been altered:

- billing now uses 'poolmanager' rather than 'poolManager'. The
  'poolManager' property has been marked as forbidden.

- pnfsmanager's defaultPnfsServer properties has been marked
  obsolete. It only applied to PNFS and we no longer support PNFS.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5623/
poolmanager: Make PSU collections thread safe and add accessors

The patch adds getters to the PSU to allow the 'bean properties' shell
commands to inspect the structure. Some entities were already exposed
through getters, others where not. Some entities were exposed through
non-getter methods.

While doing so, I discovered that many entities use non-thread safe
maps and sets internally. Even though the PSU uses a reader-writer
lock internally to protect updates, this isn't enough as we leak
references to the entities to the outside. Thus we can have concurrent
writes and reads on those collections. I replaced the collections
with thread safe equivalents.

I added setters and getters for many fields that were previously
accessed directly.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5607/
Fix typo introduced in 99c801f7b8a3f9dbeea210464f253b87b2212cd6
webadmin: do not run billing service if database flag is off

module:  webadmin (billing)

The billing service which contacts the billing cell should not attempt to retrieve data if the billing database has been disabled.

The original code had planned to enable this through Spring profiles, and the property was already in place to do so, but in the shuffle between versions the actual spring <beans> stanza was not implemented.

However, after some testing it was discovered that profiles do not seem to play with Wicket Spring injection, and the @SpringBean(required=false) annotation does nothing to stop the Spring NoSuchBeanDefinition error.

We have therefore adopted a slightly different solution.  It turns out that it is only minimally invasive and fairly clean.

Patch: http://rb.dcache.org/r/5641
Target: master
Request: 2.6
Require-book: no
Require-notes: yes
Acked-by: Gerd

Testing:

Before the fix, with billingToDB=no but generatePlots=true, one would get

13 Jun 2013 16:59:59 (System) [] Uncaught exception in thread StandardBillingServiceRefresher
java.lang.reflect.UndeclaredThrowableException: null
        at com.sun.proxy.$Proxy9.getDcBytesHistogram(Unknown Source) ~[na:na]
        at org.dcache.webadmin.controller.impl.StandardBillingService.load(StandardBillingService.java:148) ~[classes/:na]
        at org.dcache.webadmin.controller.impl.StandardBillingService.generatePlot(StandardBillingService.java:351) ~[classes/:na]
        at org.dcache.webadmin.controller.impl.StandardBillingService.refresh(StandardBillingService.java:294) ~[classes/:na]
        at org.dcache.webadmin.controller.impl.StandardBillingService.run(StandardBillingService.java:307) ~[classes/:na]
        at java.lang.Thread.run(Thread.java:722) ~[na:1.7.0_19]
Caused by: diskCacheV111.util.TimeoutCacheException: Request to [>billing@local] timed out.
        at org.dcache.cells.CellStub.sendAndWait(CellStub.java:321) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at org.dcache.cells.CellStub.sendAndWait(CellStub.java:221) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at org.dcache.cells.CellStub.sendAndWait(CellStub.java:212) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]
        at org.dcache.services.billing.histograms.data.TimeFrameHistogramDataProxy.invoke(TimeFrameHistogramDataProxy.java:93) ~[dcache-core-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT]

With the patch applied, the following combinations were tried:

billingToDb=no, generatePlots=true:   the page is not rendered (as expected)
billingToDb=yes, generatePlots=false: the page is not rendered (as expected)
billingToDb=yes, generatePlots=true:  the page is rendered (as expected)

The TimeoutCacheException does not occur, thus producing no proxy error (since the client is turned off).

RELEASE NOTES:

Fixes a bug in the new billing setup where billingToDb=no, generatePlots=true does not turn off the billing client and a TimeoutCacheException is returned since there is no database-based request service on the BillingCell side.
webadmin: fix incorrect getter in StandardBillingService

module:  webadmin (billing)

The getPopupHeight() returns the width value.

The error is so minor and the patch so new (it has not been merged to 2.6 yet) that the fix requires no release notes.

Patch: http://rb.dcache.org/r/5646/
Target: master
Request: 2.6
Require-book: no
Require-notes: no
Acked-by: Gerd
spacemanager: unknown message have to be sent to Poolmanager

as SpaceManager works as a proxy, not all messages handled as
expected.

Ticket: 7844
Acked-by: Gerd Behrmann
Target: trunk, 2.6
Require-book: no
Require-notes: yes
Merge branch 'iss7844'

Acked-by: Gerd Behrmann
libs: update jpnfs to 0.5.1

with following bug fixes and improvements:

- support for old file handles, which will allow to update nfs server without
  remount of existing clients (important for backtort to 2.2).

- reject mount from clients which do not have any export entries.
- respect root user if no_root_squash option is specified in exports file

  [3eb23ef] vfs: fix no_root_squash exports
  [5c12e5d] vfs: support old file handles
  [694da2f] vfs: construct new file handles from old one
  [513d265] test: add unit test for FileHandle and FileHandleBuilder
  [56d803b] nfsv4: consult export file on getRootInode()

Acked-by: Gerd Behrmann
Target: trunk, 2.6
Require-book: no
Require-notes: yes
webdav: Use HTTP for proxied upload

The webdav door allows uploads and downloads to be proxied throught the door
as an alternative to redirecting the client to the pool. For download we have
used HTTP for the transfer between the door and the pool, but for upload we
used an FTP data channel. We did this because originally the HTTP mover did
not support upload.

The current mover has however supported upload for quite a while and thus
it is time to let the door use HTTP for uploads too. The primary admin visible
change is that the TCP connection is now created from the door to the pool
rather than from the pool to the door. The configuration property
webdavInternalAddress is now used as a hint for the pool to select an interface
facing the door. The property is respected by both proxied downloads and
proxied uploads.

Target: trunk
Require-book: no
Require-notes: yes
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5645/
Merge remote-tracking branch 'upstream/master'
system-test: Add upload/download test for plain http

Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5643/
Require-notes: no
Require-book: no
nfs: return NFSERR_DELAY if we can't kill mover

If mover takes to much time to finish, then we will simply return OK.
In most cases this can go well. Nevertheless, it checksumming is
enabled, then file will be unavailable and file size still will be 0.

To solve this,  we return NFSERR_DELAY and enforce client to retry.
The problem here is to return NFSERR_IO if pool is unavailable.

Observed at DESY on transfer of big files
( measured checksumming seed is 1.6GB/s )

Acked-by: Gerd Behrmann
Target: trunk, 2.6
Require-book: no
Require-notes: yes
nfs: add missing properties

misses in 99c801f

Acked-by: Gerd Behrmann
Target: trunk
Require-book: no
Require-notes: no
nfs: added enable/disable switch to embeded portmat

In some cases we do not want to start embedded portmap
even if OS does not provide one.

introduced property

nfs.enable.portmap=true/false

Default it true for backward compatibility

Acked-by: Gerd Behrmann
Target: trunk
Require-book: no
Require-notes: yes
libdcap: remove redundant code

The dcap code is since middle of January 2013 has
its own repository:

https://github.com/dCache/dcap

Acked-by: Gerd Behrmann
Target: trunk
Require-book: no
Require-notes: no
libs: Upgrade to JGlobus 2.0.6-rc3.d

Addresses a compatibility issue with some versions of curl and other
clients that send empty SSL messages.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5659/
gplazma2-grid: Fix user name regression in authzdb plugin

Addresses the issue that authzdb may add a second UserNamePrincipal and thus
cause the validation phase of a login to fail. This is a regression introduced
in 2.6.1.

The patch restores the old behaviour. The logic is not exactly beatiful, but
given that space manager relies on authorization by user name, it makes sense
that the user name is consistent with the UID. Hence if UID is selected by
something different that user name, then we have to replace the user name.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Patch: http://rb.dcache.org/r/5660/
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
logging: Set remote appender log level to off by default

Unless explicitly configured, we should not attempt to send any log events to
the remote appender (both for performance reasons and as a workaround for the
problem with the socket appender connecting to itself).

Target: trunk
Request: 2.6
Require-book: yes
Require-notes: yes
Patch: http://rb.dcache.org/r/5664/
Acked-by: Albert Rossi <arossi@fnal.gov>