pool: Fix race condition that leads to orphaned files

Using FileNotInCacheException cause the chimera registration to be cleared.
Clearly we don't want this when the file is actually in the pool and just
happens to be in a state in which it isn't accessible.

This patch changes the exception to LockedCacheException.

This is an alternative to patch: https://rb.dcache.org/r/5801

Target: trunk
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7103/
(cherry picked from commit ade506eb0c921bdeb4284d42e7c247794b3e5d1a)
Merge pull request #679 from kschwank/2.7

Suppress nfs' file protocol from being published
Merge pull request #681 from gbehrmann/fix/2.7/rb7103

pool: Fix race condition that leads to orphaned files
[maven-release-plugin] prepare release 2.7.16
[maven-release-plugin] prepare for next development iteration
deb: Add python-psycopg2 as a recommended dependency

Needed by dcache-star.

Target: trunk
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7091/
(cherry picked from commit efa2de674ac1ca5aa1b18d0aba186ee98c4ead22)
alarms: Fix checksum alarm filter.

The filter provided for checksum alarms generated by the pool scrubber
is out of sync with the actual logging statement in the scrubber.

This patch fixes the regex for the filter to match.

Target: 2.7
Require-book: no
Require-notes: yes
Acked-by: Karsten
Acked-by: Paul

RELEASE NOTES

Fixes a bug present since April 2013 (but up to now undiscovered by us) preventing
reporting of checksum alarms from the pool scrubber.
Merge pull request #692 from gbehrmann/fix/2.7/rb7091

deb: Add python-psycopg2 as a recommended dependency
Merge pull request #696 from alrossi/fix/2.7/checksum_alarm_filter

alarms: Fix checksum alarm filter.
poolmanager: Avoid request leak

When a read pool request waits for a file to be replicated or staged, it
registers a callback for the reply from the pool.

We have observed a situation, in which there were many more entries in the hash
table of callbacks than there were requests. The only code path I could find that
could explain this behaviour is when resubmitting a stage request. I am not
certain when exactly this would happen, but it is the only scenario I can see
how entries in the map could leak.

The patch also removes the 'rc destroy' command. The implementation was incomplete
and dangerous to use.

Target: trunk
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7179/
(cherry picked from commit abdb8d47fb31af99b539a32369b06d94f5fbe4da)

Conflicts:
	modules/dcache/src/main/java/diskCacheV111/poolManager/RequestContainerV5.java

(cherry picked from commit ca2afba19a8255fc1bc401f871a83cfabf44c981)
poolmanager: Fix 'request clumping limit reached' failures

Pool manager has an incomplete feature in which a read pool selection is at
most used for a certain number of requests. This limit is hard coded to one,
meaning that as soon as two clients ask for the same file, the second will
receive an error. For some protocols the door silently retries. For other
protocols, the error is propagated to the client.

This patch changes the behaviour in two ways:

The limit is raised to 20, meaning up to 20 clients will receive the same pool
selection. The reason not to put this to infinity is because we still want to
trigger load based replication. 20 is an arbitrary number, that reduces the
risk of triggering the clumping warning, while also protecting against starting
too many movers on a single pool without pool manager getting a chance to
intervene.

The error code is changed from one indicating an out of resource problem to
one that simply indicates that the request is out of date - the latter causes
the door to refresh information about the file and resubmit the pool selection.

Target: trunk
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7184/
Patch: https://rb.dcache.org/r/7180/
(cherry picked from commit 38e3ac367a6ac7696bed1b863388830dfe46194c)
build: add work-around for JDK bug and PowerMock

When mocking recalcitrant classes, the unit-tests uses powermock
to generate and inject the necessary byte-code to allow testing
independent of the underlying platform.

With OpenJDK v1.7.0_60, powermock-generated byte-code functions
correctly.  With OpenJDK v1.7.0_65, this generated byte-code
fails will errors like:

    java.lang.VerifyError: Bad <init> method call from inside of a branch
    Exception Details:
      Location:
        org/dcache/gplazma/plugins/NsswitchTest$SimpleIntStorage.<init>(Lorg/dcache/gplazma/plugins/NsswitchTest;)V @39: invokespecial
      Reason:
        Error exists in the bytecode
      Bytecode:
        0000000: 2a2b b500 142a 4d12 16b8 001c 03bd 000f
        0000010: 121e b800 22b8 0028 3a04 1904 b200 2ca5
        0000020: 000e 2a01 c000 2eb7 0031 a700 092c b700
        0000030: 3301 57b1
      Stackmap Table:
        full_frame(@45,{UninitializedThis,Object[#13],UninitializedThis,Top,Object[#15]},{})
        full_frame(@51,{Object[#2],Object[#13],Object[#2],Top,Object[#15]},{})

            at sun.reflect.GeneratedSerializationConstructorAccessor7.newInstance(Unknown Source)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
            at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:40)
            at org.powermock.reflect.internal.WhiteboxImpl.newInstance(WhiteboxImpl.java:229)
            at org.powermock.reflect.Whitebox.newInstance(Whitebox.java:139)
            at org.dcache.gplazma.plugins.NsswitchTest.setUp(NsswitchTest.java:86)

This problem has been reported and will hopefully be fixed at some
point, but meanwhile, this patch adds a work-around.

Target: master
Patch: https://rb.dcache.org/r/7185/
Acked-by: Gerd Behrmann
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Merge pull request #723 from gbehrmann/fix/2.7/rb7179

poolmanager: Avoid request leak
Merge pull request #728 from gbehrmann/fix/2.7/rb7180

poolmanager: Fix 'request clumping limit reached' failures
[maven-release-plugin] prepare release 2.7.17
[maven-release-plugin] prepare for next development iteration
Update chimera-cli

Gives some more hints when invoked wrongly.
This is perhaps not a very useful patch since the command is dropped in 2.10, but I hope you like it anyway.
I hope I chose the correct branch.
bugfix: fix reverseMap not throwing NoSuchPrincipalExceptions

Instead of throwing a NoSuchPrincipalException on failed reverse
mappings the Ldap plugin would just return an empty set.
This fixed this misbehaviour.

Ticket: http://rt.dcache.org/Ticket/Display.html?id=8422
Acked-by: Paul
Target: trunk
Target: 2.7
Target: 2.8
Target: 2.9
Target: 2.10
Require-book: no
Require-notes: yes
Merge pull request #740 from kschwank/bugfix/ldapReverseMap8422/2.7

bugfix: fix reverseMap not throwing NoSuchPrincipalExceptions
xrootd: Add billing entry on delete

In contrast to other doors, the xrootd door failed to generate billing
notifications on file deletion. This patch resolves this issue.

Target: trunk
Require-notes: yes
Require-book: no
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7190/
(cherry picked from commit 3425431abcf5457c26e94b21e415b35fbcb7cd78)

Conflicts:
	modules/dcache-xrootd/src/main/java/org/dcache/xrootd/door/XrootdDoor.java