hsqldb-plugin: fix transposed varchar settings

The longer varchar (8000) should appear on errormessage but was erroneously given to a different field in a few of the tables.

Target: master
Request: 2.7
Request: 2.6
Require-notes: no
Require-book: no
Acked-by: Gerd
Acked-by: Paul
srmspacemanager: remove code path that has neever been used

    remove code path which has never been used, but was exposed in
    commit f03d1d73fdf79f066a90ac5be2d20ae8ded760f9 by mistake

    Patch: http://rb.dcache.org/r/6100/
    Acked-by: Gerd Behrmann
    Target: trunk
    Require-book: no
    Require-notes: no
dcache-fhs: fix missing chown for httpd directory in debian postinst

Target: master
Request: 2.7
Request: 2.6
Require-book: no
Require-notes: yes
Acked-by: Gerd

RELEASE NOTES
Fixes bug in debian installation preventing writes to the httpd directory and thus blocking creation of webadmin directory.
webdav: Fix error in config documentation

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6107/
webdav: Fix typos in webdav Spring configuration

Also added a webdav instance using the jglobus backend to the
system-text configuration.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6108/
spacemanager: bug fix. Always remove file from srmspacefile table

   commit f03d1d73fdf79f066a90ac5be2d20ae8ded760f9 introduced a bug:
   a boolean variable that previously has always been true, was set to
   false exposing undesirable code path resulting in deleted files
   not being removed from srmspacefile table. Since this variable has never
   been exposed to users this patch simply excised code that deals with
   the variable. Issue existed only on the head.

    Acked-by: Gerd Behrmann
    Target: trunk
    Require-book: no
    Require-notes: no
dcache: httpd service, add RequestLog to Jetty handler

If the Jetty RequestLogHandler is not given a request log implementation, it logs at the WARN level a message and then sets an internal Null implementation.

Given that the log message is annoying, we have provided a simple implementation of the log which delegates to the slf4j logger at the TRACE level.

Testing:

Before patch, with logger for domain set to WARN or higher, we see the jetty warning

15 Oct 2013 08:45:28 (httpd) [] !RequestLog

With patch, this disappears.  Logging of request/response, set at trace, looks like this:

15 Oct 2013 09:41:39 (httpd) [] request: [GET /webadmin/cellinfo]@794262320 org.eclipse.jetty.server.Request@2f577b30; response: HTTP/1.1 302
Date: Tue, 15 Oct 2013 14:41:39 GMT
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Pragma: no-cache
Cache-Control: no-cache, no-store
Location: http://localhost:2288/webadmin/cellinfo?1

Target: master
Request: 2.6
Patch: http://rb.dcache.org/r/6115
Require-notes: no
Require-book: no
Bug: http://rt.dcache.org/Ticket/Display.html?id=8050
Acked-by: Gerd
dcache: httpd service, remove ipv4 specific binding and expose httpd.net.listen

The Jetty ssl connector was hardcoded to listen on "0.0.0.0".  This is an ipv4-specific mask.  The simple connector and the ssl connector also did not allow for a different net binding.

This patch removes the hardcoded binding of host name and exposes an httpd.net.listen property which defaults to dcache.net.listen.

Since new properties cannot be ported to 2.6, a separate patch which simply removes the hard-coded ipv4 setting will be provided.

Testing:

Checked that the default settings still work.

Target: master
Request: 2.6
Patch: http://rb.dcache.org/r/6116
Require-notes: yes
Require-book: no
Acked-by: Gerd

RELEASE NOTES:
Allows for ipv6 binding and (in master) exposes a new property for alternate net binding.
javatunnel: add certificate chain to subject

Failing XACML authentication through the gsidcap door.

The issue is that XACML, being both an auth and map plugin, requires the full certificate chain to be in the Subject.  The gsi tunnel was only adding a GlobusPrincipal and an FQANPrincipal to the subject.

The tunnel now extracts the chain from the GSSCredential object and places it in the public credentials set of the subject.

Testing:

Before the patch, the XACML plugin would fail because it had incomplete information and which it processed in a faulty manner.  The error resulted from a mapping like the following:

[DCap-gsi-00-fndca4a-/DC=com/DC=DigiCert-Grid/O=Open Science
Grid/OU=People/CN=Kenneth Herner 1385-133 Login AUTH xacml] mapping
service
https://gums.fnal.gov:8443/gums/services/GUMSXACMLAuthorizationServicePort
returned localId LocalId[] for VomsExtensions[X509Subject='/fermilab',
X509SubjectIssuer='null', fqan='null', primary=false, VO='null',
VOMSSubject='null', VOMSSubjectIssuer='null']

After the patch, the dccp operation succeeds.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/6112/
Require-book: no
Require-notes: yes
Acked-by: Tigran

RELEASE NOTES:
Fixes a bug whereby a combination of gsidcap with a VOMS cert and XACML authentication/mapping failed to authorize the user.
skel: exclude unneeded freehep/aida dependencies

The freehep maven repository pulls in a number of dependencies which are not necessary for our runtime requirements.

After trial-and-error twiddling, I have determined that the following constitute the minimal set:

freehep-application-2.0.3.jar
freehep-graphics2d-2.2.1.jar
freehep-graphicsbase-2.2.1.jar
freehep-graphicsio-2.2.1.jar
freehep-io-2.2.2.jar
freehep-jaida-3.4.1.jar
freehep-swing-2.0.6.jar
freehep-util-2.0.5.jar
aida-3.3.1.jar
aida-dev-3.3.jar
jel-2.0.1.jar
openide-lookup-1.9-patched-1.0.jar

Hence the pom.xml has been modified to exclude those that are safe to eliminate.  Note that direct dependencies only is not an option as freehep-jaida-3.4.1.jar is needed, but carries many of the other unnecessary dependencies with it.

Testing:

Excluded subsidiary dependencies then added them back in until plots were generated without error.

Target: master
Request: 2.6
Patch: http://rb.dcache.org/r/6117
Require-notes: no
Require-book: no
Acked-by: Gerd
srm: Clean up and fix permission operations

Addresses issues in the implementation of srmCheckPermission,
srmGetPermission, and srmSetPermission, all of which failed to
use the proper error codes.

Addresses an issue in srmSetPermission in which permission bits
where toggled rather than cleared. We now report that the REMOVE
operation is not supported as the semantics described in the SRM
spec only is applicable to ACLs.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6094/
cells,spacemanager: Fix forwarding of messages

Addresses a bug that caused space manager to forward
messages twice.

AbstractCell contains logic to detect whether a message
handler already generated a reply/forwarded a message.
For reply detection, the current mechanism of looking at
the message UOID works well: Whent the direction of an
envelope is reversed and the envelope is sent, the UOID
is updated. For forwarding this does not work as the
UOID doesn't change.

The fix is a hack, really. Instead of looking at the
UOID we look at the current address in the destination
path. If it changed, it is a clear sign that the message
was forwarded.

For space manager we have the additional challenge that
the message is forwarded asynchronously. We solve that
problem by advancing the destination path to the next
address synchronously, and then forward it in a background
thread.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Dmitry Litinvtsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6101/
bootloader: Add check-config support for wrong service types

Addresses the problem that 'dcache check-config' doesn't provide information
about mistyped or wrong service types in the layout. Such errors would only
show up once a domain is started and the batch file cannot be found.

Added similar support for missing log configuration files.

I had to refactor some code to implement this correctly: Whether a service
type is valid cannot be seen from the section header alone, as the service
specific configuration may provide alternative paths for were to locate
batch files.

Note that the line number printed is off by one as it refers to the first
line after the section header (the beginning of the section specific
configuration). Fixing this requires more refactoring so for now I settled
on the current solution.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6109/
bootloader: Extend check-config to log non-standard properties

Addresses the issue that there is no easy way to find mistyped
property names in dCache configurations. 'dcache check-config'
now logs messages at INFO level about property declarations that
are not known to dCache.

Messages are only logged at INFO as dCache allows the user to
add custom properties used to declare other properties, or
used by third party scripts. Such properties are thus not
errors.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6111/
webdav: Exclude unused milton dependencies

The milton library also supports CalDAV. We do not utilize this
functionality in dCache and can thus exclude dependencies introduced
by it.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6114/
Make DirectoryStream AutoClosable and use try-with-resource

Also fixes a leak in RemoetNameSpaceProvider.

Target: trunk
Require-trunk: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6119/
srm: Clean up mkdir and rmdir implementations

Improves error codes and error messages of srmMkdir and srmRmdir
operations. Improves performance of srmRmdir.

By moving some functionality into the Storage class, the logic
could be simplified and made faster.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6120/
srm: Clean up srmRm implementation

Improves return codes and error messages.

The batching logic is changed to not be as bursty as the before.
Rather than submit a patch and wait for all to finish before
submitting the next batch, the code now submits new requests as
previous requests finish. Thus remove requests will be more
interleaved with other PnfsManager requests, and we can maintain
a high throughput with a lower limit than before. For this reason
I reduced the default value of srm.limits.remove-batch-size to 50.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6121/
Define default transaction isolation level for databases

BoneCP doesn't remember the default isolation level of the
actual Connection. Instead it injects a configured default,
or leaves the setting untouched if not defined. The latter
means that the isolation level is unpredictable if it is
changed by some code and then the connection is recycled
by some other code.

In the SRM this leads to errors like:

16 Oct 2013 22:33:37 (SRM-sisyphus) [131.169.72.212:51026 3Lg:9:srm2:prepareToPut] Please report this failure to support@dcache.org
org.springframework.dao.CannotSerializeTransactionException: PreparedStatementCallback; SQL [UPDATE srmrequestcredentials SET creationtime=?, credentialname=?, role=?,  numberofusers=?, delegatedcredentials=?, credentialexpiration=? where id=? ]; ERROR: could not serialize access due to concurrent update; nested exception is org.postgresql.util.PSQLException: ERROR: could not serialize access due to concurrent update
	at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:273) ~[spring-jdbc-3.2.3.RELEASE.jar:3.2.3.RELEASE]

This error is caused by the isolation level being higher than
expected.

This patch sets the default to READ_COMMITTED. This matches
the PostgreSQL default, which is what our code is developed
for.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6122/
srm: Clean up srmMv implementation

Minor improvements to return codes and error messages.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6123/