webdav: switch hard-coded JGlobus paths to those configured

The WebDAV door may use either Jetty's built-in SSL support or
support coming from JGlobus.  If the latter is chosen then
various paths are hard-coded.

This does not respect a site's configuration and, as a specific
example, results in the system-test failing.

This patch switches the JGlobus WebDAV SSL connector to use
the configured path.

Patch: http://rb.dcache.org/r/5575/
Acked-by: Gerd Behrmann
Target: master
Request: 2.6
webadmin: make query and filter options set in browser session-specific (alarms, poolqueueplots)

module: webadmin

The alarms and pool queue plots pages were erroneously storing query settings at the server level.  These were thus unintentionally being shared across user sessions.

This patch rectifies this behavior so that user settings do not interfere with each other.

Target: master
Patch: http://rb.dcache.org/r/5577
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Tigran

Testing: With two different browsers with different settings.

RELEASE NOTES

Fixes a bug in the web pages for Alarms and Pool Queue Plots where the settings selected by a user/session would become global.  The fix now isolates user/session settings from each other.
dcache: alarm command line, fix minor bug

module: dcache, alarm command line interpreter

This patch does the following:

1.  Fixes bug in add and modify whereby includeInKey was not getting updated.  It also adds "group" to the help message for the attribute (missing previously).
2.  Prints out the full definition for confirmation/review before commit in both add and modify.
3.  Allows the elimination of an attribute (XML element) via the '-' character.

Target: master
Patch: http://rb.dcache.org/r/5573
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Tigran

Testing: Tried out various options using add, remove and modify for illegal alteration by deletion of required elements; modified includeInKey, and tried to set group (with number and without, which is an error).

RELEASE NOTES:

For those who have tried out the dcache alarm definition commands, they may have noticed that "includeInKey" could not be modified once it was defined.  This bug has been fixed.  There is also a new feature allowing the deletion of a definition element using the '-' character.  The full xml for the definition is now also displayed before asking for confirmation to write.
Fix dCache FTP and SRM to be standards-compliant by default

The overwriteEnabled property affects FTP and SRM doors.  The default
is currently 'false', which prevents overwriting of data if the file
already exists.

RFC 959 is quite clear:

         STORE (STOR)

            This command causes the server-DTP to accept the data
            transferred via the data connection and to store the data as
            a file at the server site.  If the file specified in the
            pathname exists at the server site, then its contents shall
            be replaced by the data being transferred.  A new file is
            created at the server site if the file specified in the
            pathname does not already exist.

Therefore, by setting overwriteEnabled to 'false', dCache is not
RFC-959 compliant.  This is our default behaviour.

Similarly, SRM protocol is also quite clear how storage elements are
expected to behave.  Setting overwriteEnabled=false breaks that
expected behaviour.

Moreover, there is no additional security from setting
overwriteEnabled=false.  The client can simply delete the data and
upload afresh.

The net result is that dCache is, by default, not standards compliant
for no direct benefit.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/5544/
Acked-by: Dmitry Litvintsev
Acked-by: Tigran Mkrtchyan
Acked-by: Gerd Behrmann
libs: use jglobus-2.0.6-rc2

includes fixes for cert validation and race conditions
Acked-by:
Target: trunk, 2.6
Require-book: no
Require-notes: no
Ban elliptic-curve ciphers to prevent unsupported ciphers from being selected

Elliptic-Curve (EC) based ciphers offer advantages over RSA- and
DSA-based ciphers and are being introduced, albeit slowly.  OpenJDK (6 &
7) use external libraries, such as libnss3, to provide support for
various cipher-suites, including support for EC ciphers.  The libnss3
library provide only partial support for all of the EC-based ciphers.

Unfortuantely, the generic front-end believes all ciphers are supported.
So, if during an SSL/TLS negotiation, the client requests an EC cipher
that libnss3 doesn't support, the SSLEngine may choose that cipher.  The
result is an exception being thrown with the cryptic
'CKR_DOMAIN_PARAMS_INVALID' message.

Rather than trying to deploy work-arounds in the client or trying to
document how to disable EC-support in the server using JVM-related
configuration, this patch bans the EC ciphers programatically.

Patch: http://rb.dcache.org/r/5527/
Acked-by: Gerd Behrmann
Target: master
Request: 2.6
dcache: billing -- fix error in liquibase varchar changeset

module: dcache

The liquibase changeset 4.1.7 was discovered to contain an off-by-one error in the string used to test the condition for changing the varchar length in the billing tables (the actual length of the string was 256, not 257). As a result, 4.1.7. 4.1.8 and 4.1.9 never will run, and all billing installations created by liquibase (which hardcodes the default to 256) remained as such.

The reason for the requested change was that the path and errormessage attributes of certain tables can well exceed 256.

The new changesets correct the error.

Target: master
Patch: http://rb.dcache.org/r/5555
Require-book: no
Require-notes: yes
Request: 2.6
Acked-by: Dmitry

Testing done:  Ran changesets on both new and previously installed instances, observing correct attribute definitions in the table afterwards.

RELEASE	NOTES: 	Fixes string lengths in billing databse tables. Tables created automatically contained character columns limited to 256 which is too restrictive for path and errormessage columns in billinginfo, doorinfo and storageinfo tables. Additionaly, the length of pnfsid string was limited to 36.
A previous patch designed to address this issue failed to so.
gplazma: Rename dcache-gsi module to gplazma2-gsi

The gsi module used the dcache- prefix even though it is independent
of all dcache components. It is much more closely tied to the gPlazma 2
project (which is independent of dCache). Hence this patch renames the
module.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5581/
webadmin: Rename module to dcache-webadmin

Webadmin is a dCache component (it relies heavily on dCache classes) and
the module should thus be called dcache-webadmin to follow the convention
used for other modules.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5583/
packaging: Move DEB and RPM packaging to Maven and remove Ant configuration

Packaging was the only thing we still did (partially) through ant. Our
packaging is however mostly about putting the right files together and
calling the native packaging tools, and that we can do just as easily
from Maven.

This patch moves the RPM and DEB packaging to the fhs and srmclient
Maven modules. The packaging is triggered by activating the profiles
rpm and deb by using the -P option of Maven. The profiles are exclusive
and must not be activated at the same time. Like the old Ant targets,
the profiles allow the build number and local version strings to be
modified by overriding properties through the -D option of Maven.

Solaris packaging tools are moved to the FHS module too, I do however
not have a Solaris system for testing and thus I leave it to somebody
else to define the necessary Maven profile to invoke the packaging.

I removed all legacy packaging components and all Ant files. This
includes a couple of old spec files, including for dcap. Like for
the Solaris package, I hope somebody else can ressurect this within
Maven (or move the dcap client code out of the dCache repository and
into its own repo).

Finally, I moved the packaging related Maven modules into its
own top-level directory to separate them from the modules that
contain our source code.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5589/
Fix incremetal compilation

The maven compiler plugin version 3.1 contains fixes to incremental
compilation. In addition to upgrading the plugin, the patch removes
an empty file in the cells module that triggers recompilation (since
it doesn't generate a class file and thus always appears as a change
in the source).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Patch: http://rb.dcache.org/r/5590/
logback-test-config: Synchronize logback version with dCache

This is the logback version used during tests.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5592/
Use correct junit dependency

It used to be the case that junit contained the Hamcrest classes rather
than having a Maven dependency on the Hamcrest artifact. To avoid
conflicts, junit provided the junit-dep artifact.

Since junit 4.11, Hamcrest is now specified as a dependency and it
is no longer necessary to use junit-dep.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5593/
pool: Configure timeout for pool to pool transfer

Addresses the issue that a pool to pool transfer does not time out
if the underlying TCP connection "gets stuck".

Back when we used DCAP for pool to pool transfers, we configured TCP
keep alive to ensure that we eventually get a timeout if the TCP stacks
of the two operating systems disagree about the state of the TCP
connection. Ever since we switched to HTTP, this functionality was lost.

At NDGF we observed a situation in which a migration from one pool to
another got stuck because the TCP connection got into an inconsistent
state. The target pool saw the connection as established, while the
source pool no longer had a connection. The transfer forever stayed
in the DoorFinished state, as dCache expected that the TCP connection
would eventually be closed (since dCache knew it had been closed on the
source pool already).

We cannot get Sun's HTTP client to enable TCP keep alive, however we
can configure connect and read timeouts. Since we only ever need these
timeouts for the rare case when the OSes get confused, I don't see a
reason to make these values end user configurable. I have simply hard
coded some conservative timeout values.

Target: trunk
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5556/
Delete legacy external jars

For some time now all external dependencies are imported through Maven, so
we no longer need the directory of external jars.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrthchyan <tigran.mkrtchyan@desy.de>
Patch: none (ack'ed through jabber)
Added instructions on how to build dCache

Target: master
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5595/
dcache: Remove rogue logback-test.xml file

Commit 6a3276cfdcd17c73a4b9d2e57cd7913b6d2b86cf added a custom logback-test.xml
file needed for a single test. The logback configuration however applied to
all tests in the dcache module and resulted in severily bloated log output during
testing.

Since then, the test has been modified and it doesn't appear to rely on the
custom logback configuration anymore. However, the logback-test.xml file
remained.

This patch removes the file.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5594/
system-test: avoid awkward cell.name values

A few of the system-test cell.name values includes a forward
slash character, which the cells shell interprets as an
attempt to switch to the alternative addressing mode.

This patch updates these cells to use the neutral '-' character
instead.

Patch: http://rb.dcache.org/r/5585/
Acked-by: Gerd Behrmann
Target: master
Request: 2.6
GLUE2: add GLUE2EndpointImplementation information

The EGI GLUE2 profile requires us to publish the implementation name
and version for all published endpoints, which we currently don't
do.

This patch adds this information.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/5591/
Acked-by: Gerd Behrmann
Various improvements to short-lived Java invocations

This patch adds several features to improve responsiveness
for various short-lived java invocations and in particular
the chimera-cli command.

The '-client' option is used (sporadically) as a hint to
the JVM not to waste time optimising the byte-code as much
of the code won't be run multiple times.  This option only has
an effect on 32-bit JVMs.  This patch adds '-client' to the
places where it's missing and adds the '-XX:+TieredCompilation
-XX:TieredStopAtLevel=1' options that have the same effect for
64-bit JVMs.

For short-lived tasks, much of the time is spent processing
the classes.  By limiting the classpath to only the jar files
that are needed, less time is taken.  This patch adds support
for this and applies it to the chimera-cli command.

A considerable fraction of the time processing a class involves
verifying certain aspects of the byte-code.  If we trust the
Java compiler and the source of the code, we can choose to
side-step this verification by adding the classes to the
bootstrap-classpath.  This patch does this for the
chimera-cli command.

The chimera-cli creates a BoneCP database pool.  The current
options are not optimal.  This patch updates them.

The effect of these changes were tested by timing how long it
takes to run "chimera-cli ls /" 10 times (discarding output)
using the system-test deployment.  This is then repeated 5
times and the mean and std. deviation is calculated.

The initial time to run "chimera-cli ls /" on my laptop is
(2.99 +/- 0.03) s

Each of the improvements had the following improvements:

    -client -XX:+TieredCompilat... (0.64 +/- 0.07) s
    Limiting jars on classpath     (0.44 +/- 0.03) s
    Using boot-classpath           (0.45 +/- 0.02) s
    Updated BoneCP configuration   (0.07 +/- 0.02) s

    Total improvement              (1.56 +/- 0.03) s

Time to run "chimera-cli ls /" with all improvements is
(1.43 +/- 0.01) s

Acked-by: Tigran Mkrtchyan
Acked-by: Gerd Behrmann
Patch: http://rb.dcache.org/r/5584/
Target: master
Request: 2.6