common: backoff fix error in precondition check

module: common

The ExponentialBackoffAlgorithmFactory checks to make sure that max >= min.  However, the precondition is actually checking two values which may not have the same time unit.

This patch fixes the precondition check to use the delays converted to milliseconds.

Target: master
Acked-by: Tigran
Patch: http://rb.dcache.org/r/5568
Require-notes: no
Require-book: no
Request: 2.6
nfsv4: use worker thread strategy in pools

Worker thread strategy gives a better performance
when the pool under high load

Acked-by: Gerd Behrmann
Target: 2.6, trunk
Require-book: no
Require-notes: no
Patch: http://rb.dcache.org/r/5562/
nfs: add more admin command to monitor door and movers

Acked-by: Gerd Behrmann
Target: trunk, 2.6
Require-book: yes
Require-notes: yes
Patch: http://rb.dcache.org/r/5561/
skel: gplazma batch failing because it still checks gplazmaPolicy

module: skel

https://github.com/dCache/dcache/commit/9b251e30dab1ecc6daa11551fd018df5a92d0731

broke gplazma in master because it obsoleted gplazmaPolicy but failed to remove the property check and property from the gplazma.batch file.

This patch repairs the batch so gplazma can run.

Target: master
Patch: http://rb.dcache.org/r/5569
Require-notes: no
Require-book: no
Acked-by: Tigran

Testing:

Before:

(666) file:/usr/share/dcache/services/gplazma.batch: line 41: (666) env:startLogin.exe: line 3: (1) variable not define : gplazmaPolicy
16 May 2013 08:30:28 (System) [] Failure at startup: (666) file:/usr/share/dcache/services/gplazma.batch: line 41: (666) env:startLogin.exe: line 3: (1) variable not define : gplazmaPolicy

After:

gPlazma is happy.
dcache: fix billing liquibase add plpgsql changeset

module: dcache

Prior to version 8.4, postgres did not automatically contain support for plpgsql, and the language had to be explicitly added to the database.

The liquibase changeset was written to use an sqlCheck which was intended to execute the sql statement and ignore the warning that the language was already present.  For 8.4+ this worked fine, but on an actual 8.3 installation, the sql would not execute because sqlCheck required a return value and was getting none (what is shown in the interpreter after the statement is executed evidently is not a return value).  Hence the changeset fails entirely there.  Another disadvantage of this was the fact that the successful completion (where the warning is skipped) looked like an error and was confusing (see the ticket reported below).

Our approach here is to modify the original changeset by substituting for it two new ones with new ids which will behave correctly.  The absence of the old changeset from the changelog does not cause problems -- the entry for just continues to occupy a dead row in the databasechangelog table.

Target: master
Patch: http://rb.dcache.org/r/5572
Require-notes: no
Require-book: no
Request: 2.6
Request: 2.2
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7809
Acked-by: Tigran

Testing:

Ran the new changeset successfully over a previous installation without errors or warnings.
webadmin: make srmSpaceManagerEnabled aware; enable real backoff on collectors

module: webadmin

In order to populate the webadmin pages, data collector daemons are run which contact various services such as the PoolManager or the SpaceManager.  While it would be uncommon to run dCache with most of these services, it is not uncommon for the SpaceManager not to be there.  In this case, we would like to avoid running this collector; and in general, we would also like to backoff from pinging services which may be temporarily offline.

Some time ago, a backoff algorithm factory was added to use precisely for this reason.  However, the original constant recurrent message send/wait was for the moment maintained.

This patch does the following:

   - it passes through knowledge of the srmSpaceManagerEnabled property.  It is assumed here that the local dcache.conf file will reflect the actual setting, whereever the SpaceManager may be configured to run;

   - it slightly restructures several of the collectors so that backoff will occur at the correct place in the looping logic;
   - it creates and adds an exponential backoff algorithm to be used by the collectors most susceptible to not finding their single endpoints (PoolMonitor, RestoreHandler, SpaceToken).

Target: master
Patch: http://rb.dcache.org/r/5571/
Require-notes: no
Require-book: no
Request: 2.6
Acked-by: Tigran

Redeployed.

1.  With default, the SpaceToken collector does not run (srmSpaceManagerEnabled=no)
2.  Setting srmSpaceManagerEnabled=yes, but with no actual space manager running, observed the correct backoff behavior
3.  Adding space manager and starting the domain allowed the collector to connect on its subsequent iteration.
dcache: alarms package.jdo missing varchar length on text fields Testing Done:

module: dcache, alarms

Non-fatal truncation to 256 chars of info and notes fields on the database log entry fields (from DataNucleus) corrected by this patch.

Target: master
Patch: http://rb.dcache.org/r/5574
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Dmitry

Testing Done:

Reran with gPlazma warn message, now properly stored and displayed.

RELEASE NOTES:

The info and notes fields of the alarm/log entry for the alarm service were potentially getting truncated at 256.  This patch increases length to 8000, as in the billing tables.
webdav: switch hard-coded JGlobus paths to those configured

The WebDAV door may use either Jetty's built-in SSL support or
support coming from JGlobus.  If the latter is chosen then
various paths are hard-coded.

This does not respect a site's configuration and, as a specific
example, results in the system-test failing.

This patch switches the JGlobus WebDAV SSL connector to use
the configured path.

Patch: http://rb.dcache.org/r/5575/
Acked-by: Gerd Behrmann
Target: master
Request: 2.6
webadmin: make query and filter options set in browser session-specific (alarms, poolqueueplots)

module: webadmin

The alarms and pool queue plots pages were erroneously storing query settings at the server level.  These were thus unintentionally being shared across user sessions.

This patch rectifies this behavior so that user settings do not interfere with each other.

Target: master
Patch: http://rb.dcache.org/r/5577
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Tigran

Testing: With two different browsers with different settings.

RELEASE NOTES

Fixes a bug in the web pages for Alarms and Pool Queue Plots where the settings selected by a user/session would become global.  The fix now isolates user/session settings from each other.
dcache: alarm command line, fix minor bug

module: dcache, alarm command line interpreter

This patch does the following:

1.  Fixes bug in add and modify whereby includeInKey was not getting updated.  It also adds "group" to the help message for the attribute (missing previously).
2.  Prints out the full definition for confirmation/review before commit in both add and modify.
3.  Allows the elimination of an attribute (XML element) via the '-' character.

Target: master
Patch: http://rb.dcache.org/r/5573
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Tigran

Testing: Tried out various options using add, remove and modify for illegal alteration by deletion of required elements; modified includeInKey, and tried to set group (with number and without, which is an error).

RELEASE NOTES:

For those who have tried out the dcache alarm definition commands, they may have noticed that "includeInKey" could not be modified once it was defined.  This bug has been fixed.  There is also a new feature allowing the deletion of a definition element using the '-' character.  The full xml for the definition is now also displayed before asking for confirmation to write.
Fix dCache FTP and SRM to be standards-compliant by default

The overwriteEnabled property affects FTP and SRM doors.  The default
is currently 'false', which prevents overwriting of data if the file
already exists.

RFC 959 is quite clear:

         STORE (STOR)

            This command causes the server-DTP to accept the data
            transferred via the data connection and to store the data as
            a file at the server site.  If the file specified in the
            pathname exists at the server site, then its contents shall
            be replaced by the data being transferred.  A new file is
            created at the server site if the file specified in the
            pathname does not already exist.

Therefore, by setting overwriteEnabled to 'false', dCache is not
RFC-959 compliant.  This is our default behaviour.

Similarly, SRM protocol is also quite clear how storage elements are
expected to behave.  Setting overwriteEnabled=false breaks that
expected behaviour.

Moreover, there is no additional security from setting
overwriteEnabled=false.  The client can simply delete the data and
upload afresh.

The net result is that dCache is, by default, not standards compliant
for no direct benefit.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/5544/
Acked-by: Dmitry Litvintsev
Acked-by: Tigran Mkrtchyan
Acked-by: Gerd Behrmann
libs: use jglobus-2.0.6-rc2

includes fixes for cert validation and race conditions
Acked-by:
Target: trunk, 2.6
Require-book: no
Require-notes: no
Ban elliptic-curve ciphers to prevent unsupported ciphers from being selected

Elliptic-Curve (EC) based ciphers offer advantages over RSA- and
DSA-based ciphers and are being introduced, albeit slowly.  OpenJDK (6 &
7) use external libraries, such as libnss3, to provide support for
various cipher-suites, including support for EC ciphers.  The libnss3
library provide only partial support for all of the EC-based ciphers.

Unfortuantely, the generic front-end believes all ciphers are supported.
So, if during an SSL/TLS negotiation, the client requests an EC cipher
that libnss3 doesn't support, the SSLEngine may choose that cipher.  The
result is an exception being thrown with the cryptic
'CKR_DOMAIN_PARAMS_INVALID' message.

Rather than trying to deploy work-arounds in the client or trying to
document how to disable EC-support in the server using JVM-related
configuration, this patch bans the EC ciphers programatically.

Patch: http://rb.dcache.org/r/5527/
Acked-by: Gerd Behrmann
Target: master
Request: 2.6
dcache: billing -- fix error in liquibase varchar changeset

module: dcache

The liquibase changeset 4.1.7 was discovered to contain an off-by-one error in the string used to test the condition for changing the varchar length in the billing tables (the actual length of the string was 256, not 257). As a result, 4.1.7. 4.1.8 and 4.1.9 never will run, and all billing installations created by liquibase (which hardcodes the default to 256) remained as such.

The reason for the requested change was that the path and errormessage attributes of certain tables can well exceed 256.

The new changesets correct the error.

Target: master
Patch: http://rb.dcache.org/r/5555
Require-book: no
Require-notes: yes
Request: 2.6
Acked-by: Dmitry

Testing done:  Ran changesets on both new and previously installed instances, observing correct attribute definitions in the table afterwards.

RELEASE	NOTES: 	Fixes string lengths in billing databse tables. Tables created automatically contained character columns limited to 256 which is too restrictive for path and errormessage columns in billinginfo, doorinfo and storageinfo tables. Additionaly, the length of pnfsid string was limited to 36.
A previous patch designed to address this issue failed to so.
gplazma: Rename dcache-gsi module to gplazma2-gsi

The gsi module used the dcache- prefix even though it is independent
of all dcache components. It is much more closely tied to the gPlazma 2
project (which is independent of dCache). Hence this patch renames the
module.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5581/
webadmin: Rename module to dcache-webadmin

Webadmin is a dCache component (it relies heavily on dCache classes) and
the module should thus be called dcache-webadmin to follow the convention
used for other modules.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5583/
packaging: Move DEB and RPM packaging to Maven and remove Ant configuration

Packaging was the only thing we still did (partially) through ant. Our
packaging is however mostly about putting the right files together and
calling the native packaging tools, and that we can do just as easily
from Maven.

This patch moves the RPM and DEB packaging to the fhs and srmclient
Maven modules. The packaging is triggered by activating the profiles
rpm and deb by using the -P option of Maven. The profiles are exclusive
and must not be activated at the same time. Like the old Ant targets,
the profiles allow the build number and local version strings to be
modified by overriding properties through the -D option of Maven.

Solaris packaging tools are moved to the FHS module too, I do however
not have a Solaris system for testing and thus I leave it to somebody
else to define the necessary Maven profile to invoke the packaging.

I removed all legacy packaging components and all Ant files. This
includes a couple of old spec files, including for dcap. Like for
the Solaris package, I hope somebody else can ressurect this within
Maven (or move the dcap client code out of the dCache repository and
into its own repo).

Finally, I moved the packaging related Maven modules into its
own top-level directory to separate them from the modules that
contain our source code.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5589/
Fix incremetal compilation

The maven compiler plugin version 3.1 contains fixes to incremental
compilation. In addition to upgrading the plugin, the patch removes
an empty file in the cells module that triggers recompilation (since
it doesn't generate a class file and thus always appears as a change
in the source).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Patch: http://rb.dcache.org/r/5590/
logback-test-config: Synchronize logback version with dCache

This is the logback version used during tests.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5592/
Use correct junit dependency

It used to be the case that junit contained the Hamcrest classes rather
than having a Maven dependency on the Hamcrest artifact. To avoid
conflicts, junit provided the junit-dep artifact.

Since junit 4.11, Hamcrest is now specified as a dependency and it
is no longer necessary to use junit-dep.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5593/