common: backoff fix error in precondition check

module: common

The ExponentialBackoffAlgorithmFactory checks to make sure that max >= min.  However, the precondition is actually checking two values which may not have the same time unit.

This patch fixes the precondition check to use the delays converted to milliseconds.

Target: master
Acked-by: Tigran
Patch: http://rb.dcache.org/r/5568
Require-notes: no
Require-book: no
Request: 2.6
Committed: master@15848f5de6a5b120892758375f1cc27b05d51c91
Merge pull request #50 from alrossi/fix/2.6/common_backoff_precondition

common: backoff fix error in precondition check
dcache: fix billing liquibase add plpgsql changeset

module: dcache

Prior to version 8.4, postgres did not automatically contain support for plpgsql, and the language had to be explicitly added to the database.

The liquibase changeset was written to use an sqlCheck which was intended to execute the sql statement and ignore the warning that the language was already present.  For 8.4+ this worked fine, but on an actual 8.3 installation, the sql would not execute because sqlCheck required a return value and was getting none (what is shown in the interpreter after the statement is executed evidently is not a return value).  Hence the changeset fails entirely there.  Another disadvantage of this was the fact that the successful completion (where the warning is skipped) looked like an error and was confusing (see the ticket reported below).

Our approach here is to modify the original changeset by substituting for it two new ones with new ids which will behave correctly.  The absence of the old changeset from the changelog does not cause problems -- the entry for just continues to occupy a dead row in the databasechangelog table.

Target: master
Patch: http://rb.dcache.org/r/5572
Require-notes: no
Require-book: no
Request: 2.6
Request: 2.2
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7809
Acked-by: Tigran

Ran the new changeset successfully over a previous installation without errors or warnings.
webadmin: make srmSpaceManagerEnabled aware; enable real backoff on collectors

module: webadmin

In order to populate the webadmin pages, data collector daemons are run which contact various services such as the PoolManager or the SpaceManager.  While it would be uncommon to run dCache with most of these services, it is not uncommon for the SpaceManager not to be there.  In this case, we would like to avoid running this collector; and in general, we would also like to backoff from pinging services which may be temporarily offline.

Some time ago, a backoff algorithm factory was added to use precisely for this reason.  However, the original constant recurrent message send/wait was for the moment maintained.

This patch does the following:

   - it passes through knowledge of the srmSpaceManagerEnabled property.  It is assumed here that the local dcache.conf file will reflect the actual setting, whereever the SpaceManager may be configured to run;

   - it slightly restructures several of the collectors so that backoff will occur at the correct place in the looping logic;
   - it creates and adds an exponential backoff algorithm to be used by the collectors most susceptible to not finding their single endpoints (PoolMonitor, RestoreHandler, SpaceToken).

Target: master
Committed: master@92687f8010e0dfa60412aaef316e5aff80411769
Patch: http://rb.dcache.org/r/5571/
Require-notes: no
Require-book: no
Request: 2.6
Acked-by: Tigran

Redeployed.

1.  With default, the SpaceToken collector does not run (srmSpaceManagerEnabled=no)
2.  Setting srmSpaceManagerEnabled=yes, but with no actual space manager running, observed the correct backoff behavior
3.  Adding space manager and starting the domain allowed the collector to connect on its subsequent iteration.
Merge pull request #53 from alrossi/fix/2.6/collector_backoff

webadmin: make srmSpaceManagerEnabled aware; enable real backoff on coll...
Merge pull request #51 from alrossi/fix/2.6/fix_plpsql_changeset

dcache: fix billing liquibase add plpgsql changeset
dcache: alarms package.jdo missing varchar length on text fields

module: dcache, alarms

Non-fatal truncation to 256 chars of info and notes fields on the database log entry fields (from DataNucleus) corrected by this patch.

Target: master
Committed: master@7d7e188e69f87033934fe2bfd160e4421fbb8666
Patch: http://rb.dcache.org/r/5574
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Dmitry
Merge-req:

Testing Done: Reran with gPlazma warn message, now properly stored and displayed.

RELEASE NOTES:

The info and notes fields of the alarm/log entry for the alarm service were potentially getting truncated at 256.  This patch increases length to 8000, as in the billing tables.
Merge pull request #54 from alrossi/fix/2.6/alarms_jdo_char_len

dcache: alarms package.jdo missing varchar length on text fields
webdav: switch hard-coded JGlobus paths to those configured

The WebDAV door may use either Jetty's built-in SSL support or
support coming from JGlobus.  If the latter is chosen then
various paths are hard-coded.

This does not respect a site's configuration and, as a specific
example, results in the system-test failing.

This patch switches the JGlobus WebDAV SSL connector to use
the configured path.

Patch: http://rb.dcache.org/r/5575/
Acked-by: Gerd Behrmann
Target: master
Request: 2.6
webadmin: make query and filter options set in browser session-specific (alarms, poolqueueplots)

module: webadmin

The alarms and pool queue plots pages were erroneously storing query settings at the server level.  These were thus unintentionally being shared across user sessions.

This patch rectifies this behavior so that user settings do not interfere with each other.

Target: master
Committed: master@21b6b73bbcb27987891715b29657687d90114766
Patch: http://rb.dcache.org/r/5577
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Tigran

Testing Done: With two different browsers with different settings.

RELEASE NOTES

Fixes a bug in the web pages for Alarms and Pool Queue Plots where the settings selected by a user/session would become global.  The fix now isolates user/session settings from each other.
dcache: alarm command line, fix minor bug

module: dcache, alarm command line interpreter

This patch does the following:

1.  Fixes bug in add and modify whereby includeInKey was not getting updated.  It also adds "group" to the help message for the attribute (missing previously).
2.  Prints out the full definition for confirmation/review before commit in both add and modify.
3.  Allows the elimination of an attribute (XML element) via the '-' character.

Target: master
Committed: master@8b658b3caa423ecdbcbb593473e002cebe068686
Patch: http://rb.dcache.org/r/5573/
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Tigran

Testing: Tried out various options using add, remove and modify for illegal alteration by deletion of required elements; modified includeInKey, and tried to set group (with number and without, which is an error).

RELEASE NOTES:

For those who have tried out the dcache alarm definition commands, they may have noticed that "includeInKey" could not be modified once it was defined.  This bug has been fixed.  There is also a new feature allowing the deletion of a definition element using the '-' character.  The full xml for the definition is now also displayed before asking for confirmation to write.
Merge pull request #57 from alrossi/fix/2.6/alarm_cmdline

dcache: alarm command line, fix minor bug
Merge pull request #55 from paulmillar/fix/2.6/webdav-paths

webdav: switch hard-coded JGlobus paths to those configured
Merge pull request #56 from alrossi/fix/2.6/session_alarms_qpoolplots

webadmin: make query and filter options set in browser session-specific ...
Fix dCache FTP and SRM to be standards-compliant by default

The overwriteEnabled property affects FTP and SRM doors.  The default
is currently 'false', which prevents overwriting of data if the file
already exists.

RFC 959 is quite clear:

         STORE (STOR)

            This command causes the server-DTP to accept the data
            transferred via the data connection and to store the data as
            a file at the server site.  If the file specified in the
            pathname exists at the server site, then its contents shall
            be replaced by the data being transferred.  A new file is
            created at the server site if the file specified in the
            pathname does not already exist.

Therefore, by setting overwriteEnabled to 'false', dCache is not
RFC-959 compliant.  This is our default behaviour.

Similarly, SRM protocol is also quite clear how storage elements are
expected to behave.  Setting overwriteEnabled=false breaks that
expected behaviour.

Moreover, there is no additional security from setting
overwriteEnabled=false.  The client can simply delete the data and
upload afresh.

The net result is that dCache is, by default, not standards compliant
for no direct benefit.

For the 2.2 branch, we maintain the existing default value, but
include the warning text.

FOR RELEASE NOTES:

This patch adds an explanation of the overwriteEnabled configuration option.
This controls dCache's behaviour when user attempts to overwrite an existing
file via FTP or SRM.

The default behaviour is not standards-complaint.  This patch does NOT
alter this behaviour.  Sites are invited to review their configuration.

Target: master
Request: 2.6
Request: 2.2
Patch: http://rb.dcache.org/r/5544/
Acked-by: Dmitry Litvintsev
Acked-by: Tigran Mkrtchyan
Acked-by: Gerd Behrmann
Requires-notes: yes
Requires-book: no
Merge pull request #60 from paulmillar/fix/2.6/ftp_overwrite

Fix dCache FTP and SRM to be standards-compliant by default
libs: use jglobus-2.0.6-rc2

includes fixes for cert validation and race conditions
Acked-by:
Target: trunk, 2.6
Require-book: no
Require-notes: no

Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>