loginbroker: Comply with new configuration policy

Drops the service srm-loginbroker. All doors now register with
a single login broker called LoginBroker and SRM and all
monitoring services query this one LoginBroker to locate doors.
SRM is updated to exclude srm as a transfer protocol, thus
making it safe to register all doors in the same login broker.

The dcache.service.loginbroker property is updated to take
a comma separated list of loginbrokers to register with or
to query. httpd now uses this property to configure which
loginbrokers to query.

New properties

loginbroker.cell.name
srm.protocols.disallowed.get
srm.protocols.disallowed.put
srm.protocols.preferred

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5742/
cleaner: Clean up cleaner.properties

Besides cleaning up the comments, this patch turns the previously
obsoleted property cleaner.hsm into a deprecated property.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5744/
spacemanager: Allow previously obsoleted values for deprecated srmSpaceManagerEnabled

This patch adds legacy values for srmSpaceManaerEnabled. Thus one can
once again use true, on, yes, enabled, false, off, no, disabled, but
only when using the legacy property.

Also fixed a bug in srm.batch introduced when previously deprecating
srmSpaceManagerEnabled.

Require-book: no
Require-notes: yes
Target: trunk
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5748/
replica: Restored legacy values of deprecated replicaManager property

When renaming the configuration properties of replica manager, the
replicaManager was marked deprecated and the allowed values changed
to true and false rather than yes and no. This patch restored the
legacy values such that yes and no can be used. The new
dcache.enable.replica property still requires the values true and
false.

The default value of the property has been changed from yes/true to
no/false, as this better reflect most dCache instances.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5749/
alarms: Comply with new configuration policy

RELEASE NOTES:
BOOK:

Deprecated:

alarms.server.config
alarms.store.db.type
alarms.store.path
alarms.store.db.rdbms
alarms.store.db.driver
alarms.store.db.host
alarms.store.db.user
alarms.store.db.pass
alarms.store.db.properties
alarms.store.db.url

New:

alarms.cell.name=alarms
alarms.log.config.path=${alarms.server.config}
(one-of?off|error|warn|info|debug|${alarms.server.log.level})alarms.log.root-level=${alarms.server.log.level}
(one-of?off|xml|rdbms|${alarms.store.db.type})alarms.db.type=off
alarms.db.xml.path=${alarms.store.path}
alarms.db.rdbms.type=${alarms.store.db.rdbms}
alarms.db.driver=${alarms.store.db.driver}
alarms.db.host=${$alarms.store.db.host}
alarms.db.user=${$alarms.store.db.user}
alarms.db.password=${alarms.store.db.pass}
alarms.db.name=alarms
alarms.db.config.path=${alarms.store.db.properties}
alarms.db.url=${alarms.store.db.url}
alarms.db.schema.auto
alarms.db.schema.changelog

Obsolete:
alarms.store.db.changelog

Target: master
Require-notes: yes
Require-book: yes
Patch: http://rb.dcache.org/r/5743
Acked-by: Gerd
Fix CellStub timeout handling

A recent change to CellStub introduced a time unit property. Thus
the interpretation of getTimeout() changed, as the value was no
longer guaranteed to be in milliseconds. This patch updates the
code that uses getTimeout() and either lets it use getTimeoutInMillis()
instead, or updates it to be time unit aware. The patch also fixes
several calls to setTimeout().

Thanks to Dmitry for finding this.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/dashboard/
ssh2: factory-out command factory

Reworking how username is delivered to the dCache shell means that
the CommandFactory may be instantiated by UniversalSpringCell, which
simplifies Ssh2Admin.

It also solves a potential race-condition should two users attempt to
login at the same time with different usernames.

Target: master
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5761/
Acked-by: Gerd Behrmann
billing: Comply with new configuration policy

RELEASE NOTES:
BOOK:

NEW:

billing.cell.name
billing.enable.text=${billing.enable.text.when-disable-is-${billingDisableTxt}}
billing.enable.db=${billing.enable.db.when-toDb-is-${billingToDb}}

billing.service.poolmanager.timeout=${poolConnectTimeout}
billing.service.poolmanager.timeout.unit=MILLISECONDS

billing.text.dir=${billingLogsDir}
billing.text.flat-dir
billing.text.format.mover-info-message=${billing.format.MoverInfoMessage}
billing.text.format.remove-file-info-message=${billing.format.RemoveFileInfoMessage}
billing.text.format.door-request-info-message=${billing.format.DoorRequestInfoMessage}
billing.text.format.storage-info-message=${billing.format.StorageInfoMessage}

billing.db.config.path=${billingInfoAccessPropertiesFile}
billing.db.inserts.max-before-commit=${billingMaxInsertsBeforeCommit}
billing.db.inserts.timeout-before-commit={billingMaxTimeBeforeCommitInSecs}
billing.db.inserts.timeout-before-commit.unit=SECONDS
billing.db.driver=${billingDbDriver}
billing.db.url=${billingDbUrl}
billing.db.host=${billingDbHost}
billing.db.user=${billingDbUser}
billing.db.password=${billingDbPass}
billing.db.password.file=${billingDbPgPassFileName}
billing.db.name=${billingDbName}

billing.db.connections.max-per-partition=30
billing.db.connections.min-per-partition=3
billing.db.schema.auto=${updateBillingDb}
billing.db.schema.changelog=${billing.db.schema.changelog-when-${billing.enable.db}}

DEPRECATED:

billingLogsDir
billingDisableTxt
billingToDb

poolConnectTimeout

billing.format.MoverInfoMessage
billing.format.RemoveFileInfoMessage
billing.format.DoorRequestInfoMessage
billing.format.StorageInfoMessage

updateBillingDb
billingInfoAccessPropertiesFile
billingMaxInsertsBeforeCommit
billingMaxTimeBeforeCommitInSecs
billingDbDriver
billingDbUrl
billingDbHost
billingDbUser
billingDbPass
billingDbPgPassFileName
billingDbName

OBSOLETE:

billingInfoAccess
billingChangelog

TO BE REMOVED (to httpd.properties):

generatePlots
billingPlotsExportType
webadmin.billing-plot.refresh-threshold
webadmin.billing-plot.height
webadmin.billing-plot.width
webadmin.billing-plot.default-style
webadmin.billing-plot.default-scale

Testing done:

For the camel-case conversion, deployed and issued gridftp write, checked that billing text message was logged properly.

Target: master
Patch: http://rb.dcache.org/r/5750/
Require-notes: yes
Require-book: yes
Acked-by: Gerd
Merge branch 'master' of github.com:dCache/dcache
statistics: Fix NPE in Html tree creation

It has been noted that the statistics module fails to produce the top level html and to run the collection cron correctly.  In the first case, one sees an error such as the first reported below.  In the second, a different error occurs.  These two errors, however, appear to be related.

As the code currently stands, the "count" array used in several methods remains uninitialized until inside the subsequent loop.  However, initialization is not loop-invariant, as those loops may fail to run under certain conditions, in particular when the list of statistics files is empty.

This patch provides the appropriate pre-loop initialization of the array.

Note that this class has a number of dead variables and methods.  I did not eliminate them, however, as it occurs to me that one might wish to do a bit more cleanup and "modernize" this code.  I leave this to another patch.

Target: master
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Ticket: http://rt.dcache.org/Ticket/Display.html?id=785
Bug: http://rt.dcache.org/Ticket/Display.html?id=785
Acked-by: Gerd

Testing done:

Before this patch, one observes the following.

1.  Using the admin command create html, one sees:

Exception in creating html tree for : /var/lib/dcache/billing/2013/06/05/2013-06-05-13.raw
java.lang.NullPointerException: null
at diskCacheV111.services.PoolStatisticsV0.printTotal(PoolStatisticsV0.java:1204) ~[dcache-core-2.6.1.jar:2.6.1]
at diskCacheV111.services.PoolStatisticsV0.prepareDailyHtml(PoolStatisticsV0.java:1136) ~[dcache-core-2.6.1.jar:2.6.1]
at diskCacheV111.services.PoolStatisticsV0.prepareDailyHtmlFiles(PoolStatisticsV0.java:1041) ~[dcache-core-2.6.1.jar:2.6.1]
at diskCacheV111.services.PoolStatisticsV0.access$1100(PoolStatisticsV0.java:116) ~[dcache-core-2.6.1.jar:2.6.1]
at diskCacheV111.services.PoolStatisticsV0$HourlyRunner.run(PoolStatisticsV0.java:432) ~[dcache-core-2.6.1.jar:2.6.1]
at dmg.cells.nucleus.CellNucleus$1.run(CellNucleus.java:657) [cells-2.6.1.jar:2.6.1]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_09]

2.  Using the admin command create stat, one sees:

at diskCacheV111.services.PoolStatisticsV0.getPoolRepositoryStatistics(PoolStatisticsV0.java:1361) ~[dcache-core-2.6.6-SNAPSHOT.jar:2.6.6-SNAPSHOT]
at diskCacheV111.services.PoolStatisticsV0.createStatisticsMap(PoolStatisticsV0.java:838) ~[dcache-core-2.6.6-SNAPSHOT.jar:2.6.6-SNAPSHOT]
at diskCacheV111.services.PoolStatisticsV0.createHourlyRawFile(PoolStatisticsV0.java:973) ~[dcache-core-2.6.6-SNAPSHOT.jar:2.6.6-SNAPSHOT]
at diskCacheV111.services.PoolStatisticsV0.access$500(PoolStatisticsV0.java:116) ~[dcache-core-2.6.6-SNAPSHOT.jar:2.6.6-SNAPSHOT]
at diskCacheV111.services.PoolStatisticsV0 $HourlyRunner.run(PoolStatisticsV0.java:400) ~[dcache-core-2.6.6-SNAPSHOT.jar:2.6.6-SNAPSHOT]
at dmg.cells.nucleus.CellNucleus$1.run(CellNucleus.java:657) [cells-2.6.6-SNAPSHOT.jar:2.6.6-SNAPSHOT]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0]
15 Jul 2013 15:57:08 (PoolStatistics) [] prepareDailyHtmlFiles : File not found : /var/lib/dcache/statistics/2013/07/15/2013-07-15-day.drw

After the patch is applied, both of these commands work again, and the top index.html is produced.

RELEASE NOTES:

Fixes NullPointer bug which prevents under certain conditions top-level index.html from being created (and the statistics page being found in the browser).
chimera: Implement set-gid bit on directories

Target: trunk
Require-book: yes
Require-notes: yes
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7840
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5769/
pool: Manage periodic health check through Spring

Minor refactoring prior to adding new features to the periodic
health check.

Note that the output of the 'info' command of a pool has changed
as it no longer shows the 'Check Repository  : true' line.

Target: trunk
Require-notes: no
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5772/
hsqldb: (billing) comply with new configuration policy

Updates hsqldb.properties to match billing.properties

Testing done:

Tested dcache database ls with billingToDb on and off.
Ran dcache and checked logs.
Inspected var/plots for plots.

Target: master
Patch: http://rb.dcache.org/r/5780/
Require-book: no
Require-notes: no
Acked-by: Paul
system-test: Do not use deprecated billing property

Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Patch: http://rb.dcache.org/r/5787/
Target: trunk
Require-notes: no
Require-book: no
Merge branch 'master' of github.com:dCache/dcache