srm: Various minor improvements to error reporting

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6176/
doors: Fix race condition in registration of mover creation and termination

The race condition caused srmCopy transfers to hang randomly.

Target: trunk
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6178/
Add workaround for BoneCP bug

See https://github.com/wwadge/bonecp/pull/18 for details.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6180/
dcache-webadmin: fix regression affecting checkall on alarms table

Recent update to Wicket 1.6 broke the javascript for the "check all" checkboxes on the Alarms table.

This patch restores it.

Testing:

Checked by clicking.

Target: master
Patch: http://rb.dcache.org/r/6084/
Request: 2.7
Acked-by: Gerd
Require-notes: no
Require-book: no
Bug: http://rt.dcache.org/Ticket/Display.html?id=8022
dcache-webadmin: fix regression (submit failure) on PoolList introduced by auto-refresh

After application of http://rb.dcache.org/r/6085 it was discovered that on pages where administrative interaction occurs, in particular PoolList and ActiveTransfers, the addition of the Ajax autorefresh behavior interfered with the ability to issue commands to the pool.

The reason was that auto-refresh was swapping out the object on the page but the new object was not being propagated to the underlying list view.  Hence checking the checkbox in order to select the row was registered on the new object, but not the old, which was still being held onto in the table model.

    Two fixes were necessary.

    1.  Turn off the prior call to the pool DAO when form submit is taking place, allowing the old model object to be used.  The flag is reset so that refresh will occur at the next timeout.
    2.  Move the refresh call to inside the panel's list view "onBeforeRender" method.  The Ajax beforeRender(), which calls the refresh() adapter for the page, is not useful here, as it follows, rather than precedes, the beforeRender of the panel component.

Since information does not propagate back immediately from the pool manager, a "pending" marker has been added to allow the user to see that the command has been issued but that the state change is pending.  If the user does not manually refresh the page (usually a few seconds wait is sufficient), the new state from the pool manager will appear at the next auto-refresh (after 1 minute).  In the meantime, the entry has its state marked as "state + (pending)" and the row is colored red.

This patch only applies to PoolList.  A separate patch for ActiveTransfers will be posted.

Testing Done:

Before application of the patch, no admin commands can be issued from the "PoolUsage" page.

After the patch, commands are issued and the page is correctly updated.

    Tested by:

    Selecting a pool and changing its state.
    Waiting for auto-update and confirming.
    Selecting the same pool and changing its state back.
    Waiting for auto-update and confirming.
    Using the admin interface to change the state of a pool.
    Waiting for auto-update and confirming.

Target: master
Patch: http://rb.dcache.org/r/6161
Requires-notes: no
Requires-book: no
Acked-by: Tigran
dcache-webadmin: fix regression (submit failure) on ActiveTransfers introduced by auto-refresh

After application of http://rb.dcache.org/r/6085 it was discovered that on pages where administrative interaction occurs, in particular PoolList and ActiveTransfers, the addition of the Ajax autorefresh behavior interfered with the ability to issue commands to the pool.

The reason was that auto-refresh was swapping out the object on the page but the new object was not being propagated to the underlying list view.  Hence checking the checkbox in order to select the row was registered on the new object, but not the old, which was still being held onto in the table model.

Two fixes were necessary.

1.  Turn off the prior call to the pool DAO when form submit is taking place, allowing the old model object to be used.  The flag is reset so that refresh will occur at the next timeout.
2.  Move the refresh call to inside the panel's list view "onBeforeRender" method.  The Ajax beforeRender(), which calls the refresh() adapter for the page, is not useful here, as it follows, rather than precedes, the beforeRender of the panel component.

Since information does not propagate back immediately from the pool manager, a "pending" marker has been added to allow the user to see that the command has been issued but that the state change is pending.  If the user does not manually refresh the page (usually a few seconds wait is sufficient), the new state from the pool manager will appear at the next auto-refresh (after 1 minute).  In the meantime, the entry has its state marked as "state + (pending)" and the row is colored red.

This patch only applies to ActiveTransfers.  A separate patch for PoolList (6161) has been posted.

Testing Done:

Before application of the patch, no admin commands can be issued from the "ActiveTransfer" page (kill movers).

After the patch, commands are issued and the page is correctly updated.

Tested by:

Starting several long-lived concurrent writes to pools.
Waiting for mover info to appear.
Selecting a mover and canceling it.
Waiting to see final update/disappearance of mover from list on refresh

Target: master
Patch: http://rb.dcache.org/r/6162
Requires-notes: no
Requires-book: no
Acked-by: Tigran
dcache-webadmin: add select/deselect all to ActiveTransfers and PoolUsage

As per feature request. See http://rt.dcache.org/Ticket/Display.html?id=8047.

I am unsure whether this should be ported to 2.6 or not.  I will go ahead an post a separate patch, since it must be applied without the changes made to master having to do with auto-refresh.

Testing:

Deployed and tested by clicking and submitting.

Target: master
Patch: http://rb.dcache.org/r/6164
Require-notes: yes
Require-book: no
Feature-req: http://rt.dcache.org/Ticket/Display.html?id=8047
Depends-on: 6061
Depends-on: 6062
Acked-by: Dmitry

RELEASE NOTES:

Adds select/deselect all on lists on the Pool Usage and Active Transfer pages (when in authorized admin mode).
Chimera: fix out-by-one error

The limit on a filename is 255 characters.  Due to an out-by-one error, for
exactly 255 character-length filenames the JdbFs layer fails to catch the
problem and the database catches it instead.

Target: master
Requires-book: no
Requires-notes: no
Patch: http://rb.dcache.org/r/6183/
Acked-by: Tigran Mkrtchyan
SRM
     Rename properties for simplicity and compactness,
     group SRM request properties by
     equest properties rather than by requests.
     Address some issues reported uin RT tickets 8069,8070,8071,8072,8073

Patch: http://rb.dcache.org/r/6173/
Ticket: 8069,8070,8071,8072,8073
Acked-by: Gerd Behrmann, Paul Millar
Target: trunk
Require-book: yes
Require-notes: yes

RELEASE NOTES

Further Rationalization of SRM property names.

Deprecated properties:
=====================

srmGetReqThreadQueueSize
srmBringOnlineReqThreadQueueSize
srmPutReqThreadQueueSize
srmCopyReqThreadQueueSize
srmLsRequestThreadQueueSize
srmReserveReqThreadQueueSize

srmGetReqThreadPoolSize
srmBringOnlineReqThreadPoolSize
srmPutReqThreadPoolSize
srmCopyReqThreadPoolSize
srmLsRequestThreadPoolSize
srmReserveReqThreadPoolSize

srmGetReqMaxWaitingRequests
srmBringOnlineReqMaxWaitingRequests
srmPutReqMaxWaitingRequests
srmCopyReqMaxWaitingRequests
srmLsRequestMaxWaitingRequests
srmReserveReqMaxWaitingRequests

srmGetReqReadyQueueSize
srmBringOnlineReqReadyQueueSize
srmPutReqReadyQueueSize
srmReserveReqReadyQueueSize

srmGetReqMaxReadyRequests
srmBringOnlineReqMaxReadyRequests
srmPutReqMaxReadyRequests
srmReserveReqMaxReadyRequests

srmGetReqMaxNumberOfRetries
srmBringOnlineReqMaxNumberOfRetries
srmPutReqMaxNumberOfRetries
srmCopyReqMaxNumberOfRetries
srmLsRequestMaxNumberOfRetries
srmReserveReqMaxNumberOfRetries

srmGetReqRetryTimeout
srmBringOnlineReqRetryTimeout
srmPutReqRetryTimeout
srmCopyReqRetryTimeout
srmLsRequestRetryTimeout
srmReserveReqRetryTimeout

srmGetReqMaxNumOfRunningBySameOwner
srmBringOnlineReqMaxNumOfRunningBySameOwner
srmPutReqMaxNumOfRunningBySameOwner
srmCopyReqMaxNumOfRunningBySameOwner
srmLsRequestMaxNumberOfRunningBySameOwner
srmReserveReqMaxNumOfRunningBySameOwner

srmGetReqSwitchToAsynchronousModeDelay
srmBringOnlineReqSwitchToAsynchronousModeDelay
srmPutReqSwitchToAsynchronousModeDelay
srmLsRequestSwitchToAsynchronousModeDelay

srmDatabaseEnabled
srmGetDatabaseEnabled
srmBringOnlineDatabaseEnabled
srmPutDatabaseEnabled
srmCopyDatabaseEnabled
srmLsDatabaseEnabled
srmReserveDatabaseEnabled

srmCleanPendingRequestsOnRestart
srmGetCleanPendingRequestsOnRestart
srmBringOnlineCleanPendingRequestsOnRestart
srmPutCleanPendingRequestsOnRestart
srmCopyCleanPendingRequestsOnRestart
srmLsCleanPendingRequestsOnRestart
srmReserveCleanPendingRequestsOnRestart

srmKeepRequestHistoryPeriod
srmGetKeepRequestHistoryPeriod
srmBringOnlineKeepRequestHistoryPeriod
srmPutKeepRequestHistoryPeriod
srmCopyKeepRequestHistoryPeriod
srmLsKeepRequestHistoryPeriod
srmReserveKeepRequestHistoryPeriod

srmExpiredRequestRemovalPeriod
srmGetExpiredRequestRemovalPeriod
srmBringOnlineExpiredRequestRemovalPeriod
srmPutExpiredRequestRemovalPeriod
srmCopyExpiredRequestRemovalPeriod
srmLsExpiredRequestRemovalPeriod
srmReserveExpiredRequestRemovalPeriod

srmRequestHistoryDatabaseEnabled
srmGetRequestHistoryDatabaseEnabled
srmBringOnlineRequestHistoryDatabaseEnabled
srmPutRequestHistoryDatabaseEnabled
srmCopyRequestHistoryDatabaseEnabled
srmLsRequestHistoryDatabaseEnabled
srmReserveRequestHistoryDatabaseEnabled

srmStoreCompletedRequestsOnly
srmGetStoreCompletedRequestsOnly
srmBringOnlineStoreCompletedRequestsOnly
srmPutStoreCompletedRequestsOnly
srmCopyStoreCompletedRequestsOnly
srmLsStoreCompletedRequestsOnly
srmReserveSpaceStoreCompletedRequestsOnly

srmLsMaxNumberOfEntries
srmLsMaxNumberOfLevels

srmGetLifeTime
srmBringOnlineLifeTime
srmPutLifeTime
srmCopyLifeTime

New property names:
==================

srm.request.thread-queue-size
srm.request.get.thread-queue-size
srm.request.bring-online.thread-queue-size
srm.request.put.thread-queue-size
srm.request.copy.thread-queue-size
srm.request.ls.thread-queue-size
srm.request.reserve-space.thread-queue-size

srm.request.threads
srm.request.get.threads
srm.request.bring-online.threads
srm.request.put.threads
srm.request.copy.threads
srm.request.ls.threads
srm.request.reserve-space.threads

srm.request.max-waiting-requests
srm.request.get.max-waiting-requests
srm.request.bring-online.max-waiting-requests
srm.request.put.max-waiting-requests
srm.request.copy.max-waiting-requests
srm.request.ls.max-waiting-requests
srm.request.reserve-space.max-waiting-requests

srm.request.ready-queue-size
srm.request.get.ready-queue-size
srm.request.bring-online.ready-queue-size
srm.request.put.ready-queue-size
srm.request.ls.ready-queue-size
srm.request.reserve-space.ready-queue-size

srm.request.max-ready-requests
srm.request.get.max-ready-requests
srm.request.bring-online.max-ready-requests
srm.request.put.max-ready-requests
srm.request.ls.max-ready-requests
srm.request.reserve-space.max-ready-requests

srm.request.retries
srm.request.get.retries
srm.request.bring-online.retries
srm.request.put.retries
srm.request.copy.retries
srm.request.ls.retries
srm.request.reserve-space.retries

srm.request.retry-timeout
srm.request.retry-timeout.unit

srm.request.get.retry-timeout
srm.request.get.retry-timeout.unit

srm.request.bring-online.retry-timeout
srm.request.bring-online.retry-timeout.unit

srm.request.put.retry-timeout
srm.request.put.retry-timeout.unit

srm.request.copy.retry-timeout
srm.request.copy.retry-timeout.unit

srm.request.ls.retry-timeout
srm.request.ls.retry-timeout.unit

srm.request.reserve-space.retry-timeout
srm.request.reserve-space.retry-timeout.unit

srm.request.max-by-same-user
srm.request.get.max-by-same-user
srm.request.bring-online.max-by-same-user
srm.request.put.max-by-same-user
srm.request.copy.max-by-same-user
srm.request.ls.max-by-same-user
srm.request.reserve-space.max-by-same-user

srm.request.switch-to-async-mode-delay
srm.request.switch-to-async-mode-delay.unit

srm.request.get.switch-to-async-mode-delay
srm.request.get.switch-to-async-mode-delay.unit

srm.request.bring-online.switch-to-async-mode-delay
srm.request.bring-online.switch-to-async-mode-delay.unit

srm.request.put.switch-to-async-mode-delay
srm.request.put.switch-to-async-mode-delay.unit

srm.request.ls.switch-to-async-mode-delay
srm.request.ls.switch-to-async-mode-delay.unit

srm.persistence.enable
srm.persistence.get.enable
srm.persistence.bring-online.enable
srm.persistence.put.enable
srm.persistence.copy.enable
srm.persistence.ls.enable
srm.persistence.reserve-space.enable

srm.persistence.enable.clean-pending-on-restart
srm.persistence.get.enable.clean-pending-on-restart
srm.persistence.bring-online.enable.clean-pending-on-restart
srm.persistence.put.enable.clean-pending-on-restart
srm.persistence.copy.enable.clean-pending-on-restart
srm.persistence.ls.enable.clean-pending-on-restart
srm.persistence.reserve-space.enable.clean-pending-on-restart

srm.persistence.keep-history-period
srm.persistence.keep-history-period.unit

srm.persistence.get.keep-history-period
srm.persistence.get.keep-history-period.unit

srm.persistence.bring-online.keep-history-period
srm.persistence.bring-online.keep-history-period.unit

srm.persistence.put.keep-history-period
srm.persistence.put.keep-history-period.unit

srm.persistence.copy.keep-history-period
srm.persistence.copy.keep-history-period.unit

srm.persistence.ls.keep-history-period
srm.persistence.ls.keep-history-period.unit

srm.persistence.reserve-space.keep-history-period
srm.persistence.reserve-space.keep-history-period.unit

srm.persistence.remove-expired-period
srm.persistence.remove-expired-period.unit

srm.persistence.get.remove-expired-period
srm.persistence.get.remove-expired-period.unit

srm.persistence.bring-online.remove-expired-period
srm.persistence.bring-online.remove-expired-period.unit

srm.persistence.put.remove-expired-period
srm.persistence.put.remove-expired-period.unit

srm.persistence.copy.remove-expired-period
srm.persistence.copy.remove-expired-period.unit

srm.persistence.ls.remove-expired-period
srm.persistence.ls.remove-expired-period.unit

srm.persistence.reserve-space.remove-expired-period
srm.persistence.reserve-space.remove-expired-period.unit

srm.persistence.enable.history
srm.persistence.get.enable.history
srm.persistence.bring-online.enable.history
srm.persistence.put.enable.history
srm.persistence.copy.enable.history
srm.persistence.ls.enable.history
srm.persistence.reserve-space.enable.history

srm.persistence.enable.store-transient-state
srm.persistence.get.enable.store-transient-state
srm.persistence.bring-online.enable.store-transient-state
srm.persistence.put.enable.store-transient-state
srm.persistence.copy.enable.store-transient-state
srm.persistence.ls.enable.store-transient-state
srm.persistence.reserve-space.enable.store-transient-state

srm.limits.ls.entries
srm.limits.ls.levels

srm.request.lifetime
srm.request.lifetime.unit

srm.request.get.lifetime
srm.request.get.lifetime.unit

srm.request.bring-online.lifetime
srm.request.bring-online.lifetime.unit

srm.request.put.lifetime
srm.request.put.lifetime.unit

srm.request.copy.lifetime
srm.request.copy.lifetime.unit

Generic srm.request.foo and srm.persistence.foo properties are defaults for
concrete srm.request.{bring-online,copy,get,ls,put,reserve-space}.foo and
srm.persistence.{bring-online,copy,get,ls,put,reserve-space}.foo properties

srmStoreCompletedRequestsOnly and srm{BringOnline,Copy,Get,Ls,Put,Reserve-space}CompletedRequestsOnly
are redefined as inverse properties srm.persistence.enable.store-transient-state and
srm.persistence.{bring-online,copy,get,ls,put,reserve-space}.enable.store-transient-state
srm: Avoid string conversion when persisting delegated credentials

These are unnecessary, and FindBugs complains about lack of character
set encoding.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6181/
dcache-xrootd: Fix stat, fix return codes

Addresses protocol compliance issues in the xrootd implementation:

- Return kXR_NotFound if file is not found (affects stat, open, rm,
  rmdir, mv, list

- Implement kXR_offline, and kXR_poscpend in stat response. A file
  is offline if not on a read pool (ie it requires staging from tape,
  pool to pool, or that pools come online). A file is poscpend if
  currently being uploaded.

- Change kXR_writable, kXR_readable, kXR_xset of stat such that these
  are interpreted wrt the current subject, rather than just returning
  the world permissions.

- The stat changes also apply to statx.

- Change kXR_open to return kXR_FileLocked if file is currently being
  uploaded.

Target: trunk
Require-book: no
Require-notes: yes
Request: 2.6
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5817/
Upgrade to bonecp 0.8.0-dcache1

In addition to the changes in 0.8.0.RELEASE, this version adds the
fixes in the following two pull requests:

https://github.com/wwadge/bonecp/pull/18
https://github.com/wwadge/bonecp/pull/19

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6186/
pnfsmanager: Change defaults for pnfsmanager

Enables folding and full path permission check. Increases default thread
count to 12 (tests I ran years ago showed that Chimera scales well until
at least 12 threads - on modern hardware it may scale even better).

Note that full path permission check may result in access to files and
directories that was allowed before now to be denied. Sites are adviced to
check the permission settings on directories on upgrade, or to disable
the POSIX compliant full path permission check by flipping the property
pnfsmanager.full-path-permission-check.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Patch: http://rb.dcache.org/r/5771/
libs: update jglobus library

updated library contains fix which changes a way how
CA and it's signing_policy file are processed.

Changelog for v2.0.6-rc3.d..v2.0.6-rc4.d
    * [b3dd9f1] Make input streams buffered and close them after use
    * [c3bfef4] issue102: fix - index caches against domain name rather than hash of encodings

GGUS: https://ggus.eu/ws/ticket_info.php?ticket=98570
Acked-by: Albert Rossi
Acked-by: Gerd Behrmann
Acked-by: Paul Millar
Target: master, 2.6, 2.7
Require-book: no
Require-notes: yes
dcache-xrootd: Upgrade to xrootd4j 1.2.1

Target: trunk
Request: 2.6
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6192/
srm-client: fix name of directory into which spec file is written

The RPM build of srm-client is currently broken because the spec file is
written into the wrong directory.  This patch fixes this.

Target: master
Requires-nodes: no
Requires-book: no
Acked-by: Gerd Behrmann
Patch: http://rb.dcache.org/r/6194/
webdav: Log StringTemplate errors through slf4j

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6195/
[maven-release-plugin] prepare branch 2.7-ndgf
[maven-release-plugin] rollback changes from release preparation of 2.7-ndgf
rpm: Fix build of non-snapshot versions

Also converted - to ~, since RPM nowadays understands that symbol
(same semantics as for DEB, that is, it is lexicographically
smaller than the empty string).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6197/