spacemanager: Fix property annotations

The property annotation of DefaultAccessLatencyForSpaceReservation is
missing the default value. The patch also addresses the problem that
spacemanager.default-access-latency was not defined in terms of the
property it is deprecating.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5820/
spacemanager: Add missing properties and fix system-test config

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5821/
Fix time unit related properties

Fixes a number of bugs and omissions in time unit properties.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/5765/
Removed scoped db.* properties.

Replaces all db.* properties with equivalent
<service>.db.* properties. The old db.* properties
are marked forbidden.

Scripts are updated to use the new names.

Target: trunk
Require-book: yes
Require-notes: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5822/
Remove scoped port configuration properties

Marks port forbidden.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5823/
poolmanager: Minor update to performance cost calculation

The patch modifies how the performance cost of a pool is calculated. First
of all, remember that the performance cost is nowadays only used in *read*
pool selection for wass partitions, thus read pool selection is the only
aspect that is influenced by this change.

The motivation for touching the read pool selection is the rewrite of the
HSM integration on pools. The new code pushes the lowest level queueing
into the HSM integration driver. This has several consequences: First of
all, there is no guarantee that a particular driver has a queue at all.
Second, even if a driver has a queue internally, this would be a queue
per HSM, not a shared queue like it is in the current pool.

Moreover, the current use of HSM queues in the cost calculation does not
reflect actual load. First, file flush goes through several queuing
layers, with a per storage-class queue in a flush controller, followed
by a periodic flush into a shared HSM store queue. It is only the latter
queue that is accounted for in the cost calculation - the per storage
class queue is ignored. Second, the concept of active, queued and max
for the store and fetch queues does not make sense as a base for cost
calculation: The actual load on a pool is determined by the number of
tape drives in use and thus the number of data streams. How many
callouts to the HSM script we made does not reflect how busy neither
the network nor the disks are. The point being that the current
performance cost calculation is not exactly optimal.

The current patch aims at not using the max active value of the flush and
restore queues in cost calculation. This is simply because such a value
does not exist in the rewritten HSM system. Active and queued values still
make sense as any tape operation goes through a queued and active phase,
which can easily be counted and reported to pool manager. It doesn't matter
whether the HSM integration driver actually got a queue - each request still
got these phases.

Recall that performance cost was computed as an average of a fill factor for
each queue, i.e. the average of (active + queued) / max. For the flush and
restore queues, this is changed to contribute 1.0 to the average if there
are any queued requsts (intuitive interpretation is that the queue has
reached its capacity), and 1 - 0,75^active otherwise. Ie the contribution
grows from 0 and approaches 1 as the number of active flushes/restores
grows. Thus the maximum is no longer part of the calculation (as was the
intent with this patch).

The patch changes one other aspect of the cost calculation: Previously a
queue with max being 0 did not contribute to the average at all - it was
as if the queue did not exist. Hence suspending a queue by setting the
max to zero would significantly alter the cost. It also makes cost computed
from pools that may have a non-zero queue defined incomparable to a pool
that has a limit larger than zero: In other words, the cost of eg an FTP
transfer depends on whether the max on some other queue happens to be 0
or 1, even if that other queue is otherwise empty. This patch changes the
logic such that queues with max being zero contribute with 0.0 if the queue
is empty, and 1.0 otherwise.

Note that this patch does alter the value of the computed performance cost.
Since this value is used as a trigger for pool to pool replication on
load, and also as a fallback trigger for selecting other pool manager
links, this patch may mean that cost-cuts in pool manager will have to
be adjusted. This is however a perfect time for such a change, as dCache
2.7 forces a migration to WASS anyway and tuning parameters will have to
be revisited anyway. I also doubt that many sites have the cost-cuts
so finely tuned, that this patch makes much of a difference.

Target: trunk
Require-book: yes
Require-notes: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Karsten Schwank <karsten.schwank@desy.de>
Patch: http://rb.dcache.org/r/5790/
Remove remaining scoped properties.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5824/
Remove support for scoped properties

This patch removes all support for scoped properties.

Target: trunk
Require-book: yes
Require-notes: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5825/
skel: fix regression in billing properties affecting dcache database commands

The elimination of scoping and db.* properties did not take into account the ability to run the billing cell without a database.  The properties before this change relied on setting db.name and db.schema.changeset conditionally, based on the value of billing.enable.db, so that the dcache database commands would work correctly.

Specifically, when billing db is disabled, dcache database ls should not report the presence of the billing database, and more importantly, dcache database update should not attempt to run the liquibase changesets (since there may be no billing database present at all).

This patch restores the correct behavior.

Testing done:

Before the patch, with billing.enable.db=false,

dcache database ls was still reporting the presence of the database
dcache database update would fail if there is no billing database in the postgres installation (which is quite possible).

After the patch, correct behavior is observed for these commands.

Target: master
Require-notes: yes
Require-book: no
Acked-by: Paul

RELEASE NOTES:

If overriding the database name for billing, be aware that billing.enable.db will not affect the dcache database commands; in particular, update will still attempt to run liquibase.
Thus any non-default setting for this property will require the actual presence of a database by that name.