dcache: (remote logging) change default server port to be outside ephemeral port range

module: dcache (alarms)

This patch is an added precaution in connection with http://rb.dcache.org/r/5664.  It was discovered that the default port suggested by ch.qos.logback for the SocketAppender and server, 60001, is in fact not a good choice because it lies within the ephemeral port range for some OS's.  When there is no endpoint listening (I believe this only occurs when the server is supposed to be running local to the client domain), that port can be chosen as source for establishing a connection, such that the appender then ends up connecting to itself.  If events are actually sent over this connection, it quickly fills up the buffer and blocks.  This causes all threads which do logging to block, effectively freezing the system.

The above-mentioned patch sets remote logging by default to off, so that events are not sent unless it is explicitly activated.  The self-connected appender, however, is still potentially present (though at the 10 second reconnect timeout -- which is what the logback.xml is configured for -- it will take a while for it to find the 60001 port).

Setting the default port outside most ephemeral port ranges will guarantee that this connect-back does not happen.

Testing done:

Using a small test class with the remote logger reconnect delay set to 1 ms, and no server endpoint running, port 60001 was first chosen.  After 30 seconds,

[root@otfrid dcache]# netstat -anp | grep 60001
tcp        4      0 127.0.0.1:60001             127.0.0.1:60001             ESTABLISHED 6051/java

was seen.  The test waits 60 seconds, then begins to log continuously. The system freezes after 5446 events:

6001 25 Jun 2013 08:03:24 [] error message 1
...
6001 25 Jun 2013 08:03:24 [] error message 5446

Rerunning with port set to 9876 shows that after 10 minutes and more than 17M events, no connection is established and hence no freeze.

Target: master
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Gerd
Bug: http://rt.dcache.org/Ticket/Display.html?id=7856

RELEASE NOTES:

The default port for remote logging has been changed from 60001 to 9876; the latter lies outside most OS ephemeral port ranges.  This avoids a situation in which the connection host is localhost (the default), but there is no server listening on that port, thus permitting that port to be chosen as source for the appender connection and consequently establishing a connection to itself.  When this happens, the buffer quickly fills and blocks further logging, freezing all threads that do logging (effectively the entire domain).
cells: Fix login manager login limit interpretation

Addresses an issue in FTP and DCAP doors in which the login limit
was would allow n + 1 logins when n was given as a limit. Also
addresses an issue in which a limit of zero was considered to
be no limit at all. The correct value for disabling the limit is
-1.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Christian Bernardt <christian.bernardt@desy.de>
Patch: http://rb.dcache.org/r/5676/
info-provider: Escape percent sign in entity values

Percent signs are the only characters not allowed literally
in entity values, thus these must be escaped when exporting
the dCache configuration to an XML DTD.

The patch addresses the problem that info-provider is broken
by a recent patch adding logback formatting strings as
configuration values.

Target: trunk
Request: 2.6
Require-book: no
Require-notes: yes
Acked-by: Christian Bernardt <christian.bernardt@desy.de>
Patch: http://rb.dcache.org/r/5673/
poolmanager, doors: Don't retry stage requests on the same pool

Addresses a regression introduced between 1.9.12 and 2.2. The regression
means that poolmanager would not remember the previously tried stage pool
and thus could select the same pool twice in a row when retrying failed
stage requests.

The patches addresses this problem. To be effective, both pool manager
and doors have to be updated, although the patch does not break any
version compatibility.

Target: trunk
Request: 2.6
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/5678/
srm: Implement SRM_FILE_BUSY response

Addresses the issue, that the SRM is ignorant to other SRM requests
it may have on the same SURL. This leads to the dreaded "Already have 1
record" error in space management, as two concurrent puts on the same
file will be allowed by the SRM, but fail in space manager. More over,
there is currently no way for the user to abort the earlier requests if
they do not have the request number.

The SRM spec says that an upload operation should cause most other
operations on that SURL to fail with SRM_FILE_BUSY. srmRm should cause
existing get and put operations on the SURL to be aborted. This patch
implements those aspects of the SRM spec.

The patch does not implement this logic for srmCopy. This will be done
in a separate patch.

Since the patch addresses a real world issue we (and probably other
sites) suffer from, I request this to be merged into 2.6. Given the
change in observable behaviour, I am reluctant to merge this into 2.2
at this point.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5677/
dcache-webadmin: avoid rrd4j timestep error statement

module: dcache-webadmin (pool queue plots)

Under certain restart conditions, the rrd4j utility will report that the call to write is doing so with a timestamp which is less than 1 second since the last update.  While this error is not fatal, it is annoying to see in the log.

The extra check simply avoids writing the data under these conditions and waits till the next iteration.

TESTING DONE:

Without the patch, rapid multiple restarts of the webadmin service/httpd domain will probably produce a trace similar to this:

26 Jun 2013 09:32:47 (System) [] problem running collectPoolSeclectionUnit
java.lang.IllegalArgumentException: Bad sample time: 1372257000. Last update time was 1372257000, at least one second step is required
        at org.rrd4j.core.RrdDb.store(RrdDb.java:553) ~[rrd4j-2.1.1.jar:na]
        at org.rrd4j.core.Sample.update(Sample.java:197) ~[rrd4j-2.1.1.jar:na]
        at org.dcache.webadmin.model.dataaccess.util.RrdPoolInfoAgent.storeToRRD(RrdPoolInfoAgent.java:331) ~[classes/:na]
        at org.dcache.webadmin.model.dataaccess.util.RrdPoolInfoAgent.processPools(RrdPoolInfoAgent.java:291) ~[classes/:na]
        at org.dcache.webadmin.model.dataaccess.util.RrdPoolInfoAgent.run(RrdPoolInfoAgent.java:138) ~[classes/:na]
        at java.lang.Thread.run(Thread.java:722) [na:1.7.0]

With the patch, the log remains clean during multiple successive attempts to restart.

Target: master
Patch: http://rb.dcache.org/r/5680
Require-notes: yes
Require-book: no
Request: 2.6
Acked-by: Gerd

RELEASE NOTES:  The cause of the stack trace occasionally seen in the webadmin log file (on restart of the service) has been addressed.
pinmanager, other: Make pinmanager configuration comply with new policy

Addresses the issue that pin manager properties follow inconsistent
naming conventions.

This patch applies the naming conventions agreed to at the 2013 developers
workshop, with a few personal additions. The patch also affects most other
services, as some changed properties are used by most of them.

Guiding principals:

- All properties used by a service are prefixed by the service name (pinmanager
  in this case).

- All properties are lower case

- Properties are structured hierarchically

- A dash is used as a word separator

- Time units are made explicit by adding a property suffixes with .unit

- Properties that are used by many services have an equivalent property with
  the dcache. prefix.

- Old properties are kept during a trandition period, but marked deprecated.

- Eventually the scoping operator will be removed, but during a transition period
  it is still used. This is to avoid chaging all properties in one big bang.

New configuration families introduced:

service.<service> defines properties for communicating with other cells. Available
in both the shared dcache. hierarchy and in a per service hierarchy (currently only
pinmanager). Not for properties belonging to that service. This is only for properties
other services use to talk to <service>.

cell.threads. for properties related to threading of cell (in particular threads for
message delivery).

cell.queue. for properties related to message queueing.

authz. for authorization related properties.

Properties introduced:

dcache.service.pnfsmanager
dcache.service.poolmanager
dcache.service.billing
dcache.service.gplazma
dcache.service.spacemanager
dcache.service.pinmanager
dcache.service.replica
dcache.service.loginbroker
dcache.authz.staging
dcache.authz.staging.pep
pinmanager.cell.name
pinmanager.cell.threads.min
pinmanager.cell.threads.max
pinmanager.cell.threads.max-idle-time
pinmanager.cell.threads.max-idle-time.unit
pinmanager.cell.queue.max
pinmanager.service.poolmanager
pinmanager.service.poolmanager.timeout
pinmanager.service.poolmanager.timeout.unit
pinmanager.service.pnfsmanager
pinmanager.service.pnfsmanager.timeout
pinmanager.service.pnfsmanager.timeout.unit
pinmanager.service.pool.timeout
pinmanager.service.pool.timeout.unit
pinmanager.db.host
pinmanager.db.name
pinmanager.db.user
pinmanager.db.password
pinmanager.db.password.file
pinmanager.db.driver
pinmanager.db.url
pinmanager.db.schema.changelog
pinmanager.db.connections.max-per-partition
pinmanager.db.connections.min-per-partition
pinmanager.expiration-period
pinmanager.expiration-period.unit
pinmanager.max-pin-duration
pinmanager.max-pin-duration.unit
pinmanager.authz.staging
pinmanager.authz.staging.pep

Properties deprecated:

pnfsmanager
poolmanager
billing
gplazma
spacemanager
loginBroker
stageConfigurationFilePath
stagePolicyEnforcementPoint
pinManagerDbHost
pinManagerDbName
pinManagerDbUser
pinManagerDbPassword
pinManagerPasswordFile
pinManagerMaxActiveJdbcConnections
pinManagerMaxPinDuration

Target: trunk
Require-notes: yes
Require-book: yes
Merge branch 'master' of https://github.com/dCache/dcache
xrootd: Make xrootd configuration comply with new policy

Changes spill over to other services too, but the main focus of
the patch is to rename all xrootd properties.

Introduces:

dcache.service.loginbroker.update-period
dcache.service.loginbroker.update-period.unit
dcache.service.loginbroker.update-threshold
pool.mover.xrootd.timeout.idle
xrootd.cell.name
xrootd.net.port
xrootd.net.backlog
xrootd.net.listen
xrootd.threads
xrootd.max-total-memory-size
xrootd.max-channel-memory-size
xrootd.service.poolmanager
xrootd.service.poolmanager.timeout
xrootd.service.poolmanager.timeout.unit
xrootd.service.pnfsmanager
xrootd.service.gplazma
xrootd.service.gplazma.timeout
xrootd.service.gplazma.timeout.unit
xrootd.service.billing
xrootd.service.loginbroker
xrootd.service.loginbroker.update-period
xrootd.service.loginbroker.update-period.unit
xrootd.service.loginbroker.update-threshold
xrootd.service.loginbroker.family
xrootd.service.loginbroker.version
xrootd.service.pool.timeout
xrootd.service.pool.timeout.unit
xrootd.root
xrootd.authz.read-paths
xrootd.authz.write-paths
xrootd.authz.user
xrootd.plugins
xrootd.mover.queue
xrootd.mover.timeout
xrootd.mover.timeout.unit
admin.net.listen
dcache.net.listen
dcap.net.listen
gsidcap.net.listen
kerberosdcap.net.listen
authdcap.net.listen
dcap.service.loginbroker
dcap.service.loginbroker.update-period
dcap.service.loginbroker.update-period.unit
dcap.service.loginbroker.update-threshold
authdcap.service.loginbroker
authdcap.service.loginbroker.update-period
authdcap.service.loginbroker.update-period.unit
authdcap.service.loginbroker.update-threshold
kerberosdcap.service.loginbroker
kerberosdcap.service.loginbroker.update-period
kerberosdcap.service.loginbroker.update-period.unit
kerberosdcap.service.loginbroker.update-threshold
gsidcap.service.loginbroker
gsidcap.service.loginbroker.update-period
gsidcap.service.loginbroker.update-period.unit
gsidcap.service.loginbroker.update-threshold
ftp.net.listen
gridftp.net.listen
kerberosftp.net.listen
ftp.service.loginbroker
ftp.service.loginbroker.update-period
ftp.service.loginbroker.update-period.unit
ftp.service.loginbroker.update-threshold
gridftp.service.loginbroker
gridftp.service.loginbroker.update-period
gridftp.service.loginbroker.update-period.unit
gridftp.service.loginbroker.update-threshold
kerberosftp.service.loginbroker
kerberosftp.service.loginbroker.update-period
kerberosftp.service.loginbroker.update-period.unit
kerberosftp.service.loginbroker.update-threshold
nfsv41.service.loginbroker
nfsv41.service.loginbroker.update-period
nfsv41.service.loginbroker.update-period.unit
nfsv41.service.loginbroker.update-threshold
srm.net.listen
srm.service.loginbroker
srm.service.loginbroker.update-period
srm.service.loginbroker.update-period.unit
srm.service.loginbroker.update-threshold
webdav.net.listen
webdav.service.loginbroker
webdav.service.loginbroker.update-period
webdav.service.loginbroker.update-period.unit
webdav.service.loginbroker.update-threshold

Deprecated:

loginBrokerUpdateTime
loginBrokerUpdateThreshold
xrootdPort
xrootd/port
xrootdBacklog
xrootdThreads
xrootdMaxTotalMemorySize
xrootdMaxChannelMemorySize
xrootdPoolManagerTimeout
xrootdPoolTimeout
xrootdRootPath
xrootdAllowedWritePaths
xrootdAllowedReadPaths
xrootdPlugins
xrootdUser
xrootdIoQueue
xrootdMoverTimeout
listen
srm.loginbroker
srm.loginbroker.period
srm.loginbroker.threshold

Removed (previously deprecated):

xrootdAuthNPlugin
xrootdAuthzPlugin

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/5642/
poolmanager, pool: Drop classic partition and most space cost related code

This patch drops the classic partition. Wass has been the default partition
type for new partitions for a while already. During that time we have not
heard about any issues with wass. Therefore this patch drops the classic
partition type.

Technically, ClassicPartition is kept as a base class for WassPartition, but
the dead code is dropped. A lot of related code for computing the space
cost is dropped as well as it is no longer used.

A number of user visible changes should be documented in the release notes:

- Sites still using the classic partition type must manually update
  poolmanager.conf to create wass partitions instead.

- When switching from classic to wass, it is important to reset cpucostfactor
  and spacecostfactor, as the interpretation of these parameters has changed.
  We suggest to use the default value of 1.0 for both parameters.

- The alternate PoolSelectionUnitV2wgap has been dropped. It can no longer
  be used as a value for the selectionUnit configuration property. We believe
  that this functionality is no longer required when using wass. Should a
  need for such functionality arise in the future, it would better be
  implemented as a new partition type rather than a selection unit.

- The pool manager 'xcm ls' command no longer accepts any arguments or
  options.

- The pool manager 'cm ls' command no longer accepts the -size and -d
  options.

- The pool manager 'getpoolsbylink' command no longer accepts the -size
  option.

- The pool migration module no longer recognizes the source.spaceCost and
  target.spaceCost variables.

- The pool migration module no longer accepts 'best' as a value for
  the -select option.

- The pool manager 'rebalance pgroup' command no longer accepts 'sc' as a
  value for the -metric option. The new value 'free' has been added instead.

- The pool manager 'costCalculation' configuration parameter is obsolete.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/5682/
spacemanager : improve selecPool logic a little

commit 0bca233fba745c862b6c7253bfa94bb1945c241c
Author: Dmitry Litvintsev <litvinse@fnal.gov>
Date:   Fri Jun 28 11:46:00 2013 -0500

    spacemanager : improve selecPool logic a little

    what happened

    Ticket: #number
    Acked-by: Gerd Behrmann
    Target: trunk
    Require-book: no
    Require-notes: no

commit 3a4537c5fa4f0484d01aaba3ad881c94d44673f9
Author: Dmitry Litvintsev <litvinse@fnal.gov>
Date:   Mon Jun 24 16:01:16 2013 -0500

    module: summary line
    streamline selectPool logic a little

    what happened

    Ticket: #number
    Acked-by:
    Target: trunk
    Require-book: no
    Require-notes: no
pinmanager: Fix typo in pin manager property

Target: trunk
Require-notes: no
Require-book: no
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7900
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/5685/
Fix various database-related niggles

This patch fixes a number of relatively minor database-related irritations:

  1. When dCache starts, the SRM's AuthRecord persistency BoneCP db-pool emits a

         Max Connections < 1. Setting to 20

     warning.  This is fixed by specifying the maximum number of connections.

  2. Remove the unnecessary 'provider' from JDO persistence-unit stanza (we're no
     longer using TopLink, so this work-around is not needed any more).

  3. Fix various unit-tests to configure BoneCP's max connections, so they no
     longer emit the "Max Connections < 1" warning.

  4. Remove a dead-code line from database initialisation (used by space-manager
     and replica-manager).  The dead-code is likely due to a typo ("Min" vs "Max")
     but, to avoid possible side-effects, this patch simply removes the dead-code.

  5. Fix AuthRecordPersistenceUnit configuration so that DataNucleus' (JPA) enhancer
     no longer attempts to modify the byte-code of all dCache classes, but only
     those necessary.  This makes the build *much* less noisy and slightly
     faster.

When the DataNucleus' (JDO) enhancer modifies classes for the pin-manager, only a single
class needs enhancing (org.dcache.pinmanager.model.Pin), however the enhancer reports
that some 30 classes were considered.  I don't know which 30 classes are examined, nor
why the enhancer is examining classes other than Pin.

Patch: http://rb.dcache.org/r/5687/
Acked-by: Albert Rossi
Target: trunk
Requires-notes: no
Requires-book: no
webdav: Comply with new property policy

New

missing-files.timeout.unit
webdav.cell.name
webdav.service.pnfsmanager
webdav.service.pnfsmanager.timeout
webdav.service.pnfsmanager.timeout.unit
webdav.service.poolmanager
webdav.service.poolmanager.timeout
webdav.service.poolmanager.timeout.unit
webdav.service.spacemanager.enabled
webdav.service.pool.timeout
webdav.service.pool.timeout.unit
webdav.service.gplazma
webdav.service.gplazma.timeout
webdav.service.gplazma.timeout.unit
webdav.service.billing
webdav.service.loginbroker.version
webdav.service.loginbroker.family
webdav.service.missing-files
webdav.service.missing-files.timeout
webdav.service.missing-files.timeout.unit
webdav.service.missing-files.enabled
webdav.mover.queue
webdav.mover.kill-timeout
webdav.mover.kill-timeout.unit
webdav.mover.timeout
webdav.mover.timeout.unit
webdav.mover.transfer-confirmation-timeout
webdav.mover.transfer-confirmation-timeout.unit
webdav.net.port
webdav.net.internal
webdav.root
webdav.authz.allowed-paths
webdav.authz.readonly
webdav.authz.anonymous-operations
webdav.authz.anonymous-listing
webdav.authn.protocol
webdav.authn.keystore
webdav.authn.keystore.password
webdav.authn.truststore
webdav.authn.truststore.password
webdav.authn.accept-client-cert
webdav.authn.require-client-cert
webdav.authn.basic
webdav.authn.hostcert.cert
webdav.authn.hostcert.key
webdav.authn.hostcert.key.unit
webdav.authn.capath
webdav.authn.capath.refresh
webdav.authn.capath.refresh.unit

Deprecated

webdavPnfsTimeout
webdavPoolManagerTimeout
webdavPoolTimeout
webdavGplazmaTimeout
webdavIoQueue
webdavKillTimeout
webdavMoverTimeout
webdavTransferConfirmationTimeout
webdavPort
webdav/port
webdavInternalAddress
webdavRootPath
webdavAllowedPaths
webdavReadOnly
webdavAnonymousAccess
webdavAnonymousListing
webdavProtocol
webdavKeyStore
webdavKeyStorePassword
webdavTrustStore
webdavTrustStorePassword
webdavWantClientAuth
webdavNeedClientAuth
webdavBasicAuthentication

Removed

hostCertificateRefreshPeriod
trustAnchorRefreshPeriod
verifyHostCertificateChain
webdav.images.logo
webdav.images.directory
webdav.images.file
webdav.style.css
webdavLogoPath
webdavDirIconPath
webdavFileIconPath
webdavCssPath
logoPath
dirIconPath
fileIconPath
cssPath
webdav.templates.list
httpPortNumber
httpRootPath
httpAllowedPaths
webdavContextPath

Other changes

srmSpaceManagerEnabled only accepts the values true and false

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Albert Rossi <arossi@fnal.gov>
Patch: http://rb.dcache.org/r/5683/
Fix nsswitch plugin unit-tests

The nsswitch gPlazma plugin uses JNA to access some native functions
common to various BSD-like OSes (including GNU/Linux).  It implements
the identity-, map- and session- gPlazma plugin functionality.  The
identity- and map- functionality is based on username <--> uid,
group-name <--> gid and a user's group-membership.  The session
functionality provides hard-coded behaviour (which might be better
factored out as a separate plugin).

There are several problems with the current nsswitch unit-tests:

  1. The unit-tests depend on JNA.  Not all JVMs support JNA, so can
     cause problems, especially since JNA relies on a native library,
     which is not available for all OSes.

  2. The unit-tests depend on OS/native functions.  Not all OSes
     support these functions.  While this prevents such OSes from
     using the nsswitch plugin for deployed dCache instances, it
     should not prevent developers from using such OSes.

  3. The unit-tests depend on system configuration.  While most
     Unix-like systems have a user "root" with uid 0 and primary gid
     0; other users and groups are not standardised --- even gid 0 can
     have different group-name ("root" or "wheel")

  4. The existing tests cover only part of the identity functionality.
     The tests do not cover group--gid identity-reverse-mapping, all
     map and session functionality.

This patch addresses these issues by fully mocking the JNA-backed
classes.  That way, the unit-tests are completely isolated from JNA,
the underlying OS and its configuration.  This allows a complete
coverage of the nsswitch plugin's functionality in a portable fashion.

It turns out that mocking the JNA-based classes that nsswitch plugin
uses is particularly tricky as these Java classes inherit from system
classes that make use of static initialisers and make JNI calls from
the constructor.  Mockito-mocked classes solve points 2. to 4., but
will still require a valid JNA, so point 1. is still a problem.

To solve point 1., this patch adds the PowerMock framework.  This is
an extension to mockito that adds support for the byte-code
manipulation needed to avoid static initializer and suppress
superclass constructors.  The nsswitch unit-tests are still written
using junit and mockito; however, in addition, PowerMock is used as
required.  Note that all existing unit-tests continue to function as
normal and the extra PowerMock functionality is only invoked when the
@RunWith(PowerMockRunner.class) annotation is specified.

The existing nsswitch tests are rewritten as BDD tests and additional
tests are added.  Fluent classes and utility methods are used make the
tests as readable as possible, at the risk of somewhat obfuscating
what is actually going on.

Target: master
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5688/
Acked-by: Gerd Behrmann
Teach git to ignore generated content

Maven adds considerable predictability of where generated files are
stored.  These are of little interest to any SCM since they are
managed by the build process and the results should not be stored in a
SCM.

This patch adds a .gitignore file that excludes the 'target'
directories in the current modules (or sub-projects) from git's
consideration.  Following maven convention, this should be sufficient
to exclude all auto-generated material.

Target: trunk
Requires-notes: no
Requires-book: no
Patch: http://rb.dcache.org/r/5686/
Acked-by: Gerd Behrmann
Acked-by: Albert Rossi
ftp: Comply with new property policy

Besides updating all ftp related properties to comply with the
new policy, this patch collapses the various ftp services into
a single service. A property is now used to select the authentication
protocol. This is how it is done for webdav and xrootd already. The
new configuration policy encourages this approach as most ftp
properties are independent of the authentication protocol and were
shared by all serices. Collapsing them into a single service makes
the service more DRY.

New properties:

ftp.cell.name
ftp.authn.protocol
ftp.authn.kerberos.service-principle-name
ftp.authn.kerberos.key-distribution-center-list
ftp.authn.hostcert.key
ftp.authn.hostcert.cert
ftp.authn.capath
ftp.authn.kpwd
ftp.authz.stage
ftp.authz.stage.pep
ftp.authz.readonly
ftp.net.port
ftp.net.port-range
ftp.net.internal
ftp.performance-marker-period
ftp.performance-marker-period.unit
ftp.limits.retries
ftp.limits.streams-per-client
ftp.limits.clients
ftp.mover.queue
ftp.transaction-log
ftp.overwrite
ftp.security.ciphers
ftp.service.poolmanager
ftp.service.poolmanager.timeout
ftp.service.poolmanager.timeout.unit
ftp.service.spacemanager.enabled
ftp.service.pool.timeout
ftp.service.pool.timeout.unit
ftp.service.pnfsmanager
ftp.service.pnfsmanager.timeout
ftp.service.pnfsmanager.timeout.unit
ftp.service.loginbroker.family
ftp.service.loginbroker.version
ftp.service.gplazma
ftp.service.billing

Deprecated

ftpPort
kerberosFtpPort
gsiFtpPortNumber
clientDataPortRange
gsiftpAdapterInternalInterface
performanceMarkerPeriod
gsiftpMaxRetries
gsiftpMaxStreamsPerClient
gsiftpMaxLogin
gsiftpDeleteOnConnectionClosed
gsiftpIoQueue
FtpTLogDir
gridftp.security.ciphers
gsiftpPoolManagerTimeout
gsiftpPoolManager
gsiftpPoolTimeout
gsiftpPnfsTimeout

Forbidden

ftp/kerberos.service-principle-name
ftp.read-only
ftp/port

Removed

ftpBase
spaceReservation
spaceReservationStrict
gsiftpAllowPassivePool
kerberosScvPrincipal
gsiftpDefaultStreamsPerClient

Removed services

kerberosftp
gridftp

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5694/
(logging) minor adjustments to error-level logging (@author arossi)

I have gone through all the .java files carrying the annotation @author arossi, looking for logging irregularities.  I have tried to eliminate things like logger.error logging a stack trace when not clearly an unexpected exception (bug).  There are also a few other minor fixes.

Target: master
Patch: http://rb.dcache.org/r/5690/
Require-notes: no
Require-book: no
Acked-by: Paul
dcache: (billing) Fix incorrect MoverData field name

Regression introduced by last large multi-patch billing update.

The MoverData field 'p2p' was renamed to 'isPtp', following suggestion that Java convention be followed.  However, the liquibase and package.jdo files continued to assume the database table attribute was 'p2p'.  Hence when the data object gets dehydrated by DataNucleus, the insert statement fails.

This patch reverts the Java field name to 'p2p'.

Testing done:

Without the patch, writes to the billing database will produce the following error:

02 jul 2013 05:43:03 (billing) [] Insert of object "org.dcache.services.billing.db.data.MoverData@6e19f13" using statement "INSERT INTO "billinginfo"
("transfersize","fullsize","cellname","datestamp","protocol","action","errormessage","initiator","transaction","storageclass","errorcode","client","isp2p","pnfsid","isnew","connectiontime")
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)" failed : user lacks privilege or object not found: isp2p
02 jul 2013 05:43:03 (billing) [] could not not commit; some billing
data may be lost

With the patch, successful srmcp produced the correct entry in billinginfo.

Note:  this has been tested on a regular instance of dcache, not using system-test.

Target: master
Patch: http://rb.dcache.org/r/5692/
Require-book: no
Require-notes: yes
Request: 2.6
Acked-by: Gerd

RELEASE NOTES:  Fixes bug preventing inserts into the billinginfo table.
ssh-admin: quieten noisy logging

We deliver dCache without an authorized_keys2 file, yet
attempts to log in when a user presents an SSH key results in
logging like:

03 Jul 2013 14:11:40 (ssh2Admin) [] The authorized_keys2 file was not found. Please check if it exists in: /home/paul/git/dCache/packages/system-test/target/dcache/etc/admin/authorized_keys2
03 Jul 2013 14:11:40 (ssh2Admin) [] Could not find key: RSA Public Key
            modulus: b724ef9e0182b02db76ea20a0f674d83 [many hex digits follow ...]
    public exponent: 23
in /home/paul/git/dCache/packages/system-test/target/dcache/etc/admin/authorized_keys2.
03 Jul 2013 14:11:40 (ssh2Admin) [] The authorized_keys2 file was not found. Please check if it exists in: /home/paul/git/dCache/packages/system-test/target/dcache/etc/admin/authorized_keys2
03 Jul 2013 14:11:40 (ssh2Admin) [] Could not find key: DSA Public Key
            y: 51e74d6deb9b0b87456522d56f4fb952c21ec0 [many hex digits follow ...]
in /home/paul/git/dCache/packages/system-test/target/dcache/etc/admin/authorized_keys2.

This is wrong as the authorized_keys2 file is not required and
the login attempt was actually successful, using username+password
authentication.

This patch reduces the above messages to a single message per failure to
read the authorized_keys2 file (rather than two) and logs this message at
debug level, rather than error.

Some log messages were also rephrased to avoid unnecessary verbosity.

Target: master
Requires-book: no
Requires-notes: no
Patch: http://rb.dcache.org/r/5696/
Acked-by: Gerd Behrmann