ftp: Clean up address handling

Both the variable naming and how we compose various addresses is most
confusing. This patch tries to clean this up to make the logic easier to
follow.

There is also one behavioural change in how we determine the pool facing
interface for proxied data channels: If not set explicitly in the
configuration, before we defaulted to the control channel interface. Whether a
control channel is established on one or the other interface has absolutely no
relationship to which interface faces the pools, nor does establishing the
control channel on a different interface suddently make the pools available on
another interface. Hence this patch changes this to default to the default
interface of the host (as determined by resolving the host name).

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: https://rb.dcache.org/r/7594/
pool: Partially fix interface selection for FTP mover

The FTP mover tries to select an interface with the correct
protocol family and falls back to proxy mode if not possible.
Unfortunately the NetworkUtils utility class fails to return
null in case no suitable interface could be found.

On the other hand, the original protocol family less version
of the NetworkUtils method fails to handle null values.

This patch fixes the former to return null if the protocol
family could not be match, and fixes the latter by falling
back to ignoring the protocol family. The latter is likely
wrong, but it restore the previous functionality and resolves
the problem that existing callers of that method do not
handle a null return value.

There are plenty of issues with this piece of code. This
patch only sets out to fix the one.

Target: trunk
Require-notes: yes
Require-book: no
Request: 2.11
Request: 2.10
Request: 2.9
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: https://rb.dcache.org/r/7597/
chimera: do not shutdown jdbc dataSource on close

Acked-by: Gerd Behrmann
Target: master
Require-book: no
Require-notes: no
pool: Improve selection of listening address

The FTP mover uses logic that can select a local address that
likely is reachable from a given client address.

To do this, dCache uses a trick that will determine which
network interface the host would use to reach the client address.
It it is a good bet (although not a guarantee) that the client
is able to reach us on the same interface.

The code suffers from a flaw in what it does when the trick
fails. There are several cases when this may happen, such as
when the client address is an IPv6 address and the local
host doesn't have an IPv6 stack. The previous logic would
fall back to using the address to which the host name resolves.
This is fragile as we often observes that hosts do not resolve
their name to the external interface. We can do significantly
better by iterating over local interfaces and their addresses
and choose an address from the same protocol family and with
a suitable address scope.

The same improvement can be applied to a variant of this algorithm
that ignores the protocol family. Such a change is left to another
patch.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: https://rb.dcache.org/r/7598/
pool: Improve selection of listening address

Similar to https://rb.dcache.org/r/7598/, but applies the same optimisation
to dcap, http, nfs and xrootd movers.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: 	https://rb.dcache.org/r/7599/
Add caching of local addresses

Adds a 5 second cache of local addresses.

There is room for refactoring the NetworkUtils class further
to avoid the functional overlap with the existing static
enumeration of interfaces.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7600/
Refactor NetworkUtils to reduce redundancy

The refactored code also makes sure we discover changes
in network interfaces by not storing a static list of addresses
during startup.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: https://rb.dcache.org/r/7602/
Switch to using GlobusPrincipal from JGlobus 2

When we switched to JGlobus 2, we had to keep the old GlobusPrincipal
to maintain backwards compatibility with old pools. The pools we
need to maintain compatibility with now all use JGlobus 2, so we
can safely switch to the new verion.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7603/
Drop deprecated fields

Over time we have accumulated a number of fields we have kept
for backwards compatibility with old pools. Current master
only needs to maintain compatibility with 2.10 and 2.11, so
there is no need to keep these old fields anymore.

The obsolete 'psu set linkGroup attribute' command is removed
too. Sites are adviced to 'save' their pool manager config
prior to upgrading to ensure that this command is no longer
present in the configuration.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: https://rb.dcache.org/r/7604/
srm: Some automated refactorings

All identified by IntelliJ.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: https://rb.dcache.org/r/7605/
chimera: relax mtime update validatrion in the tests

do not sleep in test. We do check that generation is updated
even if mtime stays the same.

Acked-by: Gerd Behrmann
Target: master
Require-book: no
Require-notes: no
chimera: update chimera API

the biggest chimera change since day-one: remove setXXX methods and
use FsInode#setStat() or JdbcFs#setInodeAttributes(). Lot of deletes ....

Acked-by: Albert Rossi
Target: master
Require-book: no
Require-notes: no
srm: fix semi-infinite ls range with non-zero offset

The SRM protocol allows the client to request a directory listing with
an offset but no length.  Internally, we represent this situation as
if the client requested Integer.MAX_VALUE responses.

Later, we build a Range describing the requested directory listing; however,
this fails to take into account that adding a small positive integer to
Integer.MAX_VALUE results in a negative number, which triggers an
IllegalArguementException.

This is fixed so that if the index of last requested item is
Integer.MAX_VALUE or more then a semi-infinite range is generated instead.

Target: master
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7181
Ticket: http://rt.dcache.org/Ticket/Display.html?id=8550
Patch: https://rb.dcache.org/r/7613/
Acked-by: Gerd Behrmann
Request: 2.11
Request: 2.10
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
chimera: setInodeAttributes have to throw FileNotFoundChimeraException

Throw FileNotFoundChimeraException on attempt to update attributes of non existing inode.
Enable existing but disabled test cases to verify this functionality.

Allow size update only for regular files. Introduces two new exceptions:
IsDirChimeraException and InvalidArgumentChimeraException which corresponds
do POSIX  EISDIR or EINVAL and error codes.

Acked-by: Gerd Behrmann
Target: master
Require-book: no
Require-notes: no