Per ApplicationContext @Transactional configuration

Spring's AspectJ @Transactional implementation is limited to a singleton that
only allows one ApplicationContext to configure the aspect, thus only
supporting a single transaction manager for the entire ClassLoader.

This patch introduces a modified version of Spring's
AnnotationTransactionAspect, which instantiates the aspect for each
transactional bean. A BeanPostProcesser is used to inject the appropriate
transaction manager into the aspect instances.

With this patch we can begin to use @Transactional outside pin manager (and
have multiple pin manager services in the same dCache domain).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6382/
nfs: fix compatibility with 2.6

post 2.6 nfs code has changes package name from

org.dcache.chimera.nfs => org.dcache.nfs
Nevertheless one class is used inside
a message sent to a pool and brakes
compatibility with.

this change restores compatibility.

Notice, this makes 2.7.6 not compatible in mixed
setup.

Acked-by: Gerd Behrmann
Target: master, 2.7
Require-book: no
Require-notes: no
Fix gitignore
spacemanager: Use JdbcTemplate and annotation based transaction demarcation

JdbcTemplate is similar to our custom DBManager class. JdbcTemplate is however
much more feature complete. It is also much more well known. This patch replaces
DBManager with JdbcTemplate in space manager.

The patch also changes space manager to Spring's annotation based transaction
demarcation. This removes the need to pass around a Connection object between
methods. It makes it easier to ensure that all processing done for a message is
in a single transaction.

Note that I in some cases have annotated methods with @Transactional even
though they are already called from a transactional context. By default Spring
will simply let the method be part of the outer transaction. I use this for
methods that would not be correct if run outside a transactional context (eg
methods that do select for update).

The patch changes the space manager schema such that primary keys are flagged
as auto increment fields. Thus we no longer need to maintain the counter logic
ourself (the counter logic is tricky in the pressence of transactions as it
must not be rolled back in case of an aborted transaction). How this is
implemented is however database specific and the changeset thus contains some
special cases for PostgreSQL. The liquibase changelog defines rollback logic.
Upon downgrading dCache, the schema changes must be rolled back.

The patch adds unique constraint to pnfsid field of srmspacefile table. An
index on this field already existed, so making it unique does not add any extra
overhead. The column type is changed to varchar(36) to match the type used in
the rest of dCache.

Since JdbcTemplate translates all SQLException instances to a more detailed
exception hierarchy, the patch also updates lots of exception handlers to
catch the more detailed exceptions.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6395/
spacemanager: Convert tabs to spaces.

Require-notes: no
Require-book: no
Patch: none
spacemanager: Split space manager into service layer and data access layer

Introduces a clean data access object interface, separating the space manager
into a data access layer and a service layer.

Since all JDBC related code is now in a single class, I have chosen to inline
SQL expressions. I feel this makes the code easier to read as one does not
need to lookup the expressions all the time.

Given the size of the change, it is difficult not to sneak in some minor
janitorial changes in that is touched anyway.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6396/
transfermanagers: Send subject and updated file attributes to pool

In contrast to other doors, transfermanager does not inject the subject
and updated file attributes intp the pool io file message it sends to pool.
Space manager in master requires this information and thus cannot work
with transfermanagers unless the correct information is injected.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6413/
poolmanager: Remove link group attributes

Link group attributes were not used. The following commands have
been removed from the command line interface:

psu remove linkGroup attribute
psu set linkGroup attribute

The latter of these still exists in the code as a noop to alove existing
poolmanager.conf files to be read.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6418/
Reduce class path length on liquibase invocation

Reduces system-test build time by approx. 25 seconds on my laptop.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6421/
spacemanager: Retry on transient errors

Besides allowing the space manager to recover from various transient errors,
this change also enables us to use serializable transaction isolation.

Target: trunk
Require-notes: no
Require-book: no
Fix text width calculation for ANSI formatted help output

ANSI escape sequences have to be ignored to get the correct
width.

Target: trunk
Require-notes: no
Require-book: no
spacemanager: Rewrite command line interface

The patch more or less rewrites all space manager related admin commands.

The following commands are removed:

ls
ls file space tokens
listInvalidSpaces
listFilesInSpace
removeFilesFromSpace
remove file
reserve
release
update space reservation

The following commands are added:

ls spaces
ls files
reserve space
update space
release space
release file

The new commands should offer greater flexibility and more concise output.
Documentation has been added for all commands. Options for almost all
commands have changed.

Note that although commands have been rewritten to make use of the
annotation syntax, the point of this patch is the changed user experience,
not the change of implementation.

Internal database keys like the link group id and the file reservation id
are no longer exposed through the user interface. Group (which confusingly
could be both group and user name) and role fields are presented as a
unified owner, which may be a user name or an FQAN. Various list commands
try to hide irrellevant information, dynamically adding columns to the output
suitable for the requested information.

The code was refactored such that the CLI appears in its own class. The link
group import code has been moved to its own classs too. Commands with blocking
code (ie database operations) are now executed on a thread pool to prevent
blocking the message thread.

The RESERVED state of files has been renamed to ALLOCATED to better reflect
how it is used and to be consistent with the terminology of the aggregation
field in space reservations.

The database interface has been extended with a criterion API to allow
flexible querying.

Target: trunk
Require-notes: yes
Require-book: yes
spacemanager: Delete flushed file reservations when name space entry is deteled

Space manager marks file reservations as deleted when the name space entry
is deleted. The actual file reservation is not removed until space manager
is notified that the file is removed from the pool. For files flushed to
tape there may not be a disk copy and thus such a notification is never sent.
The consequence is that reservations for flushed files are never removed.

This patch fixes the problem by deleting the file reservation of flushed files
as soon as the name space entry is deleted. A liquibase changeset is added to
delete old reservations for deleted flushed files.

Target: trunk
Require-notes: yes
Require-book: no
dcache-chimera: Add interactive chimera shell

Adds an interactive shell for chimera. May also be used in non-interactive mode
with a command provided on the command line. In the latter form it supplants
the older chimera-cli utility.

Note that in some cases the argument order has changed compared to the
chimera-cli utility. The argument order now matches the order found in common
Posix utilities.

The chimera-cli utility should be documented as deprecated and eventually
removed from dCache.

The system-test populate script has been updated to use the new utility, which
avoids means the chimera skeleton can be created with a single invocation,
cutting down on the build time.

Target: trunk
Require-book: yes
Require-notes yes