alarms (common, dcache, dcache-webadmin, skel):  version 2.

This combined set of patches provides improvements to and
several new features for the alarm system.  The changes
are rather significant, but should not prevent downgrading
to previous versions. Interaction with version 1 should
be feasible, since what goes over the wire is still
a logback event, though alarm data inconsistencies could result.
If desirable, a back-port to the Golden Release 2.10 would be
possible when 2.11+ has been proven sufficiently stable.

1) common interfaces and properties

Provides a new set of interfaces for the alarms system, along
with revised property definitions.

AlarmDefinitions have been moved out of dcache core and
into commons.

Definitions are a special class of Alarm. Internal
alarms now must belong to the enum PredefinedAlarm.

Severity has been replaced by AlarmPriority.
Interfaces for the implementation of two internal maps
supporting the mapping of Alarms to AlarmPriority
and for custom definitions are provided.

2) DAO and map implementations

Maps: Implements priority map using properties file as
persistent backup. Implements definitions map
as before using JDOM and xml file as persistent
backup. Provides message vehicle for remote
request of the priority map.

DAO: LogEntry has now been entirely decoupled from
both the Alarms interfaces as well as ch.qos.logback
specifics. The Severity field and methods have
been removed. A liquibase changeset to remove
this attribute and its index from the schema
is provided.

3) Logback

Logback server: rather than wrapping the SocketServer
and giving it an entirely independent configuration
(thus preventing alarms from running in any
domain with other services), the server now uses
a Receiver which is a lifecycle component it adds
to the existing configuration. The extra guards in
the alarms.batch file have consequently been removed,
and logback-server.xml has been deleted.

Logback appender: now directly configures from
the relevant dcache properties and adds
the two embedded appenders (history and email).
The part of the appender which glues the logback-specific
event structure to the LogEntry and Alarms interfaces
has been extracted out into a package utility,
LoggingEventConverter. However, there is still
a binding between this appender and datanucleus
inasmuch as the PersistenceManagerFactory remains
injected through Spring.

The way in which an event is classified as an alarm
now is slight different. If the event is Marked
but no type is provided, it will follow the same
branch as unmarked events in trying to find a
regex match from any provided definitions; however,
failing a match, the untyped but marked event is
considered still to be an alarm, and is marked
GENERIC. The new structure also obviates the
need for an receiving filter based on the Alarm marker.

The RemoteMDCFilter no longer maintains its own
static host name lookup, but uses NetworkUtils.

4) Admin and Shell Commands

dCache Shell Commands: some refactoring of code in order to
reuse for admin commands; eliminated
use of URI to express optional source/origin
metadata. Added a new list command to
show names of internally defined alarms.
Add, modify and remove still work as before.

dCache Admin Commands: The send, add/modify/remove (definition)
commands from the shell are also reflected
here, as well as commands to control
priority mappings, and for saving and reloading
both priority and definitions.

5) Embedded Alarms

Changes calls to MarkerFactory to support new API.
Also incorporates changes from discarded patch
https://rb.dcache.org/r/7169/:

    Current trunk now logs the following when a pool starts:

    22 Jul 2014 12:33:52 (PoolManager) [pool_0 PoolManagerPoolUp] Pool pool_0 changed from mode disabled() to disabled(fetch,store,stage,p2p-client,p2p-server)
    22 Jul 2014 12:33:54 (PoolManager) [pool_0 PoolManagerPoolUp] Pool pool_0 is DOWN
    22 Jul 2014 12:33:57 (PoolManager) [pool_0 PoolManagerPoolUp] Pool pool_0 changed from mode disabled(fetch,store,stage,p2p-client,p2p-server) to disabled(store,stage,p2p-client)
    22 Jul 2014 12:33:57 (PoolManager) [pool_0 PoolManagerPoolUp] Pool pool_0 changed from mode disabled(store,stage,p2p-client) to enabled

    The 'Pool pool_0 is DOWN' message was added with patch 7137.

    However, this ALARM seems to capture disabled states which should
    not generate alarms, especially at pool start-up.

    The logic has been refined to limit the alarm to when a pool
    mode has actually changed to either disabled STRICT or DEAD,
    excluding on initialization.

    In particular: I have taken the general alarms out of the
    PoolManagerV5 messageArrived. It makes more sense to distinguish
    etween alarms at the source or origin of the problem.

    Thus:

    The watchdog thread is responsible for the POOL_DOWN
    alarm sent went the heartbeat ping is not returned.
    Specific pool failures on init, access, etc., which
    cause a shutdown raise alarms.
    Initial disabling and shutdown, as well as changes
    in state initiated through a user command, do not
    raise alarms.

6) Webadmin DAO

Changes to webadmin DAO to support modified scheme (without
Severity).

Entry types getter also eliminated since types are now
known via messaging with the alarm service.

7) Webadmin Presentation Layer

Changes to support filtering on priorty in memory,
as it is not part of the persistent DAO. These involve
changes to the AlarmTableProvider comparators and to
the AlarmDisplayService, which refreshes the provider
by sending a message to the alarm service in order to
fetch the alarm/priorty mapping.

The query panel drop-down list now holds the priority
values (rather than Severity), and the auto-complete
combo is based on the listing refreshed from the
alarm service. Hence httpd no longer requires the
presence of any alarms configuration/properties files
locally (though if the service is XML rather than
RDMBS, the store.xml file must still be accessible
locally).

Severity does not appear in the display table, but
the filtering of the query automatically sorts
by level of priority first (by default). Thus
when both alarms and non-alarms are displayed,
first CRITICAL, then HIGH, etc., alarms, and then
simple ERROR, WARN, etc. statements will appear.

8) Removes code which is no longer used.

TESTING DONE:

All previously existing JUnit tests have been modified
and rerun.

system-test built, booted, manually tested.

Integration tests conducted on my desktop (SL 6.3)
and testbed nodes (SL 5.3).  Interactive testing of
the commands were done.  The PoolManager heartbeat
message was also temporarily converted to an alarm.
Larger system testing was conducted using standalone
components.  Stress tests were conducted for both
a singleton alarm service domain (with no other
services) as well as including the alarm service in
another domain (billing and dCache/head node).
The stress test consisted of sending error log events
at approximately 100Hz with an exception trace of
depth 25, with 26 server-side regex definitions
guaranteed not to match the error message but set
to search the entire depth of the trace for each
message.  The service was able to sustain approximately
94Hz over the course of 100000+ events (about 15 minutes).
Writes were simultaneously done while the alarm
service was running inside billing.  No memory
issues were detected.

Target: master
Patch: https://rb.dcache.org/r/7255/
Patch: https://rb.dcache.org/r/7257/
Patch: https://rb.dcache.org/r/7258/
Patch: https://rb.dcache.org/r/7259/
Patch: https://rb.dcache.org/r/7260/
Patch: https://rb.dcache.org/r/7261/
Patch: https://rb.dcache.org/r/7262/
Patch: https://rb.dcache.org/r/7263/
Acked-by: Tigran
Require-notes: yes
Require-book:  yes

I will provide the book chapter changes
and the release notes.