Clean up import statements

This is a bulk update to change all import statements to the following format:

import <all other imports>;
<blank line>
import javax.**;
<blank line>
import java.**;
<blank line>
import diskCacheV111.**;
<blank line>
import dmg.**;
<blank line>
import gov.fnal.**;
<blank line>
import org.dcache.**;
<blank line>
import static <all static imports>;


No star imports, except for static imports for which three or more items are 
used.

Please update your IDEs accordingly.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: none (acked via email)
Fix log filtering initialization

dCache support per cell log levels through a custom turbo filter. This
turbo filter is initialized during cell initialization, however the current
initialization point is at the end of the cell creation. The consequence is
that messages logged before this point are not under the control of the
filter.

This patch moves the initialization point inside CellNucleus instead.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5245/
Alarms Infrastructure Revision (remote end filtering, marking, inclusion of error/warn events) (1 & 2)

This commit combines two patches, the second of which was merely a set of deletes (dead classes) [http://rb.dcache.org/r/5250].

This is the first of two patches (split into two for the sake of readability).  The second patch merely eliminates dead classes (this one must be applied first).

It rectifies objections to current alarm design:

A.  Filtering on sending end (scalability issues);
B.  Inability to raise alarm from inside the code by simply marking the error event;
C.  Current alarm definitions given in JSON (not necessary to break XML syntax);
D.  All error/warn events should be shipped to the remote/alarm server, not just alarms, in order to allow admins to inspect them in one place.

The brunt of the changes, especially in the webadmin module, have to do with renaming and relabeling (since the infrastructure now handles log messages which are not necessarily alarms).  The major changes are as follows:

1.  The logback.xml for the domain now does no filtering per se.  The socket appender sends all events warn and above*** to the alarm server.  It adds three properties to the event MDC before it does.
2.  The storing appender attached to the alarm server now examines the logging events it receives.  It classifies them on the basis of whether there is an ALARM marker and/or a match with one of the custom alarm definitions it knows about.  In the case of an error which is marked as ALARM, the "type" of the alarm will be whatever the marker string is (it should be constructed using the static method AlarmDefinition.getMarker(String subMarker).  All other (non-alarm) entries have a type simply equivalent to their log level (ERROR or WARN).  
3.  The server-side alarm definitions are uniform XML.  JSON appears neither in the configuration nor in the serialization and deserialization of the alarm.
4.  Display of the alarms now allows the user to choose all registered messages, only alarms, and only non-alarms, along with retaining the previous filtering capabilities on severity, type and string search using simple or regex pattern matching (in memory).
5.  There is now a new RDBMs table with an additional attribute.  I will provide a migration script for the rather unlikely case where the RDBMS alarms option has already been used.
6.  The default name for the XML storage has changed from alarms.xml to store.xml.
7.  The server.log is dedicated to ch.qos.logback errors and is non-additive.  All other logging messages go through the storage appender, which then delegates to a history.log (for use with the XML implementation) and an optional SMTP appender.

In terms of the Java code, the following are the most significant changes:

   IAlarmDOA                 --> ILogEntryDAO

   AlarmEntry                --> LogEntry

   AlarmDefinitionAppender  |
                            |--> LogEntryAppender
   AlarmEntryAppender       |

   AlarmDefinition               now is a full setter bean with only default constructor (for use by Joran); a few unnecessary methods have been eliminated and others   
                                 slightly modified.

   The AlarmDefinitionFilter has disappeared.  A new filter, RemoteMDCFilter, which accepts all events but just adds a few properties to the MDC, has been added.

Note also that several places in the code which were accessing a concrete logback object (LoggingEvent) have been changed to access only the interface (ILoggingEvent); this was not only more correct, but particularly necessary in view of the fact that the filtering now is on the receiving end of the socket sender (the socket server), which evidently passes a LoggingEventVO object to the logging API (it is a different implementation of ILoggingEvent from LoggingEvent).

A few adjustments were also made on the webadmin end in order to include the boolean 'alarm' attribute in the queries.

***NOTE: while the current changes are set to send warn-level events to the alarm/remote server, it is my opinion that this makes for a much too verbose log.  Perhaps this will change when we finally rectify the internal logging to conform with the recent suggestions.

Junit tests rewritten (and some eliminated), and run.
Tested on test bed with XML and RDBMS settings.

Target: trunk
Patch:  http://rb.dcache.org/r/5249
Acked-by: Dmitry
Require-notes: yes
Require-book:  yes

RELEASE NOTES and BOOK:  As there a number of significant changes here, I would be willing to help out with the editing.  Feel free to send me emails.
add useful indexes 

Target: trunk
Patch: http://rb.dcache.org/r/5246/
Acked-by: Paul Millar
Require-notes: yes
Require-book: no
Clean up CacheException class

- Add ability to have a cause

- Use message in Exception base class (taking care
  of maintaining serialization format).

- Get rid of mis-formating such as one-line methods, unused
  comments, etc.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5251/
spacemanager: Get rid of log errors that are no errors

It is common to see errors like

04 Mar 2013 00:02:41 (SrmSpaceManager) [door:GFTP-basilisk02-<unknown>-949126@gridftp-basilisk02Domain:1362383251755 basilisk06_pool08 DoorTransferFinished 0000F230C1C0790E4126AD55547851BF8715] file 0000F230C1C0790E4126AD55547851BF8715:There is no file with pnfsid=0000F230C1C0790E4126AD55547851BF8715

in the space manager log file. These are caused by transfers that are not subject
to space management and are not necessarily errors.

This patch refactors space manager to get rid of the above error. The patch also
removes some dead code to check for a null-return value from methods that can
never return null.

I am aware that relying on the SQL state code is not nice, however since the
entire space manager works with SQLException without further distinguishing
between types of errors, using the SQL state code is the simplest solution for
now.


Target: trunk
Request: 2.5
Request: 2.2
Ticket: http://rt.dcache.org/Ticket/Display.html?id=7685
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5254/
Clean up version number handling

dCache uses an intricate web of reflection code to obtain
version number information of various components. It also
relies on CVS/SVN revision number substitution, which will
be a problem when we move to other source code management
systems.

This patch injects branch and revision number information
into the JAR manifest files (which already contain the
build time and version number). The patch replaces all
version handling code in dCache, which now relies solely
on the information found in the manifest files.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5256/
Turn down log level of info and debug messages

As agreed during the developers meeting the 27th of
February, this patch replaces all info log messages
with debug log messages, and all debug log messages
with trace log messages.

This frees up info for use for informational low
volume logging.

Default log thresholds have been adjusted accordingly.

On upgrade the admin should consider adjusting their
logging configuration accordingly.

Target: trunk
Require-book: yes
Require-notes: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5232/
Add JSON API for dCache

The patch provides a response engine for the httpd service. The 
response engine translates users queries to UniversalSpringCell
queries. The query uses reflection to access bean properties. 
It uses a custom serialization scheme to provide a tree of 
public properties (public getters). The response engine 
translates the query response to JSON.

The API is read-only. It is exposed in the default httpd setup
under the api path:

set alias api class org.dcache.services.httpd.probe.ProbeResponseEngine

The API is accessed through HTTP, eg http://localhost:2288/api/PoolManager
or http://localhost:2288/api/PoolManager/psu.activePools.

In contrast to the info service, API requests trigger queries
to the actual service. Thus the information is guaranteed to be
up to date. The downside is that frequent probes would incure
overhead. The API is not guaranteed to be stable at this point (ie
we may change it in between releases). The goal is that any information
that can extracted through the ssh interface is also alvailable through
the API.

Target: trunk
Require-notes: yes
Require-book: yes
Committed files I forgot to add to SVN in previous commit.
Added files I forgot to add to SVN in previous commits.
Adjust log levels of various messages

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5266/
mbedded gplazma fragment: bug fix, pass pnfsmanager argument
Patch: http://rb.dcache.org/r/5271/
Acked-by: Gerd Behrmann
Target: trunk
Target: 2.5 
Require-notes: yes
Require-book: no

RELEASE NOTES 

Fixed handling of pnfsmanager argument in embedded gPlazma fragment
wicket: Exclude cglib dependency

Wicket uses cglib for some features in its Spring support.
It relies on cglib 2, which still uses asm 3 that happens
to not be fully compatible with Java 7. We have other
code that relies on asm 4 and we cannot mix the two versions
in dCache.

The patch exludes the cglib dependency. It appears that
webadmin works without it.

Target: trunk
Request: 2.5
Require-book: no
Require-notes: yes
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5272/
replicamanager: Fix NullPointerException

Fixes an NPE when database connection checkout fails:

08 mar 2013 19:56:57 (replicaManager) [] setHeartBeat() ERROR: Can't add/update process 'Adjuster' status in 'heartbeat' table in DB
java.lang.NullPointerException: null
        at diskCacheV111.replicaManager.ReplicaDbV1.setHeartBeat(ReplicaDbV1.java:1160) ~[dcache-core-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at diskCacheV111.replicaManager.ReplicaManagerV2$Adjuster.run(ReplicaManagerV2.java:917) [dcache-core-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at dmg.cells.nucleus.CellNucleus$1.run(CellNucleus.java:678) [cells-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]

Target: trunk
Request: 2.5
Request: 2.2
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/5273/
Reverted commit r18191

Applied wrong patch...
replicamanager: Fix NullPointerException

Fixes an NPE when database connection checkout fails:

08 mar 2013 19:56:57 (replicaManager) [] setHeartBeat() ERROR: Can't add/update process 'Adjuster' status in 'heartbeat' table in DB
java.lang.NullPointerException: null
        at diskCacheV111.replicaManager.ReplicaDbV1.setHeartBeat(ReplicaDbV1.java:1160) ~[dcache-core-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at diskCacheV111.replicaManager.ReplicaManagerV2$Adjuster.run(ReplicaManagerV2.java:917) [dcache-core-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at dmg.cells.nucleus.CellNucleus$1.run(CellNucleus.java:678) [cells-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:722) [na:1.7.0_11]

Target: trunk
Request: 2.5
Request: 2.2
Require-notes: yes
Require-book: no
poolmanager: Fixes rebalancer bug related to use of non-US locale

Rebalancer works by issuing migration commands to pools of a
pool group. The code uses String.format to build the migration
command. The formatting of floating point values is sensitive
to the default locale. For a non-US local this means that
floating points will use a comma as a decimal separator. This
will fail with the migration module.

This patch fixes the problem.

Target: trunk
Request: 2.5
Request: 2.2
Request: 1.9.12
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5274/
Fixed a few logging related loose threads

We recently changed many log statement from info -> debug
and debug -> trace. I forgot to update a couple of things.
This is rectified by this patch.

Target: trunk
Require-book: no
Require-notes: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/5275/
Revert recent changes to log levels

There were too many side effects and there is too little time
before 2.6 is released to make this work.

Reverted: r18198
Reverted: r18188
Reverted: r18184