srm: Rewrite srmPutDone implementation

Addresses spec compliance issues.

Note that the SRM spec states that a list of SURLs is required
for srmPutDone. Therefore a lot of dead code for the case of
an empty SURL list has been removed.

A lot of duplicated dead code for non-put requests has been
removed.

The new implementation supports several return codes unused by
the old implementation.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6140/
srm: clean up srmBringOnline implementation

Addresses SRM spec compliance issues.

Also updates the system-test test script to issue a bring-online
request and to work with Ruby 2.0 (warning: Ruby 2.0 may be required
now).

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6142/
srm: Clean up srmLs request implementation

Mostly cosmetical to bring the code in line with other request
implementations.

Changed the value of srm.limits.request.ls.levels.max to infinity.
This is to make the default values be spec compliant.

Target: trunk
Require-notes: yes
Require-book: yes
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6144/
srm: Clean up srmCopy request handling

Mostly cosmetic to mirror the other request implementations.
The bulk of the copy logic is unchanged and needs to be cleaned
up in a future patch.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6145/
srm: Final iteration over SRM handlers to unify implementations

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6146/
srm: Improve status messages of get requests

Addresses the issue that the description string of srmPrepareToGet
replies show the internal SRM scheduler state rather than the
description of the last request transition.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6148/
srm: Improve status messages of put and bring-online requests

Addresses the issue that the description string of srmPrepareToPut
and srmBringOnline replies show the internal SRM scheduler state
rather than the description of the last request transition.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6150/
srm: Fix regression in listing busy SURLs

Addresses an ARC compatibility issue.

ARC issues an srmLs against a SURL after uploading a file, but
before calling srmPutDone. It does this to verify file size and
checksum before releasing the SURL to other clients.

Ever since we implemented SRM_FILE_BUSY, this logic no longer works,
as we skip the meta data lookup for busy files. This issue is
addressed by this patch.

Target: trunk
Request: 2.6
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6147/
srm: Fix regression in srmGetRequestSummary

Addresses a regression in srmGetRequestSummary that caused
the return status of the summary request to be computed from
the status of the requests that were queried.

The spec is not very precise about what constitutes a failure
or partial failure of the query, but I chose to consider not
finding a matching request token a failure of the query itself.

I also updated the client to report file level request information
even if the query return status is a failure.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6153/
srm: Make srmRmdir S2 compliant

S2 expects that attempts to remove a directory to which files are
uploaded fail. The test enforcing this behaviour is
usecase.RmdirBeingPutInto.

This patch implements the required behaviour.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6154/
srm: Stylistic change in SrmRmdir

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6155/
srm: Make rmdir work with S2

Addresses an issue with extracting the path from S2 URLs.

S2 uses SURLs were the path in specified as a query parameter.
Recent code added to srmRmdir processing failed to extract
such paths.

Also fixed a bug in CacheExceptionFactory that caused the
rmdir implementation to return the wrong error message
when attempting to remove non-directory entries.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6159/
srm: Use description of last job transition in return status messages

Improves status messages returned to SRM clients.

The SRM in some places uses the internal state as a description text,
and in other places the current "error message", which is the description
of the last state transition with an added time stamp. In ls it even
uses the concatenated description messages of all state transitions.

Since this information should probably not leak to the outside (its
confusing and it makes FTS logs harder to understand), this patch uses
the description text of the last transition itself.

In subsequent patches I will try to improve the description messages
in calls to setState().

A consequence of this patch is that it is no longer necessary to maintain
the concatenated errorMessage field in the Job class - it is no longer
used.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6156/
srm: Improve error and status messages

Addresses issues in error and status messages. End-users
should not be confronted with exception names and other
internal and code specific messages.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6160/
srm: Use shared read locks for read-only property access

I verified all places in which we obtain a read-lock. As far as
I could see, we now never upgrade a read lock to a write lock.
Thus it is safe to use a shared read lock for read access.

I had to fix a number of getters, though. In particular turl
computation was done as a side-effect in a getter and would
thus cause a read-lock to write-lock upgrade attempt during
save. I now moved the turl computation into the main processing
of put and get. Note that this causes the turl to be computed
as the request moved to RQUEUD rather than READY. If this is
unacceptable, we could move the computation into the
stateChanged notification, but I find that slightly more
messy.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Patch: http://rb.dcache.org/r/6157/
Fix dependency exclusion

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6166/
pnfsmanager: Fix error type when delete is refused

Addresses an issue that causes

30 Oct 2013 23:07:32 (PnfsManager) [RiY:610:srm2:rm SRM-sisyphus PnfsDeleteEntry] Inconsistent message class diskCacheV111.vehicles.PnfsDeleteEntryMessage received form [>SRM-sisyphus@dCacheDomain]

to be logged in PnfsManager.

PnfsManager uses INVALID_ARGS as an internal error. Hence using
it to signal that the allowed file types do not match the actual
file type leads to the above error. The patch changes the logic
to other error codes. The error codes may not be optimal in all
cases (eg for links or special files), but they are better than
the existing error code.

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6171/
srm: Improve error logging of file upload

Target: trunk
Require-notes: yes
Require-book: no
Acked-by: Dmitry Litvintsev <litvinse@fnal.gov>
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6172/
chimera: fix sql exception if file name too long

in some cases (at least during testing) we see names longer that
256 symbols. This results into naasty exceptions which is fixed
by this patch.

Patch: http://rb.dcache.org/r/5490
Target: trunk
Request: 2.6
Require-book: no
Require-notes: no
srm: Fix race conditions in request credential storage

The request credential storage implementation is riddled with race
conditions. This patch addresses those issues.

The way we treat request credentials is broken as we collapse
delegated credentials based on dn and primary fqan. There is
no guarantee that the delegated credentials are always the
same and there is thus no basis on which we can collapse them.

More over we never delete delegated credentials. Due to the
collapsing mentioned above, the number of different credentials
is limited for most scenarios.

Combined, these two issues mean we have to completely rewrite
how we deal with delegated credentials. However for now I am
focusing on the race conditions as they cause random S2 test
failures.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6168/