srm,pool: improve support for 3rd-party HTTP copy

This patch improves SRM's existing support for HTTP 3rd-party copy.
With this patch, both push and pull requests are supported and both
unencrypted (http) and encrypted (https) targets will work.  For
encrypted connections where the pool is pushing a file to a third
party, a client X.509 credential will be used, if supplied.

Both transfer directions will attempt to check data integrity.  For
pull requests, all data integrety information comes from the headers
supplied in the GET response.  For push requests, data integrety
information comes from the headers in a HEAD response after a
successful transfer.

For push requests, if a PUT request is successful but the subsequent
HEAD request reveals data corruption then the pool will attempt to
delete the file with a DELETE request.  If this fails then the error
message is updated.  For pull requests no additional cleanup steps are
needed.

There are two levels of data integrety check: weak and strong.  Weak
data integrety is satisfied when

 o  remote service and dCache server agree on the content length,

 o  none of the checksums supplied by the remote server (if any)
    disagree with the checksums known by dCache.

Strong data integrety is satisfied when, in addition to satisfying
the weak conditions, the transfer also satisfies the following:

 o  there is at least one checksum supplied by the remote server
    that agrees with a known by dCache for this file.

Checksums are discovered via RFC-3230.  For pull requests the
checksums come as part of the GET response headers; for pull request
they come as part of the subsequent HEAD response headers.

The mover accepts a boolean argument 'requires-verification'.  If
set to false then weak data integrety is used; if true then strong
data integrety is required.

The mover also accepts zero or more (arbitrary) HTTP headers when
requesting a file be transferred.  This allows the requester to
customise the HTTP request that the pool sends to the remote server.
Such headers could include authorisation information or some other
information to steer how the remote server handles the HTTP request.

The SRM Copy request includes a list of arbitrary key-value pairs
called ExtraInfo.  This allows the client to specify additional
information that is platform-specific.  When processing such
remote-copy requests, the generic SRM-server layer now passes this
ExtraInfo to the AbstractStorageElement plugin.  The SRM
CopyFileRequest database storage has been updated to store this
information so it is available when restarting the SRM.  The same
approach may be used in the future for other requests, if necessary.

The dCache AbstractStorageElement plugin has been updated to use
ExtraInfo information to control the behaviour:

  "verify"    if set, must be either 'true' or 'false'.  If 'true' then
              strong data integrety is required for a successful
              transfer; if 'false' then weak data integrety is
              sufficient.  If not specified then a configurable default
              value is used.

  "header-"   any ExtraInfo element that starts "header-" are converted
              to an HTTP header.  The HTTP header key is the ExtraInfo
              key without the initial "header-"; the HTTP header value
              is the ExtraInfo value.  The HTTP headers are then used
              when making GET or PUT requests.

The srm-client has been updated to allow the user to specify arbitrary
ExtraInfo information.

Target: master
Acked-by: Tigran Mkrtchyan
Acked-by: Karsten Schwank
https://rb.dcache.org/r/7096/