admin: Fix help on error

The admin shell used to output a command summary in case of syntax errors,
however recent refactoring broke this when the admin shell was ported to
CellStub (CellStub assumes that dCache uses CacheException, while much code
that comes from cells relies on CommandException).

This patch resolves the issue.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6865/
scripts: add database updateSQL command

Print the SQL statements which will be applied by update command.

Acked-by: Paul Millar
Target: master
Require-book: no
Require-notes: yes
chimera: introduce inode generation number

on some workloads we need to detect that file or a directory have
been changed, even if mtime and ctime does not reflect that. This
can happen if time resolution is not good enough or timestams
where reseted back.

To reliably identify inode changes, introduced a new field,
igeneration, which is monotonically incremented on each inode
modification.

RELEASE NOTES:
The first start of namespace will take some time ( ~10-20 min )
for database schema update.

Acked-by: Gerd Behrmann
Target: master
Require-book: no
Require-notes: yes
gplazma2-ldap: use url rather than host+port

the ldap plugin builds the url of the ldap server using

"ldap://" + host + ":" + port

this way the protocol always is always "ldap". This prevents
the plugin from using secure ldap using "ldaps".
This patch substitutes host and port by a freely definable url.

Ticket: 8289
Acked-by:
Target: trunk
Require-book: yes
Require-notes: yes
srm: fix stored diff so that the Solaris 'patch' command works

The new delegation service uses Axis to autogenerate stub code
from the WSDL.  The code produced by Axis isn't ideal and
additional patches are applied.

The current diffs only work for the GNU version of 'patch', but
not the 'patch' command from Solaris.  There are two problems for
Solaris patch:

   because the filenames are the same, both exist in the filesystem
       Solaris requires that precisely one exists, otherwise it will
       fall back to prompting the user to understand which file to
       patch.

   the command line was missing, this is optional for GNU patch but
       Solaris requires it to be present.

Target: master
Patch: http://rb.dcache.org/r/6879/
Acked-by: Tigran Mkrtchyan
Merge remote-tracking branch 'upstream/master' into ldap-use-url/master
Kill workers when calling BoundedExecutor#shutdownNow

Solves a problem with hanging FTP sessions.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Patch: http://rb.dcache.org/r/6883/
ftp: fix unit tests so they run on IPv4-only machine

The unit tests for the AbstractFtpDoor are, on the most part,
designed to run independently of whether the testing machine has
IPv6 support or not.

Unfortunately one test relied on the host machine having the IPv6
loop-back interface enabled.  This patch fixes that problem.

Target: master
Request: 2.8
Request: 2.7
Patch: http://rb.dcache.org/r/6882/
Acked-by: Gerd Behrmann
srm: tidy up SRM CopyRequest and CopyFileRequest classes

Some very basic refactoring of code structure:
  Place braces in consistent fashion,
  Adjust white-space,
  Choose Java-conforming variable names,
  Move field members to start of class,
  Improve some method names

Patch: http://rb.dcache.org/r/6877/
Acked-by: Gerd Behrmann
Target: master
srm/transfermanager: make errors less noisy

If a third-party transfer fails, it gets logged in the pool,
transfermanager general log, transfermanager activity log and
the SRM.  The log messages are also needlessly long and include
unnecessary information.

This patch quietens this considerably.

Target: master
Patch: http://rb.dcache.org/r/6886/
Acked-by: Tigran Mkrtchyan
srm/transfermanager: make errors less noisy

This patch adds a part of an earlier patch that was mistakenly
omitted in commit 092c797.

Patch: http://rb.dcache.org/r/6886
Acked-by: Tigran Mkrtchayn
scripts: fix shell-specific expansion

A recent commit (c0f05a82) introduced a non-standard shell expansion
that broke the 'dcache' command on many platforms.  This patch fixes
that.

Target: master
Patch: http://rb.dcache.org/r/6888/
Acked-by: Gerd Behrmann
libs: update to nfs4j-0.7.5

Changelog for nfs4j-0.7.4..nfs4j-0.7.5
    * [b9a38a2] nfs: propagate nfs exception on readdir
    * [fa04d9e] vfs: chimera: do extra access check only for regular inodes
    * [56513fe] vfs: fix io on levels

Acked-by: Alber Rossi
Target: master, 2.8, 2.7
Require-book: no
Require-notes: no
info-provider: fix capitalisation of SRM in ControlProtocol

The glue ControlProtocol has a Type field that describes what kind
of protocol is being published; or SRM, this value should be 'SRM'.
Commit 9eab932c altered the capitalisation to 'srm', which is
picked up as an error.

This patch restores the published value to SRM.  It also removes a
mapping since all supported versions of dCache have an SRM family
of either 'SRM' or 'srm'.

Target: master
Patch: http://rb.dcache.org/r/6892/
Acked-by: Tigran Mkrtchyan
Request: 2.8
Request: 2.7
srm: tidy up toString methods

The SRM provides string representation of the current status of requests.  The formatting is rather
verbose and, for container requests, somewhat difficult to follow.

This patch tidies up the output by:
   o  removing user-priority (it isn't used anywhere),
   o  only print request priority if has a non-default value,
   o  include relative time information for creation and expiry dates,
   o  remove error message as it is simply the last line of the state
      history, which is printed anyway,
   o  fix indentation so relationship between file-requests inside a
      container request is more obvious,
   o  tidy up history printing,
   o  provide concised information about remove source and destination
      SURLs and/or TURLs for CopyFileRequest objects.

Target: master
Patch: http://rb.dcache.org/r/6890/
Acked-by: Gerd Behrmann
chimera: fix behavior on create of '.(fset)' files

the fset/pset files are special commands and we
do not allow to create them. Nevertheless, current
behavior is to throw IOError.

With this change will will throw Exists which will force client
to fall-back to regular open-for-write.

Fixes fset over NFSv4.1

Acked-by: Albert Rossi
Acked-by: Paul Millar
Target: master, 2.8, 2.7, 2.6
Require-book: no
Require-notes: no
libs: use nfs 0.8.0

a major update of NFS code with bugfix and fuctional changes:

  - simplify data types for performace
  - BETA level support for read/write with NFSv4.0
  - improved spec compliance
  - use inodes generation for change detection
  - added lookup and attribute cache
  - new oncrpc-2.2 with error handling of bad xdr streams

all fixes for 0.7 branch included as well.
The most visible change for dCache is better NFSv4.0, which allows
IO for v4.0-only clients, like mac osx 10.9, and VfsCache.

Changelog for nfs4j-0.7.0..nfs4j-0.8.0
    * [0cb05a2] core: fix data file creation in simple-server
    * [8b534f7] vfs: chimera: let inode to call setStat()
    * [1454aa5] nfs4: implement missing RECLAIM_COMPLETE
    * [4710b27] nfs4: fix open CLAIM_PREVIOUS
    * [d56193f] pom: enforce  maven-release-plugin 2.4.2
    * [1c916c5] vfs: add a special option to hide dcache from dcap clients
    * [1c89149] vfs: fix regression introduced in 1c916c5
    * [39fcca6] nfs4: refactor/optimize data structures
    * [45f0a62] nfs4: simplify nfstime4 structure
    * [9b8420d] nfs4: fix millis to nfstime4 conversion
    * [fefb630] nfs4: simplify data types fattr4_{files,space} and Co.
    * [00118a8] nfs4: fix nfstime4 encoding
    * [6d05b98] nfs4: simplify bitmap4
    * [44e4d54] nfs4: fix lookupp on the root of the tree
    * [0222c7d] nfs4: fix handling of illegal operations
    * [dc38de7] nfs4: client: add read-no-state command
    * [7e865b5] nfs4: simplify data types
    * [ab1a807] nfs4: fix secinfo_no_name operation
    * [2b161a3] nfs4: fix behavior on undefined operation
    * [59046c9] nfs4: fix parent lookup behavior on a symlink
    * [1119117] nfs4: fix open_claim after 'reclaim complete'
    * [1fd3d92] nfs4: fix regular open before 'reclaim complete'
    * [cde8648] nfsv4: fix regression introduced in 7e865b5
    * [57153a6] nfs4: fix create_session of expired clients
    * [f0b201e] nfsv41: add support for IO without pNFS
    * [f6dfa96] nfsv4: some operation must be solo in the compound
    * [eddcec4] nfsv4: DESTROY_CLIENTID must return BUSY on existing sessions
    * [b73fc63] nfsv: fix client principal for non GSS credentials
    * [9854b6d] nfsv4: open_owner clientid used for v4.0 only
    * [9c342a9] vfs: chimera: translate exceptions to nfs errors
    * [7ec5cb3] nfsv4: add a special operation factory for local IO
    * [333c6af] nfs4: simplify data types
    * [3ae2ea2] vfs: update interface to match chimera changes
    * [15e274e] src: remove jna based nss-id-mapping
    * [7a54d56] vfs: fix export idx on get parent
    * [cddf937] nfs4: verify seqid in OPEN_CONFIRM as per RFC
    * [ad27d17] nfsv41: add check for max number of operations in a COMPOUND
    * [b61060a] nfs4: commit allowed only for file objects
    * [2efcd7d] nfs4: consolidate object name checks into a single place
    * [2345ce5] nfs4: allow IO only on file objects
    * [c0e40b4] nfs4: validate nanos in nfstime4
    * [fe439ec] nfs4: validate readdir cookie
    * [8d2e6ac] pom: update junit to support single test run from IDE
    * [61d15cb] nfs4: do strict utf8 check
    * [f9156b2] nfs4: limit max session slots
    * [46540c3] nfs4: respect client's requested back channel reply cache size
    * [8c0d4ab] nfs4: remove redundant List object
    * [852f4d0] nfs4: remove redundant autoboxing
    * [20c12d4] nfs4: fix hard link constrains
    * [f8f0999] nfs40: fix setclientid
    * [aea34e8] nfs40: fix open_confirm with bad stateid
    * [ee87871] nfs41: implement putpubfh
    * [f91b989] nfs4: fix secinfo
    * [0d746c1] nfs4: allow open only for regular files
    * [202c4b9] nfs4: verify/nverify should check that all requested attributes evaluated
    * [1bfaaf7] nfs4: hidden and system are unsupported file attributes
    * [c7ad329] nfs4: verify/nverify must return error if rdattr_error is used
    * [f0f11e5] nfs4: add bitmap4#isEmpty() to check is any bit set
    * [7ba3337] nfs: propagate nfs exception on readdir
    * [d3ac4a1] vfs: chimera: do extra access check only for regular inodes
    * [119f722] nfs4: introduce stateid4#equalsWithSeq()
    * [b3efb2c] nfs40: update stateid on read and write
    * [ff190f3] nfs4: check state's sequence id when updating lease time
    * [009a32c] nfs4: do not set size on non file objects
    * [3d1a845] nfs: add a possibility to shutdown Cache's cleanup thread
    * [b6a6e12] nfs4: add utility method to check for stateless stateids
    * [93e8059] vfs: fix io on levels
    * [41def7d] nfs4: fix change_info on open-create
    * [c73e09b] vfs: call stat only when it's needed
    * [414c947] nfs: remove useless rpc program check
    * [71d00a6] libs: update to oncrpc4j 2.2
    * [ab2118c] nfs4: validate attributes on setattr
    * [a4461fa] vfs: update cache decorator to cache stat information
    * [d2d365b] vfs: use backend filesystem file id as a cache key
    * [25ed237] pom: enforce utf8 for source files
    * [61015d9] nfs: reject readdir requests if cached listing not found
    * [be616ab] nfs4: set the file mode with create operation
    * [306ea49] vfs: introduce AclCheckable interface to VirtualFileSystem
    * [3084d3a] nfs4: use inode generation to indicate file/direcory change

Acked-by: Gerd Behrmann
Target: master
Require-book: no
Require-notes: yes
nfs: use vfs cache

make use of attribute caching. Three new options are added:

    nfs.namespace-cache.time
    nfs.namespace-cache.unit
    nfs.namespace-cache.size

The default value for caching time is 3 sec, which an optimal for most workloads.
The default cache size (number of cached objects) is zero, which is effectively, switching off
the caching.

Each caches entry takes 120 bytes of additional memory.

Acked-by: Paul Millar
Target: master
Require-book: no
Require-notes: yes
TransferManager: fix error-handling

An earlier patch (5ed882c6cd56bd9cdc50a9017a2cb74047e00607) switched from
using CellAdapter#sendMessage to the recently introduced call-back mechanism.
This separates the normal and error response handling into separate call-back
methods.  Unfortunately the patch didn't also refactor the error handling
in TransferManagerHandler so the existing error handling remained in the
successful code-path, where it was never called.

This patch moves the error handling to the error call-back method.  It
tidies up the error messages and reduces the logging somewhat.  The retry
if the delete of a file fails is refactored slightly.

Target: master
Patch: http://rb.dcache.org/r/6889/
Acked-by: Gerd Behrmann
Enable IPv6 stack by default

The patch enables the use of the IPv6 network stack if available. This
means that dCache can be used with IPv6 sockets.

The JVM still defaults to preferring IPv4 addresses, even when using the
IPv6 stack.

On installation the old behavior can be reactivated by configuring dcache.java.options.extra=-Djava.net.preferIPv4Stack=true.

Target: trunk
Require-notes: no
Require-book: no
Acked-by: Paul Millar <paul.millar@desy.de>
Patch: http://rb.dcache.org/r/6901/