httpd: log bugs with stacktrace

Log a stack-trace and return appropriate HTTP response status code if
a bug is detected.

Target: master
Patch: http://rb.dcache.org/r/6973/
Acked-by: Tigran Mkrtchyan
systemtest: regenerate host credentials, add user credentials

Patch adds new host credentials so the system-test continues to work
out-of-the-box.  The patch also adds user-certificates generated by
the same Disposable CA.  This should allow people without an X.509
certificate to use system-test without first regenerating credentials.

Target: master
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
webdav: work-around race-condition in StringTemplate library

StringTemplate has a lazy-load for parsing the template file.  It
turns out that this is very racy, with at least two race-conditions.

Once race-condition has the effect that if two threads attempt to
fetch a template at about the same time with the template file not
loaded then the template being requested by the second thread (the one
that lost the race, so not loading the template file) will be never be
available.

Since dCache uses STGroupFile object to render only one template, this
means that if two threads attempt to fetch the template close enough
together then the template is never available and null is returned.
Because of another problem, this triggers a NPE.

A symptom of this problem is that dCache (> 2.6) will log a single
entry

   "redefinition of template <name>"

where <name> is the template that is no longer available.

The work-around is to force the loading and parsing the template file
by the thread creating the cell.  This removes the race-condition and
the file is loaded correctly.

There is another race-condition that results in multiple threads
attempting to load and parse the file.  This will result in dCache
logging a line like:

   "redefinition of template <name>"

for each template and for each additional thread that attempts to load
the file. While noisy, this should only happen (if at all) for "a
while" after initialising and does not affect the correctness of the
rendered response.

Target: master
Request: 2.9
Request: 2.8
Request: 2.7
Request: 2.6
Bug-report: http://rt.dcache.org/Ticket/Display.html?id=8315
Bug-report: (private correspondance with DESY people)
Patch: http://rb.dcache.org/r/6986/
Acked-by: Gerd Behrmann
webdav: generate error page when template couldn't be found

The webdav door uses StringTemplate to generate HTML pages for
directory listing and error pages, using hard-coded template
names that the corresponding template files must provide.

One reason for using StringTemplate is to allow site customisation
of the output.  However, dCache behaves badly if the hard-coded
template name isn't found; rather than logging a meaningful
error, a NPE is triggered.

This patch provides a hard-coded "broken dCache" page that describes
the problem so that the admin can fix their own site.

Target: master
Patch: http://rb.dcache.org/r/6981/
Acked-by: Tigran Mkrtchyan
billing: do not generate billing message if file based store is disabled

do not log billing messages to pinboard

Acked-by: Albert Rossi
Target: master
Require-book: no
Require-notes: yes