puppetlabs/puppetdb — pull request #198 by grimradical (2012-07-04 00:17:38+00:00)
Title: Improved "logging of last resort"

There are certain points in the lifecycle of PuppetDB where it's
critical that we properly log an exception, even if that means we spam
different log targets (logfiles, stdout, stderr, etc) and duplicate
output.

Daemon startup is one of these critical points. An error during
startup should be logged as thoroughly as possible, as there's no
really good way to debug the issue otherwise (as the daemon will
terminate upon error, making it hard to do meaningful
live-debugging). This becomes doubly-important when you consider that
99% of users are launching PuppetDB via an init script of some sort,
and init scripts can do funky things with the stdout and stderr
streams of launched processes. On some platforms, that stuff is
captured for you. On others, it goes to the users interactive
terminal. And still on others, it ends up in the aether never to be
seen again.

The other critical point is when a thread dies due to an unhandled
exception. I'd prefer that never occur but, well, shit happens yo. You
can register a callback with the JVM whenever a unhandled exception
occurs, so we can use the same verbose logging routines we employ
during daemon startup to ensure that unhandled exceptions get some
degree of debuggability.

Better safe than sorry, and I'd prefer to err on the side of logging
too much during these critical sections of code than log too little.

Signed-off-by: Deepak Giridharagopal