Don't NPE when closing a badly setup appender.

Log4J can have its configuration reloaded, if the appender wasn't setup
correctly then log4j will attempt to close the appender and this NPEs.

Same thing goes with JUL
Check that we don't close multiple times.