Add close flag to configuration

This allows the application to control when the listeners are flush/closed.

Needed because the installed shutdown hook maybe run first before files are closed during shutdown.