Allow config setting to rethrow any errors

Swallowing error messages can make it hard to find errors.