close handler when log is disabled

When a log is disabled using `enable=False` , the log handler should also be close, not only removed.

Required for EasyBuild, see also https://github.com/hpcugent/easybuild-framework/pull/620.