allow unsetting the logger

As the created object is not forced to have a logger in the first place,
it should be valid to use the mutator to unset the logger again.