Clean up listeners, even if the plugins don't.