Make sure not to re-create notifier in rescue-block