Safer removal of notification observer.