Detach service listener to avoid notify in illegal state