Move RemoveAllofAllListeners logic to RemoveAllListeners