remove old refs
special case removeAllListeners for '**' and return false for non-mutation event listeners