Fix removeListener for listeners not registered through band dispatcher