Remove the reference to the observer