Remove exception on removing nil. It's common for the observer being removed to be weakly referenced.