Only remove observers from KVO if we have a superview in dealloc