Tear down KVO with an unsafe reference

A weak reference won’t live long enough for us to do so.