Use weak references in KVO observation

Hopefully fixes #1206 and #1548.
To make sure I understand the timeline:

1. Weak ref is nil'd.
2. We end up in here.
3. Object is deallocated.