Use weak references in KVO observation

Hopefully fixes #1206 and #1548.
You so fancy.