Clean up the KVO a bit more

No need for key path check, same for getting the correct point