Don't follow references in _get_changed_fields (#422, #417)

A better fix so we dont follow down a references rabbit hole.