Use remove_change instead of changed_attributes direct access