Remove inverse_of_field related metadata