Use remove_group_association(..) in the DELETE /invite/<invite_id>