Only unflag the aliased index we're dealing with