minor update

don’t allocate large memory for num_vote_loss_permuted_correct_class if
we don’t compute permuted importance