Before revision:  In this paper, we propose a new training objective " Guided Complement Entropy " (GCE) that has dual desirable effects: (a) neutralizing the predicted probabilities of incorrect classes,and (b) maximizing the predicted probability of the ground-truth class , particularly when (a) is achieved. Training with GCE encourages models to learn latent representations where samples of different classes form distinct clusters, which we argue, improves the model robustness against adversarial perturbations. Furthermore, compared with the state-of-the-arts trained with cross-entropy, same models trained with GCE achieve significant improvements on the robustness against white-box adversarial attacks, both with and without adversarial training . When no attack is present, training with GCE also outperforms cross-entropy in terms of model accuracy .
After revision:  In this paper, we propose a new training objective " Guided Complement Entropy " (GCE) that has dual desirable effects: (a) neutralizing the predicted probabilities of incorrect classes,and (b) maximizing the predicted probability of the ground-truth class like cross-entropy, we neutralize its probabilities on the incorrect classes along with a "guided" term to balance between these two terms. We show in the experiments that our method achieves better model robustness with GCE also outperforms cross-entropy in terms of model accuracy .