  The discovery of adversarial examples revealed one of the most basic
vulnerabilities of deep neural networks. Among the variety of techniques
introduced to tackle this inherent weakness, adversarial training was shown to
be the most common and efficient strategy to achieve robustness. It is usually
done by balancing the robust and natural losses. In this work, we aim to
achieve better trade-off between robust and natural performances by enforcing a
domain invariant feature representation. We present a new adversarial training
method, called Domain Invariant Adversarial Learning (DIAL) that learns a
feature representation which is both robust and domain invariant. DIAL uses a
variant of Domain Adversarial Neural Network (DANN) on the natural domain and
its corresponding adversarial domain. In a case where the source domain
consists of natural examples and the target domain is the adversarially
perturbed examples, our method learns a feature representation constrained not
to discriminate between the natural and adversarial examples, and can therefore
achieve better representation. We demonstrate our advantage by improving both
robustness and natural accuracy compared to current state-of-the-art
adversarial training methods.
