  In deep learning applications, the architectures of deep neural networks are
crucial in achieving high accuracy. Many methods have been proposed to search
for high-performance neural architectures automatically. However, these
searched architectures are prone to adversarial attacks. A small perturbation
of the input data can render the architecture to change prediction outcomes
significantly. To address this problem, we propose methods to perform
differentiable search of robust neural architectures. In our methods, two
differentiable metrics are defined to measure architectures' robustness, based
on certified lower bound and Jacobian norm bound. Then we search for robust
architectures by maximizing the robustness metrics. Different from previous
approaches which aim to improve architectures' robustness in an implicit way:
performing adversarial training and injecting random noise, our methods
explicitly and directly maximize robustness metrics to harvest robust
architectures. On CIFAR-10, ImageNet, and MNIST, we perform game-based
evaluation and verification-based evaluation on the robustness of our methods.
The experimental results show that our methods 1) are more robust to various
norm-bound attacks than several robust NAS baselines; 2) are more accurate than
baselines when there are no attacks; 3) have significantly higher certified
lower bounds than baselines.
