  Deep Neural Networks (DNNs) have achieved remarkable performance on a variety
of applications but are extremely vulnerable to adversarial perturbation. To
address this issue, various defense methods have been proposed to enhance model
robustness. Unfortunately, the most representative and promising methods, such
as adversarial training and its variants, usually degrade model accuracy on
benign samples, limiting practical utility. This indicates that it is difficult
to extract both robust and accurate features using a single network under
certain conditions, such as limited training data, resulting in a trade-off
between accuracy and robustness. To tackle this problem, we propose an
Adversarial Feature Stacking (AFS) model that can jointly take advantage of
features with varied levels of robustness and accuracy, thus significantly
alleviating the aforementioned trade-off. Specifically, we adopt multiple
networks adversarially trained with different perturbation budgets to extract
either more robust features or more accurate features. These features are then
fused by a learnable merger to give final predictions. We evaluate the AFS
model on CIFAR-10 and CIFAR-100 datasets with strong adaptive attack methods,
which significantly advances the state-of-the-art in terms of the trade-off.
Without extra training data, the AFS model achieves a benign accuracy
improvement of 6% on CIFAR-10 and 9% on CIFAR-100 with comparable or even
stronger robustness than the state-of-the-art adversarial training methods.
This work demonstrates the feasibility to obtain both accurate and robust
models under the circumstances of limited training data.
