Title: Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Authors: Dan Hendrycks
Thomas Dietterich
Venue: ICLR 2019
Keywords: robustness
benchmark
convnets
perturbations

Abstract: In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations. We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers. Afterward we discover ways to enhance corruption and perturbation robustness. We even find that a bypassed adversarial defense provides substantial common perturbation robustness. Together our benchmarks may aid future work toward networks that robustly generalize.

TL;DR: We propose ImageNet-C to measure classifier corruption robustness and ImageNet-P to measure perturbation robustness

Review discussion:
Summary: This paper observes that a major flaw in common image-classification networks is their lack of robustness to common corruptions and perturbations. The authors develop and publish two variants of the ImageNet validation dataset, one for corruptions and one for perturbations. They then propose metrics for evaluating several common networks on their new datasets and find that robustness has not improved much from AlexNet to ResNet. They do, however, find several ways to improve performance including using larger networks, using ResNeXt, and using adversarial logit pairing.
Quality: The datasets and metrics are very thoroughly treated, and are the key contribution of the paper. Some questions: What happens if you combine ResNeXt with ALP or histogram equalization? Or any other combinations? Is ALP equally beneficial across all networks? Are there other useful adversarial defenses?
Clarity: The novel validation sets and reasoning for them are well-explained, as are the evaluation metrics. Some explanation of adversarial logit pairing would be welcome, and some intuition (or speculation) as to why it is so effective at improving robustness.
Originality: Although adversarial robustness is a relatively popular subject, I am not aware of any other work presenting datasets of corrupted/perturbed images.
Significance: The paper highlights a significant weakness in many image-classification networks, provides a benchmark, and identifies ways to improve robustness. It would be improved by more thorough testing, but that is less important than the dataset, metrics and basic benchmarking provided.
Question: Why do authors do not recommend training on the new datasets?
We thank you for your careful analysis of our paper.
“Question: Why do authors do not recommend training on the new datasets?”
We do not suggest this as the datasets are corrupted or perturbed forms of clean ImageNet validation images, and that training on these specific corruptions would no longer provide a test of generalization ability to novel forms of corruptions. Researchers could train on various other corruptions, such as film grain, adversarial noise, HSV noise, uniform noise, high-pass filtering, median blur, spherical camera distortions, pincushion distortions, out-of-distribution object occlusions, stylized images ( https://openreview.net/forum?id=Bygh9j09KX ), lens scratches, image quilting, color quantization, etc.
“Are there other useful adversarial defenses?”
Different adversarial training schemes can degrade accuracy so much that they performed worse on these benchmarks. Many other adversarial defenses which do not use train on adversarial or benign noise have been shown not to provide robustness on noise corruptions (see the thorough work of https://openreview.net/pdf?id=S1xoy3CcYX Figure 3). In the coming month, we intend to explore more combinations of techniques to increase robustness, such as the combinations you suggest. In the appendix we explicate four attempts which did not lead to added robustness.
This paper introduces new benchmarks for measuring the robustness of computer vision models to various image corruptions.
In contrast with the popular notion of “adversarial robustness”, instead of measuring robustness to small, worst-case perturbations this benchmark measures robustness in the average case, where the corruptions are larger and more likely to be encountered at deployment time. The first benchmark “Imagenet-C” consists of 15 commonly occurring image corruptions, ranging from additive noise, simulated weather corruptions, to digital corruptions arising from compression artifacts. Each corruption type has several levels of severity and overall corruption score is measured by improved robustness over a baseline model (in this case AlexNet). The second benchmark “Imagenet-P” measures the consistency of model predictions in a sequence of slightly perturbed image frames. These image sequences are produced by gradually varying an image corruption (e. g. gradually blurring an image). The stability of model predictions is measured by changes in the order of the top-5 predictions of the model.
More stable models should not change their prediction to minute distortions in the image. Extensive experiments are run to benchmark recent architecture developments on this new benchmark. It’s found that more recent architectures are more robust on this benchmark, although this gained robustness is largely due to the architectures being more accurate overall. Some techniques for increasing model robustness are explored, including a recent adversarial defense “Adversarial Logit Pairing”, this method was shown to greatly increase robustness on the proposed benchmark. The authors recommend future work benchmark performance on this suite of common corruptions without training on this corruptions directly, and cite prior work which has found that training on one corruption type typically does not generalize to other corruption types.
Thus the benchmark is a method for measuring model performance to “unknown” corruptions which should be expected during test time.
In my opinion this is an important contribution which could change how we measure the robustness of our models. Adversarial robustness is a closely related and popular metric but it is extremely difficult to measure and reported values of adversarial robustness are continuously being falsified [1,2,3].
In contrast, this benchmark provides a standardized and computationally tractable benchmark for measuring the robustness of neural networks to image corruptions. The proposed image corruptions are also more realistic, and better model the types of corruptions computer vision models are likely to encounter during deployment. I hope that future papers will consider this benchmark when measuring and improving neural network robustness. It remains to be seen how difficult the proposed benchmark will be, but the authors perform experiments on a number of baselines and show that it is non-trivial and interesting. At a minimum, solving this benchmark is a necessary step towards robust vision classifiers.
Although I agree with the author’s recommendation that future works not train on all of the Imagenet-C corruptions, I think it might be more realistic to allow training on a subset of the corruptions. The reason why I mention this is it’s unclear whether or not adversarial training should be considered as performing data augmentation on some of these corruptions, it certainly is doing some form of data augmentation. Concurrent work [4] has run experiments on a resnet-50 for Imagenet and found that Gaussian data augmentation with large enough sigma (e.g. sigma = .4 when image pixels are on a [0,1] scale) does improve robustness to pepper noise and Gaussian blurring, with improvements comparable to that of adversarial training. Have the authors tried Gaussian data augmentation to see if it improves robustness to the other corruptions? I think this is an important baseline to compare with adversarial training or ALP.
Few specific comments/typos:
Page 2 “l infinity perturbations on small images”
The (Stone, 1982) reference is interesting, but it’s not clear to me that their main result has implications for adversarial robustness. Can the authors clarify how to map the L_p norm in function space of ||T_n - T(theta) || to the traditional notion of adversarial robustness?
1. https://arxiv.org/pdf/1705.07263.pdf
2. https://arxiv.org/pdf/1802.00420.pdf
3. https://arxiv.org/pdf/1607.04311.pdf
4. https://openreview.net/forum?id=S1xoy3CcYX&noteId=BklKxJBF57
Thank you for your interest in this topic and your analysis of our paper.
“I think it might be more realistic to allow training on a subset of the corruptions.”
Researchers could train on various other corruptions, such as film grain, adversarial noise, HSV noise, uniform noise,
high-pass filtering, median blur, spherical camera distortions, pincushion distortions, out-of-distribution object occlusions, stylized images ( https://openreview.net/forum?id=Bygh9j09KX ), lens scratches, image quilting, color quantization, etc. We have updated the text to make it clearer that researchers can train on more than just cropped and flipped images, but we still do not want researchers training on the test corruptions. In the paper we experimented with uniform noise data augmentation in the stability training experiment and found minor perturbation robustness gains, but not with Gaussian noise with a large standard deviation.
Thank you for pointing out that the brief Stone comment requires much more context.
For that reason we have removed the citation. Essentially, if f is a model and f^\hat is an approximation, and if input x is d-dimensional, then if we want | f(x) - f^\hat (x) | < epsilon, then in some scenarios the number of samples necessary is ~ epsilon^{-d}. Other context is on slide 10 of https://github.com/joanbruna/MathsDL-spring18/blob/master/lectures/lecture1.pdf
“l infinity perturbations on small images”
Thanks to your suggestion, we have changed this to “perturbations on small images.” We kept the word “small” as the images often have side length 32 pixels. We removed “l_infinity” since that method has had some success for perturbations which are small in an l_2 sense.
This paper introduces two benchmarks for image classifier robustness, ImageNet-C and Image-P. The benchmarks cover two important cases in classifier robustness  which are ignored by most current researchers. The authors' evaluations also show that current deep learning methods have wide room for improvement. To our best knowledge, this is the first work that provides systematically a common benchmarks for the deep learning community.  The reviewer believes that these two benchmarks can play an important role in the research of image classifier robustness.
We thank you for taking time to review our work.
We should like to thank all of the reviewers and commenters for their constructive comments and kind reception. Independent from their comments, we have created CIFAR-10-C and CIFAR-10-P which could be adequate for rapid experimentation.
Also in the revised version is a new appendix where we briefly analyze a different notion of robustness separate from our main contributions. We will respond to each reviewer’s comments individually.
A parallel submission proposes to train classifiers on stylized ImageNet images. The aim is to make classifiers rely less on texture and more on shape. https://openreview.net/pdf?id=Bygh9j09KX
We have found that this method indeed improves corruption robustness. A ResNet-50 obtains an mCE of 76.70%, while a ResNet-50 trained on both ImageNet images and stylized ImageNet images has an mCE of 69.32% (with general improvements noise, blur, weather, and digital categories).

Meta-review:
role: AC
data: metareview: The reviewers have all recommended accepting this paper thus I am as well. Based on the reviews and the selectivity of the single track for oral presentations, I am only recommending acceptance as a poster.

Decision: Accept (Poster)