When Robustness Doesn’t Promote Robustness: Synthetic vs. Natural Distribution Shifts on ImageNet
Authors: Rohan Taori
Achal Dave
Vaishaal Shankar
Nicholas Carlini
Benjamin Recht
Ludwig Schmidt
Venue:  (2020)
Keywords: robustness
distribution shift
image corruptions
adversarial robustness
reliable machine learning

Abstract: We conduct a large experimental comparison of various robustness metrics for image classification. The main question of our study is to what extent current synthetic robustness interventions (lp-adversarial examples, noise corruptions, etc.) promote robustness under natural distribution shifts occurring in real data. To this end, we evaluate 147 ImageNet models under 199 different evaluation settings. We find that no current robustness intervention improves robustness on natural distribution shifts beyond a baseline given by standard models without a robustness intervention. The only exception is the use of larger training datasets, which provides a small increase in robustness on one natural distribution shift. Our results indicate that robustness improvements on real data may require new methodology and more evaluations on natural distribution shifts.

Under review as a conference paper at ICLR 2020
WHEN ROBUSTNESS DOESN'T PROMOTE ROBUST- NESS: SYNTHETIC VS. NATURAL DISTRIBUTION SHIFTS ON IMAGENET
Anonymous authors Paper under double-blind review
ABSTRACT
We conduct a large experimental comparison of various robustness metrics for image classification. The main question of our study is to what extent current syn- thetic robustness interventions (lp (fp-adversarial examples, noise corruptions, etc.) promote robustness under natural distribution shifts occuring in real data. To this end, we evaluate 147 ImageNet models under 199 different evaluation settings. We find that current robustness intervention improves robustness on natural dis- tribution shifts beyond a baseline given by standard models without robustness intervention. The only exception is the use of larger training datasets, which pro- vides small increase in robustness on one natural distribution shift. Our results indicate that robustness improvements on real data may require new methodology and more evaluations on test sets representing natural distribution shifts.
INTRODUCTION
Reliable classification under distribution shift is still out of reach for current machine learning (Tor- ralba et al. 2011 Recht al.) 2019). As result, the research community has proposed a wide range of evaluation protocols that go beyond single, static test set. Common examples include adversar- ial examples (Szegedy et 2013 Biggio et al., (2013), noise perturbations (Geirhos et al. 2018 Hendrycks & Dietterich 2019), and spatial transformations (Fawzi & Frossard, 2015 Ennssrom
al. 2019). Encouragingly. the past few years have seen substantial progress in robustness to these distribution shifts, e.g., see (Madry et al.) 2018 Zhang et al.) 2019 Geirhos et al.) 2019 Zhang
2019 Engstrom et al. 2019: Yang et al. 2019) and many others. However, an implicit assumption underlying this research direction is that robustness to such synthetic distribution shifts will lead to models that also perform more reliably on natural distribution shifts.
We challenge that assumption. We conduct large experimental study involving 147 ImageNet mod- els evaluated under 199 different evaluation settings for a total 29,253 test set evaluations. Our model testhed contains wide range of standard models and most proposed robustness interventions (adversarial training, various forms of data augmentation, etc.). In order to measure robustness under natural distribution shift, we utilize two recently proposed variants of ImageNet (Deng et al. [0099] Russakovsky et al. 2015) and ImageNet (Berg et al. 2015). Both test sets consist entirely of un- perturbed images drawn from the same sources as the original datasets, but also exhibit substantial
accuracy drops for all current model architectures (Recht et al. 2019 Shankar et al. 2019).
Figure I shows the main result our evaluation. The plots display model accuracies for the two natural distribution shifts. As noted in prior work, there is substantial drop in accuracy when going from the original test set to the test set with distribution shift. A priori, one may hope that a more robust model would see a smaller drop than baseline approaches without a robustness intervention. But we find that in both cases, current robustness interventions offer little to no benefit over stan- dard models. In particular, the accuracy on the original test set alone almost perfectly predicts the accuracy on the test sets with distribution shift. Importantly, the same relationship between original and "shifted" accuracy holds for both standard models and models with an explicit robustness in- tervention. This implies that the robustness interventions do not close the gap between original and "shifted" test accuracies any more than standard models with the same accuracy.

---

Under review as a conference paper at ICLR 2020
Accurasy on Imaa Imagetetv2 (top-1. %6
ImageNet (logit axis scalling)
Accuracy on Robust [pm- %61
Imagenet (logit scaling)
65
80
75-
60
65
55
40
30
55
60
05
70
75
80
40
50
60
70
80
Accuracy on ImageNet (top-1, %)
Accuracy on Vid-Robust (pm-0, %)
Perfect robustness
Lp adversarially robust
Trained with more da
Standard training
Other robustness intervention
Linear fit
Figure 1: Model accuracy on the two natural distribution shifts, ImageNetV2 (left) and Ima- geNetVidRobust (right). Each data point corresponds to model in testbed and is shown with 99.5% Clopper-Pearson confidence intervals. The plots demonstrate that the standard test accuracy (x-axis) almost perfect predictor for the test accuracy under distribution shift (y-axis). This holds regardless of whether the model was trained with robustness intervention. Current robust- ness interventions reduce the accuracy drops under these distribution shifts only by small amount (on ImageNet VidRobust) or not at all (on ImageNetV2) The axes were adjusted using logit scaling and the linear fit was computed the scaled space. The red shaded region is 95% confidence region for the linear fit from 100,000 bootstrap samples.
There are only two significant deviations away from the otherwise universal relationship between original and shifted accuracy. The first is that fp-adversarially robust models do offer increased accuracy against the distribution shift on ImageNet of overall low accuracy, As we will demonstrate Section 5 interpolating between standard model and random classifier yields comparable robustness increase in this accuracy regime. So the current fp-robust models not provide a truly new robustness trade-off.
However, this increase is in a regime
The second class of outliers models trained substantially different data. In particular, the two most pronounced outliers on ImageNetV2 correspond to models trained on 10x to 1,000 more training examples. This shows that adding more training data does indeed increase robustness ImageNetV2 more than any other currently proposed explicit robustness intervention. However, adding data also yields only small gains: the accuracy drops of around 10% from ImageNet to ImageNetV2 shrink by - 2%, even when adding 1,000x more data.
Overall, results show that current robustness gains on synthetic distribution shifts do not transfer to improved robustness on the natural distribution shifts presently available as test sets. This suggests that research on reliable machine learning may currently be focusing on interventions that do not promote robustness on natural distribution shifts. Achieving robustness on real data may instead require new methodology and more evaluations on natural distribution shifts.
To aid this development, we will release and maintain our robustness testbed as platform for proposing and evaluating new models and datasets. total, our testbed includes prediction data for 29,253,000 pairs of models and image inputs. We hope that a comprehensive repository for robust- ness evaluations will simplify the process of comparing proposed models and evaluation settings and enable further analysis of robustness questions.
In the next section, we describe our experimental setup in detail. Section D] introduces our main measure of robustness. The following three sections then investigate three main questions:
synthetic robustness measures predictive of performance on natural distribution shifts?
What robustness interventions are effective for natural distribution shifts?
How does the amount of training data impact robustness?
2 EXPERIMENTAL SETUP
We first formally define experimental setup. In the standard classification setting, model f is first trained i.i.d. samples from fixed data distribution. We then test on another set S₁ of i.i.d.
2

---

Under review as a conference paper at ICLR 2020
samples from the same distribution to compute the test accuracy accs; (f) = = (a.))s, Iff(x) v)- We refer to any deviation from this setup distribution shift.
More specifically, our robustness evaluation involves distribution shifts defined as follows. Instead of using the original test set S1, may also evaluate the accuracy a test set S2 that is collected via a similar but not identical procedure to the original test set S (i.e., S2 represents a different distribution). Moreover, we allow each data point (either from Si S2) be pertubed before passing it to the classifier f. We consider general setup where the perturbation (( y, f) may depend on the model f to evaluated. For instance, we can generate adverssaaia examples [[f(x') = v]- Alternatively, we can simply perturb each --=Irl |
by setting π (ar, y. f) = arg data point with Gaussian noise, i.c., (2, y, f) x + NO o²) independently of y and f.
Combining the choice of test set S and perturbation π, the test accuracy under distribution shift S, is then defined as
a
Σ If((((, y1 (2.0)
Depending on S and g. this "shifted" test accuracy may be higher or lower than the original accuracy accs, The focus of this paper is distribution shifts where there is reasonable hope to classify the images correctly (e.g., because humans can do so), but standard models achieve only a substantially lower accuracy than the original test Reducing these accuracy gaps between original and shifted accuracy an important goal of robust machine learning.
2.1 TYPES OF DISTRIBUTION SHIFTS
At high level, we distinguish between two main types of distribution shift. The crucial distinction between the two whether the distribution shift involves a synthetic intervention at the pixel level. We use the term natural distribution shift for cases that rely only unmodified images. In contrast, we refer to distribution shifts as synthetic if they involve modifications of existing test images. To be conerete, now provide overview of the various distribution shifts in robustness evaluation.
2.1.1 NATURAL DISTRIBUTION SHIFTS
Our testbed includes two distribution shifts that involve only unmodified natural images.
Vma. The first example is the new I2 test set recently collected by Recht et al. (019). The goal of this test set was to closely reproduce the original ImageNet dataset creation process, c.g., matching the data source (Flickr) and the data cleaning process (Mechanical Turk). Nevertheless, the authors found that state-of-the-an models still suffer a performance drop at least 1 11% on the new test set. This makes the dataset interesting example for studying robustness to natural distribution shifts. Testing on ImageNet fits into our evaluation framework by letting be the new ImageNetV2 test set (denoted by S2) and leaving to be the identity function.
Image VidRouust. In contrast to precisely defined, synthetic notions of robustness such as lp- adversarial examples, currently an open problem to characterize the distribution shift arising in ImageNetV2. To narrow this gap between the real and synthetic robustness challenges, (Shankar cet al. (2019) have introduced variant of the ImageNetVid dataset (Berg et al. 2015).. The authors assembled consecutive video frames that are all highly similar to human yet cause classifiers to make errors on some of the images. Inspired by advavsssarial examples, the authors propose the following pm, (plus-minus k) metric evaluate the accuracy of classifier as
pm sf.
Σ (Ru)es
min IIf(x') = =y]
E'EB (m)
where (x) contains the 2k frames around the anchor frame (I frames in each temporal di- rection). This corresponds perturbation function 7 ((. y. f) arg miggiigg.B.ee I[f(2') y). Under this metric, the authors find that even the best models accuracy drop of at least 13%.
3

---

Under review as a conference paper at ICLR 2020
2.1.2 SYNTHETIC DISTRIBUTION SHIFTS
The research community has developed wide range of synthetic robustmess notions for image classification over the past five years. In our study, we consider the following classes of synthetic distribution shifts.
Adversarial examples. One prominent example of synthetic distribution shifts are adversarial examples, which demonstrate that current image classifiers can be fooled by small image perturba- tions that are (almost) imperceptible to human (Szegedy al. 2013 Biggio & Roli 2018). In our robustness evaluation, we focus on untargeted adversarial perturbations bounded in foc or eornorm.
Image corruptions. Since fp-adversarial examples are unlikely to occur outside a truly worst- case setting, the research community has proposed various synthetic image corruptions less adversarial distribution shifts. The goal of these corruptions is to test robustness to distribution shifts that are more realistic and hopefully predictive of performance on real data (we will investigate this assumption in Section T In our testbed, we include all corruptions from (Hendrycks & Dietterich 2 and additionally some corruptions from (Geirhos et al.. 2018). These include common examples of image noise (Gaussian, shot noise, etc.), various blurs (Gaussian, motion), simulated weather conditions (fog, snow), and "digital" corruptions such as various JPEG compression levels. We refer the reader to Appendix AA663 for a full list of the 38 corruptions.
Style transfer. We also include a style transfer version of the ImageNet test set (Huang & Belongie, 2017). This distribution shift was proposed to evaluate whether classification models are relying more on shape or texture features (Geirhos et al. 2019).
2.2 CLASSIFICATION MODELS
Our model testbed includes 147 ImageNet models covering a variety of different architectures and training methodologies. Our goal was to include most relevant pretrained ImageNet models available online. In multiple cases we also contacted paper authors with various degrees of success. At a high level, the models can be divided into the following three categories (see Appendix A.5.1 for list of all models).
Standard models. We to models trained on the ILSVRC 2012 training set without a specific robustness focus as standard models. This category includes 74 models, ranging from the seminal AlexNet with top-1 accuracy 56.5% through widely used architectures such VGG, ResNet, and
Inception to the state-of-the-an EfficientNet with top-1 accuracy 84.4% (Krizhevsky et al. 2012 Simonyan & Zisserman 2014 He et al. 2016 Szegedy et al. 2015 Tan & Le 2019).
Robust models. This category includes models with an explicit robustness intervention. We subdivide this class further into two types of robustness interventions:
Models with increased adversarial robustness. We include both models trained with projected gradient descent (Xic al. 22119] Shafahi al. (219)) and models with evaluation via random- ized smoothing (Cohen et al. 2019 Salman et al. 2019), which yields a total of 14 models.
Further robustness interventions such as data augmentation schemes (Geirhos et al. 2019 Yun ct al. 2019) and anti-aliasing (Zhang 2019). This subset contains 51 models.
Models trained on more data. Finally, our testbed also contains three types of models that utilized substantially more training data than the aforementioned models. The ResNet152- ImageNetl Ik model was trained on 12.4 million images for 11,221 classes from the full ImageNet dataset (Wu 2016). Like all other models, only evaluate it over the 1,000 classes from ILSVRC 2012. Facebook recently released models trained on billion images from Instagram; we refer to these models as Instagram models (Mahajan al. 2018). We also include evaluations from the JFT-300M model trained on 300 million images at Google (Sun et al. 2017).
3 MEASURING THE EFFECT OF ROBUSTNESS INTERVENTIONS
When comparing interventions to increase robustness, crucial question is how to measure their effect in meaningful way. One approach would be to simply use the absolute accuracy numbers accs,= under the distribution shift specified by S and я (see Doction[]aaover If the goal is to select the best performing model (e.g., for a concrete deployment), this is indeed relevant criterion.
4

---

Under review as a conference paper at ICLR 2020
On the other hand, relying on absolute accuracy numbers becomes less relevant when comparing various robustness interventions with different unperturbed accuracies accs, As can be seen from Figure [ the accuracy under distribution shift varies substantially with the standard accuracy: mod- els with higher standard accuracy already achieve higher accuracy under distribution shift. So to isolate the effect of training intervention on the robustness to distribution shift, would like to measure how much a robustness intervention increases the shifted accuracy beyond what is possible with standard models at the same standard accuracy
To quantify this notion of robustness, we define a "baseline" function 05,x(11 (1) to return the shifted accuracy that a standard model with test accuracy z' on the standard test set S] expected to achieve under the distribution shift S, æ. So if f is a model without robustness intervention, / should satisfy 35.n (accs, (f)) = acCs,=
A priori, it is unclear if such function 3 exists. The accuracy of a trained model f on the dis- tribution shift (S, π) could depend on variety of properties besides the standard accuracy of f (model architecture, data augmentation scheme used during training, etc.). However, both natural distribution shifts we study in our paper have the intriguing property that the standard test accuracy is a nearly perfect predictor of the test accuracy under distribution shift for all standard models in our testbed. In particular, fitting a simple logistic model to the (accs, (f), accs,, (f)) pairs yields a good linear fit in the logit domain (see Figure []]. This shows that we can obtain an accurate estimate from data, which we will rely on in the remainder of the paper.
With the baseline robustness function /3 in place, we can now define the effective robustness p(f, S, T of trained model f to distribution shift (S, π) as
x) = accs (f) Bs, (accs, (f))
If a model f is truly more robust under distribution shift S, x (in particular, more so than standard models without robustness intervention), should have effective robustness p(f, S, T) substan- tially larger than O. Graphically, p can be interpreted as the vertical deviation away from the linear fit in Figure I In the following, we will study to what extent various synthetic robustness measures are predictive of the effective robustness on natural distribution shifts, and which models achieve the largest effective robustness on natural distribution shifts.
4 DOES SYNTHETIC ROBUSTNESS PREDICT NATURAL ROBUSTNESS?
Given the difficulty of measuring a model's robustness to natural distribution shifts, an important question is whether there are simple synthetic proxies. We therefore study to what extent robustness to synthetic distribution shifts predicts robustness on natural distribution shift.
A simple approach would be to directly compare different models' robustness to synthetic distri- bution shift and their robustness to natural distribution shift. However, we run into an important confounding factor. As mentioned earlier, models that have higher accuracy have smaller accuracy drop on the natural distribution shifts we consider. Moreover, we find that models with higher aceu- racy also see smaller accuracy drops under various synthetic distribution shifts. Therefore it would not be methodologically sound to directly compare synthetic robustness to natural robustness and draw any conclusions about transferability between the two. Simply the fact that a model is more accurate will simultaneously predict both that (i) the accuracy drop under natural distribution shift will be smaller, and that (ii) the model will be more robust synthetic distribution shift.
To control for the confounder standard accuracy, we rely on the effective robustness notion intro- duced in Section 3 i.e., the robustness increase that cannot be attributed an increase in standard accuracy. In Figure 2 we analyze the relationship between two synthetic robustness notions and effective robustness on ImageNetV2. The resulting plot shows that the two quantities are almost entirely uncorrelated: a model being more robust to synthetic perturbations does not imply that the model will have a smaller accuracy drop on ImageNetV2. Appendix A. we repeat the above experiment for the ImageNet VidRobust distribution shift and reach similar conclusions.
We note that this corresponds to the problem of estimating a treatment effect as studied in causal inference. We do not explicitly rely on knowledge about causal inference to make our exposition more accessible but refer the reader to textbooks on causal inference for hackground (Pearl et al. 2016 Heran & Robins 2019).
5

---

Under review as a conference paper at ICLR 2020
Effective robustriess (ImageNetv2)
2.5
Effective robustness (ImageNetv2)
2.5
2.0
2.0
1.5
1.5
1.0
1.0
0.5
0.5
0.0
0.0
-0.5
-1.5
-40
-35
-30
-25
-2
-15
-60
-50
-40
-30
-0
-10
Accuracy change under corruptions
Accuracy change under Ip attacks
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
Figure 2: Robustness synthetic distribution shift (X-axis) vs. effective robustness the Ima- geNetV2 distribution shift (y-axis). The left plot shows synthetic robustness as measured by an average over image corruptions from (Hendrycks & Dietterich 2019; Geirhos al., 2018 2019). The right plot shows synthetic robustness as measured by PGD attacks for 82- and foo robustness and two perturbation sizes each. In both cases, synthetic robustness not predictive of effective robustness under the natural distribution shift. Appendices A.] and A.2 contains similar plots for individual synthetic robustness measures and the ImageNerVidRRous distribution shift.
5 WHAT HELPS WITH NATURAL DISTRIBUTION SHIFT?
As can be seen in Figure T most models did not achieve substantial effective robustness Nevertheless, a few model types stood out with significant effective robustness on one of the two datasets (no model in our testbed that showed non-trivial gains for both natural distrbution shifts). We discuss the main outliers in more detail.
on ImageNetV2 and MmageNetVidRRobue
ResNet152- 5eammmameeett Ik and Instagram models. The left plot in Figure 3 zooms to the two model families that displayed high effective robustness on ImageNerV2. All of these models are trained on additional data beyond the standard ILSVRC 2012 training set. The models exhibited an effective robustness of 2.0 and 1.4 (median robustness for the four Instagram models), which was substantially larger than any other model with similar accuracy. However, despite their promising performance on Vm2 these models show little to no effective robustness for Ima- geNetVidRobust. Moreover, the effective robustness gain is larger for the model trained on less data than the Instagram models. To further investigate the effect of data on model robustness, conduct series of controlled experiments in Section
lk
fp-robust models. In the right plot of Figure 1 we see a cluster of l. apaavverrrrillly robust classifiers that all exhibit substantial effective robustness for ImageNetVidRobeee also in a low accuracy regime, in particular lower than the standard classifiers they were derived from (this trade-off is discussed in Tsipras et al., 20199 Raghunathan et al. 2019) among others).
However, the models are
This trade-off poses potential confounding factor: although the f--robust models show high effec- tive robustness, it is not clear if the resulting accuracy trade-off is meaningful. To put the trade-off into context, we compare the models to simple baseline. In particular, consider the family of classifiers given by interpolating between random guessing and the standard classifier the l ro-buttt models are derived from. Since random guessing is not affected by the distribution shift, this interpo- lation can trade off standard accuracy for effective robustness. In the right plot of Figure we illus- trate this family of interpolated classifiers (the dotted line) and show that standard ResNetl with no robustness interventions can be interpolated with a random classifier to achieve the same trade-off as an b robust model. So for the purpose of the natural distribution shift on ImageNet obuuus the Ey robust models offer no benefit beyond interpolating with random classifier.
6 How DOES THE AMOUNT OF TRAINING DATA IMPACT ROBUSTNESS?
In the previous section we have seen that the ResNet152- 5mmm:155t1mme and Instagram models achieve non-zero effective robustness ImageNetV2. This plausible effect since larger training sets provide more thorough sampling real world images. However, the JFT-300M model does not display this effect despite being trained on 300x more data than standard models.

---

Under review as a conference paper at ICLR 2020
ImagetterVa 7oom #1
Imagetier Vid Robust Zoom
aeteerarally rebaot (frice retuctiric commaition
Inperpstate
Standard
Sdversafllly
Other rebustress intervention
Imagehes
Imagelict
50
Vid Robust Benign
ImageNetVZ Accuracy
Figure 3: The left plot zooms into the range of the ResNet/52-ImageNelll Ik and Instagram models on ImageNetV2 and shows that the models achieve significant effective robustness, i.c., they lie significantly above the linear fit. The right plot zooms into the range of the fy-robust models on ImageNet VidRobust with high effective robustness. The dotted line shows the family of models achievable by interpolating between standard ResNet- and random classifier. It achieves the same robustness trade-off as the b robustt models.
Subsampling number images per dass (logit scaling)
Subsampling classes (logit axis scalling)
Perfect
Sucsesmple
Imegetet-vi v2 125 val "S)
liwwer tt
Sussomplle 1-4
claspes
Stanzard bransing
Sussumple 1/8
Stancord training
classes
Trained orth mone date Ne subsempling
Suzeemple 1/36
70
Suzeample 1.32
90
20
80
20 20
30
70 80
80
50
ImageNtet val 2tp--1.2 %6
Imagetit 125 class val Itop-1, %)
Figure 4: To investigate the impact of training data robustness, vary the ILSRVC 2012 data along two axes: the number of images per class (left), and the number of classes (right). Although models trained on more data (e.g.. the Instagram and ResNet152-ImageNNer Ik models) provide improvements in effective robustness, we find that subsampling of the ILSVRC training has no impact on effective robustness.
A possible explanation could be that differences in label diversity or quality play a role in promoting robustness. For example, the Instagram models trained dataset collected to overlap with the ILSVRC categories, and the 12sSeet15-ImageNeetl 1k model is trained on a superset of the ILSVRC data. Meanwhile, the JFT model trained on long-tail, weakly-labeled dataset with 18,291 categories, which, to the best of our knowledge, were not collected to align with ILSVRC categories. To investigate the role of data in more detail, we now conduct two experiments.
Varying the number of images per class. We start by subsampling the number of images per class in the ILSVRC training set by factors of {2. 4, 16. 32} and show the impact on accuracy and robustness ImageNetV2 in Figure As expected, increasing the number of images per class consistently leads to higher accuracy. However, the impact on robustness is exactly as predicted by fit on standard ImageNet models. This indicates that varying the the size of training in an i..dd manner affects accuracy but not effective robustness, at least for ImageNetV2.
Varying the number classes. Next, we consider the diversity of the data by varying the set of classes. In particular, we create three successive subsets of the ILSVRC training set with 500, 250, and 125 classes (see Appendix [..6.5] for the list of classes). We then evaluate all models in our testbed on the 125 classes present in all subsets. We present results in Figure 4 which shows that varying the number of classes again affects accuracies but not effective robustness.
Our experiments suggest that neither growing the number of images per class nor the number of classes in an i.i.d. fashion are effective robustness interventions. Nevertheless, Section B shows that larger dataset can provide meaningful robustness improvements. This disparity may be due to limi- tations of emulating changes the training set by subsampling ILSVRC. For one, our experiments consider i.i.d. subsets of the training images or classes and do not consider growing datasets in
7

---

Under review as a conference paper at ICLR 2020
manner that may cover different image distributions, in the case of Instagram data. Another pos- sibility is that increases in dataset size may only improve robustness after the dataset size is large enough such that the accuracy on the original distribution is close to saturated. Our experiments only observe dataset sizes smaller than ILSRRC, which may fall below this inflection point.
7 RELATED WORK
There are several related papers that study the relationship between various synthetic robustness measure, e.g., Geirhos et al. 2018: Engstrom et al. 2019; Hendryeks & Dietterich] 2019 Kang et al. 2019; Tramer & Bonch 2019; Maini et 2019). To the best of our knowledge, paper is the first to compare synthetic and natural distribution shifts. Since our findings show that synthetic robustness currently does not promote natural robustness, our paper offers complementary view to the aforementioned comparisons focused on synthetic robustness.
Our work relies on two recent papers introducing datasets with natural distribution shifts (Recht) eet al. 2019, (Shankar et al. 2019). Recht et al. (019)) already note that there is a clear linear rela- tionship between original and new test set accuracy but do not evaluate any models with robustness intervention or models trained on more data. Shankar et al. (2019) compare only a small set of robust models. Due to the limited size of their test set and model testbed, their results do not conclusively show that l. fp-robust models offer effective robustness on their dataset. In contrast, our testbed of robustness interventions is substantially larger and aims to encompass all publicly available models. We also include models trained on more data and conduct experiments to investigate the effect of training data on robustness. While our experiments shows that f,-robust models do offer effective robustness
mmageNeVVidRobuss we also put this improvement in context by demonstrating that it is comparable to interpolating with a random classifier. Moreover, none of the two papers study to what extent synthetic robustness is predictive of robustness under natural distribution shifts.
8 CONCLUSION
One aspiration of robustness interventions is to improve performance on real distribution shifts. Since this is challenging problem, synthetic notions of robustness offer natural starting point. While it expected that corresponding interventions will be less effective on real distribution shifts, the hope is that these interventions still convey at least some robustness on real data. Unfortunately, our experiments show that this is not the case for two recent examples of real distribution shifts. On both datasets, models with robustness intervention perform no better than baselines without such interventions. The fact that progress on synthetic distribution shifts is essentially uncorrelated with progress on real distribution shifts raises the question whether further improvements for synthetic robustness will yield benefits on real data.
To deploy machine learning in safety-critical environments, it will be necessary to address this ques- tion and improve robustness under real distribution shifts. Our experiments suggests multiple direc- tions for future work to achieve this goal.
One important direction is to assemble more examples of natural distribution shifts in order to test robustness real data in more detail. While it is possible that other types of natural distribution shifts are more correlated with synthetic robustness, this hypothesis needs to be tested with carefully curated datasets.
Our experiments on ImageNetV2 suggest that adding training data can improve robustness. A sim- ilar phenomenon has also been demonstrated for adversarial robustness (Schmidt al. 2018 Car- mon et al. 2019 Uesato et al. 2019; Najafi et al., 2019 Zhai et al. 2019). However, it is currently unclear what type data to add: the JFT-300N model trained on 300x more data offers no robust- ness improvement. The ResNerl 52- Net 1k model was trained on roughly 10x more data and offers robustness gain comparable or larger than the Instagram models trained on 1,000x more data. Understanding what additional training data is most effective could significantly improve ro- bustness to real distribution shifts.
Finally, it is important to note that the synthetic robustness interventions do offer robustness on the distribution shifts they are designed for. This raises the hope that accurately characterizing real distribution shifts will allow to leverage existing techniques for training more robust models under well-specified distribution shifts. The distribution shifts in our comparison and our extensive model testhed offer starting point for such investigation into improved training methodology.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Alexander C. Berg, Michael Bernstein, Jia Deng, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Jonathan Krause, Li Fei-Fei, Wei Liu, Sean Ma, Olga Russakovsky, Sanjeev Satheesh, and Hao Su. ILSVRC development kit. http:://image net http:///a/a/////////eaa..e.. SBSVVR22015dewvkitee tar gz 2015.
Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning. Puttern Recognition, 2018.
03141
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šındić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases, 2013. https: 1ink. springer, com/
chapter 10.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems (NeurIPS), 2019. htttps://arxtti http://wwwrr//////////oe 905. 13736
Jeremy M Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via ran- domized smoothing. International Conference on Machine Learning (ICMI.), 2019. https:
//arxiv.org abs///b///// 902.02918
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: large-scale hierarchical image database. In Conference on Computer Vision and Pattern Recognition (CVPR),
2009. http: /www. imagmmmaaaiae. net. org.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rota- tion and a translation suffice: Fooling CNNs with simple transformations. In International Con- ference Machine Learning (ICML), 2019. http://://axxxx..//aaaaaa /1cbh 12717722221979999
Alhussein Fawzi and Pascal Frossard. Manitest: Are classifiers really invarianc? British Machine Vision Conference (BMVC), 2015. attps://axxx..oorraab//11111
Nic Ford, Justin Gilmer, Nicholas Carlini, and Ekin Dogus Cubuk. Adversarial examples are a natural consequence of test error in noise. In International Conference on Machine Learning (ICML), 2019. http:// htrp/arxvvvvir... warg//bbs/ abs 190119911110 1ttp/19911111111311
Robert Geirhos, Carlos R. M. Temme, Jonas Rauber, Heiko H. Schütt, Matthias Bethge, and Fe- lix A. Wichmann. Generalisation in humans and deep neural networks. Advances in Neu- ral Information Processing Systems (NeurIPS), 2018. http::/ nips. cc/paper/ 1e82-generalisationn inw.ssttiiio-----aasaaas---a- and -tt--aaan--dddedee---eeaaaaaa---
pdf
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained enns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations (ICLR), 2019. https: http:://axivvv.. org/ aag/abs//
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
https://arxiis org/ alos /15l//1/1... 03385
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In Intermational Conference on Learning Representations (ICLR), 2019.
Miguel A. Herman and James M. Robins. Causal Inference. Chapman & Hall/CRC, 2019. https www. hsph. ha rva rd. edu
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normal- ization. International Conference on Computer Vision (ICCV), 2017. https: /tttps://axiv..../// org/
abs/ 2as//703..mm 06868

---

Under review as a conference paper at ICLR 2020
Daniel Kang, Yi Sun, Dan Hendrycks, Tom Brown, and Jacob Steinhardt. Testing robustness against unforeseen adversaries, 2019. https://arxii: org/ abs 1w1.8.9000....
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. tion with deep convolutional neural networks. In Advances in Neural Informa- tion Processing Systems (NIPS), 2012. https: http:////appprss.ssssccocceeeeee 1e44-mimgeeetccii
Imagenet classifica-
wwp.. cc/pper// fiaiiiicatt..... con-winee th- deep- -eeual
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR), 2018. https://arxis wwtp:/pwwwwwwww//////oooooosse
/ttp:s///arrvoooor//bba///////// www.o///a///...../oo 06083
Dhruv Kumar Mahajan, Ross B. Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. Exploring the limits of weakly supervised pretraining. In European Conference on Computer Vision (ECCV), 2018. https:
farxiv.org/a att//bba///// 800.01105.000 00932
Pratyush Maini, Eric Wong, and J. Zico Kolter. Adversarial robustness against the union of multiple perturbation models, 2019.
abs/ 909. 10000//0099900088888
Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. Robustness to adversarial perturbations in learning from incomplete data. In Advances in Neural Information Processing
Systems (NeurIPS), 2019. https://aaxix: org/ /90..//95.... 13021
Judea Pearl, Madelyn Glymour, and Nicholas P. Jewell. Causal Inference in Statisties: A
Primer. Wiley, 2016. htttps://ww..dd. wi www.y.lww....................... com/ en- ent-uss/aauall+iffeeeeee
Aditi Raghunathan, Sang Michael Xie, Fanny Yang, John C. Duchi, and Percy Liang. Adversarial training can hurt generalization, 2019. https: htttps:/aaxx...or////ataa wttp:/wwwwwgg/bb///o/baa 06032)
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classi- fiers generalize imagenet? In International Conference on Machine Learning (ICML), 2019.
https://arxiv. httpg//bs/19900 10811
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, 2015.
Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, and Se- bastien Bubeck. Provably robust deep learning via adversarially trained smoothed classifiers, 2019. https::/arxiv..rg//////
906 04584
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adver- sarially robust generalization requires more data. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
11285
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! 2019. https:
//arxiv.org/aat abs / 904.1.41 12843
Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt. A systematic framework for natural perturbations from videos, 2019. https://
arxiv. htt.voorg/abb////// 910...9966..22 02168
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image 556 2014.
Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable ef- fectiveness of data in deep learning era. In International Conference on Computer Vision (ICCV), 2017. https: :/tps:///axiiec wttp:/iwworgrooaaoooooasee 02968
10

---

Under review as a conference paper at ICLR 2020
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2013. https::/ /attpp://arxi...o org/ abs/
2512.6199
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolu- tions. In Conference on Computer Vision and Pattern Recognition (CVPR), 2015. https:
www.ii.oig/abba..aaaaaaaaaaaaa.a a409 4842v1
Mingxing Tan and Quoe V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks, International Conference on Machine Learning (ICML), 2019. https://arxiv.
atrg/bb/// /905/1905.00 11946
Antonio Torralba, Alexei A Efros, et al. Unbiased look at dataset bias. In Conference Com- puter Vision and Pattern Recognition (CVPR), 2011. https://teeexxlore.. wew.rreeieeeerreo..
document /5995344//99
Florian Tramer and Dan Boneh. Adversarial training and robustness for multiple perturbations.
In Advances in Neural Information Processing Systems (NeurIPS), 2019. https://arxiv.
org/ ha////bb///904 13000
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Alcksander Madry. Robustness may be odds with accuracy. In International Conference Learning Representa- tions (ICLR), 2019.
12152
Jonathan Uesato. Jean-Baptiste Alayrac, Po-Sen Huang, Robert Stanforth, Alhussein Fawzi, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neu- ral Information Processing Systems (NeurIPS), 2019.
13725
Wei Wu. www.tp/sss/.hhha..ccccoooooooooe
and http::///aaaaeeeoooontttts//////
aws Swslabs//e hrt.ap/...eeee.aarr..//c///////r hwt.://seaaaaaaeaaascacccaaaiao hatnioon/mndeezoo.pyy.
2016.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustness. In Conference on Computer Vision and Pattern Recognition
(CVPR), 2019.
Fanny Yang, Zuowen Wang, and Christina Heinze-Deml. Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness. In Advances in Neu- ral Information Processing Systems (NeurIPS), 2019. https::/arxxiv.or///aass
11235
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy train strong classifiers with localizable features. In Interna- tional Conference on Computer Vision (ICCV), 2019.
04899
Runtian Zhai, Tianle Cai. Di He, Chen Dan. Kun He, John Hopcroft, and Liwei Wang. Adversarially robust generalization just requires more unlabeled data, 2019. https:://rxxis org.abs/ 1906. 1906.00555) 00555
Hongyang Zhang, Yaodong Yu, Jiantao Jiso, Eric Xing, Laurent El Ghaoui, and Michael I. Jor- dan. Theoretically principled trade-off between robustness and accuracy. In International Con- ference on Machine Learning (ICML), 2019.
zhang 1aagt9pphtttm html
Richard Zhang, Making convolutional networks shift-invariant again. In International Conference on Machine Learning (ICML), 2019,
http://as/191....../ 11486
11

---

Under review as a conference paper at ICLR 2020
A
APPENDIX
12

---

Under review as a conference paper at ICLR 2020
Figure overview of testbed. Each row is model, and each column is evaluation setting. For the corruptions, we aver- age over the five severities defined in (Hendrycks & Dietterch 2019). We also plot in-memory and on- disk versions of each cor- ruption as jpeg compres- sion was found to con- founding factor in (Ford et al. 2019). We leave out the class-subsampled models and evaluations de- scribed in Section 6 for brevity.
13

---

Under review as a conference paper at ICLR 2020
A.1
SYNTHETIC CORRUPTIONS VS. EFFECTIVE ROBUSTNESS (IMAGENET-VID-ROGE ROBUSTT)
Effective robustress (Imageree--id ridrrousss)
Effective robustress (Imageretoeid
-2
-2
Accuracy change under wwg.corruptions
Accurecy change avg.
Standard training
Other robustness intervention
adversarially robust
Trained with more data
Figure 6: Robustness synthetic distribution shift (x-axis) vs. effective robustness the Ima- geNetVid distribution shift (y-axis). The left plot shows synthetic robustness as measured by an average over image corruptions from (Hendrycks & Dietterich 2019: Geirhos al., 2018 2019). The right plot shows synthetic robustness as measured by PGD attacks for 82- and foo robustness and two perturbation sizes each. In both cases, there almost correlation, with the except of the Ip adversarially robust models explored earlier.
14

---

Under review as a conference paper at ICLR 2020
A.2 SYNTHETIC ROBUSTNESS VS. EFFECTIVE NATURAL ROBUSTNESS
Effectie robustmeess (Imngeretten www.eret--trrrooooooooo
Effeclive
Accuracy change under avg corruption
Accuracy change under avg comrustios
EHfeciive robustrees emmaggeeett www.------bbboooo
Effective rebusines (Imzgelcta))
under
Effective robustrrss (wwagene--weteeeeeee.ee receeetttiieeoott
Effeciive rebuttress (Inaçoteev2)
2-
8%
-0
-6
Accuracy change brightness in-memory
Accuratty change triss inmmmorry
Effective rebustness (Imageret-vidd rectttt robbot)
Effactive (maggleer/2)
0-
-1-
-20
-18
-18
-14
32
10
-18
-66
-14
-12
-00
-8
Accuracy aay change under brightness on-dik
Accuracy change brightness on-disk
Efective rob seeeess mmmgee-e-vv---o....
Effective robuztrezs (mmgghhut22)
-50
-00
-30
-20
-1.
-50
-40
-30
20
-10
Accuracy change under contrast inmeemory
Accuracy charge under contrest in-memory
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
15

---

Under review as a conference paper at ICLR 2020
Effect mibostress (maagene---it wareeene-------rrrooeee www.-bbboottt
2
Effeciive
Accuracy change
Accurary change
Effective rebuctres Umaggemet aeddddddde (od--ddddoot)))) roburs))
Effective (Inaghhce22
40
-30
20
10
50
-2
10
10
Accuracy change aator memory
Accuracy change defocus in memory
Effectie rob stteess timageret--oee www.--oooooooo
Effecdive roburtress
-50
-40
-30
-20
-10
-50
-40
-30
-30
-10
Accuracy change under detocus blu on disk
Accuracy change under detocus__bug bur
an-dissk burr_nn-dik
Effective roburtess (imageneteiiiee www.geeeeeeerrrrrtettt.t
Effective robustice IimageNcc/21
-35
-30
-25
20
-10
-30
-20
-35
-10
Accuracy change under elastic transform n-momary
Accuracy charge under elastic transform In-memory
Effective robustmess Ummager wmw.geeat-ttt-rrrooee
Efeccdive
Accuracy change undor 01022c transfom on disk
Accuracy change undor clastic transform on dikk
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
16

---

Under review as a conference paper at ICLR 2020
Effect ribustress (mwggen---iidrrbeueee
2
Effeciiv
30
-50
-00
20
Accuracy change /n-menory
Accurary charge
in-memorry
Effective rebuttness Dimggenet-eee www.eettwwddeeee -ewteeeee--ddoooootoeoooo (ohoo))
Effective r00k (Inazhhcv2)
-50
30
10
-50
40
-30
Accuracy change tog
Accuracy change fog_on
Bffectie rot sttress timagerett www.--ooooooooo
Effecdive robustress
-10
-10
-20
-10
-40
-30
-10
-10
Accuracy change under frost Jn-oorrry
Accuracy change uncer frost n-memory
Effective rebustess (ww.geeettetteeeee www.roood--ooottt
Effective robudtice ImmageNec/21
11
-40
40
Accuracy change under frost on-disk
Accuracy change under frost on-disk
Effective robustmess Ummnger wmw.genattttrrrraaa
Efecaiive
Accuracy change under gauss on Dlur memory
Accuracy change under gaussian blur in mommory
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
17

---

Under review as a conference paper at ICLR 2020
Effect rebostrees (mwggene--ii---ee -id-en------bbuut
Effecivv
-0
Accuratyy change gaussian
Accuracy change gaussiar
Effective rebuctmes Dmmggeeet sedd--dd-- -od----dddoodd)) mobort))
Effective (Imazchctv2)
30
20
50
-00
30
-20
Accuracy change under goussian noss memory
Acourary change gausstan_notse memory
Effectie rob streess timageret www.--oooooonnn
Effecdive roburtress
-30
-00
-30
-20
-10
-50
-40
-20
-10
Accurecy change under gaussian_ oolse_on disk
Accuracy change under goussion
cecessoonnnnsseeeeeeeeeee noise on--dokkkk
Effective robustes (imagenet www.aneeeewvttrrrreeeee www.ettt----orttt
Effective robustice IimageNec/21
11
-30
-30
-30
44
-20
Acouracy change under gloss blu onidisk
Accuracy change under glass blur on-disk
Effective robustrees dos timngereet-tidrrdooeee wtn.eet-t----ooussiitt eobbustt
Effeciive rabusines timagetet221
Accuraty change greyacale
Accuracy under greyscale
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
18

---

Under review as a conference paper at ICLR 2020
Effect rubustress Smwagen---ii--rbbuuee
Effecivv (InageNesv2)
50
30
Accuracy change Inmmenory
Accuracy change
jn-memory
Effective roboctrenn Dmmggemet wed wet---ddddddds)oooo -oboot))))
Effective (Imazchetv2)
30
-00
40
30
20
-00
-00
20
-10
Accuracy change mouusse_nosse_____
Accuracy change wwiseseeoooeeeoooom
Bffectie rob sotresteessss timageret--eeee www.--ooooooooo
Effecdive roburtress
-35
-30
-25
-20
15
-10
-35
-30
-25
-20
-55
-10
Accuracy change under jpt___oopreesooonnnooeee
Accuracy change under joeg poopppessstnnnn memory
Effective robustice ImmageNccr22
Effective roburmess www.geeeewwwwttrrreeeeee
-30
-25
10
-30
-35
-30
-15
-10
Accuracy change under lo0g compression on-disk
Accuracy change under doal compress on on-disk
Effective robustmess Ummager tew.geeetttttrrreeet
Erecadive
Accuracy change under motion blur memory
Accuracy change under metion blr memoryy
Standard training
Other robustness intervention
adversarially robust
Trained with more data
19

---

Under review as a conference paper at ICLR 2020
Effect robustreess (Imagene-----i--- -wdpene------bbutt
Effecivv
29
15
Accuracy change
Accuracy charge
Effeccive rebustmens Dime www.geeee-viiideee gen.---dd-dddoott robust)
Effective (Imagotees2)
-1-
-40
-00
-20
40
-00
-0
Accuracy change under apd.12.pp40.1
Acturacy change pgd 22epps..1
Effective rots steesss timagenet-vd--omm
Effccliv Dobucurezz
-70
-60
-30
-40
-00
20
-10
-70
-50
-00
-40
-30
-10
Accuraty change under pgd aps0.5
Acturacy change under pgd 12.eps0.5
Effective rebustress Omargenet wii---o----bb
Effective robustr (mmgeeett221
-40
Aocuua A altueee under Sogd 20605
050
-20
-70
-50
-20
-10
Acturacy change under pgd inf.eps0.5
Eifective robustress eos Ulmngeree--ttrrroeeeee.. wtr.geeet-ttt--rroesittit enbbustt
Effeciive rabusines timagehetr21
under Inp
under inf.aes
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
20

---

Under review as a conference paper at ICLR 2020
Effect robostrees wmaagene--ii---rbbuee
Effecivv
20
20
20
Accuracy change jo-memory
Accuracy change pizelate jn-menory
Effective rebuctien Dimagemet wedee-idddeee wiw-----dd-oooos)) roburt)
Effective (Imagohcev2)
-30
25
10
-00
20
Accuracy change under pixclate
Accuracy charge pixelete
Blfectie rob stuusteeess www.aeeee--vioooonee
Effecdive roburtrezz
-20
-15
-10
-20
-15
-10
Accuracy change under saturate__ wwaprtteenmmmeee in-memory
Accurscy change under saturate, in Jn-memory
Effective rebustess (ww.gemeeeeedeeee. www.eeeeewerrooootttt..oo
Effectivv robustide ImnageNcc/21
11
-25.
-22.5
-200
-173
-15.
-125
-00
-7.5
-25.0
-223 -20.9
-175
-3.0
-12
-100
-73
Accuracy change under saturate on-dikk
Accuracy change under saturate onidisk
Effective robustmess Uimngemet-einn wewngeeetttttrrreeeii
Efecciive (maggohetr2
Accuracy change under shot nosse in memory
Accuracy change under shot noice memory
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
21

---

Under review as a conference paper at ICLR 2020
Effectio mibostress (maggene---it wotpeeee--i--rbbuuuuee
Effective
Accuracy thange
Accuracy change
Effective robuttmes Dmmggemet seddd-dd--- -ow-ooot---ooooott robort))
Effective (Imazchet22)
50
-43
40
30
-30
-00
35
50
23
20
Acouracy change momory
Accuracy change show, memorry
EHfective rob streesteesss timageret--ooeee www.--o-oooooooo
Effecdive roburtresz
2-
-50
-43
-40
-25
-50
-45
-00
-35
-30
Accurscy change under now-on-disk
Accuraty change under show, on-disk
Effective robustice Imageeeeo/21
Effective rebustees www.ggeeetwiwttrreeeeeee
11
-30
-10
-20
-10
Accuracy change under Spattar In-memory
Accuracy change under spattor insmemory
Effective robustmess Uimigerett menngertttitrrrreaet
Efecciive rabustnnss (imggehetr2
Accuracy change under spatter diicisk
Accuracy change under spatter on disk
Standard training
Other robustness intervention
Lp adversarially robust
Trained with more data
22

---

Under review as a conference paper at ICLR 2020
Effect we rebostreess (maggene---tt weteeeee---drrbuuuueee www--rbboottt
Effeciiv
40
20
Accuracy thange
jn-memory
Accuracy change specide jn-menory
Effective rebocteenn Dmmggemet aed -et---dd-odd))) -obooo))))
Effective (Imagchcev2)
30
20
-00
-00
-30
20
-10
Accuracy change specill, notee
Accuracy change cotoccenootsoooo
Bffectie rob streess timageret--oee www.--ooooooto
Effecdive roburtress
-00
-50
-40
-30
-20
-10
-50
-30
-60
-30
-20
-10
Accuraty change under stylized imagenet
Accuracy change under styllteddmaaaeeeeeeeee magenet
Effective rebustes (imagemee www.geeeetteweeeeeeeee... www.eettttrrrrttttoi
Effective robustice IimageNccr22
11
-40
-20
-10
-40
-30
-20
Acouracy change under zoom blur In-mamory
Accuracy change under zoom blur in-momory
Effective robustmess Ummngeett-ete-eee
Efecaiive
Accuracy change under 20om blur clsk
Accuracy chango under zoom blur on disk
Standard training
Other robustness intervention
adversarially robust
Trained with more data
23

---

Under review as a conference paper at ICLR 2020
A.3
ORIGINAL ACCURACIES DISTRIBUTION SHIFT ACCURACIES
10
0
45
Copines PUROROY 1000 91
Accurnty mageleer 1tpp
Accuraty Imsgaliee htp-1
Imageleer 1by7 couing)
Imagelist Jngt
ACCLOS COTTOO Lwmoooe
Accurety connect and
Accurdey meggrees Rlp 111
ACCUFOO Imagemest Siap-
Imaggete dlgil, sualing
Iragufist Pogi uns sccllig)
ACUURR DebrUu (nteeer tue 2
Accuricy mageeeer Rop
Accurnty Imsgeker 2.pp-1.00
11 PURETT MT 1-003, hd)
Imagalist Togt seating)
2
magelee
Accueet 1.50
Perfect robustness training
Lp adversarially robust robustness
Trained with more do
24

---

Under review as a conference paper at ICLR 2020
Inasgette thypt suding)
Iragefict Sugt
65
ACCUPT 4 10g pinonoe
Accurecy or fos
Accurscy imageles Htp
Accuracy Imagaliet Stp-1.
Imageliet (bgt scaling
Imagalist logt seating)
Ascurisy mepeles Hap
Acca a0y Imagefist lip-. 1.
Imageleet (bg2 sait scuin))
Imagalitt Togt seeliig)
DM000n 2NV enne WTT Don
Acoureyy mageleer Rep
Aerurary Imagetee 2tp-11
Imagellet (bgi soaling
Imageliet 2007t acck areling)
ICC/O0T guoeae rebe
Accuor
[tpp-1
Imagelae ecaling
Inagelist Zegit
1.70
25
40
55
80
65
0
25
50
35
62
05
10
Perfect robustness
Lo adversarially robust
Trained with more de
Standard training
Other robustness intervention
Linear
25

---

Under review as a conference paper at ICLR 2020
tnsigertes Hogi, scring)
Imagalist Togt
R20mnyy moge moogritea. REOLE Ue0 2.00
72
Accursty impelien ttpp.
Accuracy Imagaltet hop-1.
birer War.oalllpp............. S1oce (000 100.. 00pe
Imagetiet Zegt eesting)
95-
23
20
Acoursty Imagelen Htp
35
Imagaitat Zogt acaliig
Imagetiet lbgi cralin)
64
A0003 woreeeee togps erovee Cip3. Scourayy worooaoooo
Accrrasy Imagelee Hap-1
Aecurayy Imagettet 200-11
Imageliet 2egt vesting)
Imagelee
6.
12
AcoutsEy Imwgelee Htp-
Imaggeiet
Imagelint log'
RACHROC RED
Accurnr
40
55
90
05
35
42
30
35
66
65
10
15
Perfect robustness
Lo adversarially robust
Trained with more de
Standard training
Other robustness intervention
Linear
26

---

Under review as a conference paper at ICLR 2020
Rooracy co conies Unteere 2003.
Imsgeeas thag. suiing
Imagatet Togt eca seating
Accursty imselien ttop
Accuracy Imagaliet hip-1.
Imagetiet (bgt scaling)
Imagalist logt seating)
82
Accurisy mugeleet Hop
Accu 80y Imagafus lipp.. 70
Imspetiet Higit scaling
Imagaliat Zogt sceling)
85
Accumey
AcoureEy Imaggeeee Hep-1
Accurny Imagette 200-11 1,
Imagelee 1897 salk oraling)
Imagelitt 20g7 venling)
Accurecy
Itep
Imagelint Imageliee 200p-3
Imagelae ecalngg
MEDEIC SN (ererrr (1004.
25
40
45
53
55
80
Ø
0
30
35
60
65
10
Perfect robustness
Lo adversarially robust
Trained with more de
Standard training
Other robustness intervention
Linear
27

---

Under review as a conference paper at ICLR 2020
Insppeted thypt suling
Iragefist logt
Accurscy impelee ttop
Accuracy Imagaliet htp-1.
Imsgeliet (bgg scaling!
Imagalist logt seating)
Ascurisy mupeled Hap
Accu 90y Imagefus lip-1 1.
Imspefiet (lgght scaling)
Imagalist Togt seeling)
ALCURT 01 Crewe www.e//wwoee.... pummtpoe
Actureey mageleer Hep
Aomirayy Imagetee 2tp-1 10
Imagelee 18o2 sale srulin)
Imageliet 2009t acck oreling)
11
Imagelint [tpp-1
175
40
55
80
Ø6
0
35
42
30
35
66
65
10
Perfect robustness
Lo adversarially robust
Trained with more de
Standard training
Other robustness intervention
Linear
28

---

Under review as a conference paper at ICLR 2020
50
89
85
IraggMet
Perfect robustness
Lo adversarially robust
Trained with da
Standard training
Other robustness intervention
Linear fit
29

---

Under review as a conference paper at ICLR 2020
A.4 CORRUPTION ROBUSTNESS
Figure 22: A detailed view of corruption robustness, with cells sampled from the main grid in Figure
5 Here we present resnet50s trained on some of the corruptions from the imagenet-C benchmark, as well as the best model trained on more data, instagram-resnext10) 33x448d, and the best model trained on just the standard training set, efficientnet-b7.
We have already seen that corruption robustness does not promote effective robustness, or robustness real distribution shift. Here, we analyze whether robustness to some corruptions transfers to others, and what may contribute to corruption robustness. Figure 22 shows the result of training various resnet50 on few corruptions from imagenet-c.
In line with prior work, this plot here tells us that training against one type of synthetic corruption or one set of synthetic corruption does not transfer well to other corruptions. There are cases where transfer does happen, but overall the models are only robust to the corruption they are trained on. This fact is even more prominent when looking at the difference between the instagram model and efficientnet- while both are amongst the top in terms of robust to synthetic corruptions, instagram is more robust to the on disk versions while efficientnet is more robust to the in memory versions. At first glance, this phenomenon seems strange, but in fact it could be that since the instagram model is trained on images pulled from instagram, which are jpeg compressed, it is more resistant to on disk perturbations while the efficientnet is trained using autoaugment, which is augmentation strategy done in memory, explaining its greater robustness to in memory perturbations.
A.5 DETAILS ON OUR MODELS
'Each resnet50 was trained with a batch size of 256 for 120 epochs, starting with a learning rate of 0.1 and decaying by a factor of 10 every 30 epochs. For the resnet50s trained on corruptions, randomly sample corruption and severity for each image. Refer to A.6.3 for details corruptions and severities. We use custom fast gpu implementations of these corruptions for training.
30

---

Under review as a conference paper at ICLR 2020
Top-1 model accuracies on the original ImageNet validation set, the Imagenetv2 set mmatched-frequent an average over all the corruptions, and an average over all the pgd attacks. Note that since we take an average of many attacks, the PGD column can no longer be considered worst-case attacker for the model (look to A.62 for specific attacks). The confidence intervals are 99.5% Clopper-Pearsee
Table I:
intervals; they are not provided for the average columns since iid assump- tions for the intervals may be violated. References for the models can be found in Appendix A.5. We exclude the subsampled class models for brevity. The table continues for more pages.
ImageNet Aggregated Top-1 Accuracies
Validation Accuracy
Imagenetv2 Accuracy
Avg. Corr. Accuracy
Avg. PGD Accuracy
Model
FixResNeXt101 32x48d
86.0 [85.6, 86.4]
77.7 [76.5, 78.9]
65.1
19.9
FixResNeXt1013222
85.9 1855, 86.4]
77.6 [7.44, 78.7]
65.4
20.3
instagram resnext101 32x48d
85.4 [85.0, 85.9]
77.0 [75.8, 78.1]
65.5
24.1
instagram-resnextl0) _32x32d
85.1 [84.6, 85.5]
76.8 [75.6, 77.91
64.8
24.4
efficientnet-b7
84.4 [83.9, 84.9)
74.4 73.2, 75.6]
63.1
30,9
instagram-restie 10 36d
84.2 [83.7, 84.6]
75.5
[74.3, 76.7)
63.2
29.2
efficientnet-b6
84.0 183.6, 84.5]
74.1
72.88 75.3]
63.7
33.4
efficientnet-b5
83.7 [83.2, 84.1]
73.3
77..0, 74.5]
62.5
31.1
FixPNASNet
83.5 [83.0, 83.9)
73.2 [71.9, 74.4]
61.3
22.5
puasnetSlarge
82.7 182.3, 83.2]
72.3 [71.0, 73.6]
61.8
29.5
instagram-resnexille .3x8dd
82.7 [82.2, 83.2)
73.6 [72.4, 74.9]
60.8
30.1
efficientnet-b4
82.6 [82.2, 83.1
71.5 [70.2, 72.7]
60.0
33.5
nasnetalarge
82.5 182.0, 83.0]
72.3 [71.0, 73.5]
61.7
36.8
senet154
81.3 [80.8, 81.8)
70.2 [68.9, 71.5]
54.1
30.6
polynet
80.9 180.4, 81.3]
70.1 [68.8, 71.3]
54.0
23.0
efficientnet-b3
180.3, 81.3]
69.7 [68.4, 71.0]
55.9
30.2
inceptionresnetv2
80.3 [79.8, 80.8)
69.3 [68.0, 70.6]
56.9
34.8
1.e3seext10132x44
80.2 [79.7, 80.7]
69.4 [68.1, 70.7]
52.3
28.8
inceptionv4
[79.6, 80.6]
69.2 [67.9, 70.5]
55.5
27.9
Desnet10l_uutmis
79.8 [79.3, 80.3)
68.0 [66.7, 69.3]
50.1
25.6
dpn107
79.7 [79.2, 80.2]
67.8 [66.5, 69.2
52.4
30.8
FixResNerSOCuMMix.W
79.4 [78.9, 80.0]
67.0 [65.7, 68.3]
43.5
dpn13]
79.4 [78.9, 79.9]
67.7 [66.4, 69.0)
52.1
30.4
FixReNNet5CuuMMix
79.4 [78.9, 79.9]
66.8 [65.5, 68.2]
43.4
18.2
dpn92
79.4 [78.9, 79.9
67.2 [65.9, 68.6]
49.3
25.7
resnext [01 32xx88d 32x8d
79.3 [78.8, 79.8]
67.4 [66.1, 68.8)
49.7
25.4
efficientnet-h2
79.3 [78.8, 79.8]
67.4 [66.1, 68.7]
53.6
29.3
dpn98
79.2 [78.7, 79.7]
67.8 [66.4, 69.1]
51.8
30.2
jft-300M
79.2 [78.7, 79.7]
67.4 [66.1, 68.77
53.5
26.8
se_resnext50_224e
79.1 [78.6, 79.6]
67.8 [66.4, 69.
50.6
24.7
resnext101 64x4d
79.0 [78.4, 79.5]
67.1 [65.8, 68.5]
52.1
23.6
wide_resnet101.2
78.8 [78.3, 79.4]
66.4 [65.0, 67.7]
48.2
25.2
xception
78.8 [78.3, 79.3]
67.2 [65.9, 68.6]
51.7
26.3
se se.resnet1.52
78.7 [78.1, 79.2]
67.4 [66.1, 68.8]
50.9
28.4
resnet500cutmiix
78.6 [78.1, 79.1]
65.8 [64.5, 67.
44.7
26.4
FixResNet50.v2
78.6 [78.1, 79.1]
66.5 [65.2, 67.8]
43.3
15.3
FixResNee50
78.5 [78.0, 79.1]
66.2 [64.9, 67.6]
43.2
15.1
2ide_resnet50.22
78.5 (77.9, 79.0]
66.2 [64.8, 67.5
46.2
26.1
se se.resnet101
78.4 [77.9, 78.9]
[65.8, 68.4]
50.1
28.2
resnet152
78.3 [77.8, 78.8]
66.8 [65.5, 68.2]
47.8
22.5
resnett50_fcatureccumm
78.2 [77.7, 78.7]
66.0 [64.7, 67.4]
44.3
25.1
resnext101 32x4d
78.2 [77.7, 78.7]
66.3 [64.9, 67.6]
51.0
22.4
resnet10] 1pf3
78.1 [77.6, 78.6]
66.1 [64.7, 67.4]
46.5
22.5
resnet101 Ipf5
77.9 [77.4, 78.4]
66.2 [64.8, 67.5
46.5
23.1
efficientnet-bl
[77.3, 78.3]
66.3 [65.0, 67.7]
51.2
29.0
resnet101 Jp12
77.8 [77.3, 78.3]
66.3 [64.9, 67.6]
46.1
22.0
31

---

Under review as a conference paper at ICLR 2020
ImageNet Aggregated Top-1 Accuracies
Validation Accuracy
Imagenetv2 Accuracy
Avg. Corr. Accuracy
Avg. PGD
Model
Accuracy
se.resnet50
77.6 [77.1, 78.2]
65.8 [64.5, 67.2]
48.1
27.4
resnext50
77.6 [77.1, 78.1]
65.8 664.5, 67.2]
45.6
22.5
resnet50.minn mixup
77.5 [76.9, 78.0]
65.0 [63.7, 66.4]
48.2
22.2
[bresnet152
71.4 [76.9, 77.9
65.5 [64.1, 66.8]
50.0
23.5
resnet101
77.4 [76.8, 77.9]
63.5 [64.1, 66.8]
46.1
21.8
FixResNet550 adaaatttai
77.3 [76.8, 77.9]
65.4 [64.1, 66.8]
45.2
20.5
inceptionv3
77.3 176.8, 77.8]
65.7 [64.3, 67.0]
49.8
25.7
densenet161
77.1 [76.6, 77.7]
65.3 [63.9, 66.6]
49.4
22.2
resnet50_cuiie cutouu
77.1 [76.5, 77.6]
64.4 [63.1, 65.8)
43.8
19.9
dpn68b
(76.5, 77.6]
64.8 [63.4, 66.1]
45.7
18.7
resnet50_Ipf3
77.0 [76.5, 77.6]
64.5 [63.1, 65.8]
43.5
21.7
densenet201
76.9 [76.4, 77.4
64.8 [63.4, 66.
47.6
23.9
resnet50_Ipf3
76.8 176.3, 77.3
64.7 [63.3, 66.0)
43.3
21.8
resnet50.Iple 1p12
76.8 [76.3, 77.3]
64.5 [63.2, 65.8]
42.2
20.8
resnet50_trainee SIN
76.7 [76.2, 77.2]
64.6 [63.3, 66.0]
44.0
22.6
nd. Be__IN_then__iiee www.tuned.dd.
cafferesnet101
76.2 [75.7, 76.7]
64.1 [62.8, 65.5]
44.8
25.5
resnet 12-mman1--i
76.2 [75.6. 76.7]
66.1 [64.8, 67.5]
47.3
30.7
resnet50_aw 1attt500awsbbssllnng baseline
[75.6, 76.7]
63.6 [62.3, 65.0]
42.1
21.3
resnet50
76.1 [75.6, 76.7]
63.2 [61.8, 64.6]
41.6
21.4
Eesnet50_imageeee 10Uperce
76.0 [75.4, 76.5]
63.2 [61.9, 64.6]
41.6
21.3
dpn68
75.9 [75.3, 76.4]
63.1 661.7, 64.5]
45.5
17.7
efficienttet-b0
75.8 [75.3, 76.3)
63.2 [61.9, 64.6]
45.9
29.5
Senet50-randomizee
75.7 [75.1, 76.2]
63.8 [62.4, 65.1]
41.8
20.9
smoothing_noies _atgggooooieee00e00
densenet169
75.6 [75.1, 76.1]
63.5 [62.2, 64.9]
46.7
21.8
resnet50. with brightness.
75.3 [74.7, 75.8)
62.7 [61.3, 64.1]
43.9
22.2
Dssnet50_with.spattee
75.2 [74.7, 75.8]
62.6 [61.3, 64.0]
42.8
22.8
densenet121 Ipf3
75.1 [74.6, 75.7]
62.5 [61.], 63.8]
40.5
20.0
densenet121_ppp
75.0 [74.5,
62.8 [61.5, 64.2]
41.8
21.1
denseneti21_Ipf2
75.0 [74.5, 75.6]
63.1 [61.7, 64.5]
41.2
20.8
resnet50. with saturate aws
74.9 [74.3, 75.4]
62.3 [60.9, 63.6]
42.4
20.6
resnet50_trainee Stt.oitrriiedddoneeeeeeo SIN
74.6 [74.0, 75.1]
62.8 [61.4, 64.1]
47.9
22.9
resnet34.1pt2
74.5 [73.9, 75.0]
62.2 [60.8, 63.6]
41.5
21.0
densenet121
74.4 [73.9, 75.0)
62.1 [60.8, 63.5]
43.5
20.0
resnet34_Ipf3
[73.8, 74.9)
62.2 [60.8, 63.5]
42.2
20.7
vgg19.bn
74.2 [73.7, 74.8)
62.0 [60.6, 63.3]
37.9
16.5
resnet.34_pph
74.2 [73.6, 74.7]
62.2 [60.8, 63.6]
41.2
20.9
nasnetamobile
74.1 [73.5, 74.6]
[59.9, 62.7]
44.8
22.8
vgg16.bn Ipf5
74.0 [73.5, 74.6]
61.2 [59.8, 62.5]
36.2
19.0
vggl6_bn__pp
74.0 (73.5, 74.6]
61.7 [60.3, 63.0
36.1
17.5
vgg16.bn. Ipf
73.9 [73.4, 74.5]
61.9 [60.6. 63.3]
36.3
18.3
fesnet50_with wwth_frosttattt frost_aws
73.8 [73.2, 74.3]
61.6 [60.2, 62.9]
42.4
21.5
73.6 [73.1, 74.2]
60.9 [59.5, 62.3
41.8
39.5
bninception
73.5 [73.0, 74.1]
62.0 [60.6. 63.4]
40.6
21.3
mnasnet].0
73.5 [72.9, 74.0]
60.4 [59.1, 61.8)
36.4
18.8
vgg16.bn
73.4 [72.8, 73.9
60.7 [59.3, 62.
35.7
16.1
resnet34
73.3 [72.8, 73.9]
60.9 [59.5, 62.2]
40.5
21.2
resnet50_with wrrtthhaussiannnoee noise.aws
73.0
[72.4, 73.5]
60.6 [59.2, 61.9]
45.6
45.5
resnet50_ wit500.wwtttt. gwwssiannunonoo noise. contra
72.7
[72.2, 73.3]
60.1 [58.7, 61.5]
51.8
24.0
st.motion.N blur jlur.jpeg compression..cc..
mobilenet_v2 1ippnntttv2__p2
72.6 [72.1, 73.2]
59.4 [58.0, 60.8]
34.5
17.4
mobilenet.vv..ptt
72.6 [72.0, 73.1]
59.7 [58.3, 61.1]
34.8
17.6
32

---

Under review as a conference paper at ICLR 2020
ImageNet Aggregated Top-1 Accuracies
Validation Accuracy
Imagenetv2 Accuracy
Avg. Corr. Accuracy
Avg. PGD
Model
Accuracy
mobilenett.v2.Ip5
72.5 [71., 73.1]
59.8 [58.4, 61.1]
34.9
17.8
vgg19
72.4 [71.,, 72.9]
59.7 [58.3, 61.1]
32.4
20.6
vgg16_JpfS
72.3 [71.8, 72.9
59.8 [58.4, 61.2]
31.9
19.7
vgg 16 Jp13
72.2 [71.,, 72.7
59.3 [57.9, 60.7]
32.2
19.1
vggl6.1pf22
72.2 [71.6, 72.7
59.3 [57.9, 60.6]
32.0
19.2
resnet50_wwid withet50.wwthh.omm wwt..wtthhhomeee.eee contrastaws
72.0 [71.4, 72.6
58.9 [57.5, 60.3]
40.8
17.0
mobilenet.v2
71.9 171.3, 72.4]
59.0 [57.6, 60.4]
34.0
17.8
resnet50.with.fogr
aws
718 [71.,, 72.3]
58.2 [56.8, 59.6]
37.9
16.8
resnet18_|pp 1__e118_pp33
71.7 [71.1, 72.2]
58.5 [57.1, 59.9]
36.8
20.1
vggl6
7L.6 171.0, 72.2]
58.5 [57.1, 59.91
31.3
20.1
vgg13.bn
71.6 [71.0, 72.2]
58.8 [57.4, 60.2]
31.8
15.2
resnet18_ Ipf2
71.4 170.8, 72.0]
58.5 [57.1, 59.9]
36.9
19.6
resnet18_Ipt5
71.4 (70.8, 72.0]
58.1 156.7, 59.5]
36.9
19.9
resnet50 1rrer500mmmmeeeell subsample_
70.5 [69.9, 71.1]
58.0 [56.6, 59.3]
35.4
21.8
.of 1.l.2.aach6..orrt. original images
vggl1_bn
70.4 169.8, 70.9
57.4 156.0, 58.8]
31.7
18.0
rcsnet50-randomizzd.
70.3 [69.7, 70.9]
57.7 [56.3, 59.1]
40.7
63.2
smoothing_noise.0.25
v3
69.9 [69.3, 70.5]
56.8 [55.4, 58.2]
28.5
19.3
30oglenet/incepiion/
69.8 (69.2, 70.4]
57.9 [56.5, 59.3]
38.8
21.8
resnet18
69.8 [69.2, 70.3]
57.3 [55.9, 58.7]
35.0
19.5
shuff
69.4 [68.8, 69.9]
56.0 [54.5, 57.3]
30.9
16.4
vggll
69.0 [68.4, 69.6]
55.7 [54.3, 57.1]
28.6
22.4
resnet50.ww with withlaixelate.ae aws
68.5 [67.9, 69.1]
56.7 [55.3, 58.1]
39.6
19.6
facebook adv -dviied
[67.7, 68.9]
55.2 [53.8, 56.6]
40.9
41.4
resnext101 denoiseAAl
Ssaett0--smoothingaddeeeee
67.9 [67.3, 68.5]
54,5 [53.1, 55.9]
40.6
61.8
_DNN_2steps.eps_b 20t..oos.p...2..os.....5
mnasnetO_5
67.6 [67.0, 68.2]
54.2 [52.8, 55.6]
27.9
17.4
resnet50, Bhch:00wwtthhhheeeeee srrrthhmttionttt blatrionnbuuuuee blur_aws aaws
67.5 [66.9, 68.0)
55.9
[54.5, 57.3]
38.7
15.9
facebook_ady
65.3 (64.7, 65.9]
52.6 [51.2, 54.0]
38.0
39.5
d_resnet152_denooise
resnet50-randomized
64.2 [63.6, 64.8]
51.2 [49.8, 52.6]
39.8
61.4
smoothing noise 0.50
resnet50 with greyscale
63.3 [62.7, 63.9
50.7
449.3 52.1]
28.3
18.7
resnet50 aanaaanne_suusammpeee
63.2 [62.6, 63.8]
50.5
[49.1, 51.9]
27.4
21.4
2144__atc6___orrr_ images
facebook adv traine
62.3 [61.7, 63.0]
49.8
[48.4, 51.2]
35.8
37.6
d.resnet152_basse -baseline
resnet50-smonthing adversarial
62.2 [61.6, 62.8]
49.1
[47.7, 50.5
39.1
59.2
DNN 2steps eps 512 noise 0.50
resnet50_with Bator/00wwthhoooomo_o
61.3 [60.6, 61.9)
49.4 [48.0, 50.8
33.3
12.4
shufflenet.v2.x0
60.6 [59.9, 61.2]
47.2 [45.8, 48.6]
23.6
16.1
resnet50) ady-train-free
60.5 [59.9, 61.1]
47.4 [46.0, 48.8)
29.4
57.4
resnet50-smmottinggaaveeee addversaraaaaa
60.5 [59.9, 61.1]
47.0 [45.6, 48.4]
37.2
58.8
L_GD_ 1 http__ppp_112_1111111__1 wttps_552nnnis_... 2.0iscc_0..5
Drsaet5O.taiined.on.
60.2 [59.6, 60.8)
48.6
[47.2. 50.0]
39.4
19.0
squeczenet1
58.2 [57.6, 58.8)
45.4 [44.0, 46.8
20.2
16.1
Squeezenet1_0
58.1 [57.5, 58.7]
44.9 [43.5, 46.3]
20.2
18.1
alexnet 1pf2
57.2 [56.6, 57.9]
44.0 [42.6, 45.4]
22.5
29.3
alexnet_Ipf3
56.9 [56.3, 57.5
43.7 [42.3, 45.1]
22.8
30.6
alex
56.6 [56.0, 57.2]
43.4 [42.0, 44.7]
22.8
32.0
alexnet
56.5 [55.9, 57.1
43.4 [42.0, 44.8)
21.6
24.1
Sesnet50-smmoohingg_vveeee adversaria
54.7 [54.0, 55.3]
41.7 [40.3, 43.1]
35.7
53.8
LPGD_ stseDDDsstpppp
33

---

Under review as a conference paper at ICLR 2020
ImageNet Aggregated Top-1 Accuracies
Validation Accuracy
Imagenetv2 Accuracy
Avg. Corr. Accuracy
Avg. PGD
Model
Accuracy
resnel50-randomized.
53.1 [52.5, 53.7]
40.7 [39.4, 42.1]
34.9
52.1
smoothing sanoothing-noiie.... noise.1.00
resnet50-smoothing.advv adversarial
51.9 [51.2, 52.5]
39.8 [38.4, 41.1]
34.4
51.6
DNNN2steps, wot.s51222iie__u. 1.00
51.7 (51.1, 52.3]
39.3 138.0, 40.7]
19.8
19.7
baatch64ooriginaal...
resnet50-smootting adversaria
44.3 (43.7, 44.9]
33,2
[31.9, 34.6]
29.8
43.7
resnet50 imagenet subsample.1
36.7 [36.1,
27.0
[25.8, 28.3]
12.7
16.0
of 16 batch64 original images
resnet50_ 1it-:00wwttt_hhh wrrth_dffous_hle _hluraws
31.9 [31.3, 32.5
24.6 22.44, 25.8]
18.2
9.6
htw.:/aannnnennnaaannaaee .o..22.batch44.orrppp.. immage
21.4
[20.9, 21.9]
15.6 [14.5, 16.6]
7.2
11.1
-magess
A.5.1 FULL LIST MODELS EVALUATED IN TESTBED
The following list contains all models we evaluated on ImageNet with references and links to the corresponding source code.
FixResNeXt101 33x48ddxxy v2 [tttee//Attuuucco//ccccbbreerree
Model
Type: Trained with more data.
2. FixResNeXt101 .32x48d
Model
Type: Trained with more data.
3. instagram-resneex101 _32x48d http:://ghhu.ccc///eeeetttreeses Images
Model Type: Trained with more data.
instagram-resnextt0) _32x32d http::///stttiicoooeeeeettrrrera
Model Type: Trained with more data.
5. efficientnet-b7 lattps://githun
http:////ttttoooo/////////////tt Model
Type: Standard training.
6. instagram-resnextt00 _32xx66 http:://fttiiicooocccceedeeeeerr
Model Type: Trained with more data.
7. efficientnet-h6 Type: Standard training.
Model
8.
efficientnet-bS http:///tttuuuoo//laaaaa///aaaee Model
Type: Standard training.
9. FixPNASNet htt.K//www/////t/ooo//oo/o/o/oo. Type: Standard
training. http:////ttttt////////aaaaaaaaat Model
Type: Standard training.
11.
instagram-resnextile 1atprrrmmmrrsneel1111132022 _3xxxxd https///hhha.ccc/ccccooorr////es
Model Type: Trained with more data.
12.
http::///ttttioooollaeeeeaaaaaad Model
Type: Standard training.
http::///ttttceee//tt/////taaaer Model
Type: Standard training.
14.
5enet154 http:////...ttoooee///////aaaaat Model Type: Standard training.
15.
https////ttthccoooad//eeeeeeeeer Model Type:
Standard training.
efficientnet-b3 Type: Standard training.
http:///tthhoooo////ooo/////aaat Model
17. inceptionresnetv2
http:s///hhhhoooooooeeeeeeeeeeer p.yoorch
Model Type: Standard training.
34

---

Under review as a conference paper at ICLR 2020
18. 1reseext10132xx6
Model Type: Standard training.
19. inceptionv4 Type: Standard training.
Model
20. resnet101 cutmix ottpp//ppes//tttuuooo///o//oaaaa Model Type: Other
robustness intervention.
21.
http:///ohhghtttooeee/e/eeerrerr Model Type:
Standard training.
FixResNec50CutMix..2
Model
Type: Other robustness intervention.
23. dpn131 http:////tttuuccccree///////rrar Model Type: Standard training.
24. FixResNeeSOCulMix /ttpsseeeCCuiiattttpppss//////es
httpss//gghtuaaccc/ssseeeterrrrs Model Type:
Other robustness intervention.
25.
http://:ss/..ttccoc/////////aaaa Model
Standard training.
1amext101_32xx2 http:///hhecc..cc///////t/tttttt
html Model Type: Standard trainig.
efficientnet-b2
Model
Type: Standard training.
28.
Model Type:
Standard training.
29. google resnet101 jft-30M Model not publicly released internal correspondence. Model Type:
Trained with more data.
30. se. resnext50_32sss
Model Type: Standard training.
31. #ssnxtt01_64x44
Model Type: Standard training.
22
[tmmi Type: Standard training.
33. xception htt.r///oooccccccoooooceeeaaeear Model Type:
Standard training.
34. se_ se_resnet152 Type: Standard training.
Model
35.
h/tpa//a//aopa//c/ooo/o/ooooooo/ Model Type: Other
robustness intervention.
36.
FixResNec50vv htt.a//wwooesee////.ooo/ooooooon Model Stan-
dard training.
FixResNec50 htte://fftteees/cccooco//////ett Model Type: Standard
training.
38. wide resnet50 2 http:///goooocccc/oo////sttaaaaa Model Type: Standard training.
39. resnet101 http:///chhaaa...cec//////aaaaaa
Model
Type: Standard training.
40.
hrtpp///pepp//ctccccc/////////or Model Type:
Standard training.
41. resnet50_featur http://ww...aaro//o/ooo/ooo//ooa Model Type:
Model
Other robustness intervention.
resnext101_32x44
Model Type: Standard training.
43. resnet101_ httppp///eettta..ccmm/////aaaaaa Model Type: Other bustness intervention.
44. resnet101 httf///pee//ggbbbb.cc//////aaaas i |n Model Type: Other bustness intervention.
35

---

Under review as a conference paper at ICLR 2020
45. efficienttet-bl Type: Standard training.
Model
46. resnet101_ bustness intervention.
http://ppp/..tttoccc/////aaaaaaa Model Type: Other ro-
47. se se_resnet50 Type: Standard training.
httt:://tttiiccccceeeee//ataatae Model
48. resnext50 http:///e//.ccccccc////////////a
Model Type: Standard training.
49. ossnet50_mixu http:////oaaaaaaao/////////////a Model Type: Other
robustness Intervention.
fbresnet152 htte:r//cttaiiiarreeeee///s/aaae Model
Type: Standard training.
51. resnet101 http:////hhhccccoo/////////eaaoa Model Type:
Standard training.
52. CereeseeeSO..o.adaaaaiiiee
htte:///ttthhoccc///eeeeeeeettts Model
Type: Standard training.
53. inceptionv3 Type: Standard training.
http:///htthhcccoocaaaeeeeerrras Model
54. densenet161 http::///hhuuuucccaaaaaaaaaaaaaa
Model
Type: Standard training.
55. resmet50 cutout httpppas//ttaaaaaao///o//////taa Model Type: Other
robustness intervention.
56. dpn68b http:////ctttooooooc///////aaaat Model Type:
Standard training.
resmet50 hggi:///s/stttuacc///oooo/aaaais Model Type: Other ro- bustness intervention.
densenet201 https///aghaiaaoooccaaaaaaaaaaea pytorch Model
Type: Standard training.
59.
resnet50.Ipph btteas///htttuucccccccdd///aaaar Model Type: Other
bustness intervention.
60. resnet50_ http://////gitii..ooo//////aaaaa Model Type: Other
bustness intervention.
61.
resnec50 SIN SIN_and and www.he.ffnntuuee... https://gitnub.. com htw.tnuub..co//ggirhoo...... rgeirhos/////// Cexture-vs- saappe Model Type: Other robustness intervention.
62. cafferesnet101 http:////sotoooaoce/////e//eee/r
Model
Type: Standard training.
63. resnet152- 105--mmmeenet111 Ik
Model Type:
Trained with more data. For this model, we took the 1000 logits corresponding to imagenet classes
and computed the accuracies those logits (as opposed to taking the max over 11k logits).
64. resnet50 aws baseline
Model Type: Standard training.
resnet50 http:////gggggoooooooooooo//aaar Standard training.
Model
resnet50 imagenet. httpeeeeeetttt0peeeeeeem.baaiiia .origiiaa images
httpa://gLthub.commCCC
pro Type: Standard training.
67.
Model Type:
Standard training.
68.
efficientnet-b0
Model
Type: Standard training.
69. resserSDrandommzee wowee/ooooiieeeee.
Model Type: Lp adversarially robust. During evaluation, called predict with alpha=l and 10000.
70.
https//////g.hhhccc//////e//pppr models pytorch
myyorr
Model
Type: Standard training.
36

---

Under review as a conference paper at ICLR 2020
71.
pytorch Model Type: Other robustness intervettoo............
72. resnet50 www.wwtttttteaaaa.m. aws
pytorch Model Type: Other robustness intervention. intevvttosssneeeeeetioo.o....
73. 1ensenet121 prt.:aa/a..aaaao.c.c.../////aaaa Model Type: Other
robustness intervention.
74,
densenet121.Ipp http::///.E.ll...oooooooooooolaa Model Type: Other
robustness Intervention.
htt.a///../aa...ooooo.oooooooaar Model Type: Other
robustness intervention.
76. resectOO.witbh.sturaaee...v
pytorch Model Type: Other robustmess intervention.
77. resmet50 trained on MIN and IN httppr///ggttbboco.c...gahaeetes
Model Type: Other robustness intervention.
78. http.h//...cccssssc.cccccccoc/ct Model Type: Other ro-
bustness intervention.
79. 12nsenet121 hetp::///sttibboooo/////aaeaetaa pytorch Model
Type: Standard training.
80. resmet34_ http://:/s//////.ooo/ooooo///aa/ Model Type: Other
bustness intervention.
81. 1gg19. http:///w..tttece////////aaaaaaa Model Type: Standard training.
Model
82. resnet34 htf.:aa///gtttaa.ccoooo/////a1as Model Type: Other
bustness intervention.
83. nasnetamobile Type: Standard training.
httpp://tttttt...ca/a///////aaaa pytoochh Model
84. vg http://ttwe////tt...//////////at Model Type: Other ro- bustness intervention.
Xtt.S//ppte/////tttaa.ooooooaoaa Model Type: Other ro-
bustness intervention.
/t htt.::/ss/..thccccooo////ooaaaaa Type: Other or
86. Ipp_
bustness intervention.
87.
pytorch Model Type: Other robustness intervention.
88.
http:://giithbb.com/ceeeee
preetrainedd-cceess.yyyyoe Model Model Type: Other robustness intervention.
intereetoo.......nnioo....
89.
Model
Type: Standard training.
mnasnetl.0
Model Type: Standard training.
91.
16 bn http:///wo//ttttacaccaa//ee/aeen Model Type:
Standard training.
92. resnet34 Standard training.
http:///ppp....oooo////////aa/aa Model Type:
93. resnet50_wiihe wtt.://wwwwwassi.nniiii...c preccraineddmccess.ppp pytrcp Model Type: Other robustness intervention.
94.
/atpps://sithbb
com/ octp://cceeeeeeecc.....oooooeeeo Other robustness intervention.
htw.hhheehhhhhhahtttttttoooooaaa Model Type: Other
robustness intervention.
96. httppiie.....ggoooe/tccccccc/oee Model Type: Other
robustness intervention.
97. mobilenet_y2.ng http://ttt////gghh..oo///a/a//aa --add-ooo cons Model Type: Other
robustness intervention.
37

---

Under review as a conference paper at ICLR 2020
98. vgg19 Standard training.
http://s/tttaaccccccceecraaaaaar Model Type:
1 intervention.
99.
httfa///we/.tttti.c/////////aaaa Other robust-
100.
4tt44//1110/////aaceccooo/////a0 Other robust-
intervention.
101.
httpa///////////..aa////aaaaaaaa Other robust-
ness intervention.
102.
resmet50 coathh.ontrssaww
pytorch Model Type: Other robustness intervention.
103.
mobilenet v2
Model Type: Standard training.
104.
Dssnet50_wtth_fog.ut
Model Type: Other robustness intervetioon.
105.
resmet18_.ple httpa/////aoooocccoco/c////aaaaa Model Type: Other
bustness intervention.
106. vgg16 http:p/:/.ccccccccoceeeeeeeeeaer Model Type:
Standard training.
107.
hgg.tt/wwe//tttaacccaeeee///tsse Model Type:
Standard training.
108.
18smet18 httpp///////tttt.aaooooo/oaaaaaa Model Type: Other ro-
bustness intervention.
109.
http://ff.aaa////a../..ooo///oaa Model Type: Other ro-
bustness intervention.
110. resnet50 imagenet subsample batch64 original images
attps://github.com
Type: Standard training.
111.
htt.a//htsa......c////////////aa Model Type:
Standard training.
112.
resnetSO-anddomizel.smmee smo.oossoottohiniiseeeee http:///aa.////////taaaaa//aaaaa
Model Type: Lp adversarially robust. During evaluation, we called predict with alpha=l and n=100.
113.
http:///ss..ttoocco////e//aaaaaa Model Type:
Standard training.
114.
googlenet/ineeptionnl
html
Model Type: Standard training.
115.
resnet18
http::/ss...ccccccocooecaaaaaaaa Model Type:
Standard training.
116.
haml Model Type: Standard training.
117. vggll
Model Type:
Standard training.
118.
resnet50.wiin minnt50.with.pixxlatc..o pixelaxtxxxwaww aws
pytorch Type: Other robustness intervention.
119.
facebook adv .ww.indiidddddd.... adw.tnineddddrddeddeedaae traiineedeemmeettoo..eoneeellll
Type: Lp adversarially robust. robusst
120.
resnet50-smoothing adversarial DNN 2steps eps 512 noise 0.25
httpa://github..om com/
Hadisa o
Model Type: L.p adversarially rohust. During
evaluation, we called predict with alpha=l n=100.
121.
mmasneto_5
Model Type: Standard training.
122.
wisher5O.with..nn motion motion.blur.aa. aww
pytorch Model Type: Other robustness intervention.
123.
ImageNet- Law.NNet-Avveesariaa
Model Type: Lp adversarially robust.
38

---

Under review as a conference paper at ICLR 2020
124.
resner5O-randomiizeessnmoooo
http:///gttppppgtttib.bbbb. ht sww.sthhaa.tmoooniiinnngoo
Model Type: Lp adversarially robust. During evaluation, we called predict with alpha=l and n=100.
125.
resmet50.with greyscale aws
Myderl Type: Other robustmess intervention.
126.
jimages
nt.ps://github.com
C pytocch Model Type: Standard training.
127.
LnaggNeeG=-dvverssriiall---- Model
Model Type: Lp adversarially robust.
128.
SsneccOOssmoothiingadvverssriiri DNN 2steps.eps 512.p1s..pp...1e..........11 moise 0.50
httpa::/github.com
Model Type: Lp adversarially robust. During
evaluation, called predict with alpha=1 and n=100.
129.
somecto,wwithzzoomhhhl
pytorch Model Type: Other robustness intervention.
130.
shuffleneeLv2.x0.55
html Model Type: Standard training.
131.
resnet50.
http:://gghhbbb.cc////aaaaaaaaa
Freekbbersgariallraiiinnni Model Type: Lp adversarially robust.
132.
noise_0.25
https://github..co
Model Type: Ipp adversarially rohust. During
evaluation, wc called predi ct with I
133.
resmet50.trained..or
Model
Type: Other robustness intervention.
134.
squeezenetl_1
http::///ftaaaaccaaaaaaaaacaaaae Model
Type: Standard training.
135.
squeezenet1_ _0
Model
Type: Standard training.
136. alp2 http://sssattti...oo././//ooaaaa Model Type: Other ro-
bustness intervention.
137.
alp http:://///tttiiccooo/////aaaaaa cons Model Type: Other ro-
bustness intervention.
138.
http://///tttta.ao/////////aaaaa Model Type: Other ro-
bustness intervention.
139. alexnet http:////ttttte/e//////////aaatr
Model Type:
Standard training.
140.
adwessssooooiiinnansssisilllllll Isttsstppppsssspspppppppp sssteepeps_51....... Sls_noise___. 0.50
https://github.. com/
Model Type: Lp adversarially robust. During
evaluation, we called predict with alpha=1 and n=100.
141.
http://ggttbb... htwp:/hubbooo///eeeeeeeeee htt.://oo...ooooooooao///////ooo smotthinn Model Type: Lp adversarially robust. During evaluation, we called predict with alpha=1 and n=100.
resnet50-randomized. htw.oommizedsmoootiiinn..ooooo -oiiiii_nnooie__..000
142.
resnee50-mmooohingg.adeeeee wtt.eeesarraall.NNaatteeepp.... m0s.m............
https://github. com/ Model Type: Lp adversarially robust. During
evaluation, we called predict with alpha=l and n=100.
143.
resnet50 Desttttt.mmmmeeee...bbaaaaeo.oL Calear/ppeetaannccccoooessss.aaa
images
https://github.com
Type: Standard training.
144.
httpa://github. com/
haddiiaaiiaan/mmmoochhhnnaaerrrr Model Type: I.p adversarially rohust. During
evaluation, we called
www.lcllllpccccccccccccc www.:ppeee.o with alpha=l and n=100.
145.
cemn550.mmeennt.sxsssmlll...... htt.://o.......ccccccoocoogginno immages
httpa://github.co
Badeene/pretraannd--nooeee wwt.::ilsss..oorccchoho.o Model Type: Standard training.
146.
pytorch Model Type: Other robustness intervention.
147.
renne50_inngenneuuussmmpplll htt.a/ee________aaa_a_aaaaae www.aahhhooooiiiiaaaaaaa
ht.pa://github.com
Cadene/pretraiinet pytorch Model Type: Standard training.
39

---

Note about the FixRes models: the github repo for the fixres code uses slightly different implemen- tation of the Resize ze () method than the PyTorch default. Our testbed was built with the default resizing method, and thus the topl numbers report here around 0.35% lower than what is Under review as a conference paper at ICLR 2020
claimed in the paper. We plan to fix this soon.
A.6 DETAILS ON OUR EVALUATION SETTINGS
A.6.1 REAL DISTRIBUTION SHIFT
For Imagenetv2, we evaluate on the following datasets: imagenetv2-matchee freeeeaameccccee-eueeeeeee-eee frequency-format-val Bougenetv2-thresoold-e format-val, umagenetv2-top-image
(magenetv-mmatche---rqqeeee
-aa-enettt-ttrehoodd-ooo The format-val versions are variants of the original dataset encoded with jpeg settings similar to the original one. Unless oth- erwise stated, results in our paper referring to imagenetv2 are for wawpeett---mcccd-ffreeeeee vol.
mmagenev2-too--mage--ormee
For Imagenet-Vid-Robure we look at the 1109 anchor frames in the dataset and evaluate the benign accuracy for pm0. For pm10, we look at up to 20 nearest frames marked "similar" to the anchor frame in the dataset and count it as misclassification if any one of the predictions is wrong.
A.6.2 PGD
We the following 4 pgd attacks each model with these settings:
pgd. web.linf.epae e. Norm: 0.5/255, Step size: 5.88e-5, Num steps: 100
pgd. fod.linf.ege eps2 Norm: 2/255, Step size: 2.35e-4, Num steps: 100
pgd.12. Norm: 0.1, Step size: 0.01, Num steps: 100
12. Norm: 0.5, Step size: 0.05, Num steps: 100
Most of the models were attacked with only 10% of the dataset (in a class-balanced manner) due to computational constraints. These models are displayed with larger error bars in the plots.
A.6.3 CORRUPTIONS
We include 38 different corruption types: greyscale. gaussian noise (in memory and disk), shot noise (in memory and on disk), impulse noise (in memory and on disk), speckle noise (in memory and disk), gaussian blur (in memory and on disk), defocus blur (in memory and on disk), glass blur (in memory), motion blur (in memory and on disk), zoom blur (in memory and on disk), snow (in memory and disk), frost (in memory and disk), fog (in memory and disk), spatter (in memory and on disk), brightness (in memory and on disk), contrast (in memory and on disk), saturate (in memory and on disk), pixelate (in memory and on disk), jpeg compression (in memory and on disk), elastic transform (in memory and on disk).
For each corruption, we average over the five severities. Unfortunately we were not able im- plement glass blur efficiently in memory and so that entry is missing. We make sure to make the distinction between memory corruptions, for which we provide custom fast gpu implementations, and on disk corruptions, for which use the publicly available imagenet-c dataset, since it was ported Ford al. (2019) that jpeg compression can have a significant impact on model accuracies (indeed, as evidenced by Figure
A.6.4 STYLIZED IMAGENET
We use the stylized imagenet dataset used by (Geirhos al. 2019) another evaluation dataset.
A.6.5 125 CLASS EVALUATION
For the 125 class evaluation, we evaluate the following classes from ILSVRC:
n01677366 n01 007457757 n01742172 nC1796340 n01829413 n01871265 n01 92491
n01494475 n01630670 nc1644373 n01644900 n01669191
40

---

Under review as a conference paper at ICLR 2020
n01944390
n01978287
nC1980166
n02007558
n02009229
n02017213
102033041
n02037110
n02056570
n02071294
n02085936
n02086079
n02093428
n02093991 nC2095314
n02095570
n02096294
n02096437
n02097474
n02100236
n02100583
n02102318
n02105056
n02107574
n02112706
n02113023
nC2114855
n02128925
n02134418
n02138441
n02165105
n02219486
nC2226429
n02264363
n02280649
n02441942
n02483708
n02486261
nC2488291
n02492035
n02641379
n02730930
n02777292
n02790996
nC2795169
n02808440
n02814533
n02814860
n02837789
n02859443
nC2892201
n02895154
n02948072
A02951585
n02977058
n03000247
n03110669
n03201208
n03208938
m03216828
n03240683
n03250847
nC3272562
n03297495
n03337140
nO3376595
n03379051
n03447721
nC3492542
n03527444
n03535780
nO3642806
n03670208
n03673027
nC3692522
n03710193
n03775071
n03832673
n03838899
n03840681
n03868242
n03873416
n03877845
n03884397
n03908714
n03920288
n03933933
n04004767
n04009552
n04037443
n04041544
n04067472
nC4074963
nO4099969
n04125021
n04141975
n04149813
n04204238
n04208210
n04229816
n04266014
n04310018
n04330267
n04335435
nC4336792
n04355338
n04417672
n04479046
n04505470
n07715103
nc7875152
n09256479
n12620546
41


Reviews:
[{"review_id": "HkxRhsD5YS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper tries to evaluate whether robustness ‘interventions’ such as robustness to adversarial examples and other synthetic distribution shifts on natural distribution shifts. \nOverall this paper is generally well written and focuses on an important direction of understanding when the toy/synthetic settings used for convenience actually transfer to the real world. However, I have some concerns with the current version of this work. \n\nDecision: I vote for rejecting this paper. My main concerns are regarding both the motivation of this work and the quality and soundness of the claims. \n\n— Soundness: My major concern is with the datasets that are used for “natural” distribution shift. ImageNetV2 is essentially from the same distribution as the original Imagenet. It’s unclear what the distribution shift is and whether there is a distribution shift. In fact, the paper that produces Imagenet V2 tries very hard to minimize the distribution shift. Hence it seems unreasonable to use this dataset for a shift. Similarly, the other dataset used is from Imagenet Videos. Here again, it’s unclear what the shift actually is, and hence using these datasets to measure performance to natural distribution shifts seems questionable. \n\n— Motivation: I am a little confused about the motivation of this work as presented. It does not seem unreasonable that robustness to one kind of shift doesn’t transfer to another. It seems like the right questions to ask would be the following: Do the synthetic robustness goals seem like they would occur in natural settings? Does robustness on synthetic datasets and synthetic generations of such perturbations transfer to perturbations (of the same kind) but occurring in natural world. \n\nFinally, it seems like the novelty/contribution of this work is very small. The paper uses existing datasets and existing robustness measures. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A"}, "extra_text": {}}, {"review_id": "Hke8aQMpKS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n============================================ Update after rebuttal =========================================\n\nI thank the authors for their detailed rebuttal. I'm in general satisfied with the authors' response to my concerns, so as promised, I'm happy to increase my score.\n\n========================================================================================================\n\nThis paper considers the relationship between various measures of synthetic robustness and two distinct measures of natural robustness in large scale image classification models. The authors argue that the synthetic robustness measures considered in this paper are not predictive of natural robustness when the effect of baseline accuracy is subtracted. I think, if true, this is an important message that researchers in this area need to be aware of. However, I have a number of questions and concerns about the results. I would be happy to increase my score if the authors could address some of these issues:\n\n1) Another important recent benchmark not mentioned in the paper is ImageNet-A (https://github.com/hendrycks/natural-adv-examples). I would encourage the authors to include this benchmark among their natural robustness measures (in addition to ImageNetV2 and ImageNetVidRobust). The advantage of this dataset is that because it samples from the error distribution of a high-performing ImageNet model, to a large extent, it already comes with the baseline subtracted, so it essentially obviates the need for the indirect effective robustness measure introduced here. The raw accuracies on ImageNet-A would be directly interpretable and they would also answer the question “why should we care?” in a more visceral way, because even the Instagram trained state-of-the-art ImageNet models seem to achieve a mere 17% accuracy on this benchmark: https://arxiv.org/abs/1907.07640\n\n2) There seems to be a direct conflict between the main conclusion of this paper (that synthetic robustness measures do not predict natural robustness) and an opposite conclusion reached in an earlier paper (https://arxiv.org/abs/1904.10076) where the authors claim that robustness against synthetic perturbations like translation, hue, and saturation are actually highly predictive of video robustness (not sure if this would generalize to ImageNetV2). As far as I can see, these particular perturbation types are not included among the synthetic perturbations considered in this paper. Can you please clarify this discrepancy? \n\n3) Relatedly, looking at the scatter plots of effective robustness vs. robustness against individual perturbations in the appendix, especially for the video robustness measure, some of the correlations seem to be pretty significant (for example, video robustness vs. jpeg compression robustness, p. 19). So, I am wondering to what extent the main conclusion of this paper might just be driven by the averaging of a large number of non-predictive perturbations and a smaller number of more predictive perturbations. \n\n4) Also, ImageNet-P perturbations are not included in the paper. If the authors want to make their claims more reliable, I would encourage them to consider these among their synthetic perturbations as well. The translation perturbation in ImageNet-P, in particular, would be particularly important to consider for video robustness given the results from the arxiv pre-print mentioned in 2) above.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "BkgJfUqRYH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studied an interesting question, whether the gain of robustness from synthetic distribution shifts can be transferred/generalized to the robustness under natural distribution shifts. It was shown that in the context of natural distribution shifts, no current robustness intervention can really outperform standard models without a robustness intervention. The main strength of this paper is its extensive experimental study. However, I still have concerns on this work.\n\n1)  The authors mentioned \"an implicit assumption underlying this research direction is that robustness to such synthetic distribution shifts will lead to models that also perform more reliably on natural distribution shifts.\" However, I am uncertain about this point. Let us take adversarial robustness as an example, I am NOT surprising that the robustness on crafted adversarial examples cannot be generalized to the robustness over natural distribution shifts. Thus, I do not think that adversarial robustness obeys the 'implicit assumption'. In other words, the studied problem should be better connected to adversarial robustness.\n\n2) The significance of 'effective robustness' is not clear. It seems that most of insights were learnt from Figure 1, 3 and 4. Do they rely on the metric 'effective robustness'? \n\n3) In Figure 1, 'Trained with more' -> 'Trained with more data'\n\n############# Post-feedback ################\nThanks for the response. I am still not fully convinced by the significance of the findings in this work.\nIn the paper, the authors highlighted that \"our results show that current robustness gains on synthetic distribution shifts do not transfer to improved robustness on the natural distribution shifts presently available as test sets.\"  I am not surprising at this point, since the synthetic shift, e.g., introduced from adversarial examples, may only characterize the short-cut shift for misclassification. Thus, robustness learnt from this synthetic distribution shift might not transfer to the natural distribution shift. \n\nI decide to keep my original score. \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors show that models trained to satisfy adversarial robustness properties do not possess robustness to naturally occuring distribution shifts. The majority of the reviewers agree that this is not a surprising result especially for the choice of natural distribution shifts chosen by the authors (for instance it would be better if the authors compare to natural distribution shifts that look similar to the adversarial corruptions). Moreover, this is a survey study and no novel algorithms are presented, so the paper cannot be accepted on that merit either.
Decision: Reject