Models Out of Line: A Fourier Lens on Distribution Shift Robustness
Authors: Sara Fridovich-Keil
Brian R. Bartoldson
James Diffenderfer
Bhavya Kailkhura
Peer-timo Bremer
Venue: NeurIPS 2022 Accept (2022)
Keywords: OOD robustness
effective robustness
deep neural networks
spectral analysis
CLIP models

Abstract: Improving the accuracy of deep neural networks on out-of-distribution (OOD) data is critical to an acceptance of deep learning in real world applications. It has been observed that accuracies on in-distribution (ID) versus OOD data follow a linear trend and models that outperform this baseline are exceptionally rare (and referred to as ``effectively robust”). Recently, some promising approaches have been developed to improve OOD robustness: model pruning, data augmentation, and ensembling or zero-shot evaluating large pretrained models. However, there still is no clear understanding of the conditions on OOD data and model properties that are required to observe effective robustness. We approach this issue by conducting a comprehensive empirical study of diverse approaches that are known to impact OOD robustness on a broad range of natural and synthetic distribution shifts of CIFAR-10 and ImageNet. In particular, we view the "effective robustness puzzle" through a Fourier lens and ask how spectral properties of both models and OOD data correlate with OOD robustness. We find this Fourier lens offers some insight into why certain robust models, particularly those from the CLIP family, achieve OOD robustness. However, our analysis also makes clear that no known metric is consistently the best explanation of OOD robustness. Thus, to aid future research into the OOD puzzle, we address the gap in publicly-available models with effective robustness by introducing a set of pretrained CIFAR-10 models---$RobustNets$---with varying levels of OOD robustness.

Models Out Of Line: A Fourier Lens On Distribution Shift Robustness
Sara Fridovich-Keill; Brian R. Bartoldson', James Diffenderfer', Bhavya Kailkhura', Peer-Timo Bremer University of California, Berkeley
'Lawrence Livermore National Laboratory
Abstract
Improving the accuracy of deep neural networks on distif-disttiuutiin (OOD) data is critical acceptance of deep learning in real world applications. It has been observed that accuracies on in-distribution (ID) versus OOD data follow a (and
linear trend and models that outperform this baseline are exceptionally referred to as "effectively robust"). Recently, some promising approaches have been developed improve OOD robustness: model pruning, data augmentation, and ensembling or zero-shot evaluating large pretrained models. However, there still is no clear understanding of the conditions on OOD data and model properties that are required to observe effective robustness. We approach this issue by conducting a comprehensive empirical study of diverse approaches that are known to impact OOD robustness on broad range of natural and synthetic distribution shifts of particular, we view the "effective robustness puzzle"
CIFAR-10 and ImageNet. through Fourier lens and ask how spectral properties of both models and OOD data correlate with OOD robustness. We find this Fourier lens offers some insight into why certain robust models, particularly those from the CLIP family, achieve OOD robustness. However, our analysis also makes clear that no known metric is consistently the best explanation of OOD robustness. Thus, to aid future research into the OOD puzzle, we address the gap in publicly-availate models with effective robustness by introducing set of pretrained CIFAR- 10 models RobustNets with varying levels OOD robustness.
Introduction
Deep learning (DL) holds great promise for solving difficult real-world problems domains such as healthcare, autonomous driving, and cyber-physical systems. The major roadblock in adopting DL for real-world applications is that real-world data often deviates from the training data due to noise, corruptions, or other changes in distribution that may have temporal spatial causes [3, 20, 41, 44]. DL. models are known to be highly brittle under such distribution shifts, which raises the risk of making incorrect predictions with harmful consequences. Designing approaches learn models that achieve both high accuracy on in-distribution data and high robustness to distribution shifts is paramount the safe adoption of DL in real-world applications.
Several recent works [1, 35] have noticed an intriguing phenomenon: accuracy on in-distribution (ID) versus accuracy distribution-shifter data obeys linear trend across wide range of models, corruption benchmarks, and prediction tasks. Unfortunately. as nearly all current DL models bear substantial loss under distribution shifts, this linear relationship implies that current training strategies are insufficient to bridge the gap between ID and out-of-distributiio (OOD) performance. However, despite being exceedingly rare, models that break this linear trend obtain effective robustness [1] exist-such models defy explanation by the wisdom that OOD accuracy and II)
*Corresponding author: sfkêeecs. bew.eeey...........ee..ee.eee.ee edu.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

accuracy should correlate strongly: they are "out of line". Identifying when and why such models arise is key to overcoming OOD brittleness in DL models.
Related research efforts have focused on improving OOD robustness via new techniques such as model
pruning [8], data augmentation [5, 21], J. and ensembling large pre-trained models [37, While these efforts have created significant excitement in the ML robustness community, it remains unclear (a) under which conditions on OOD data these methods can improve effective robustness (ER), and (b) which underlying model properties make a DNN effectively robust. A more in-depth understanding of these phenomena is likely to help in bridging the gap between ID and OOD performance.
Towards achieving this goal, we carry out a comprehensive empirical study leveraging models trained using each of the affremmnnioned ofate-of-hhe-art approaches to improving OOD robustness. We analyze the impact of these robustness improvement methods on various model architectures, two clean datasets (CIFAR-10 and ImageNet), and a broad range of natural and synthetic distribution shifts. In particular, we view the OOD robustness puzzle through a Fourier lens and ask how spectral properties of OOD data and models influence their effective robustness. In this process, we design new metrics that capture Fourier sensitivity of models as test data moves farther away from the training data manifold. We make the following contributions:
A new perspective on the state of OOD robustness: across diverse models, metrics, and distribution shifts, we observe that no one single metric rules them all, not even in-distribution accuracy. This accents the potential need for multi multi-faceted approach to understanding OOD robustness, contrast to ID generalization where single properties like model flatness can enjoy predictive success across an array of models [28].
Design of Fourier sensitivity metrics: our new metrics quantify various model properties, including spectral ones that strongly correlate with the OOD robustness of ensembled CLIP models [52]. explaining this robustness much better than ID accuracy.
A public collection of effectively robust models: to help crack the OOD robustness puzzle, we make available (at https: //s: //github..... attps://github. weithub.com//e cwwccomm/aaaaeeeee comm/arafridor 'RobustNets, RobustNets dataset of CIFAR- 10 models with effective robustness that stems from training under a variety of data augmentation and pruning schemes.
2 Related work
Distribution shift fragility is a well-documented phenomenon among neural networks [40, 41, 20, 19, 34] in which a model trained on an ID dataset performs markedly worse when evaluated on OOD data, even when humans find the OOD data just as easy to classify. These OOD accuracy gaps often follow linear relationship between ID and OOD accuracy, but the slope and intercept of the linear trendline varies depending on the specific ID and OOD datasets [34, 15]. In our work, we focus on two popular image classification benchmark ID datasets, CIFAR-10 [30] and ImageNet [6], and an array of OOD benchmarks for each.
While most models follow the linear ID-OOD trendlines of Miller et al. [34], some models are "effectively robust" and appear above the trendline, with OOD accuracy higher than expected given ID accuracy [1]. These models are quite rare, and thus far have only been shown to arise through a few training paradigms: by pruning a model as a sort of regularization [8]. by training with data augmentation intended to mimic the expected OOD test data [5, 21], or by pretraining on a larger and more diverse dataset and evaluating zero-shot [37], partially finetuning [1], or interpolating the weights of zero-shot and a finetuned model [52].
Although as a community we are beginning to uncover robust training methods, it remains a mystery why these particular methods achieve robustness on particular distribution shifts. We study this mystery via Fourier lens based on prior work involving both image frequency and finction frequency analyses. Ortiz-Jimenez al. [36], Yin et al. [S5]. and Sun et al. [47] study model sensitivity to Fourier perturbations of the input images and analyze how different data augmentations produce different Fourier sensitivities (and correspondingly robustness to image perturbations of different frequencies). Chen et al. [4] showed that exchanging amplitude and phase information between images (as a regularization scheme) has connections robustness as well. Another line of research focuses on spectral bias [13, 2, 54, 38], wherein models prioritize learning low frequency

---

functions over the input space. This bias towards low function frequencies can also affect robustness and is influenced by training hyperparameters like data augmentation and weight decay [13].
Among the model properties we study as potential predictors of OOD accuracy are ID accuracy [34], model Jacobian norm [24], the linear pixel-space interpolation metrics of Fridovich-Kell et al. [13], and our own Fourier amplitude and phase interpolation metrics. We study these metrics over wide range of robust and nonrobust models, including sparse models from [8] on CIFAR-IO and pretrained
models from CLIP [37, 52] and RobustBench [5] on ImageNet.
3 Methods
Datasets. We consider two standard image classification tasks: CIFAR-10 [30] and ImageNet [6].
For each of these "in-distribution" (ID) datasets we consider variety of "out of distribution" (OOD) datasets, both natural and synthetic. For CIFAR-10 we consider CIFAR-10.1 [40] and CIFAR-IO-C
[20]; for ImageNet we consider ImageNetV2 [41], ImageNet-C [19]. ImageNet-R [23], ImageNet-A [9]. ImageNet-Skeee Sketchhhhhhhhhhtt [50], and ObjectNet [10].
Models. Our CIFAR-10 experiments use 3 different model architectures: Conv8 [12], ResNet18 [17], and VGG16 [45]. For each model architecture, we consider a range of model pruning strategies that have been studied for their effect on OOD robustness [8, 25] as some of these pruned models were demonstrated to have higher OOD robustness than dense models. Namely, lottery-ticket style pruning methods [12, 42, 39, 7] were able to provide robustness gains on CIFAR-10- C [8]. We outline the various pruning techniques and sparsity levels we utilized in detail in Section 4.1.
In our ImageNet experiments we use 28 standard (nonrobust) pretrained models from Torchvision [33], pretrained models from the ImageNet-C leaderboard on RobustBench [5], and 33 models obtained by rohustness-enhancin finetuned CLIP ViT-B/16, ViT-B/32, and ViT-L/14 [37]. More details on the ImageNet models we use from Torchvision and RobustBench can be found in Section A.I.
weight-space interpolation [52] between zero-shot and ImageNet-
metrics: ID accuracy, Jacobian norm, pixel interpolation. Because OOD robustness is a complex and multi-faceted problem, we make use of multiple previously-propen model measurements and apply them to study effectively robust models. To the best of our knowledge, our work is the first application of most of these metrics (except for in-distribution accuracy) to the study of effective robustness. Specifically, we evaluate four previously prooposed metrics that have been observed to correlate with some notion of generalization: accuracy unseen in in-distribution data ("ID accuracy") [34]. Jacobian norm [24], and within- and between-class pixel interpolation high frequency fraction [13]. These latter metrics analogous to the high frequency fractions compute for amplitude and phase interpolation (described below), except that interpolating paths simply interpolate in pixel space between two images, where the two images are from either the same class ("within-class") or different classes (" ("between-class") [13]. Prior work observed that in-distribution generalization typically improves as within-class high frequency fraction decreases and between-class high frequency fraction increases [13]. Reducing the norm of the Jacobian of model outputs with respect input data can push the decision boundary away from the training points, providing robustness to random perturbations of the input data [24]. Thus, models with smaller Jacobian norms may perform better on corrupted/shifted data. We estimate the norm of the Jacobian using a random- projection- based approach [24]; further details can be found in Section A.4.
Fourier interpolation metries. We also introduce novel Fourier interpolation metrics to capture more information about model behavior that may help answer the robustness puzzle. Inspired by the pixel-space interpolation procedure of Fridovich-Kan Keil et al. [13], we propose evaluating the smoothness of model's predictions along image paths that perturb the Fourier amplitude or phase information of one test image towards another, while preserving the Fourier phase or amplitude (respectively) information of the original image. The intuition for this type of interpolation is that most of the semantic content of an image is contained within its Fourier phases, which encode structural information like edges. Accordingly, we can perturb the Fourier amplitude of an image without destroying its semantic meaning, and we can reliably destroy semantic content by perturbing Fourier phase. Example Fourier amplitude and phase interpolating paths on ImageNet are shown in Figure I; example paths on CIFAR-10 and further computational details are included in Section A.3.
3

---

Figure 1: Example Fourier amplitude (top) and phase (bottom) interpolating paths from ImageNet. Each path includes 100 images; every 15th image visualized here. The first image along each path is an unaltered image from the original validation set; the last image has the same Fourier phases (top) or amplitudes (bottom) as the original but has some of the Fourier amplitudes (top) or phases (bottom) of random other image from the validation set. Amplitude interpolation preserves semantic content across usually the entire path, whereas phase interpolation destroys semantic content more rapidly. Note that the human visual system is robust to both types of corruption to some extent (see [5]]..
More precisely, we compute an interpolating path by first randomly selecting two images Xy and X from the IDD test/validation set. We perform standard preprocessing on each image, which involves cropping to a standard size (for ImageNet) and normalizing each pixel by the training mean and stan- dard deviation (for both CIFAR-10 and ImageNet). We then compute the two-dimensional discrete Fourier transform (DFT) of each image and separate the complex-valued results into amplitude and phase components a, and P; for i {0, 1]. To construct an image along an amplitude interpolating path, we retain the original phase Po and interpolate the low-frequency amplitude as (1 - A) + Xa (while preserving the high frequency amplitude), where À E [0. 1]. and produce an image via the inverse discrete Fourier transform. Phase interpolation is defined analogously; we retain the original amplitude and interpolate the low-frequency phase. We interpolate the lowest 40% of image frequen- cies for both amplitude and phase on CIFAR-I01 10, the lowest 20% of image frequencies for phase on Im- ageNet, and all image frequencies for amplitude on ImageNet. We choose 100 evenly-spaced À values to define each path, and choose 5000 random paths for CIFAR- 10 and 7000 for ImageNet. The images in Figure are rescaled for visualization (essentially undoing the initial pixel-wise normalization).
We evaluate each model on each path, producing a probability vector over the available classes for each image along the path. We compute two metrics based on these path predictions: high frequency fruction (HFF) and consistent distance (CD). For high frequency fraction, we take the one-dimensional DFT of the 100 model predictions along the path, average the Fourier amplitudes among all the classes, and compute the fraction of the total amplitude above a frequency threshold (10% of the maximum fireuueccy.. This metric is always between 0 and (usually between 0.1 and 0.3). The higher the average high frequency fraction, the more sensitive model is to the given Fourier perturbation of its input images.
We also consider consistent distance, which is defined as the index of the first image along the path that the model classifies differently (by highest softmax prediction) than the original image. This metric is always between and 100, and is often at least 40. Consistent distance is intended as more intuitive metric to capture the same notion of robustness corruptions of image Fourier content. In particular, Fourier amplitude corruptions change image statistics but do not hamper human semantic identification; a more robust model in this sense should have lower high frequency fraction and a higher consistent distance on average compared to a less robust model.
4 Results
We are interested in understanding when (under what conditions on the distribution shift) and why (via model properties) neural networks can exhibit effective robustness. Although our experiments are correlational rather than causal, they provide evidence for and against various hypotheses regarding why effective robustness might emerge. We hope that future work will make use of our proposed

---

metrics and released model database RobustNets to answer the causal questions raised by our experiments, perhaps following Dziugaite et al. [11].
We perform a systematic study of three model interventions that have been shown to impact OOD robustness: pruning [8], data augmentation [5, 21J, and weight ensembling [52]. For each intervention, we evaluate benchmark set of pretrained CIFAR-I0 10 and/or ImageNet models on the original in-distribution test set, several OOD test sets, and a suite of model property metries capturing local smoothness (via model Jacobian norm), frequency response to pixel-space interpolation within and between classes [13], and Fourier amplitude and phase sensitivity (via high frequency fraction and consistent distance). Representative results from our analysis of these three interventions are presented in Sections 4.1, 4.2, 4.3, and 10 For each dataset we consider a mixture of natural and synthetic distribution shifts: CIFAR-10, [40] and CIFAR-10-C [20], and ImageNet- V2 [41], ImageNet-R [23], ObjectNet [10], ImageNet-A [9), ImageNet-Skench [50], and ImageNet-C [19]. Since CIFAR-1O-C and ImageNet-C: are each composed of 15 synthetic corruptions ranging from low to high frequency, we include results on representative corruptions of each frequency band in the main text and defer results on the remaining corruptions to the appendix. In all figures, we show 95% confidence intervals around each measurement (Clopper-Pearse) for accuracy measurements and Gaussian bounds for averages of other metries).
4.1 When and why does model pruning confer effective robustness?
Diffenderfer et al. [8] demonstrated the nuanced behavior that model pruning has on OOD robustness, specifically with respect to the distribution shift from CIFAR- 10 to CIFAR-10-C. Notably, pruned models are capable of providing superior OOD robustness compared to dense, or unpruned, models. Furthermore, the effect, either positive or negative, and degree of robustness arising from model pruning is dependent on both the model architecture and the pruning algorithm. It remains unknown which properties of these pruned models contribute to their robustness on OOD data.
In an effort to better understand these findings, we further investigate this behavior by considering the three categories of pruning used in Diffenderfer et al. [8]: traditional [16, 57], rewinding lottery- tickets [12, 42], and initialization lottery-tickets [39, 7]. Note that we often abbreviate "lottery tickets" as LT. We provide details on each of the pruning techniques in Section A.2. For each pruning strategy and each architecture, we prune models to 50% 60%, 80%, 90%, and 95% sparsity. For all pruning methods, pruning is performed in an unstructured (i.e.. individual weights are pruned) and global manner (i.e.. prune to a given sparsity across the entire network). Additionally, for traditional and initialization LTs, pruning was performed in a layerwise fashion (i.e., where each network layer is pruned to the given sparsity level). In Figure 2 we investigate when (on which types of distribution shifts) and in Table I we study why (via corresponding model property measurements) these differently-prunte CIFAR- 10 models achieve their varying degrees of robustness.
Because different model architectures respond differently to different pruning methods, in each subplot of Figure we color-code models by their architecture and fit a separate probit-domain OOD vs. ID accuracy regression line to each architecture, so that each regression captures range of different pruning amounts (what fraction of weights are pruned). We display these regressions with opacity proportional to the fit quality R², so that regression lines that fit the data well are dark and easily seen, while lower-quality linear fits are more transparent. In the corresponding Tables 1, each reported slope (m) and R22 value is the average of the corresponding values among the three architectures we consider. For ease of reading, we bold average R2 values above 0.5, and among these entries highlight the best (bright green) and second- pruning method on each OOD dataset. Although these results are nuanced and raise additional open questions, we summarize few key takeaways from our CIFAR- 10 model pruning experiments:
(faded green) R values for each
As reported in Miller et al. [34], ID accuracy is often a strong predictor of OOD accuracy; this trend holds across different pruning methods. However, the linear trend breaks down for mid and high frequency image corruptions like pixelate, Gaussian noise, and impulse noise.
Even for low-frequency corruptions, different model architectures might lie on either the same or different robustness lines (as evidenced by comparing brightness and contrast).
When trained without data augmentation, higher effective robustness is present on several mid- and high-frequency corruptions by non-residual architectures (i.e. Conv8 and VGG16) (see Figures 2
5

---

and 8). However, when training with data augmentation, less variation in effective robustness is realized by different architectures (see Figures and 9).
R2"
43
Traditioral
Rewindding IT
Inttialization IT
0.55
-00
016
-840
08.3
0.0
0.13
CIIAR10 (N6)
0.80
85
Aap
Arpp 187
4.11
672
-50
16311
-6.75
0.33
73
77
Phane CD
0.33
033
300
0..30
6422
04
89
50
50
80
85
90
95
50
85
50
95
80
85
90
ID acceray
14
1036
0.5
11:9
01:37
07
97
Juctbe
0.2
-001
Brighnees (%)
Pagl betwoan
9.33
0.31
0.5
0.34
0.32
Asp--88
-873
0.7
-647
0.55
-3.9
8.00
App
Pleac- CD
0.08
0.27
002
0.23
0.02
Austina
-0.0
0.15
.0.01
0.4
0.0
0:25
Contres (9))
3
Rac- waliis
.5.99
8:74
-4.15
0.64
-7.4
0.81
.6.33
Amp CD
0.6
0.65
0.04
0.54
005
0.83
39
Pesso HFF
Plaae
CU03
6883
0:36
-9.17
0.02
0.3
-7.18
50
80
85
90
95
30
83
95
50
85
90
Datccu' Blur (N)
888
0.95
0.51
0.86
0.61
-.54
Raci betwaa
5.35
0.35
0.52
0.35
7.54
18
Atep- HPP
4.01
0.42
-3.96
0.43
-2.79
0.23
Amp-CCD
17
72
HFF
4.76
Plac- CD
6.02
0.19
0.01
0.37
.0.0
6.12
0.73
Plae (5a)
84
Rel sitin
Taodkas
.6.8
.00
0.66
0:11
40.0
-8.09
16.
3.67
0.00
0.13
28
76
13
Paa HIF
Ang CD
0.00
0:31
0:36
38.95
0:05
0.63
11:11
-0/77
0.45
90
85
95
85
90
Gaussis in Nooe (%)
84
84
accenac
Noke
0.06
Poul witii
-6.17
0324
-3.00
0.14
16.5
6.30
Paul beraaee
7.68
0.11
50
Asp-CD CD
6.01
0.12
0.89
6.17
-0.06
0447
Phac- HRF
-8.3
0.18
-11.0
0.23
1608
Imnultse Nabse (9W
Facabian
0.11
0.5
-1.0
E:
Risd widke
-1.26
0.10
1.31
18.0
5.67
033
baween
3.1
-17.81
-3.2
CD
-1.0
-02
01.0
-010
D.56
Haiz: HFF
4.05
-7.5
0.41
8.61
0.43
42
9C
CPAR-10 Text Accuracy (%)
CHAR-11 Text Ascunaty (%)
EIAR 10 Test Accurecy (54)
Figure 2: When does pruning affect OOD robustness?
Table 1: How well do metrics for pruned models correlate with OOD bustness?
Initialization LT pruning confers robustness to high-frequen figh-frequeccy corruptions and can induce negative correlation between ID and OOD accuracy for these corruptions.
Jacobian is rarely a good predictor of OOD accuracy, but it does correlate negatively (lower
norm is better) for models pruned by rewinding methods on mid-frequency corruptions.
As reported Fridovich-Keil et al. [13], lower high frequency fraction HFF (i.e. smoother predictions) for within-class pixel interpolation is good predictor of OOD accuracy, at least for low- low-freque fww-freyynccy distribution shifts where ID accuracy is also predictive.
Amplitude and phase interpolation are sometimes predictive of OOD accuracy, particularly for initialization LT pruning, and in the expected direction of correlation (a better model should have lower HFF and bigher CD). However, even these metries fail to explain most the robustness behavior for high (and some mid) frequency distribution shifts CIFAR-10.

---

Please see Figure 10 for visualizations of the data fit associated with the most predictive metric for each distribution shift and pruning approach.
4.2
When and why does data augmentation confer effective robustness?
Cless
Asglis
Clean
Augmixx
Gaussian
93
53
93
ID acermcy lacthoo
1:41
8800
0.50
CIFAR10.1 198)
84
64
84
Ptal: betuso
6.55
0.45
Arrp 1337
-5.3
1.79
0.83
633
4.88
8.71
03
69
69
Arrp-
Phose LD
0.29
50
80
85
90
95
50
88
85
90
95
50
183
85
90
92
brightness (95)
07
67
07
6.09
0.0
011
0.01
0.31
000
93
93
93
Bigghoee
Paui watla
4.36
8:91
0.51
6.74
187
8.76
84
84
84
Amp CD
001
8.78
0.06
0.7
0.
0.15
89
80
85
35
80
85
90
05
80
85
93
07
67-
07
0:8
Contraxt (76)
03
Jaodkas
Radi wilher
-988
0.79
-5.70
-3.22
Rad- betwaee
11.35
0.5
12.76
8.75
3.12
016
64
64
Aupp-
02
69
69
Mase HPF
-9.0
0.44
5.98
0.11
0.35
603
Plase CD
6.03
8.53
0/02
0.12
0.02
0.22
180
25
93
1h
181
25
90
182
25
90
07
57-
077
ID acceracy
0-47
0.34
6.68
1139
0.7
IU6
Defocus Blur
03-
03
03-
Defice Sixe
0.36
-31
0.12
Porl: betownn
0.34
9.5
0.76
5-13
0:11
Arpp IPP
-.29
0.13
1.19
0.55
-.19
84
84
84
Axpp
0.36
6166
0.67
Phose CD
0.56
69
69
10
25
95
181
25
60
191
182
255
V3
9.3
1.3
acurray
Piwelat (16)
89
89
Jacatun
0.13
1654
4103
0.14
64
84
Poolac
Paal: watts
Paal: between
-1.12
3.03
0.34
0.16
-3.7
0.39
0.49
4.33
531
0.3
77
77
Aop-
CD)
6.54
60
69
Plosn HEF
54
0.7
7.3
0.5
5.78
01
39
59
Phoso
0.35
0.01
0.58
083
0.55
95
181
25
90
182
Geussian Noise 195)
47
47
47
ID Paaearacyy
-1.33
0.33
0.68
0.11
0.38
891
84
64
84
Conmara Noae
Jaodsas
Rid bawwees
-13.6
0.2
-034
1.32
6.57
50
50
50
Aug- HFF
Aup CD
369
0.2
1.26
008
-3.74
6.79
15
Mlacc CD
4069
0.2
0.02
0.00
025
80
85
90
95
10
85
90
93
18
85
90
93
93
93
93
VSGL6
-C.6
0.26
0135
6A7
Inpulse hatee
Foncl
ReshatLB
Lagabe Nooe
Justban
vitia
-011
08.7
0.1
0.13
-00)
8:1
01
Miller ctal. 2021
HPF
3.86
-3.5
6.1
50
50
50
App- CD -000
0.31
100
0.2
0103
8.55
30
80
25
90
35
30
80
85
90
55
30
80
25
90
93
CIFOR-10
Eneuney (96)
FIRAR-10
Arnursy (55)
Arr aney (9))
Table 2: How well metries for data- Figure 3: When does augmentation affect OOD robustness? augmented models correlate with OOD robustness?
common practice protect against distribution shift is train with data augmentation [S, 21J. Accordingly, we revisit each of the CIFAR-10 models considered in Section 4.1 with three different variations of data augmentation used during training: augmentation (clean), AugMix [21], and Gaussian noise augmentation [29]. In Figure 3 we investigate when (on which types of distribution shifts) and in Table we consider candidate explanations for why (via corresponding model property measurements) these As in Figure 2 and Table we fit separate regression line to each model architecture and average the resulting slope and R² values. The models that contribute to each regression line therefore share an architecture and an augmentation strategy, but vary by pruning method and amount. Our findings in this experiment echo those in Section 4.1, I, but we make the following additional observations:
CIFAR-10 models achieve varying degrees of robustness.

---

The closer the match between the training data and the OOD data in terms of their spectral profile (see Section A.5 for visualizations), the better ID accuracy, within-class pixel interpolation, and amplitude interpolation HFF and CD are at predicting OOD accuracy. This is evident from the high R2 values for these metrics when predicting defocus blur accuracy for models trained with AugMix, and when predicting Gaussian or impulse noise accuracy for models trained with Gaussian noise.
The closer the spectral match between the training data and the OOD data, the better the OQI) accuracy. This unsurprising result is evident from the higher accuracies of AugMix-trained models on contrast and defocus blur corruptions and the higher accuracies of GGaussan-nnose--ree models on Gaussian noise and impulse noise corruptions.
Please see Figure 11 for visualizations of the data fit associated with the most predictive metric for each distribution shift and data augmentation approach.
4.3 CLIP: Weight ensembling a robust model on ImageNet
Perhaps the most stunning example of OOD robustness for image classification is CLIP [37]. which showed that a large model pretrained achieves effective robustness on many OOD test sets simultaneously. Andreassen et al. [1] found that finetuning these robust models towards ImageNet improves ID accuracy on ImageNet but erodes effective robustness, and Wortsman et al. [52] offered a "best of both worlds" solution by interpolating in weight space between a pretrained zero-shot model and its finetuned counterpart, improving ID accuracy on ImageNet without sacrificing robustness. Accordingly, we study a set of CLIP models obtained by interpolating the pretrained and finetuned versions of three CLIP ViT models (B/16, B/32, and L/14) [37, 52] with various relative weights on the two models. We compare this CLIP model set benchmark set of 28 standard (nonrobust) ImageNet pretrained models from Torchvision [33] as well as 5 pretrained models from the RobustBench [5] ImageNet- leaderboard; these models typically use data augmentation to improve corruption performance but are not nearly as effectively robust as the CLIP models. We make the following observations in Figure 4 and Table 3:
massive dataset and evaluated zero-shot on ImageNet
Although in-distribution accuracy is a reasonable predictor of OOD accuracy for most models and distribution shifts, it performs markedly worse when predicting OOD accuracy for CLIP models (and a couple of effectively robust RobustBench models on some OOD shifts).
Fourier interpolation metrics for both amplitude and phase interpolation are strong predictors (often R²2 > 0.9) of OOD performance for CLIP models across OOD shifts. This suggests a hypothesis for why CLIP models are more robust: they are less sensitive perturbations of Fourier amplitude and phase [51], which preserve image semantics for humans but confuse nonrobust models.
We also consider the ability of these model metrics to predict robustness in ImageNet models trained with various data augmentation approaches, which can produce effective robustness many of the distribution shifts we consider [5]. Figures 5 and 13 show the most predictive metric's performance for these non-CLIP models from Figure 4. We again find that, although no single metric is best across all distribution shifts, our proposed Fourier interpolation metrics are often most predictive, outperforming in nn-distribution accuracy. See Appendix A0 for full results and discussion.
5 Discussion
Our work is only the beginning of a full understanding of what makes models effectively robust to distribution shifts, starting with the domain of image classification where some models have demon- strated effective robustness. For example, Scction 4. showed diffcrent types of model pruning can confer robustness to different distribution shifts, and while some of this robustness can be explained by improved behavior in response to amplitude and phase perturbations, much remains an open mystery.
As we saw in Section 4.2 (and has been noted by prior work [55, 21I), models can achieve robustness to some distribution shifts by training with data augmentations designed imitate the expected OOD data. This approach can be powerful way to mitigate expected test-time distribution shifts but leaves open the question how to prepare for the unexpected. We learned in Section 4.3 that more robust CLIP models are less sensitive to perturbations of Fourier amplitude and phase, perhaps making them more attuned to human perceptions of semantic meaning. This finding provides an exciting opportunity for future work to confirm our results with both additional out-of-line models and causal
8

---

experiments; and to incorporate our Fourier interpolation metrics into training paradigms, for instance as regularizers or data augmentation strategies, to explicitly encode robustness in future models.
All
CLIP
Nea-LIP
m
76
ID maaceeg
0:93
0.85
8:96
0.91
Imagjeha
Ptal-vithe -yibbn
-3975
0.83
-8.58
0:38
-035
6392
50
57
65
Piud bete-n
Anp
8.83
0.2
0:75
0.00
1001
9.6
14.21
0:00
6:73
50
Plano HRF
.83
1.50
16
6:71
19465
0:47
ImageNet Val. Accuracy ImageNet Val. Accuracy
Consistert Distances
Phac-
0G22
0.75
0..00
0.54
0.14
6.57
0.77
Inageta
Rad walbi
18.17
0.76
18.9
0.57
-7.95
0:55
6,3
15.9
20.03
Angp CD>
000
6.75
003
69
94
50
60
B4
50
0.22
0.24
0.26
-304
-17.34
800
1544
Imagevst Val. Art mray (%) Imagetet Val. accurary (%6
High Freq Frans Phase
Cihjectorr (86)
1.76
0.8
8.56
0.0
0.75
0.80
1001
Pial berwae
16.44
8.14
78.46
Arpp- Map-399
-19-2
0.68
2013
6.11
0.41
0.0
1EF
-9.03
13440
74
6
04
60
30
50
Mascc CD
08.
6012
0296
0,25
0.55
Imagetet Val. Accuracy (8) magelett val, Accuracy (%) Consistent Distances Phase
Imagetet (46)
30
accracy
3.14
6:36
1.6
01:13
0.81
011
50
Padi videe
-0099
0.26
03335
0.35
-9.00
0.15
Padi- boowazs
41.20
806
13:21
0.83
18.87
0.29
CD
0.68
8.74
0.55
0885
0.15
15
15
Paax- RPF
-30
8.53
-23.0
$2.35
0.36
052
0.06
0.34
ImageNet Val. Accuracy
ImageNet Val. Accuracy
Fracs Phase
Inagetur PS)
8 i v
à 1 V
accuracy
0.0
-6.0
30
Bad eiiia
-1434
0.36
-201
0166
-8.15
15
-7.76
-18.2
Arg -CD
0022
6.89
0.54
Iteue
84
60
84
0.22
0.24
0.26
0.55
ImageNet Vol. Anccy (%)
Imagefelt Val, Accuracy (%6)
High Fres Fracs Phase
Contrest PS1
ID Duuuuryy
1,45
0.62
0.86
0.5
1.26
8.6
50
8 5
15
4.0
Pad bdwace
14.09
8.43
24.5
Arpp HPF
-2235
8.6
-119
-36
Phase -1F
-38.6
-11.57
8.55
25.25
1.22
69
94
69
0,200
0.225
0,250
Maac .C
0003
0.72
0.00
8.94
007
0.59
Imagetet Vol. AcC ancar (8) imagelet Val, Accuracy [9)
High treq Fracs Amp
Deloxus ur
50
8 5 9 8 3
ID auneyy
1822
0.5
8.64
1.30
30
5 18
Radl: bntaice
Rad. wilis
-8383
6621
8.6
1.26
8.88
0.33
-14 11
34.5
0.78
8.1
15
Arep-CD CD3
0025
Paac H0F
-1.44
00.3
10..0
33.53
ImageNet Val. Accuracy
ImageNet Val. Accuracy
High Freq Amp
Goussiar Nows (Ne)
5o-
it i8 v5
Jastia
0.0
0.1
-60
25
Stencard Batch
Padl vitais
-13.0
0.51
-8007
8.56
-1675
6.51
-15T1
-0.62
CUP
Aipp CD
003
8.03
0.00
Phanc
-11.6
0.24
35.45
0.30
AM
0 200 5.9 n P90
ImageNet Val. Accuracy (96
ImageNet Val.. Accuracy (96)
High Freq Fracs Amp
Figure 4: When does CLIP ensembling affect OOD robust- ness?
Table 3: How well do metrics for CLIP models correlate with OOD robustness?
Our results provide new perspective on the effective robustness problem by showing that no one single metric rules them all. This dictates the need for a multi-faceter analysis to understand OOD robustness, which is in sharp contrast to the in-distribution setting where a single property such as model flatness can reasonably explain generalization performance.
RobustNets benchmark. The robustnesses of our analyzed CIFAR-10 10 models remain out- line. To track and aid progress on effective robustness prediction, we release our code as well as a database of our model weights and metric values at https: //gts: thub..u..... com/ http://saaaarooooooooo/ooooooset wow.ddootootttt....ooooooo The models we study on ImageNet are publicly available.

---

224
214
0:94
0.85
8:96
0.91
Ryjus/Bewh
Jacdaa
2M
0.41
0.4
0.5
Pial- vataa
-2075
-8.3
8.38
.553
0%
4%
8
Anp 18F
-701
0.63
-186
0:81
0.0
Imagehu
52
Are
082
837
0.00
0:
Phace CD
0.22
0.75
0.50
0.34
0.14
30
<
M
50
77
69
ImageNet Val. Accuracy (%)
ImageNet Vol. Accuracy (96) Imageket Val. Accuracy (h)
ID IDuaare
1.2
014
002
0.31
0.77
Imagatat (Ng
6.75
4.0
Had -hteece
24.33
8.76
6.33
038
150
0.56
Axgp
-1.36
-3144
27
Maac 2PP
+17.4
1554
0,55
Maaic CD
646
8.76
003
6.57
0034
0.55
15
B4
50
17
12
to Consistent Distences Amp
ImageNet Val. Accurecy (1))
ImageNet Val. Accuracy (16)
accumaay
0.29
1.38
Posl -iibls
16.01
0-47
0.41
-33.3
1457
Objecthul
15
0.75
88
C3
696
000
0.88
0155
0,97
Phusa
-0.03
-3..0
6.96
<>
50
17
59
6
ImagaNat Val. Accuracy (%)
Imageett Val. Accuracy (96)
Imagekot Vol. Accuracy (1)
Imagatez (S)
Jucetian
3.34
0:36
8:53
881
1.6
01:12
8.7
0.35
0.0
0.15
0.0
B g
0.26
Arp- HIT
-6677
0.4
13.00
1885
-1.2
0.0
Asp-CC
0125
Pane- CTD
0.06
8.71
0.94
0.55
0/0
59
7)
0.26
ImageNet Val. Accuracy (96)
Imagenen Val. Accuracy 196)
High Friq tracs Betwaen
ID accuracy
1.55
0..
0.78
0.52
0.53
15
0.73
-00
Rad betnnce
1727
890
1.17
0.25
19.03
M37
Imagehaa
-15.4
-7.28
-18.5
1
Plane HFF
-0123
6.40
11..3
28 73
0.33
Place
59
to Consistent Distances Amp
Imagetet Val. Accurasy (1)) Inr areeeet Vol. Accuracy 190
92
Contrast 196)
Dracare
-0.0
8 8
Paad nitls
-5445
0145
-1.5
0.59
-12 12
1L45
13
17
Arep CD
1423
0.02
1.53
-11.57
0t Consistent Cistances Amp
ImageNet Val. Accurscy (1)
ImageNet Val. Accuracy 196)
Delocua Blur PS)
1.90
g
52
0.0
1443
100
0.67
4.0
001
Rail berawe
01
34.5
12
Arsp H17
-20.1
0.57
-3.24
-57.03
Assp-CCD
10.09
13.63
59
17
2s Consistent Distances Amp
Paoor (3)
0.07
8.7
0.00
8.00
6/7
0.61
ImageNet Val. Accurecy (16)
ImageNeet Val. Accuracy (36)
Gaussien Noose (N)
1.40
8.60
0.5
8.66
1.3
666
18 2
30
Jastin
PocustBanch
Pool Netwer
Anp 130
13.12
-0311
0.41
8.65
8.000
-537
0-15
33.05
ImagaNat Val. Accuracy (96 Imagehet Val. Accuracy (9)
1
Condistant Distances
Paas- HFF
Phac- CD
-2..6
0.33
0.44
0.5
-8.19
001
6.5
0.5
55.45
6.72
0.20
Figure 5: When does ImageNet data augmentation affect OOD robustness?
Table 4: How do metrics for ImageNet
models correlate with OOD robustness?
Acknowledgments
We would like to thank Jonathan Frankle, Mitchell Wortsman, and Ludwig Schmidt for helpful discussion and pointers to resources. SFK acknowledges support from NSF GRFP. This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC55-07NNA and was supported by the L.NNL-LDRD Program under Project No. 2022-S1-004 (LLNL-CONF-8855))
10

---

References
[1] Anders Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roclofs. The evolution of out-of- distribution robustness throughout fine-tuning, 2021. URL https:://aaxi...rr//////////////
aba////..////./////// 15831.
[2] Ronen Basri, David Jacobs, Yoni Kasten, and Shira Kritchman. The convergence rate of neural networks for learned functions of different frequencies. 2019. doi: (048850/ARXXI111966004222 URL https:
// //arxiv. wwtpiii.oog//baae argg/66/1906..... 1906.00425.
[3] Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K Varshney, and Dawn Song. Anomalous example
detection in deep learning: A survey. IEEE Access, 8:132330-13233 2020.
[4] Guangyao Chen, Peixi Peng. Li Ma, Jia Li, Lin Du, and Yonghong Tian, Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain. CoRR, abs/2108.08489
2021. URL https://arxxx org/ att/aaas///// 2ttp./////..../ 20880088888888....
[5] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robusthench: a standardized adversarial robustness benchmark, 2020. URL. https: //ariiv.////v///////e/a org/ aa/a/aaa/01...0 /ttp//000..0007.....
[6] Jia Deng, Wei Dong. Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.
Ieee, 2009.
[7] James Diffenderfer and Bhavya Kailkhura. Multi lottery ticket hypothesis: Finding accurate bi- nary neural networks by pruning a randomly weighted network. International Conference on Learning Representations, 2021.
[8] James Diffenderfer, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. A winning hand: Compressing deep networks can improve distri-iitt robustness. In Advances in Neural In- formation Processing Systems, 2021. URL https: //poocedin//////gggg/ wwt.igs.nerrr.... hww..pp.ccpppppe//2222000000 222/1/ie/02// pdf.
0607f4c705595691 1a4f311112444444eeee1ee
[9] Josip Djolonga, Jessica Yung, Michael Tschannen, Roh Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver. Matthias Minderer, Alexander Amour, Dan Moldovan. Sylvain Gelly, Neil Houlsby, Xiaohua Zhai, and Mario Lucic. On robustness and transferability of convolutional neural networks. CoRR, abs/2007.08558, 2020. URL https: : //txx/aaxvvv///////////////aa.la httgg/bs/2000 00w000000000.....
[10] Josip Djolonga, Jessica Yung. Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D' Amour, Dan Moldovan, Sylvain Gelly, Neil Houlsby, Xiaohua Zhai, and Mario Lucic. On robustness and transfcrability of convolutional neural networks. CoRR, org/ hb////ba//// 2007. 08558.
abs/2007.08558, 2020. https:tttt///ttt
[11] Gintare Karolina Dziugaite, Alexandre Drouin, Brady Neal, Nitarshan Rajkumar, Ethan Cahallero, Linbo Wang. Ioannis Mitliagkas, and Daniel M. Roy. In search of robust measures of generalization. CoRR, abs/2010.11922 2020. URL https://arxix org/
11924.
[12] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arviv preprint 1rXiv:1803.0363. 2018
[13] Sara Fridovich-Ken Keil, Raphael Gontijo Lopes, and Rebecca Roelofs. Spectral bias in practice: The role of function frequency in generalization. CORR, abs/21 10.02424, 2021. URL http::t/aaaatetee org/abs/ .2424..
[14] Robert Geirhos, Patricia Rubisch, Claudio Michaclis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained cmns are biased towards texture; increasing shape bias improves accuracy and robustness. CORR, abs/1811. 12211, 2018 URL http:// org/ wttgg/abs/8811. 12231.
[15] Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Predicting with confidence unseen distributions, 2021. URL https:
orgo htg//abs21007 003115
[16] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural networks. arXiv preprint arXiv:
2015.
[17] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
[18] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CoRR, ahs/1512.033355, 2015. URI. http::/ttaa/ii...o//// http:://ariiv.../aaaa/////
[19] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. 2019. URL
org/abs/ haab.//9/...... 12261.
[20] Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019, URL /ttttss//oppnreeeeee net net/forum? id-HJz6tiCQYm.

---

[21] Dan Hendrycks, Norman Mu, Ekin Dogus Cubuk. Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method improve robustness and uncertainty. In International Conference on Learning Representations. 2019.
[22] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song. Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of -ustof-distiiuttion generalization. CoRR, abs/2006. 16241, 2020. URL https://arxiv. org/ abe/ 2206. 16241.
[23] Dan Hendryeks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. CoRR, ftss/2006.12411, 2020. URL
attps://arxiv. org/ aba/ 220000000.0.. 16241.
[24] Judy Hoffman, Daniel A Roberts, and Sho Yaida. Robust learning with jacobian regularization. arXiv preprint arXiv: 02729, 2019.
[25] Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome. What do compressed deep neural networks forget? arXiv preprint arX1v:1911.05248, 2019.
[26] Gao Huang. Zhuang Liu, and Kilian Q. Weinberger. Densely connected convolutional networks. CoRR, abs/1608. 106.60088..6..000 2016. URL http: wwtp://..rrgab//////oe/ 069993.
[27] Forrest N. landola, Matthew W. Moskewicz. Khalid Ashraf, Song Han, William J. Dally, and Kurt Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <Imb model size. CoRR, abs/1602.07360, 2016. URI.
[28] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic general- ization measures and where to find them. arXiv preprint arXiv: 2019.
[29] Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion. On the effectiveness of adversarial training against common corruptions. arXiv preprint arXiv:2103..22325, 2021.
[30] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
[31] Alex Krizhevsky. One weird trick for parallelizing convolutional neural networks. CoRR, abs/1404.5997,
2014. URL http://arxiv, . org/ ab//14044. 1444. 5997.
[32] Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet V2: practical guidelines for efficient CNN architecture design. CoRR, abs/1807.11101 2018. URL /ttp////vvvv/v////////ee/e//a org/ haaaabba//81 1807/ 11164.
[33] Sébastien Marcel and Yann Rodriguez. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM International Conference on Multimedia, MM '10. page 1485-1488, New York, NY. USA, 2010. Association for Computing Machinery. ISBN 9781605589336. doi: 10.1145/187395111874455
URL https: //tp:.///oooo/oo///// http://ooooo////////////// 10. 11155/18739511.772555.
[34] John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang. Yair Carmon, and Ludwig Schmidt. Accuracy on the line: On the strong correlation between out-of- distribution and in-distrihution generalization, 2021. URL :tttpp///arxvv... orgg/ba//////eeee ab2/2107 .04649.
[35] John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In International Conference on Machine Learning. pages 7721-7735. PMLR, 2021.
[36] Guillermo Ortiz-limene,, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Hold me tight! influence of discriminative features on deep network boundaries, 2020. URL https: // /arxiv. . org/abs. httg//bbs2000...0
[37] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transfcrable visual models from natural language supervision, 2021. URL 2103.00020
org/abs/
[38] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral hias of neural networks. 2018. doi: 10.48550/ARXIV.118. 08734. URL https: http://aavvvvor//eeeeeae org/ aba/ 11101000660.4...
[39] Vivek Ramanujan, Mitchell Wortsman, Aniruddha Kembhavi, Ali Farhadi, and Mohammad Rastegari. What's hidden a randomly weighted neural network? In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 11893-11902 2020.
[40] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize to 102, 2018. URL https:/ haapss:///axxxii /tt///vvvogg/eeeeeeeee/.e abs/ /abe///0...... 00451.
12

---

[41] Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize imagenet?, 2019. URL https://aaxiii wwtp:/iiiorraaeeeeeee hargg/bb//te abs//9/2. 10811.
[42] Alex Renda, Jonathan Frankle, and Michael Carbin. Comparing rewinding and fine-tuning in neural network pruning. 2020.
[43] Mark Sandler, Andrew Howard. Menglong Zhu, Andrey Zhmoginov, and Liang- Ching Chen. Mobilenetv2:
Inverted residuals and linear bottlenecks. 2019.
[44] Vaishaal Shankar, Achal Dave, Rebecca Roclofs, Deva Ramanan, Benjamin Recht, and Ludwig Schmidt.
Do image classifiers generalize across time?, 2019. URL https:/ http///axvvvv/////////eee/eaaeva www.v.org/aaaae httg//b//1990 1906. 02168.
[45] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv 1409.1550 09 2014.
[46] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion, 2014. URL https: //twx//aavvvv///////a//eeeeawala wwwg/aa////// http://////ar....orraaa/a/////// 1556.
[47] Jiachen Sun, Akshay Mchra, Bhavya Kailkhura, Pin- Chen, Dan Hendryeks, Jihun Hamm, and Morley Mao. Certified adversarial defenses meet out-of-distribution corruptions: Benchmarking robustness and simple baselines. arXiv preprint arXiv:21 12.00659, 2021.
[48] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott
Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. CoRR. hts/1409.4842, 2014. URL http://arxiv arxiv. org/ aba/aaa/a//// hetp//49..4888
[49] Mingxing Tan, Bo Chen, Ruoming Pang. Vijay Vasudevan, and Quoc V. Le. Mnasnet: Platform-aware neural architecture search for mobile. CoRR, 0bbs/180..1666 2018. URL http://arxiv.. http:///aaxiv...g/b///////// 1807 11626.
[50] Haohan Wang, Songwei Ge. Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. Advances Neural Information Processing Systems, pages 10506-10518, 2019.
[51] Felix A. Wichmann, Doris I. Braun, and Karl R. Gegenfurtner. Phase noise and the classifica- tion of natural images. Vision Research, 46(8):1520-1555 1529, 2006. ISSN 0042-6989. doi: https:
!/doi.or// 101.. 1.008. URL https:/:/// htt.:tttt///w..ceeeeeiieeeee httpa///irreettccccccceeeee/eeee pi1//8002298995505544.
[52] Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Korblith, Rehecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi. Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models, 2021. URL https: / arxiv. wwwxiii.rg/aaaaaee ttg/abb/21099 /ttb:///99...993..
[53] Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transfor- mations deep neural networks. CoRR, abs/1611 1,04311,, 2016. URL http: org/ aba/ /161. 05431.
[54] Zhi Qin John Xu, Yaoyu Zhang, and Yanyang Xiao. Training behavior of deep neural network in frequency domain, 2018. URL https://arxii: wwwreiiiv.or/aaee abs// /11701177.001 01251.
[55] Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, and Justin Gilmer. A fourier perspective model robustness in computer vision, 2019. URL https://arxiv. org/ abs//906.. 08988.
[56] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. CoRR, abs/1605.07 07646, 2016. URL http://arxiv wwwrxiv..or/aaae abs/1605. 07146.
[57] Michael Zhu and Suyog Gupta. To prune, or not to prune: exploring the efficacy of pruning for model compression, 2017.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Limitations are included in the discussion section.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
13

---

(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proof's of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes] Code and pre- cow/sarafrideeeeooooocooo /Robus tNets. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Most of these are provided in the methods section, and some are in the supplement.
trained models are available at htap:::///ttii
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes] We use 95% confidence intervals in all of our figures.
(d) Did you include the total amount of compute and the type of resources used (c.g.. type of GPUs, internal cluster, or cloud provider)? [No] Our experiments did require substantial compute to evaluate all of our models on all of the interpolation paths, but we use pretrained models so we do not require any compute for training.
4. If you using existing assets (e.g., code, data, models) or curating/releasing
assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We cite the creators of all the models and datasets we use.
(b) Did you mention the license of the assets? [No] Licenses for individual assets can be found by visiting the websites of the corresponding creators.
(c) Did you include any new assets either in the supplemental material or as a URL? [No] We release our RobustNets database of pretrained models evaluated in this paper.
(d) Did you discuss whether and how consent obtained from people whose data you'l 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "mW-8Q0AiPk", "reviewer": "Reviewer_YyCi", "summary": "This paper carries out a comprehensive study of the Distribution Shift Robustness through a Fourier lens and designs new 'Fourier sensitivity metrics' that capture Fourier sensitivity of models as test data moves farther away from the training data manifold. It provides the theoretical analysis of the OOD robustness on pruning, data augmentation, and weight ensembling.", "questions": "The questions are listed in the pros and cons.", "limitations": "N.A.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n(1) This paper considers the OOD robustness on pruning, data augmentation, and weight ensembling.\n(2) This paper designs new metrics that capture the Fourier sensitivity of models.\n\nWeaknesses:\n(1) To the best of the reviewer's knowledge, this is one existing work [1*] considering both the amplitude and phase on the robustness in the frequency, while [1*] is not in this paper's reference list. The design difference is needed, at least.\n\n(2) The Power spectral densities in Figure 2 are nearly the same as those in [36]. A citation of [36] is needed in Sec. 4.1.\n\n(3)  The experiment setting is a little strange. The experiments in Secs. 4.2 and 4.3 are conducted on CIFAR-10, while ImageNet is used in Sec. 4.4. From the reviewer's view, it is better to conduct the experiments on one consistent dataset or on both datasets.\n\n(4) Since all the experiments are conducted on corruption benchmarks (CIFAR10-C and ImageNet-C), the separated experimental results on each type of corruption are suggested to add.\n\n[1*] Amplitude-Phase Recombination: Rethinking Robustness of Convolutional Neural Networks in Frequency Domain. ICCV 2021\n\n\n-------------------------------------------------------\nPost-rebuttal reviews:\nMy main concerns are on the experimental settings, which have been well-addressed in the rebuttal. I will raise the recommendation to 'weak accept'.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Fa21d5ve9sj", "reviewer": "Reviewer_RxwV", "summary": "**High level goal:** The authors are interested in an empirical study of models that display *effective robustness* (those that are \"above the line\" that's traced by in and out-of-distribution accuracies of models that don't display effective robustness) in the image domain. \n\n**Gap in literature that the paper contributes in addressing:** The conditions (on OOD data, model and training techniques) under which effective generalization is achieved is not well understood. \n\n**Main contributions:**\n  * Using previously proposed metrics that are known to correlate with OOD generalization (including some new metrics based on the Fourier spectra of the images and models), the authors study how different types of pruning, data augmentation and ensembling affect effective generalization. \n  * The authors release the RobustNets suite of models that display effective robustness to facilitate further research in this space. \n  * The Fourier-based metrics proposed by the authors correlate well with effective generalization of models form the CLIP family. It's also interesting that these metrics don't correlate particularly strongly when other (mostly convnet based) models are used, which is useful in gaining indirect knowledge about the types of functions represented by these networks. \n\n\n\n\n\n", "questions": "* Could you explain which models you evaluated on ImageNet? The description (between lines 109 and 112) is a bit vague. Which architectures are included, for example? \n* Line 153: Could you clarify what you mean by \"we interpolate the lowest 40% of image frequencies\"? Are these the frequencies that occupy 40% of the energy? Or perhaps something else? Also, do you leave the rest of the frequencies untouched?\n* Line 280: Are the 11 CLIP models obtained by interpolating the same pretrained and finetuned versions of CLIP? \n\n\nNitpick: It perhaps could be interesting to add a plot in the Appendix that supplements Figure 1 in that instead of tracing amplitude or phase interpolations along a single axis, a grid of images can be provided where on the left top and right bottom, we have the unperturbed images, and in the middle we have various (mixed) interpolations. ", "limitations": "**Only image-based distribution shifts:** The proposed metrics (and the analysis) is only concerned with image data. \n**Experiments have no causal power:** Despite authors' claims, the experiments are fully correlational, and this should be acknowledged so. (see weaknesses for above)\n", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS:\n* **Relevance:** The focus of the paper is well within the scope of NeurIPS, especially given the recent interest in gaining a deeper understanding of OOD generalization. \n* **Comprehensive evaluation:** The experiments are quite extensive, providing a plethora of useful data for understanding the conditions under which effective generalization occurs. \n* **Usefulness of the spectral characteristics of in and OOD data:** It turns out that the spectral characteristics of in and OOD data gives useful information regarding on which distributions shifts are more likely to induce \"accuracy-on-the-line' style phenomenon. \n* **Evidence that each distribution shift is \"unique\"** The authors make a compelling case that each distribution shift has it's own problem structure, and there's not single metric that correlates well with generalization across all shifts (at least among the ones explored in the paper)\n\n\nWEAKNESS:\n* **Lack of causal explanations:** Even though the authors use words that imply the results provide a causal understanding of why effective generalization happens, all of the results are correlational. It's misleading to claim that the experiments can answer \"why\" questions. I believe it's very important that this is fixed before the camera-ready version, should the paper be accepted. (I'm happy to hear arguments against this criticism) In my opinion, this is an important limitation that can be fixed simply by removing any causal claims from the paper - the existing results are still worthwhile. \n* **No emphasis on architecture:** It'd be interesting to see a discussion of whether model architecture has any impact on effective generalization. The authors likely have enough empirical data to address this - a discussion of this would have made the paper stronger. \n**Lack of mechanistic understanding:** Despite a plethora of empirical data, I don't think I've acquired a deeper (i.e. mechanistic) understanding of why/how effective robustness occurs. I also am not sure if a practitioner also leaves the paper with a solid understanding of which techniques to use to achieve effective robustness, beyond using methods from already existing literature. \n  * This is not to say that the presented results are non meaningful - perhaps follow-up works will fill these gaps. \n\n================\n**Post rebuttal update**\nI thank the authors for their response. Since some of my important concerns are addressed, which I've reflected on my review by increasing the score. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2EsEX4qUJtp", "reviewer": "Reviewer_N1xd", "summary": "This paper takes metrics which predict OOD robustness (such as ID accuracy, model jacobian norm, etc.) and then looks at the correlation between these metrics and OOD accuracy for methods which can produce robust models. This paper observes that while ID accuracy is a good predictor of robustness for natural distribution shifts (ImageNet -> ImageNetV2), for some synthetic distribution shifts there are better metrics. In general they find \"no single metric rules them all\". The paper also introduces new metrics which correlate will with the robustness of ensembled CLIP models.", "questions": "- Why do the authors think that proposed metrics correlate well for ensembled CLIP models?\n- Can any of the proposed metrics be used as objectives to train better models?\n- What happens on additional natural distribution shifts such as ImageNetR or ObjectNet or a shift from WILDS?\n", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nOverall the results are very interesting and the empirical findings are likely useful to the community. Moreover, the proposed metrics seem that they may be potentially useful in further understanding CLIPs robustness. Finally, robustnets could be a useful resource.\n\nWeaknesses:\nMy thoughts are that the main weakness of this paper was the lack of distribution shifts, in particular natural distribution shifts which are more likely to occur in the world. I would have appreciated results for additional distribution shifts including ImageNetR or ObjectNet. In addition, it's not clear if there is a takeaway for designing better models. Finally, while a conclusion of this paper is that there is no one metric to rule them all, there is only one column of plots without ID accuracy on the x-axis. Consider showing other metrics on the x-axis in the appendix.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept