Models Out Of Line: A Fourier Lens On Distribution Shift Robustness
Abstract
Improving the accuracy of deep neural networks on out-of-distribution (OOD) data is critical to an acceptance of deep learning in real world applications.
It has been observed that accuracies on in-distribution (ID) versus OOD data follow a linear trend and models that outperform this baseline are exceptionally rare (and referred to as “effectively robust”).
Recently, some promising approaches have been developed to improve OOD robustness: model pruning, data augmentation, and ensembling or zero-shot evaluating large pretrained models.
However, there still is no clear understanding of the conditions on OOD data and model properties that are required to observe effective robustness.
We approach this issue by conducting a comprehensive empirical study of diverse approaches that are known to impact
OOD robustness on a broad range of natural and synthetic distribution shifts of
CIFAR-10 and ImageNet.
In particular, we view the “effective robustness puzzle” through a Fourier lens and ask how spectral properties of both models and OOD data correlate with OOD robustness.
We find this Fourier lens offers some insight into why certain robust models, particularly those from the CLIP family, achieve
OOD robustness.
However, our analysis also makes clear that no known metric is consistently the best explanation of OOD robustness.
Thus, to aid future research into the OOD puzzle, we address the gap in publicly-available models with effective robustness by introducing a set of pretrained CIFAR-10 models— RobustNets — with varying levels of OOD robustness.
Introduction
Deep learning (DL) holds great promise for solving difficult real-world problems in domains such as healthcare, autonomous driving, and cyber-physical systems.
The major roadblock in adopting DL for real-world applications is that real-world data often deviates from the training data due to noise, corruptions, or other changes in distribution that may have temporal or spatial causes [3, 20, 41, 44].
DL models are known to be highly brittle under such distribution shifts, which raises the risk of making incorrect predictions with harmful consequences.
Designing approaches to learn models that achieve both high accuracy on in-distribution data and high robustness to distribution shifts is paramount to the safe adoption of DL in real-world applications.
Several recent works [1, 35] have noticed an intriguing phenomenon: accuracy on in-distribution (ID) versus accuracy on distribution-shifted data obeys a linear trend across a wide range of models, corruption benchmarks, and prediction tasks.
Unfortunately, as nearly all current DL models bear a substantial loss under distribution shifts, this linear relationship implies that our current training strategies are insufficient to bridge the gap between ID and out-of-distribution (OOD) performance.
However, despite being exceedingly rare, models that break this linear trend to obtain effective robustness [1] exist—such models defy explanation by the wisdom that OOD accuracy and ID accuracy should correlate strongly: they are “out of line”.
Identifying when and why such models arise is key to overcoming OOD brittleness in DL models.
Related research efforts have focused on improving OOD robustness via new techniques such as model pruning [8], data augmentation [5, 21], and ensembling large pre-trained models [37, 51].
While these efforts have created significant excitement in the ML robustness community, it remains unclear (a) under which conditions on OOD data these methods can improve effective robustness (ER), and (b) which underlying model properties make a DNN effectively robust.
A more in-depth understanding of these phenomena is likely to help in bridging the gap between ID and OOD performance.
Towards achieving this goal, we carry out a comprehensive empirical study leveraging models trained using each of the aforementioned state-of-the-art approaches to improving OOD robustness.
We analyze the impact of these robustness improvement methods on various model architectures, two clean datasets (CIFAR-10 and ImageNet), and a broad range of natural and synthetic distribution shifts.
In particular, we view the OOD robustness puzzle through a Fourier lens and ask how spectral properties of OOD data and models influence their effective robustness.
In this process, we design new metrics that capture Fourier sensitivity of models as test data moves farther away from the training data manifold.
We make the following contributions:
• A new perspective on the state of OOD robustness: across diverse models, metrics, and distribution shifts, we observe that no one single metric rules them all, not even in-distribution accuracy.
This accents the potential need for a multi-faceted approach to understanding OOD robustness, in contrast to ID generalization where single properties like model flatness can enjoy predictive success across an array of models [28].
• Design of Fourier sensitivity metrics: our new metrics quantify various model properties, including spectral ones that strongly correlate with the OOD robustness of ensembled CLIP models [51], explaining this robustness much better than ID accuracy.
• A public collection of effectively robust models: to help crack the OOD robustness puzzle, we make available (at https://github.com/sarafridov/RobustNets )
a RobustNets dataset of CIFAR10 models with effective robustness that stems from training under a variety of data augmentation and pruning schemes.
Related work
Distribution shift fragility is a well-documented phenomenon among neural networks [40, 41, 20, 19, 34] in which a model trained on an ID dataset performs markedly worse when evaluated on OOD data, even when humans find the OOD data just as easy to classify.
These
OOD accuracy gaps often follow a linear relationship between ID and OOD accuracy, but the slope and intercept of the linear trendline varies depending on the specific ID and OOD datasets [34, 15].
In our work, we focus on two popular image classification benchmark ID datasets, CIFAR-10 [30] and ImageNet [6], and an array of OOD benchmarks for each.
While most models follow the linear ID-OOD trendlines of Miller et al.
[34], some models are “effectively robust" and appear above the trendline, with OOD accuracy higher than expected given
ID accuracy [1].
These models are quite rare, and thus far have only been shown to arise through a few training paradigms: by pruning a model as a sort of regularization [8], by training with data augmentation intended to mimic the expected OOD test data [5, 21], or by pretraining on a larger and more diverse dataset and evaluating zero-shot [37], partially finetuning [1], or interpolating the weights of a zero-shot and a finetuned model [51].
Although as a community we are beginning to uncover robust training methods, it remains a mystery why these particular methods achieve robustness on particular distribution shifts.
We study this mystery via a Fourier lens based on prior work involving both image frequency and function frequency analyses.
Ortiz-Jimenez et al.
[36], Yin et al.
[54], and Sun et al.
[47] study model sensitivity to Fourier perturbations of the input images and analyze how different data augmentations produce different Fourier sensitivities (and correspondingly robustness to image perturbations of different frequencies).
Chen et al.
[4] showed that exchanging amplitude and phase information between images (as a regularization scheme) has connections to robustness as well.
Another line of research focuses on spectral bias [13, 2, 53, 38], wherein models prioritize learning low frequency functions over the input space.
This bias towards low function frequencies can also affect robustness and is influenced by training hyperparameters like data augmentation and weight decay [13].
Among the model properties we study as potential predictors of OOD accuracy are ID accuracy [34], model Jacobian norm [24], the linear pixel-space interpolation metrics of Fridovich-Keil et al.
[13], and our own Fourier amplitude and phase interpolation metrics.
We study these metrics over a wide range of robust and nonrobust models, including sparse models from [8] on CIFAR-10 and pretrained models from CLIP [37, 51] and RobustBench [5] on ImageNet.
Methods
Datasets.
We consider two standard image classification tasks: CIFAR-10 [30] and ImageNet [6].
For each of these "in-distribution" (ID) datasets we consider a variety of "out of distribution" (OOD) datasets, both natural and synthetic.
For CIFAR-10 we consider CIFAR-10.1 [40] and CIFAR-10-C [20]; for ImageNet we consider ImageNetV2 [41], ImageNet-C [19], ImageNet-R [23], ImageNet-A [9], ImageNet-Sketch [50], and ObjectNet [10].
Models.
Our CIFAR-10 experiments use 3 different model architectures: Conv8 [12], ResNet [17], and VGG16 [45].
For each model architecture, we consider a range of model pruning strategies that have been studied for their effect on OOD robustness [8, 25] as some of these pruned models were demonstrated to have higher OOD robustness than dense models.
Namely, lottery-ticket style pruning methods [12, 42, 39, 7] were able to provide robustness gains on CIFAR-10-C [8].
We outline the various pruning techniques and sparsity levels we utilized in detail in Section 4.1.
In our ImageNet experiments we use 28 standard (nonrobust) pretrained models from Torchvision [33], 5 pretrained models from the ImageNet-C leaderboard on RobustBench [5], and 33 models obtained by robustness-enhancing weight-space interpolation [51] between zero-shot and ImageNetfinetuned CLIP ViT-B/16, ViT-B/32, and ViT-L/14 [37].
More details on the ImageNet models we use from Torchvision and RobustBench can be found in Section A.1.
Previously-proposed metrics: ID accuracy, Jacobian norm, pixel interpolation.
Because OOD robustness is a complex and multi-faceted problem, we make use of multiple previously-proposed model measurements and apply them to study effectively robust models.
To the best of our knowledge, our work is the first application of most of these metrics (except for in-distribution accuracy) to the study of effective robustness.
Specifically, we evaluate four previously-proposed metrics that have been observed to correlate with some notion of generalization: accuracy on unseen in-distribution data ("ID accuracy") [34], Jacobian norm [24], and within-class and between-class pixel interpolation high frequency fraction [13].
These latter metrics are analogous to the high frequency fractions we compute for amplitude and phase interpolation (described below), except that interpolating paths simply interpolate in pixel space between two images, where the two images are from either the same class ("within-class") or different classes ("between-class")
Prior work observed that in-distribution generalization typically improves as within-class high frequency fraction decreases and between-class high frequency fraction increases [13].
Reducing the norm of the Jacobian of model outputs with respect to input data can push the decision boundary away from the training points, providing robustness to random perturbations of the input data [24].
Thus, models with smaller Jacobian norms may perform better on corrupted/shifted data.
We estimate the norm of the
Jacobian using a random-projection-based approach [24]; further details can be found in Section A.4.
Fourier interpolation metrics.
We also introduce novel Fourier interpolation metrics to capture more information about model behavior that may help answer the robustness puzzle.
Inspired by the pixel-space interpolation procedure of Fridovich-Keil et al.
[13], we propose evaluating the smoothness of a model’s predictions along image paths that perturb the Fourier amplitude or phase information of one test image towards another, while preserving the Fourier phase or amplitude (respectively) information of the original image.
The intuition for this type of interpolation is that most of the semantic content of an image is contained within its Fourier phases, which encode structural information like edges.
Accordingly, we can perturb the Fourier amplitude of an image without destroying its semantic meaning, and we can reliably destroy semantic content by perturbing
Fourier phase.
Example Fourier amplitude and phase interpolating paths on ImageNet are shown in
Figure 1; example paths on CIFAR-10 and further computational details are included in Section A.3.
More precisely, we compute an interpolating path by first randomly selecting two images x 0 and x 1 from the ID test/validation set.
We perform standard preprocessing on each image, which involves cropping to a standard size (for ImageNet) and normalizing each pixel by the training mean and standard deviation (for both CIFAR-10 and ImageNet).
We then compute the two-dimensional discrete Fourier transform (DFT) of each image and separate the complex-valued results into amplitude and phase components a i and p i for i ∈ { 0 , 1 } .
To construct an image along an amplitude interpolating path, we retain the original phase p 0 and interpolate the low-frequency amplitude as (1 − λ ) a {+
+} + λ a 1 (while preserving the high frequency amplitude), where λ ∈ [0 , 1] , and produce an image via the inverse discrete Fourier transform.
Phase interpolation is defined analogously; we retain the original amplitude and interpolate the low-frequency phase.
We choose a frequency threshold for what is considered low frequency for interpolation based on visual inspection to ensure that amplitude interpolation preserves semantic meaning and phase interpolation destroys it.
We interpolate the lowest 40% of image frequencies for both amplitude and phase on CIFAR-10, the lowest 20% of image frequencies for phase on ImageNet, and all image frequencies for amplitude on ImageNet.
We choose 100 evenly-spaced λ values to define each path, and choose 5000 random paths for CIFAR-10 and 7000 for ImageNet.
The images in Figure 1 are rescaled for visualization (essentially undoing the initial pixel-wise normalization).
We evaluate each model on each path, producing a probability vector over the available classes for each image along the path.
We compute two metrics based on these path predictions: high frequency fraction (HFF) and consistent distance (CD).
For high frequency fraction, we take the one-dimensional DFT of the 100 model predictions along the path, average the Fourier amplitudes among all the classes, and compute the fraction of the total amplitude above a frequency threshold (10% of the maximum frequency).
This metric is always between 0 and 1 (usually between 0.1 and 0.3).
The higher the average high frequency fraction , the more sensitive a model is to the given
Fourier perturbation of its input images.
We also consider consistent distance , which is defined as the index of the first image along the path that the model classifies differently (by highest softmax prediction) than the original image.
This metric is always between 1 and 100, and is often at least 40.
Consistent distance is intended as a more intuitive metric to capture the same notion of robustness to corruptions of image Fourier content.
In particular, Fourier amplitude corruptions change image statistics but do not hamper human semantic identification; a more robust model in this sense should have a lower high frequency fraction and a higher consistent distance on average compared to a less robust model.
Results
We are interested in understanding when (under what conditions on the distribution shift) and why (via model properties) neural networks can exhibit effective robustness.
Although our experiments are correlational rather than causal, they provide evidence for and against various hypotheses regarding why effective robustness might emerge.
We hope that future work will make use of our proposed metrics and released model database RobustNets to answer the causal questions raised by our experiments, perhaps following Dziugaite et al.
We perform a systematic study of three model interventions that have been shown to impact
OOD robustness: pruning [8], data augmentation [5, 21], and weight ensembling [51].
For each intervention, we evaluate a benchmark set of pretrained CIFAR-10 and/or ImageNet models on the original in-distribution test set, several OOD test sets, and a suite of model property metrics capturing local smoothness (via model Jacobian norm), frequency response to pixel-space interpolation within and between classes [13], and Fourier amplitude and phase sensitivity (via high frequency fraction and consistent distance ).
Representative results from our analysis of these three interventions are presented in Sections 4.1, 4.2, 4.3, and A.10.
For each dataset we consider a mixture of natural and synthetic distribution shifts: CIFAR-10.
[40] and CIFAR-10-C [20], and ImageNet-V2 [41],
ImageNet-R [23], ObjectNet [10], ImageNet-A
[9], ImageNet-Sketch [50], and ImageNet-C [19].
Since CIFAR-10-C and ImageNet-C are each composed of 15 synthetic corruptions ranging from low to high frequency, we include results on representative corruptions of each frequency band in the main text and defer results on the remaining corruptions to the appendix.
In all figures, we show 95% confidence intervals around each measurement (Clopper-Pearson for accuracy measurements and Gaussian bounds for averages of other metrics).
4.1 When and why does model pruning confer effective robustness?
Diffenderfer et al.
[8] demonstrated the nuanced behavior that model pruning has on OOD robustness, specifically with respect to the distribution shift from CIFAR-10 to CIFAR-10-C.
Notably, pruned models are capable of providing superior OOD robustness compared to dense, or unpruned, models.
Furthermore, the effect, either positive or negative, and degree of robustness arising from model pruning is dependent on both the model architecture and the pruning algorithm.
It remains unknown which properties of these pruned models contribute to their robustness on OOD data.
In an effort to better understand these findings, we further investigate this behavior by considering the three categories of pruning used in Diffenderfer et al.
[8]: traditional [16, 56], rewinding lotterytickets [12, 42], and initialization lottery-tickets [39, 7].
Note that we often abbreviate "lottery tickets" as LT.
We provide details on each of the pruning techniques in Section A.2.
For each pruning strategy and each architecture, we prune models to 50%, 60%, 80%, 90%, and 95% sparsity.
For all pruning methods, pruning is performed in an unstructured (i.e., individual weights are pruned) and global manner (i.e., prune to a given sparsity across the entire network).
Additionally, for traditional and initialization LTs, pruning was performed in a layerwise fashion (i.e., where each network layer is pruned to the given sparsity level).
In Figure 2 we investigate when (on which types of distribution shifts) and in Table 1 we study why (via corresponding model property measurements) these differently-pruned CIFAR-10 models achieve their varying degrees of robustness.
Because different model architectures respond differently to different pruning methods, in each subplot of Figure 2 we color-code models by their architecture and fit a separate probit-domain
OOD vs. ID accuracy regression line to each architecture, so that each regression captures a range of different pruning amounts (what fraction of weights are pruned).
We display these regressions with opacity proportional to the fit quality R 2 , so that regression lines that fit the data well are dark and easily seen, while lower-quality linear fits are more transparent.
In the corresponding Tables 1, each reported slope ( m ) and R 2 value is the average of the corresponding values among the three architectures we consider.
For ease of reading, we bold average R 2 values above 0.5, and among these entries we highlight the best (bright green) and second-best (faded green) R 2 values for each pruning method on each OOD dataset.
Although these results are nuanced and raise additional open questions, we summarize a few key takeaways from our CIFAR-10 model pruning experiments:
• As reported in Miller et al.
[34], ID accuracy is often a strong predictor of OOD accuracy; this trend holds across different pruning methods.
However, the linear trend breaks down for mid and high frequency image corruptions like pixelate, Gaussian noise, and impulse noise.
• Even for low-frequency corruptions, different model architectures might lie on either the same or different robustness lines (as evidenced by comparing brightness and contrast).
• When trained without data augmentation, higher effective robustness is present on several mid- and high-frequency corruptions by non-residual architectures (i
e. Conv8 and VGG16)
(see Figures
and 8).
However, when training with data augmentation, less variation in effective robustness is realized by different architectures (see Figures 3 and 9).
[Figure]
[Table]
• Initialization LT pruning confers robustness to high-frequency corruptions and can induce a negative correlation between ID and OOD accuracy for these corruptions.
• Jacobian norm is rarely a good predictor of OOD accuracy, but it does correlate negatively (lower norm is better) for models pruned by rewinding methods on mid-frequency corruptions.
• As reported in Fridovich-Keil et al.
[13], lower high frequency fraction HFF (i.e. smoother predictions) for within-class pixel interpolation is a good predictor of OOD accuracy, at least for low-frequency distribution shifts where ID accuracy is also predictive.
• Amplitude and phase interpolation are sometimes predictive of OOD accuracy, particularly for initialization LT pruning, and in the expected direction of correlation (a better model should have lower HFF and higher CD).
However, even these metrics fail to explain most of the robustness behavior for high (and some mid) frequency distribution shifts on CIFAR-10.
Please see Figure 10 for visualizations of the data fit associated with the most predictive metric for each distribution shift and pruning approach.
4.2 When and why does data augmentation confer effective robustness?
[Figure] [Table] A common practice to protect against distribution shift is to train with data augmentation [5, 21].
Accordingly, we revisit each of the CIFAR-10 models considered in Section 4.1 with three different variations of data augmentation used during training: no augmentation (clean), AugMix [21], and
Gaussian noise augmentation [29].
In Figure 3 we investigate when (on which types of distribution shifts) and in Table 2 we consider candidate explanations for why (via corresponding model property measurements) these differently-augmented CIFAR-10 models achieve varying degrees of robustness.
As in Figure 2 and Table 1, we fit a separate regression line to each model architecture and average the resulting slope and R 2 values.
The models that contribute to each regression line therefore share an architecture and an augmentation strategy, but vary by pruning method and amount.
Our findings in this experiment echo those in Section 4.1, but we make the following additional observations:
• The closer the match between the training data and the OOD data in terms of their spectral profile (see Section A.5 for visualizations), the better ID accuracy, within-class pixel interpolation, and amplitude interpolation HFF and CD are at predicting OOD accuracy.
This is evident from the high R 2 values for these metrics when predicting defocus blur accuracy for models trained with AugMix, and when predicting Gaussian or impulse noise accuracy for models trained with Gaussian noise.
• The closer the spectral match between the training data and the OOD data, the better the OOD accuracy.
This unsurprising result is evident from the higher accuracies of AugMix-trained models on contrast and defocus blur corruptions and the higher accuracies of Gaussian-noise-trained models on Gaussian noise and impulse noise corruptions.
Please see Figure 11 for visualizations of the data fit associated with the most predictive metric for each distribution shift and data augmentation approach.
4.3 CLIP: Weight ensembling a robust model on ImageNet
Perhaps the most stunning example of OOD robustness for image classification is CLIP [37], which showed that a large model pretrained on a massive dataset and evaluated zero-shot on ImageNet achieves effective robustness on many OOD test sets simultaneously.
Andreassen et al.
[1] found that finetuning these robust models towards ImageNet improves ID accuracy on ImageNet but erodes effective robustness, and Wortsman et al.
[51] offered a "best of both worlds" solution by interpolating in weight space between a pretrained zero-shot model and its finetuned counterpart, improving ID accuracy on ImageNet without sacrificing robustness.
Accordingly, we study a set of 33 CLIP models obtained by interpolating the pretrained and finetuned versions of three CLIP ViT models (B/16, B/32, and L/14) [37, 51] with various relative weights on the two models.
We compare this CLIP model set to a benchmark set of 28 standard (nonrobust)
ImageNet pretrained models from
Torchvision [33] as well as 5 pretrained models from the RobustBench [5] ImageNet-C leaderboard; these models typically use data augmentation to improve corruption performance but are not nearly as effectively robust as the CLIP models.
We make the following observations in Figure 4 and Table 3:
• Although in-distribution accuracy is a reasonable predictor of OOD accuracy for most models and distribution shifts, it performs markedly worse when predicting OOD accuracy for CLIP models (and a couple of effectively robust RobustBench models on some OOD shifts).
• Fourier interpolation metrics, particularly HFF and CD for amplitude interpolation, are strong predictors (often R 2 ≥ 0 . 9 ) of OOD performance for CLIP models across OOD shifts.
This suggests a hypothesis for why CLIP models are more robust: they are less sensitive to perturbations of Fourier amplitude, which preserve image semantics for humans but confuse nonrobust models. We
also consider the ability of these model metrics to predict robustness in ImageNet models trained with various data augmentation approaches, which can produce effective robustness on many of the distribution shifts we consider [5].
Figures 5 and 13 show the most predictive metric’s performance for these non-CLIP models from Figure 4. We again find that, although no single metric is best across all distribution shifts, our proposed Fourier interpolation metrics are often most predictive, outperforming in distribution
accuracy.
See Appendix A.10 for full results and discussion.
Discussion
Our work is only the beginning of a full understanding of what makes models effectively robust to distribution shifts, starting with the domain of image classification where some models have demonstrated effective robustness.
For example, Section 4.1 showed different types of model pruning can confer robustness to different distribution shifts, and while some of this robustness can be explained by improved behavior in response to amplitude and phase perturbations, much remains an open mystery.
As we saw in Section 4.2 (and has been noted by prior work [54, 21]), models can achieve robustness to some distribution shifts by training with data augmentations designed to imitate the expected OOD data.
This approach can be a powerful way to mitigate expected test-time distribution shifts but leaves open the question of how to prepare for the unexpected.
We learned in Section 4.3 that more robust CLIP models are less sensitive to perturbations of Fourier amplitude (and sometimes phase), perhaps making them more attuned to human perceptions of semantic meaning.
This finding provides an exciting opportunity for future work to confirm our results with both additional out-of-line models and causal experiments; and to incorporate our Fourier interpolation metrics into training paradigms, for instance as regularizers or data augmentation strategies, to explicitly encode robustness in future models. [Figure] [Table]
Our results provide a new perspective on the effective robustness problem by showing that no one single metric rules them all.
This dictates the need for a multi-faceted analysis to understand OOD robustness, which is in sharp contrast to the in-distribution setting where a single property such as model flatness can reasonably explain generalization performance
[Figure]
RobustNets benchmark.
The robustnesses of our analyzed CIFAR-10 models remain out-of-line. To track and aid progress on effective robustness prediction, we release our code as well as a database of our model weights and metric values at https://github.com/sarafridov/RobustNets . The models we study on ImageNet are publicly available.
[Figure]
We would like to thank Jonathan Frankle, Mitchell Wortsman
[Figure]
and Ludwig Schmidt for helpful discussion and pointers to resources. SFK acknowledges support from NSF GRFP. This work was performed under the auspices of the U [Figure]
S. Department of Energy by Lawrence Livermore National
Laboratory under Contract DE-AC52-07NA27344 and was supported by the LLNL-LDRD Program under Project No. 2022-SI-004 (LLNL-CONF-835574)
[Figure]
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] Limitations are included in the discussion section.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Code and pretrained models are available at https://github.com/sarafridov/RobustNets .
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] Most of these are provided in the methods section, and some are in the supplement.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We use 95% confidence intervals in all of our figures.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] Our experiments did require substantial compute to evaluate all of our models on all of the interpolation paths, but we use pretrained models so we do not require any compute for training.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] We cite the creators of all the models and datasets we use.
(b) Did you mention the license of the assets?
[No] Licenses for individual assets can be found by visiting the websites of the corresponding creators.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] We release our RobustNets database of pretrained models evaluated in this paper.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
A Appendix A.1 ImageNet Model Details
We use the following models from Torchvision (details at https://pytorch.org/ vision/0.8/models.html ): alexnet [31], vgg11 [46], vgg11_bn , vgg13 , vgg13_bn , vgg16 , vgg16_bn , vgg19 , vgg19_bn , resnet18 [18], resnet34 , resnet50 , resnet101 , resnet152 , squeezenet1_0 [27], squeezenet1_1 , densenet121 [26], densenet169 , densenet161 , densenet201 , googlenet [48], shufflenet_v2_x1_0 [32], mobilenet_v2 [43], resnext50_32x4d [52], resnext101_32x8d , wide_resnet50_2 [55], wide_resnet101_2 , and mnasnet1_0 [49].
We use the following models from the RobustBench [5] model zoo on ImageNet corruptions (details at https://github.com/RobustBench/robustbench#model-zoo ): Geirhos2018_SIN [14], Geirhos2018_SIN_IN , Geirhos2018_SIN_IN_IN , Hendrycks2020Many [22], and
Hendrycks2020AugMix [21].
From the CLIP family [37], we use 33 models formed by weight-space interpolation between zero-shot and finetuned ViT-B/16, ViT-B/32, and ViT-L/14 (11 models from each architecture).
A.2 Pruning methods
In Section 4.1 we present results considering three categories of model pruning used in Diffenderfer et al.
[8]: traditional (fine-tuning [16], gradual magnitude pruning [56]), rewinding lottery-tickets (weight-rewinding [12], learning-rate rewinding [42]), and initialization lottery-tickets (edgepopup [39], biprop [7]).
Here we provide a brief description of each pruning method.
Traditional.
As the name suggests, fine-tuning prunes a model at the end of the regular training period by removing p % of the weights with the smallest magnitude, then fine tunes the remaining weights using the learning rate at the end of the regular training period.
Gradual magnitude pruning progressively removes weights during training in accordance with a sparsity scheduler (see Fig. 1 in
Zhu and Gupta [56]) until the target sparsity is reached.
Typically, some training takes place before and after pruning in gradual magnitude pruning.
Rewinding lottery tickets.
Rewinding lottery tickets perform repeated training-then-pruning steps in which some percentage of the remaining weights are pruned at each pruning step (traditionally 20% of the remaining weights are pruned at each step).
After each training-then-pruning step, either the weights and learning rate are rewound to their values at initialization (i.e. weight-rewinding) or the learning rate is rewound to a value earlier in the training process (learning-rate rewinding).
The first rewinding lottery ticket method supported the Lottery Ticket Hypothesis [12] which suggests that sparse subnetworks exist within randomly initialized dense networks that can be trained to the same accuracy as dense networks.
Initialization lottery tickets.
Stronger versions of this hypothesis, such as the Strong [39] and
Multi-Prize Lottery Ticket Hypotheses [7] suggest that such subnetworks exist at initialization that achieve comparable accuracy to the dense network without requiring any training and, further, that these subnetworks’ weights can be binarized.
Edgepopup identifies such subnetworks using surrogate scores to identify the most important weights at initialization.
Biprop integrates weight and/or activation binarization into the search process for such subnetworks resulting in binary-weight or binary-weight and activation networks (BNNs).
Note that biprop networks in this paper only make use of binarized weights.
A.3 Fourier interpolation on CIFAR-10 and computational details
Figure 1 in the main paper shows example Fourier amplitude and phase interpolating paths on
ImageNet.
Figure 6 shows example Fourier amplitude and phase interpolating paths on CIFAR-10, where again the images are rescaled for visualization (undoing the pixel-wise normalization).
Relatedly, note that in Section 3, the “lowest 40% of image frequencies” we reference is with respect to the maximum spatial frequency of the image (a function of the pixel resolution) and not based on the actual power spectral density of the image.
Further, the interpolation is done using a square mask on the DFT of the image, which interpolates the lower image frequencies but leaves the higher image frequencies unperturbed.
A.4 Jacobian norm computational details
We estimate the norm of the Jacobian using a random-projection-based approach that estimates the Jacobian norm as a function of a batch of samples of size B and n proj projection vectors [24].
For CIFAR-10 and ImageNet, we set n proj to 10 and 50 , respectively; we set B to 400 and 1000 , respectively.
Error bars for our estimates are 95% confidence intervals constructed under the assumption that the n proj B unique Jacobian norm estimates made by the projection method are i.i.d., which is consistent with the error order of ( n proj B ) − 12 expected by Hoffman et al.
[24] when B ≫ 1 .
Under this assumption, which may lead to underestimation of the true error if violated, our choice of n proj and B ensures that the average error bar is less than 2% of the size of the estimated Jacobian norm (and all error bars are less than 8% of the size of the estimated Jacobian norm).
Our estimated
Jacobian norms are between 60 and 2300 for ImageNet models and between 2 and 50 for CIFARmodels.
A.5 Power spectral density of distribution shifts
To explore the behavior of models on OOD data, it is helpful to characterize the nature of the OOD data with respect to the ID test data; indeed this type of characterization is common practice [54].
Specifically, considering the difference in power spectral density (PSD) between each of the OOD test sets and the original ID test set allows us to categorize each OOD test set as being concentrated on low, mid, or high frequency perturbations with respect to the ID test set.
We perform this characterization for CIFAR-10.1 and each of the 15 corruptions in CIFAR-10-C.
The PSDs shown in Figure 7 reflect the distribution shift between each OOD dataset and the original
CIFAR-10 test set, and are sorted roughly in order of increasing frequency.
Of note, this PSD characterization illustrates that the OOD shift encountered on CIFAR-10.1 data is composed of low-frequency information, much like the brightness and contrast CIFAR-10-C corruptions.
As each of the CIFAR-10-C corruptions are transformations applied to the original CIFAR-10 test images, the PSD for each CIFAR-10-C corruption is computed by taking the difference between each corrupted and original test image, computing the PSD of the resulting difference image, then averaging the PSDs for all images of the given corruption.
As CIFAR-10.1 images are not corruptions of the original CIFAR-10 test images, the PSD characterization must be computed in a different manner.
First, for each of the ten CIFAR-10 classes we compute the average PSD for CIFAR-10 test images and CIFAR-10.1 images and take the difference of these class PSDs.
The final approximated
PSD representing the CIFAR-10.1 shift is taken to be the average of the difference PSDs for the ten classes.
The dramatic variations in frequency content among these different distribution shifts underscore the difficulty of the robustness puzzle; models may need different tools or properties to achieve robustness on different types of distribution shifts.
[Figure]
A.6 Pruning results on additional corruptions from CIFAR-10-C
Figure 8 and Table 5 extend Figure 2 and Table 1 to the nine remaining corruptions from CIFAR-10-C
[Figure]
[Table]
[Figure]
[Table]
A.7 Augmentation results on additional corruptions from CIFAR-10-C
Figure 9 and Table 6 extend Figure 3 and Table 2 to the nine remaining corruptions from CIFAR-10-C.
[Figure]
[Table]
A.8 Best metric performance visualized for CIFAR-10 distribution shifts
Figures 10 and 11 visualize the fit to the data associated with the best metrics from the CIFARpruning analysis figures (Figures 2 and 8) and data augmentation analysis figures (Figures 3 and 9), respectively.
Note that Tables 7 and 8 are duplicates of the original figures’ associated tables and are reproduced here for convenience.
[Figure]
[Figure] [Table] [Figure] [Table]
[Figure]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
A.9 CLIP results on additional corruptions from ImageNet-C
Figure 12 and Table 9 extend Figure 4 and Table 3 to the remaining corruptions from ImageNet-C. [Figure] [Table] [Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
A.10 ImageNet data augmentation results on all corruptions
Figures 5 and 13 and Tables 4 and 10 visualize the best metric for non-CLIP ImageNet models, including models from RobustBench [5] trained with data augmentation.
Specifically, the RobustBench models are ResNet-50 models trained using the following data augmentations: Stylized-ImageNet [14], DeepAugment [22], and AugMix [21].
Importantly, our baseline models include a ResNettrained without special data augmentations, allowing us to see the effect that the aforementioned data augmentations have on the predictability of robustness while holding architecture constant.
Similar to the robustness of CLIP models, the robustness of non-CLIP ImageNet models is often best predicted by our Fourier interpolation metrics (in particular, CD for amplitude interpolation does well in this setting).
In-distribution accuracy can also show strong R 2 , but it apparently has more trouble predicting the robustness of out-of-line models than our metrics.
This phenomenon is illustrated on distribution-shifts like Pixelate and Gaussian Noise (see the corresponding rows of Figures 5 and 13), where in-distribution accuracy can predict many of the models’ robustnesses but clearly fails on the effectively robust models from RobustBench.
Notably, in these cases, our amplitude CD metric can predict these effectively-robust models’ performances as well as it can predict the non-effectively-robust models’ performances.
Finally, we emphasize that, despite analyzing a variety of architectures, the out-of-line non-CLIP models on most distribution shifts are almost always the RobustBench models.
Importantly, this is consistent with the focus on data augmentation for adding robustness [21, 14, 22], but it also suggests that the construction of architectures with effective robustness is a potential direction for future work.
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure] [Table] [Figure] [Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
[Figure]