Models Out Of Line: A Fourier Lens On Distribution Shift Robustness
Anonymous Author(s)
Abstract
Improving the accuracy of deep neural networks on out-of-distribution (OOD)data is critical to an acceptance of deep learning in real world applications.
It hasbeen observed that accuracies on in-distribution (ID) versus OOD data follow alinear trend and models that outperform this baseline are exceptionally rare (andreferred to as “effectively robust”).
Recently, some promising approaches have beendeveloped to improve OOD robustness: model pruning, data augmentation, andensembling or zero-shot evaluating large pretrained models.
However, there still isno clear understanding of the conditions on OOD data and model properties thatare required to observe effective robustness.
We approach this issue by conductinga comprehensive empirical study of diverse approaches that are known to impact OOD robustness on a broad range of natural and synthetic distribution shifts of CIFAR-10 and ImageNet.
In particular, we view the “effective robustness puzzle”through a Fourier lens and ask how spectral properties of both models and OODdata correlate with OOD robustness.
We find this Fourier lens offers some insightinto why certain robust models, particularly those from the CLIP family, achieve
OOD robustness.
However, our analysis also makes clear that no known metric isconsistently the best explanation of OOD robustness.
Thus, to aid future researchinto the OOD puzzle, we address the gap in publicly-available models with effectiverobustness by introducing a set of pretrained CIFAR-10 models— RobustNets —with varying levels of OOD robustness.
Introduction
Deep learning (DL) holds great promise for solving difficult real-world problems in domains such ashealthcare, autonomous driving and cyber-physical systems.
The major roadblock in adopting DLfor real-world applications is that real-world data often deviates from the training data due to noise,corruptions, or other changes in distribution that may have temporal or spatial causes [3, 20, 41, 44].
DL models are known to be highly brittle under such distribution shifts, which raises the risk ofmaking incorrect predictions with harmful consequences.
Designing approaches to learn modelsthat achieve both high accuracy on in-distribution data and high robustness to distribution shifts isparamount to the safe adoption of DL in real-world applications.
Several recent works [1, 35] have noticed an intriguing phenomenon: accuracy on in-distribution(ID) versus accuracy on distribution-shifted data obeys a linear trend across a wide range of models,corruption benchmarks, and prediction tasks.
Unfortunately, as nearly all current DL models beara substantial loss under distribution shifts, this linear relationship implies that our current trainingstrategies are insufficient to bridge the gap between ID and out-of-distribution (OOD) performance.
However, despite being exceedingly rare, models that break this linear trend to obtain effectiverobustness [1] exist—such models defy explanation by the wisdom that OOD accuracy and ID accuracy should correlate strongly: they are “out of line”.
Identifying when and why such modelsarise is key to overcoming OOD brittleness in DL models.
Related research efforts have focused on improving OOD robustness via new techniques such as modelpruning [8], data augmentation [5, 21], and ensembling large pre-trained models [37, 51].
While theseefforts have created significant excitement in the ML robustness community, it remains unclear (a)under which conditions on OOD data these methods can improve effective robustness (ER), and (b)which underlying model properties make a DNN effectively robust.
A more in-depth understandingof these phenomena is likely to help in bridging the gap between ID and OOD performance.
Towards achieving this goal, we carry out a comprehensive empirical study leveraging models trainedusing each of the aforementioned state-of-the-art approaches to improving OOD robustness.
Weanalyze the impact of these robustness improvement methods on various model architectures, twoclean datasets (CIFAR-10 and ImageNet), and a broad range of natural and synthetic distributionshifts.
In particular, we view the OOD robustness puzzle through a Fourier lens and ask how spectralproperties of OOD data and models influence their effective robustness.
In this process, we designnew metrics that capture Fourier sensitivity of models as test data moves farther away from thetraining data manifold.
We make the following contributions:
• A new perspective on the state of OOD robustness: across diverse models, metrics, and distribution
shifts, we observe that no one single metric rules them all, not even in-distribution accuracy.
This accents the potential need for a multi-faceted approach to understanding OOD robustness,
in contrast to ID generalization where single properties like model flatness can enjoy predictive
success across an array of models [28].
• Design of Fourier sensitivity metrics: our new metrics quantify various model properties, including
spectral ones that strongly correlate with the effective robustness of ensembled CLIP models [51],
explaining this robustness much better than ID accuracy.
• A public collection of effectively robust models: to help crack the OOD robustness puzzle, we makeavailable (upon publication) a RobustNets dataset of CIFAR-10 models with effective robustnessthat stems from training under a variety of data augmentation and pruning schemes.
Related work
Distribution shift fragility is a well-documented phenomenon among neural networks[40, 41, 20, 19, 34] in which a model trained on an ID dataset performs markedly worsewhen evaluated on OOD data, even when humans find the OOD data just as easy to classify.
These OOD accuracy gaps often follow a linear relationship between ID and OOD accuracy, but the slopeand intercept of the linear trendline varies depending on the specific ID and OOD datasets [34, 15].
In our work, we focus on two popular image classification benchmark ID datasets, CIFAR-10 [30]and ImageNet [6], and an array of OOD benchmarks for each.
While most models follow the linear ID-OOD trendlines of Miller et al.
[34], some models are“effectively robust" and appear above the trendline, with OOD accuracy higher than expected given
ID accuracy [1].
These models are quite rare, and thus far have only been shown to arise througha few training paradigms: by pruning a model as a sort of regularization [8], by training with dataaugmentation intended to mimic the expected OOD test data [5, 21], or by pretraining on a largerand more diverse dataset and evaluating zero-shot [37], partially finetuning [1], or interpolating theweights of a zero-shot and a finetuned model [51].
Although as a community we are beginning to uncover robust training methods, it remains amystery why these particular methods achieve robustness on particular distribution shifts.
We studythis mystery via a Fourier lens based on prior work involving both image frequency and functionfrequency analyses.
Ortiz-Jimenez et al.
[36], Yin et al. [54], and Sun et al.
[47] study modelsensitivity to Fourier perturbations of the input images and analyze how different data augmentationsproduce different Fourier sensitivities (and correspondingly robustness to image perturbations ofdifferent frequencies).
Chen et al.
[4] showed that exchanging amplitude and phase informationbetween images (as a regularization scheme) has connections to robustness as well.
Another lineof research focuses on spectral bias [13, 2, 53, 38], wherein models prioritize learning low frequencyfunctions over the input space.
This bias towards low function frequencies can also affect robustnessand is influenced by training hyperparameters like data augmentation and weight decay [13].
Among the model properties we study as potential predictors of OOD accuracy are ID accuracy
[34],model Jacobian norm [24], the linear pixel-space interpolation metrics of Fridovich-Keil et al.
[13],and our own Fourier amplitude and phase interpolation metrics.
We study these metrics over a widerange of robust and nonrobust models, including sparse models from [8] on CIFAR-10 and pretrainedmodels from CLIP [37, 51] and RobustBench [5] on ImageNet.
Methods
Datasets.
We consider two standard image classification tasks: CIFAR-10 [30] and ImageNet [6].
For each of these "in-distribution" (ID) datasets we consider a variety of "out of distribution" (OOD)datasets, both natural and synthetic. For CIFAR-10 we consider CIFAR-10.1 [40] and CIFAR-10-C[20]; for ImageNet we consider ImageNetV2 [41], ImageNet-C [19], ImageNet-R [23], ImageNet-A[9], ImageNet-Sketch [50], and ObjectNet [10].
Models.
Our CIFAR-10 experiments use 3 different model architectures: Conv8 [12], ResNet18[17], and VGG16 [45].
For each model architecture, we consider a range of model pruning strategiesthat have been studied for their effect on OOD robustness [8, 25] as some of these pruned modelswere demonstrated to have higher OOD robustness than dense models.
Namely, lottery-ticket stylepruning methods [12, 42, 39, 7] were able to provide robustness gains on CIFAR-10-C [8].
Weoutline the various pruning techniques and sparsity levels we utilized in detail in Section 4.1.
In our ImageNet experiments we use 28 standard (nonrobust) pretrained models from Torchvision[33], 5 pretrained models from the ImageNet-C leaderboard on RobustBench [5], and 33 modelsobtained by robustness-enhancing weight-space interpolation [51] between zero-shot and ImageNetfinetuned CLIP ViT-B/16, ViT-B/32, and ViT-L/14 [37]. More details on the ImageNet models weuse from Torchvision and RobustBench can be found in Section A.1.
Previously-proposed metrics: ID accuracy, Jacobian norm, pixel interpolation.
Because OODrobustness is a complex and multi-faceted problem, we make use of multiple previously-proposedmodel measurements and apply them to study effectively robust models.
To the best of our knowledge,our work is the first application of most of these metrics (except for in-distribution accuracy) to thestudy of effective robustness.
Specifically, we evaluate four previously-proposed metrics that havebeen observed to correlate with some notion of generalization: accuracy on unseen in-distributiondata ("ID accuracy")
[34], Jacobian norm [24], and within-class and between-class pixel interpolationhigh frequency fraction [13].
These latter metrics are analogous to the high frequency fractions wecompute for amplitude and phase interpolation (described below), except that interpolating pathssimply interpolate in pixel space between two images, where the two images are from either thesame class ("within-class") or different classes ("between-class")
Prior work observed thatin-distribution generalization typically improves as within-class high frequency fraction decreasesand between-class high frequency fraction increases [13].
Reducing the norm of the Jacobian ofmodel outputs with respect to input data can push the decision boundary away from the trainingpoints, providing robustness to random perturbations of the input data [24].
Thus, models withsmaller Jacobian norms may perform better on corrupted/shifted data.
We estimate the norm of the Jacobian using a random-projection-based approach [24]; further details can be found in Section A.4.
Fourier interpolation metrics.
We also introduce novel Fourier interpolation metrics to capturemore information about model behavior that may help answer the robustness puzzle.
Inspired bythe pixel-space interpolation procedure of Fridovich-Keil et al.
[13], we propose evaluating thesmoothness of a model’s predictions along image paths that perturb the Fourier amplitude or phaseinformation of one test image towards another, while preserving the Fourier phase or amplitude(respectively) information of the original image.
The intuition for this type of interpolation is thatmost of the semantic content of an image is contained within its Fourier phases, which encodestructural information like edges.
Accordingly, we can perturb the Fourier amplitude of an imagewithout destroying its semantic meaning, and we can reliably destroy semantic content by perturbing
Fourier phase.
Example Fourier amplitude and phase interpolating paths on ImageNet are shown in
Figure 1; example paths on CIFAR-10 and further computational details are included in Section A.3.
More precisely, we compute an interpolating path by first randomly selecting two images x 0 andx 1 from the ID test/validation set.
We perform standard preprocessing on each image, which involves cropping to a standard size (for ImageNet) and normalizing each pixel by the training mean andstandard deviation (for both CIFAR-10 and ImageNet).
We then compute the two-dimensionaldiscrete Fourier transform (DFT) of each image and separate the complex-valued results intoamplitude and phase components a i and p i for i ∈ { 0 , 1 } .
To construct an image along an amplitudeinterpolating path, we retain the original phase p 0 and interpolate the low-frequency amplitude as(1 − λ ) a 0 + λ a 1 (while preserving the high frequency amplitude), where λ ∈ [0 , 1] , and producean image via the inverse discrete Fourier transform.
Phase interpolation is defined analogously;we retain the original amplitude and interpolate the low-frequency phase.
We choose a frequencythreshold for what is considered low frequency for interpolation based on visual inspection to ensurethat amplitude interpolation preserves semantic meaning and phase interpolation destroys it.
Weinterpolate the lowest 40% of image frequencies for both amplitude and phase on CIFAR-10, thelowest 20% of image frequencies for phase on ImageNet, and all image frequencies for amplitudeon ImageNet.
We choose 100 evenly-spaced λ values to define each path, and choose 5000 randompaths for CIFAR-10 and 7000 for ImageNet.
The images in Figure 1 are rescaled for visualization(essentially undoing the initial pixel-wise normalization).
We evaluate each model on each path, producing a probability vector over the available classesfor each image along the path.
We compute two metrics based on these path predictions: highfrequency fraction (HFF) and consistent distance (CD).
For high frequency fraction, we take theone-dimensional DFT of the 100 model predictions along the path, average the Fourier amplitudesamong all the classes, and compute the fraction of the total amplitude above a frequency threshold(10% of the maximum frequency).
This metric is always between 0 and 1 (usually between 0.1 and0.3).
The higher the average high frequency fraction , the more sensitive a model is to the given Fourier perturbation of its input images.
We also consider consistent distance , which is defined as the index of the first image along the paththat the model classifies differently (by highest softmax prediction) than the original image.
Thismetric is always between 1 and 100, and is often at least 40.
Consistent distance is intended as a moreintuitive metric to capture the same notion of robustness to corruptions of image Fourier content.
Inparticular, Fourier amplitude corruptions change image statistics but do not hamper human semanticidentification; a more robust model in this sense should have a lower high frequency fraction and ahigher consistent distance on average compared to a less robust model.
Results
We are interested in understanding when (under what conditions on the distribution shift) and why(via model properties) neural networks can exhibit effective robustness.
Although our experiments arecorrelational rather than causal, they provide evidence for and against various hypotheses regardingwhy effective robustness might emerge. We hope that future work will make use of our proposed metrics and released model database RobustNets to answer the causal questions raised by ourexperiments, perhaps following Dziugaite et al. We perform a systematic study of three model interventions that have been shown to impact OOD robustness: pruning [8], data augmentation [5, 21], and weight ensembling [51].
For eachintervention, we evaluate a benchmark set of pretrained CIFAR-10 and/or ImageNet models on theoriginal in-distribution test set, several OOD test sets, and a suite of model property metrics capturinglocal smoothness (via model Jacobian norm), frequency response to pixel-space interpolation withinand between classes [13], and Fourier amplitude and phase sensitivity (via high frequency fractionand consistent distance ).
Representative results from our analysis of these three interventions arepresented in Sections 4.1, 4.2, 4.3, and A.10.
For each dataset we consider a mixture of naturaland synthetic distribution {+shifts: CIFAR-10.
+}
[40] and CIFAR-10-C [20], and ImageNet-V2 [41],
ImageNet-R [23], ObjectNet [10], ImageNet-A
[9], ImageNet-Sketch [50], and ImageNet-C [19].
Since CIFAR-10-C and ImageNet-C are each composed of 15 synthetic corruptions ranging fromlow to high frequency, we include results on representative corruptions of each frequency band inthe main text and defer results on the remaining corruptions to the appendix. In all figures, we show95% confidence intervals around each measurement (Clopper-Pearson for accuracy measurementsand
Gaussian bounds for averages of other metrics).
4.1 When and why does model pruning confer effective robustness?
Diffenderfer et al.
[8] demonstrated the nuanced behavior that model pruning has on OOD robustness,specifically with respect to the distribution shift from CIFAR-10 to CIFAR-10-C.
Notably, prunedmodels are capable of providing superior OOD robustness compared to dense, or unpruned, models.
Furthermore, the effect, either positive or negative, and degree of robustness arising from modelpruning is dependent on both the model architecture and the pruning algorithm.
It remains unknownwhich properties of these pruned models contribute to their robustness on OOD data.
In an effort to better understand these findings, we further investigate this behavior by consideringthe three categories of pruning used in Diffenderfer et al. [8]: traditional [16, 56], rewinding lotterytickets [12, 42], and initialization lottery-tickets [39, 7].
Note that we often abbreviate "lottery tickets"as LT.
We provide details on each of the pruning techniques in Section A.2.
For each pruningstrategy and each architecture, we prune models to 50%, 60%, 80%, 90%, and 95% sparsity.
Forall pruning methods, pruning is performed in an unstructured (i.e., individual weights are pruned)and global manner (i.e., prune to a given sparsity across the entire network).
Additionally, fortraditional and initialization LTs, pruning was performed in a layerwise fashion (i.e., where eachnetwork layer is pruned to the given sparsity level).
In Figure 2 we investigate when (on which typesof distribution shifts) and in Table 1 we study why (via corresponding model property measurements)these differently-pruned CIFAR-10 models achieve their varying degrees of robustness.
Because different model architectures respond differently to different pruning methods, in eachsubplot of Figure 2 we color-code models by their architecture and fit a separate probit-domain
OOD vs. ID accuracy regression line to each architecture, so that each regression captures a rangeof different pruning amounts (what fraction of weights are pruned).
We display these regressionswith opacity proportional to the fit quality R 2 , so that regression lines that fit the data well are darkand easily seen, while lower-quality linear fits are more transparent.
In the corresponding Tables 1,each reported slope ( m ) and R 2 value is the average of the corresponding values among the threearchitectures we consider.
For ease of reading, we bold average R 2 values above 0.5, and amongthese entries we highlight the best (bright green) and second-best (faded green) R 2 values for eachpruning method on each OOD dataset.
Although these results are nuanced and raise additional openquestions, we summarize a few key takeaways from our CIFAR-10 model pruning experiments:
• As reported in Miller et al.
[34], ID accuracy is often a strong predictor of OOD accuracy; this trend holds across different pruning methods.
However, the linear trend breaks down for mid andhigh frequency image corruptions like pixelate, Gaussian noise, and impulse noise.
• Even for low-frequency corruptions, different model architectures might lie on either the same ordifferent robustness lines (as evidenced by comparing brightness and contrast). • When trained without data augmentation, higher effective robustness is present on several mid- and high-frequency corruptions by non-residual architectures (i e. Conv8 and VGG16)
(see Figures
and 7).
However, when training with data augmentation, less variation in effective robustness isrealized by different architectures (see Figures 3 and 8). [Figure] [Table] • Initialization LT pruning confers robustness to high-frequency corruptions and can induce a negative
correlation between ID and OOD accuracy for these corruptions.
• Jacobian norm is rarely a good predictor of OOD accuracy, but it does correlate negatively (lower
norm is better) for models pruned by rewinding methods on mid-frequency corruptions.
• As reported in Fridovich-Keil et al. [13], lower high frequency fraction HFF (i [Table]
e. smootherpredictions) for within-class pixel interpolation is a good predictor of OOD accuracy, at least forlow-frequency distribution shifts where ID accuracy is also predictive. • Amplitude and phase interpolation are sometimes predictive of OOD accuracy, particularly for initialization LT pruning, and in the expected direction of correlation (a better model should havelower HFF and higher CD).
However, even these metrics fail to explain most of the robustness behavior for high (and some mid) frequency distribution shifts on CIFAR-10.
Please see Figure 6 for visualizations of the data fit associated with the most predictive metric foreach distribution shift and pruning approach.
4.2 When and why does data augmentation confer effective robustness?
A common practice to protect against distribution shift is to train with data augmentation [5, 21].
Accordingly [Figure] we revisit each of the CIFAR-10 models considered in Section 4.1 with three differentvariations of data augmentation used during training: no augmentation (clean), AugMix [21], and
Gaussian noise augmentation [29].
In Figure 3 we investigate when (on which types of distributionshifts) and in Table 2 we consider candidate explanations for why (via corresponding model propertymeasurements) these differently-augmented CIFAR-10 models achieve varying degrees of robustness.
As in Figure 2 and Table
[Figure] we fit a separate regression line to each model architecture and averagethe resulting slope and R 2 values.
The models that contribute to each regression line therefore sharean architecture and an augmentation strategy, but vary by pruning method and amount.
Our findingsin this experiment echo those in Section 4. [Figure] but we make the following additional observations: [Figure]
• The closer the match between the training data and the OOD data in terms of their spectral profile(see Section A.5 for visualizations), the better ID accuracy, within-class pixel interpolation, andamplitude interpolation HFF and CD are at predicting OOD accuracy.
This is evident from the highR 2 values for these metrics when predicting defocus blur accuracy for models trained with AugMix,and when predicting Gaussian or impulse noise accuracy for models trained with Gaussian noise.•
The closer the spectral match between the training data and the OOD data, the better the OODaccuracy.
This unsurprising result is evident from the higher accuracies of AugMix-trained modelson contrast and defocus blur corruptions and the higher accuracies of Gaussian-noise-trainedmodels on Gaussian noise and impulse noise corruptions. Please see Figure 7 for visualizations of the data fit associated with the most predictive metric foreach distribution shift and data augmentation approach.
4.3 CLIP: Weight ensembling a robust model on ImageNet
Perhaps the most stunning example of OOD robustness for image classification is CLIP [37], whichshowed that a large model pretrained on a massive dataset and evaluated zero-shot on ImageNetachieves effective robustness on many OOD test sets simultaneously.
Andreassen et al.
[1] foundthat finetuning these robust models towards ImageNet improves ID accuracy on ImageNet buterodes effective robustness, and Wortsman et al.
[51] offered a "best of both worlds" solution byinterpolating in weight space between a pretrained zero-shot model and its finetuned counterpart,improving ID accuracy on ImageNet without sacrificing robustness.
Accordingly, we study a setof 33 CLIP models obtained by interpolating the pretrained and finetuned versions of three CLIP ViTmodels (B/16, B/32, and L/14) [37, 51] with various relative weights on the two models.
We comparethis CLIP model set to a benchmark set of 28 standard (nonrobust)
ImageNet pretrained models from Torchvision [33] as well as 5 pretrained models from the RobustBench [5] ImageNet-C leaderboard;these models typically use data augmentation to improve corruption performance but are not nearly aseffectively robust as the CLIP models.
We make the following observations in Figure 4 and Table 3:
• Although in-distribution accuracy is a reasonable predictor of OOD accuracy for most models and
distribution shifts, it performs markedly worse when predicting OOD accuracy for CLIP models(and a couple of effectively robust RobustBench models on some OOD shifts).
• Fourier interpolation metrics, particularly HFF and CD for amplitude interpolation, are strongpredictors (often R 2 ≥ 0 . 9 ) of OOD performance for CLIP models across OOD shifts.
Thisobservation offers a candidate explanation for why CLIP models are more robust: they are lesssensitive to perturbations of Fourier amplitude, which preserve image semantics for humans butconfuse nonrobust models.
Separately, in Appendix A.10, we analyze the ability of these model metrics to predict robustness in ImageNet models trained with various data augmentation approaches, which can produce effectiverobustness on many of the distribution shifts we consider [5]. Note that this analysis (see Figure 12)visualizes the most predictive metric’s performance for the non-CLIP models from Figure 4.
Discussion
Our work is only the beginning of a full understanding of what makes models effectively robust todistribution shifts, starting with the domain of image classification where some models have demonstrated effective robustness.
For example, Section 4.1 showed different types of model pruning canconfer robustness to different distribution shifts, and while some of this robustness can be explained byimproved behavior in response to amplitude and phase perturbations, much remains an open mystery.
As we saw in Section 4.2 (and has been noted by prior work [54, 21]), models can achieve robustnessto some distribution shifts by training with data augmentations designed to imitate the expected OOD data.
This approach can be a powerful way to mitigate expected test-time distribution shifts butleaves open the question of how to prepare for the unexpected.
We learned in Section 4.3 that morerobust CLIP models are less sensitive to perturbations of Fourier amplitude (and sometimes phase),perhaps making them more attuned to human perceptions of semantic meaning.
This finding providesan exciting opportunity for future work to both confirm our results across additional robust modelarchitectures and with causal experiments, and to incorporate these Fourier interpolation metrics into training paradigms, for instance as regularizers or data augmentation strategies, to explicitlyencode robustness in future models.
Our results provide a new perspective on the effective robustnessproblem by showing that no one single metric rules them all.
This dictates the need for a multi-facetedanalysis to understand OOD robustness, which is in sharp contrast to the in-distribution setting wherea single property such as model flatness can reasonably explain generalization performance [Figure] [Figure]
RobustNets benchmark.
To aid further research into understanding, modeling, and designing for
OOD robustness, we will publish a database of our pretrained models on CIFAR-10 (with diverseeffective robustness) with the full range of model architectures, data augmentations [Figure] and pruningtypes we study.
The models we study on ImageNet are already publicly available.
We will alsopublish the ID and OOD accuracy values for each model, and our code, so that future researchers canapply it to study new and more robust models as they become available.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] Limitations are included in the
discussion section.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] Code will be
made public upon acceptance.
We do also provide sufficient details in the text that
anyone could reproduce our code.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] Most of these are provided in the methods section, and some are
in the supplement.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] We use 95% confidence intervals in all of our figures.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] Our experiments did require
substantial compute to evaluate all of our models on all of the interpolation paths, but
we use pretrained models so we do not require any compute for training.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We cite the creators
of all the models and datasets we use.
(b) Did you mention the license of the assets?
[No] Licenses for individual assets can be
found by visiting the websites of the corresponding creators.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No] We do not include new assets in the submission; however, we will release our
RobustNets database of pretrained models evaluated in this paper upon acceptance.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation? [N/A]
A AppendixA.1 ImageNet Model Details
We use the following models from Torchvision (details at https://pytorch.org/vision/0.8/models.html ): alexnet [31], vgg11 [46], vgg11_bn , vgg13 , vgg13_bn ,vgg16 , vgg16_bn , vgg19 , vgg19_bn , resnet18 [18], resnet34 , resnet50 , resnet101 ,resnet152 , squeezenet1_0 [27], squeezenet1_1 , densenet121 [26], densenet169 ,densenet161 , densenet201 , googlenet [48], shufflenet_v2_x1_0 [32], mobilenet_v2 [43],resnext50_32x4d [52], resnext101_32x8d , wide_resnet50_2 [55], wide_resnet101_2 , andmnasnet1_0 [49].
We use the following models from the RobustBench [5] model zoo on ImageNet corruptions (details at https://github.com/RobustBench/robustbench#model-zoo ): Geirhos2018_SIN[14], Geirhos2018_SIN_IN , Geirhos2018_SIN_IN_IN , Hendrycks2020Many [22], and
Hendrycks2020AugMix [21].
From the CLIP family [37], we use 33 models formed by weight-space interpolation betweenzero-shot and finetuned ViT-B/16, ViT-B/32, and ViT-L/14 (11 models from each architecture). A.2 Pruning methods In Section 4.1 we present results considering three categories of model pruning used in Diffenderferet al. [8]: traditional (fine-tuning [16], gradual magnitude pruning [56]), rewinding lottery-tickets(weight-rewinding [12], learning-rate rewinding [42]), and initialization lottery-tickets (edgepopup[39], biprop [7]).
Here we provide a brief description of each pruning method.
Traditional.
As the name suggests, fine-tuning prunes a model at the end of the regular trainingperiod by removing p % of the weights with the smallest magnitude, then fine tunes the remainingweights using the learning rate at the end of the regular training period.
Gradual magnitude pruningprogressively removes weights during training in accordance with a sparsity scheduler (see Fig. 1 in
Zhu and Gupta [56]) until the target sparsity is reached.
Typically, some training takes place beforeand after pruning in gradual magnitude pruning.
Rewinding lottery tickets.
Rewinding lottery tickets perform repeated training-then-pruning stepsin which some percentage of the remaining weights are pruned at each pruning step (traditionally20% of the remaining weights are pruned at each step).
After each training-then-pruning step, eitherthe weights and learning rate are rewound to their values at initialization (i.e. weight-rewinding) orthe learning rate is rewound to a value earlier in the training process (learning-rate rewinding).
Thefirst rewinding lottery ticket method supported the Lottery Ticket Hypothesis [12] which suggeststhat sparse subnetworks exist within randomly initialized dense networks that can be trained to thesame accuracy as dense networks.
Initialization lottery tickets.
Stronger versions of this hypothesis, such as the Strong [39] and
Multi-Prize Lottery Ticket Hypotheses [7] suggest that such subnetworks exist at initialization thatachieve comparable accuracy to the dense network without requiring any training and, further, thatthese subnetworks’ weights can be binarized.
Edgepopup identifies such subnetworks using surrogatescores to identify the most important weights at initialization.
Biprop integrates weight and/oractivation binarization into the search process for such subnetworks resulting in binary-weight orbinary-weight and activation networks (BNNs).
Note that biprop networks in this paper only makeuse of binarized weights.
A.3 Fourier interpolation on CIFAR-10 and computational details
Figure 1 in the main paper shows example Fourier amplitude and phase interpolating paths on
ImageNet.
Figure 5 shows example
Fourier amplitude and phase interpolating paths on CIFAR-10,where again the images are rescaled for visualization (undoing the pixel-wise normalization).
Relatedly, note that in Section 3, the “lowest 40% of image frequencies” we reference is with respectto the maximum spatial frequency of the image (a function of the pixel resolution) and not based on the actual power spectral density of the image.
Further, the interpolation is done using a square maskon the DFT of the image, which interpolates the lower image frequencies but leaves the higher imagefrequencies unperturbed.
A.4 Jacobian norm computational details
We estimate the norm of the Jacobian using a random-projection-based approach that estimatesthe Jacobian norm as a function of a batch of samples of size B and n proj projection vectors[24].
For CIFAR-10 and ImageNet, we set n proj to 10 and 50 , respectively; we set B to 400 and1000 , respectively.
Error bars for our estimates are 95% confidence intervals constructed under theassumption that the n proj B unique Jacobian norm estimates made by the projection method are i.i.d.
,which is consistent with the error order of ( n proj B ) − 12 expected by Hoffman et al.
[24] when B ≫ 1 .
Under this assumption, which may lead to underestimation of the true error if violated, our choice ofn proj and B ensures that the average error bar is less than 2% of the size of the estimated Jacobiannorm (and all error bars are less than 8% of the size of the estimated Jacobian norm).
Our estimated
Jacobian norms are between 60 and 2300 for ImageNet models and between 2 and 50 for CIFAR-10models.
A.5 Power spectral density of distribution shifts
To explore the behavior of models on OOD data, it is helpful to characterize the nature of the OODdata with respect to the ID test data; indeed this type of characterization is common practice [54].
Specifically, considering the difference in power spectral density (PSD) between each of the OOD testsets and the original ID test set allows us to categorize each OOD test set as being concentrated on low,mid, or high frequency perturbations with respect to the ID test set.
We perform this characterizationfor CIFAR-10.1 and each of the 15 corruptions in CIFAR-10-C.
The PSDs shown in Figure 6 reflect the distribution shift between each OOD dataset and the original
CIFAR-10 test set, and are sorted roughly in order of increasing frequency.
Of note, this PSDcharacterization illustrates that the OOD shift encountered on CIFAR-10.1 data is composed oflow-frequency information, much like the brightness and contrast CIFAR-10-C corruptions.
As each of the CIFAR-10-C corruptions are transformations applied to the original CIFAR-10 testimages, the PSD for each CIFAR-10-C corruption is computed by taking the difference betweeneach corrupted and original test image, computing the PSD of the resulting difference image, thenaveraging the PSDs for all images of the given corruption.
As CIFAR-10.1 images are not corruptionsof the original CIFAR-10 test images, the PSD characterization must be computed in a differentmanner.
First, for each of the ten CIFAR-10 classes we compute the average PSD for CIFAR-10 testimages and CIFAR-10.1 images and take the difference of these class PSDs.
The final approximated
PSD representing the CIFAR-10.1 shift is taken to be the average of the difference PSDs for the tenclasses.
The dramatic variations in frequency content among these different distribution shifts underscore
[Figure]
the difficulty of the robustness puzzle; models may need different tools or properties to achieve
[Figure]
robustness on different types of distribution shifts
[Figure]
A.6 Pruning results on additional corruptions from CIFAR-10-C
[Figure]
Figure 7 and Table 4 extend Figure 2 and Table 1 to the nine remaining corruptions from CIFAR-10-C.
[Figure]
[Figure]
[Table]
[Figure]
[Table]
[Table]
[Table]
Figure 8 and Table 5 extend Figure 3 and Table 2 to the nine remaining corruptions from CIFAR-10-C.
[Figure]
[Table]
[Table]
A.8 Best metric performance visualized for CIFAR-10 distribution shifts
Figures 9 and 10 visualize the fit to the data associated with the best metrics from the CIFAR-10pruning analysis figures (Figures 2 and 7) and data augmentation analysis figures (Figures 3 and 8), respectively.
Note that Tables 6 and 7 are duplicates of the original figures’ associated tables and arereproduced here for convenience.
[Figure]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
[Table]
A.9 CLIP results on additional corruptions from ImageNet-C
Figure 11 and Table 8 extend Figure 4 and Table 3 to the remaining corruptions from ImageNet-C.
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
A.10 ImageNet data augmentation results on all corruptions
Figure 12 and Table 9 visualize the best metric for non-CLIP ImageNet models, including models from
RobustBench [5] trained with data augmentation.
Specifically, the RobustBench models are ResNet50 models trained using the following data augmentations: Stylized-ImageNet [14], DeepAugment[22], and AugMix [21].
Importantly, our baseline models include a ResNet-50 trained without specialdata augmentations, allowing us to see the effect that the aforementioned data augmentations have onthe predictability of robustness while holding architecture constant.
Similar to the robustness of CLIP models, the robustness of non-CLIP ImageNet models is oftenbest predicted by our Fourier interpolation metrics (in particular, CD for amplitude interpolationdoes well in this setting).
In-distribution accuracy can also show strong R 2 , but it apparently hasmore trouble predicting the robustness of out-of-line models than our metrics.
This phenomenonis illustrated on distribution-shifts like Pixelate and Gaussian Noise (see the corresponding rows of
Figure 12), where in-distribution accuracy can predict many of the models’ robustnesses but clearlyfails on the effectively robust models from RobustBench.
Notably, in these cases, our amplitude
CD metric can predict these effectively-robust models’ performances as well as it can predict thenon-effectively-robust models’ performances.
Finally, we emphasize that, despite analyzing a variety of architectures, the out-of-line non-CLIPmodels on most distribution shifts are almost always the RobustBench models.
Importantly, this isconsistent with the focus on data augmentation for adding robustness [21, 14, 22], but it also suggeststhat the construction of architectures with effective robustness is a potential direction for future work.
[Figure]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]