A Closer Look at Model Adaptation using Feature Distortion and Simplicity Bias
Authors: Puja Trivedi
Danai Koutra
Jayaraman J. Thiagarajan
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Transfer Learning
Robustness
Adaptation
Data Augmentation

Abstract: Advances in the expressivity of pretrained models have increased interest in the design of adaptation protocols which enable safe and effective transfer learning. Going beyond conventional linear probing (LP) and fine tuning (FT) strategies, protocols that can effectively control feature distortion, i.e., the failure to update features orthogonal to the in-distribution, have been found to achieve improved out-of-distribution generalization (OOD). In order to limit this distortion, the LP+FT protocol, which first learns a linear probe and then uses this initialization for subsequent FT, was proposed. However, in this paper, we find when adaptation protocols (LP, FT, LP+FT) are also evaluated on a variety of safety objectives (e.g., calibration, robustness, etc.), a complementary perspective to feature distortion is helpful to explain protocol behavior. To this end, we study the susceptibility of protocols to simplicity bias (SB), i.e. the well-known propensity of deep neural networks to rely upon simple features, as SB has recently been shown to underlie several problems in robust generalization. Using a synthetic dataset, we demonstrate the susceptibility of existing protocols to SB. Given the strong effectiveness of LP+FT, we then propose modified linear probes that help mitigate SB, and lead to better initializations for subsequent FT. We verify the effectiveness of the proposed LP+FT variants for decreasing SB in a controlled setting, and their ability to improve OOD generalization and safety on three adaptation datasets.

Published as a conference paper at ICLR 2023
A CLOSER LOOK AT MODEL ADAPTATION USING FEA- TURE DISTORTION AND SIMPLICITY BIAS
Puja Trivedi
Danai Koutra
Jayaraman J. Thiagarajan
CSE Department
CSE Department
Center of Applied Scientific Computing Lawrence Livermore Natl. Laboratory j jayaram 1n1. gov
University of Michigan pu jujattumi ch. edu
University of Michigan
dkout ummriiimmii ch. edu
ABSTRACT
Advances in the expressivity of pretrained models have increased interest in the design of adaptation protocols which enable safe and effective transfer learning. Going beyond conventional linear probing (LP) and fine tuning (FT) strategies, protocols that can effectively control feature distortion, i.c., the failure to update features orthogonal to the in-distribution, have been found to achieve improved out- of-distribution generalization (OOD), In order to limit this distortion, the LP+FT protocol, which first learns linear probe and then uses this initialization for subsequent FT, was proposed. However, in this paper, we find when adaptation protocols (LP, FT, LP+FT) also evaluated calibration, robustness, etc.), complementary perspective to feature distortion is helpful explain protocol behavior. To this end, we study the susceptibility of protocols to simplicity bias (SB), i.c. the well-known propensity of deep neural networks to rely upon simple features, as SB has recently been shown to underlie several problems in robust generalization. Using synthetic dataset, we demon- strate the susceptibility of existing protocols SB. Given the strong effectiveness of LP+FT, we then propose modified linear probes that help mitigate SB, and lead better initializations for subsequent FT. We verify the effectiveness of the proposed LP+FT variants for decreasing SB in a controlled setting, and their ability to improve OOD generalization and safety on three adaptation datasets.
variety of safety objectives (e.g.,
INTRODUCTION
Through the of larger datasets (Yalniz et al., 2019), better architectures (Zhai et al., 2022; Chen et al., 2022; Steiner et al., 2022; Tolstikhin al., 2021), and different self-supervised learning (SSL) approaches (He et al., 2020; Chen et al., 2020; Grill al., 2020; Caron al., 2020), the quality of pretrained representations available for transfer learning tasks has dramati- cally improved. Indeed, representations from such high-quality SSL. models have been found to be more robust (Hendrycks et al., 2019; Liu et al., 2021), transferable (Ericsson et al., 2021) and semantically consistent (Caron et al., 2021) than their su- pervised counterparts. In this regard, there is growing need for adaptation protocols that explicitly capitalize pretrained features to induce similar beneficial properties, e.g., inducing more than just high accuracy on the target task, after models have been trained on the downstream task.
Dara
OGD Datta
Comuptonns
Conapptonn Bar
Adveraartee
Anomalis
4
these improved
Figure 1: Strong and Safe Adap- tation. Practical deployment in high risk applications requires that adapted models not only generalize well to in- and out-of distribution data of the downstream task, but they do so safely.
However, standard adaptation protocols that rely upon fine- tuning (FT) all model parameters or training only a linear probe (LP) while freezing the network parameters do maximize the potential of high-quality representations. For example, while high high-tyality, pre-trained models have sufficiently expressive features to perform well on
Correspondence pu ja jatt umi edu.

---

Published as a conference paper at ICLR 2023
both in-distribution (ID) and -ittt--fdistiittiinnn (OOD) data, LP and FT not able to effectively induce this property in adapted models (Andreassen et al., 2021). Recently, however, Kumar et al. (2022) proved that by modifying features only in the ID representation subspace, FT can lead to higher OOD error as it distorts directions outside the ID subspace that are needed for OOD generalization. As both ID and OOD subspaces represented by the pretrained model, Kumar al. demonstrate that limiting feature distortion, or controlling updates towards the ID subspace, can lead to improved ID and OOD performance. To this end, they propose new protocol which performs LP prior to FT (abbrev. LP + FT). By first performing LP, this two-step process ensures that subsequent FT will remain in the vicinity of the original LP solution. This reduces the overall distortion towards the ID distribution subspace and improves performance.
While strong and OOD generalization on the target task is indeed an important aspect of transfer learning, practical, high-risk applications require that models are also safe (Hendrycks et al., 2021). For example, adapted models should also be well-calibrated, robust to corruptions or adversaries and able to reliably detect anomalous samples (see Figure 1). Given that existing adaptation protocols are primarily focused on improving generalization, it is unclear how existing protocols utilize high-quality pretrained features to promote safe adaptation, and if current protocol design perspectives, such as mitigating feature distortion, will also enable safe generalization.
Our Work: In this work, seek to understand the factors relevant to the design of adaption protocols that promote effective and safe generalization. We take the first step towards this aim by (i) demonstrating limitations in existing LP,, FT, and LP+FT protocols through an extensive, joint evaluation, and (ii) studying adaptation protocols through the complementary lens of avoiding simplicity bias, i.e., the problematic tendency of deep neural networks (DNNs) to prefer simple, potentially brittle features over complex features (Soudry et al., 2018; Gunasekar et al., 2018; 8; Geirhos al., 2019; Hermann et al., 2020; Shah al., 2020). Using the insights from our analysis, we propose three variants of the LP+FT protocol that jointly improve safety and generalization on three datasets. Our contributions can be summarized as follows:
Joint Analysis of Adaptation Protocol Safety and Generalization (Sec. 3). We show that when adaptation protocols are evaluated with respect both ID/OOD generalization and safety, LP+FT trails behind LP or FT on several safety metrics. This demonstrates that solely mitigating feature distortion may not be sufficient for safe generalization. We also observe that keeping subsequent FT close to LP solution is crucial for the improved OOD generalization of LP LP+FT. This motivates us to focus on improving the LP initialization as a mechanism for improving both safety and OOD performance.
Role of Simplicity Bias in (Unsafe) Adaptation (Sec. 4). To understand how protocols may induce safe adaptation, we study how different protocols avoid simplicity bias. While simplicity bias (Shah al., 2020; Geirhos et al., 2019) has been shown to underlie several problems in machine learning safety, to the best of our knowledge, we are the first to consider its role in adaptation settings. We demonstrate that protocols must not only reduce distortion, but also should mitigate simplicity bias for effective adaptation.
Improved Protocols for Mitigating Simplicity Bias and Distortion (Sec. 5). We propose three, simple modified LP+FT protocols that help mitigate both simplicity bias and distortion (Sec. 4.1). In particular, we consider modifying the I.P step with uncertainty-driven perturbations (Pagliardini et al., 2022), virtual adversarial training (Miyato et al., 2017) and model-soups (Wortsman et al., 2022), they are simple and effective strategies. Across synthetic and real datasets, the modified protocols help improve safety and generalization to some extent.
2 RELATED WORK AND BACKGROUND
Here, discuss the most relevant work on adaptation protocols and simplicity bias; we discuss additional related work in Sup. A.
Adaptation Protocols. For a comprehensive overview of transfer learning, please see the surveys of Zhuang et al. (2021) and Pan & Yang (2010). Here, we discuss the works that are most relevant to our own. Kirichenko et al. (2022) recently demonstrated that models are able to learn both core features and spurious features. However, classifiers can rely upon spurious features, harming
2

---

Published as a conference paper at ICLR 2023
performance on minority groups. To reduce the reliance on spurious features, they propose to retrain the classifier on a small amount of "reieeigting" data, which allows the model to leverage the core features instead of the spurious features. Other modifications and heuristics have also been proposed to improve FT's performance, including side-tuning (Zhang et al., 2019), which tunes a small secondary network that is then combined with the original model, using larger/smaller learning methods (Jiang et al., 2020). In this work, we
rates for the classifier, as well as regularization-basse focus on two popular and effective protocols, LP and FT. We additionally study the LP+FTT protocol as it is theoretically snaretically-roundddd. grounded, does not require re-weighting data, is designed to exploit high-quality pre-trained representations and achieves SOTA OOD performance during adaptation.
Simplicity Bias. It is well-known that DNNs demonstrate bias toward simple, potentially less expressive features (Brutzkus et al., 2017: Soudry et al., 2018; Gunasekar et al., 2018; Geirhos et al., 2019; Hermann et al., 2020; Lubana et al., 2023), such as textures and backgrounds, and that this bias can lead to shortcuts that limit the generalization of DNNs. Indeed, recently Shah et al. (2020) formalized this intuition by more precisely defining simplicity bias, based on the number of linear components to define a decision boundary, and showed that SB leads to non-robust decision boundaries that affects model's sensitivity to distribution shifts and adversarial perturbations. In brief, by learning simple features first, models become invariant to complex features, potentially leading to narrow decision boundaries which can fail to generalize under data shifts. Notably, DNNs exhibit this bias even when complex features are more expressive and necessary for fitting the distribution. While various techniques have recently been proposed to mitigate simplicity bias when training from scratch or in the context of pretraining (Teney et al., 2021), we are, to the best of our knowledge, the first to rigorously study the role of simplicity in the context of model adaptation.
JOINT ANALYSIS OF PROTOCOL SAFETY AND GENERALIZATION
In this section, we evaluate the performance of adaptation protocols across several additional safety objectives (Hendrycks et al., 2021), as practical transfer learning applications require both strong and safe generalization. Through this expanded evaluation, we find that no single protocol is optimal across all safety objectives. Indeed, the inability of LP+FTT to induce safe adaptation indicates that complementary perspective to feature distortion, namely simplicity bias, is necessary when designing generalizable and safe protocols (see Sec. 4). We further argue that by constraining models around the LP initialization during FT, LP FFT may inadvertently harm safety performance by hampering models' abilities to learn complex, task-specific features needed for robust generalization. While we expand upon the role of LP initialization in Secs. 4 and 5, we begin, here, by introducing the expanded evaluation and experimental setup.
Experimental Setup. Three downstream adaptation tasks (and their respective OOD distributions) are considered: CIFAR-I0 (ID) - (STLIO, CIFARIO.1) (OOD), Domainnet-Sketch - (Domainnet- and Living17 (Source) -> Living17 (Target). These
ClipArt, Sommainnet-Painting Domainnet-Reat datasets are selected as they correspond to two different types of distribution shifts (standard domain adaptation and subpopulation) and three levels of distortion (low, medium, high). A MoCo-V2 ResNet-50 (He et al., 2020) pretrained on ImageNet- IK is used as the base-feature extractor for CIFAR1O and Living17 experiments, and the CLIP ResNet-50 image encoder pretrained on 400 million (image,text) pairs is used for Domainnet-Sketch These models are selected as they provide sufficiently high-quality representations capable of generalizing to both ID and OOD downstream data (Kumar et al., 2022). We perform grid-search to find the best hyper-parameters, and average over 3 seeds. See Sup. B.2 for additional details.
Expanded Evaluation. In addition to OOD accuracy on the aforementioned distribution shifts, we report performance on the following metrics in order to evaluate adapted models on key problems in machine learning safety (Hendrycks et al., 2021). Our evaluation setup is inspired by Hendrycks et al. (2022):
Mean Corruption Accuracy (mCA//CA): We consider two sets of corruptions: the 15 naturalistic corruptions (Corr) (Hendrycks & Dietterich, 2019), and 10 perceptually dissimilar corruptions (Corr) (Mintun et al., 2021). Corruptions are applied to the ID test dataset and the average accuracy over each set is reported.
3

---

Published as a conference paper at ICLR 2023
Calibration Error (RMSE): It is important that models well-calibrated so that practitioners may trust the provided predictions in high-risk applications Guo et al. (2017). We measure the root mean square error of calibration as follows: /2c [ = & c e3] where C indicates the confidence scores, while Y and denote the model's predictions and ground-truth labels, respectively.
Anomaly Detection Performance (AUROC): Recognizing when samples are anomalous allows models to abstain from making uninformed and inapplicable predictions. We consider samples from Blobs, Gaussian, LSUN, Places69, Rademacher, Textures, and SVHN datasets as anomalies and report the AUROC (area under the ROC curve) of the binary classification problem of detecting such samples as anomalies.
Adversarial Accuracy: DNNs well-known to be fooled by imperceptible distortions (lyys et al., 2019). We use a 2/225, 10-step PGD (Madry et al., 2018) 8) attack to measure the robustness of models such perturbations.
We make the following observations regarding the behavior of different protocols using this expanded evaluation. In brief, we find that no single protocol is effective across all datasets in jointly ob- taining strong and safe adaptation, and that, on low distortion adaptation tasks, the quality of the LP initialization is critical as pre-trained feature extractor is not substantially updated during LP FFT.
OBSERVATION 1: MITIGATING FEATURE DISTORTION MAY NOT INDUCE SAFE ADAPTATION.
Here, we ask how protocols perform when we consider both safety and generalization objectives to better understand the feature distortion perspective. In particular, LP+FT is able to outperform LP and FT in this expanded evaluation, then it suggests that solely mitigating feature distortion during may be sufficient to induce robust adaptation. To test this claim, we rank protocol performance for each safety metric, where ranks are first computed for each dataset separately, and then averaged. Results are shown in Fig. 2. Smaller ranks correspond to better performance.
1.4 (Better)
ID
Acc.
2.33
2.33
1:33
Acc.
00D
2.00
2.67
1.33
1.6
1.33
Carr.
MCA
2.00
2.33
1.67
-.8
Adv. Acc.
2.33
1.67
2.00
2.0
ID
Results. LP+FT obtains the best rank for ID and OOD accuracy as expected, well as Corr and Corr accuracy. However, we also see that FT is better ranked for Adversarial Accuracy and OOD calibration, while LP is better ranked for ID calibration and Corr calibration. However, given that LP+FT trails behind protocols that are not explicitly designed to limit distortion on some safety metrics, it is clear that a complementary perspec- tive is needed to better understand protocol behavior. Indeed, LP+FT has the best average rank, indicating that it is good starting point improve upon.
RMSE
Corr. RMSE
2.00
2.00
2.00
Carr. RMSE
1.67
2.33
2.00
2.4
AMSE
Anom. AUROC
2.67
2.00
1.33
(Worse)
LP+FT
Figure 2: Disparate Performance of Protocols. We plot the average rank of each protocol for different safety and generalization metrics. We see no single protocol achieves top rank across all metrics.
The above results are aggregated across different types of dis- tribution shifts; we extend this analysis next by considering the interplay between individual datasets and protocol performance. These detailed results are presented in Table
OBSERVATION 2: LINEAR PROBING SOLUTIONS MATTER.
Naturally, the amount of distortion required to effectively adapt a pretrained model to downstream task will vary in accordance to the similarity of the downstream and pretraining data. Here, we seek to understand how protocols behave under different levels of distortion. In particular, we hypothesize that the LP initialization becomes more influential for LP FTT in low distortion settings, as subsequent FT remains in the vicinity of initialization. To this end, we compute the batched centered kernel alignment (CKA) score (Nguyen et al., 2021) with respect to the adapted and pretrained models, and take a closer look at performance across metrics. We note that while CKA is better suited for measuring distortion than the L2 norm as used by Kumar et al. (2022), other neural representation metrics can also be used (Ding et al., 2021: Davari al., 2023).
4

---

Published as a conference paper at ICLR 2023
Results. As shown in Fig. 3, we see that minimal distortion (CKA 2 0.9) is required to obtain competitive LP+FT performance on DomainNet and Living17. However, on CIFARIO, which requires the most distortion evidenced by lower CKA scores, FT is the most effective protocol for safety measures and is very comparable on generalization performance (see Table 1).
The effectiveness of LP and LP+FT Living17 in improving OOD generalization over FT is hardly surprising, as Living 17 is subset of ImageNet, on which the base feature-encoder was already trained.
In contrast, on DomainNet, the difficulty of FT in matching the ID test task performance, despite achieving high training accu- racy. suggests FT may learn solution that relies upon shortcuts (or simple features) that do not generalize. We emphasize that LP+FT greatly benefits from strong LP initializations on these low- distortion datasets as corresponding CKA scores show that very limited updates are made during FT. While LP+FT does induce meaningful improvements over LP on Living17 and performs com- parably to LP on DomainNet, we stress the model must be kept close the LP initialization during FT. Indeed, to obtain acceptable LP +TT performance, small learning rates (3e-7.1e-5) and frozen batch-norm parameters during FT necessary.
LP4FT
8 0.5
0.99
0.08
0.79
CIFARLO
DomainNet
Livin17
DATASET
Figure 3: Dataset Distortion. We plot the CKA similarity be- tween adapted and pretrained models. DomainNet and Liv- ing17 require low distortion, as seen by performance of LP+FT across metrics with high CKA (> 0.9).
Summary. Taken jointly, these results suggest that while solely mitigating feature distortion may not be sufficient ensure that adapted models perform well on safety metrics across different levels of shift, improving the LP initialization may be viable solution to obtaining strong and safe generalization. Indeed, the effectiveness of LP+FTT on low distortion datasets and its high average ranking indicates that it is promising protocol to build upon. To understand how to build better protocols, we next introduce simplicity bias as a complementary perspective to feature distortion.
Generalization
Robustmess
Calibration
Anomaly
Rep. Similarity
Daraset
Protecol
ID
ODD*
C.
Adv.
ID
C
ODD.
Out Class
ID
Acc:
Ace.
Acc.
Aoc.
Act:
J-RMS
I-RMS
1-RMS
1-RMS
AUROC
CKA
CIFARID
D.9138
8888888819
0.6012
0.6553
0.0003
0.9945
0.83225
0.8666
0.6236
1,0000
CIFAR10
FT
0.9539
0.5962008545
0.7434
0.7553
0.0231
0.9668
0.8335
0,8453
0,9232
1.0000
0.6811
CIFARID
IP:FT
0.9422
0.775/086511
0.6921
0.6790
0.0018
0.9521
0.784)
0.7721
0.8863
0.5111
0.7853
DomainNet
L2
0.8913
0.8013
0.0019
0,6020
0,1768
0.9638
0.9645
0.8571
0.9264
0.8679
10000
DomainNet
FT
0,7613
0.1522
0.5186
0,2744
0.4164
0.8368
0.7234
0.7234
0.6379
0.8811
0.6092
DomainNet
LP-F7
0,8985
0.2990
0,6343
0,5979
0,1927
0.9566
0.8455
0.8445
0,8899
0,9022
0.9222
Living1?
0.9521
0.124
0.7010
0,7377
0,2350
0.9313
0 0.8693
0.880]
0.9117
0.9907
1,00000
Living17
F7
0.9518
0.7168
0.701L
0.7164
0.1563
0.8873
0.9019
0.8604
0.9295
0.9794
0.7847
17
LP-F7
0.8261
0,7671
0,2135
0.9782
0.9472
0.8742
Table 1: Safety and Generalization Performance of Adaptation Protocols. (Best in bold. Second best underlined.) While LP+FT indeed achieves strong ID and OOD performance across datasets, we see that different protocols may perform better when safety evaluation is also considered. For CIFAR-10, which requires the most distortion evidenced by lower CKA scores, see that FT is the most effective; LP+FT and LPare most effective. respectively, on Living17 and DomainNet, which require significantly less distortion. This suggests that, while mitigating feature distortion is effective for improving generalization, it is not always sufficient for also improving safety.
MITIGATING SIMPLICITY BIAS & FEATURE DISTORTION FOR SAFE ADAPTATION
As discussed in Sec. 2, simplicity bias (SB) underlies various safety issues in machine learning as models may learn to rely upon simple features that often do not generalize under distribution shifts, such corruptions or adversaries (Shah et al., 2020). Therefore, we argue that mitigating feature distortion in a way that minimizes this bias can be a valid mechanism to improve both safety and generalization performance. Correspondingly, in this section, propensity of different protocols to simplicity bias in a controlled setting. In particular, given our previous observation that LP+FT models remain in close vicinity of the I.P solution after FT,
first measure the
5

---

Published as a conference paper at ICLR 2023
focus on improving the performance of this initial LP initialization that may capitalize upon LP+FT strong OOD performance, while simultaneously improving safety. To this end. we propose three light-weight LP +FT variants that are able to both reduce distortion and SB. We begin by introducing our synthetic dataset and experimental setup.
Dataset. As shown in Fig. 4, we create "dominoes" of complex and simple features by pairing each class (Shah et al., 2020) from CIFARIO (complex) with the corresponding "digit" class in MNIST (simple), e.g.. "bird" samples are paired with digit "2" samples, where the label for each domino is determined by the complex, CIFARIO sample. Datasets with three levels of correlation (95%, 99%, 100%) between the simple and complex features are constructed for training. While 100% correlation allows models to only learn the simple feature for perfect generalization, the more realistic lower correlation settings require models learn at least some aspect of the complex features.
Experimental Setup. For evaluation, we also construct a random- ized (10% correlation) variant, where simple features are randomly paired with complex features. We give two examples in panels 3 and 4 of Fig. 4. To assess OOD generalization, create variant where complex features are sampled from STL10, instead of CIFARIO, e.g. panels and in Fig. 4.
CIFARTO Corr.
STL10 Corr.
CIFAR10
STL10 Rand.
Rand.
Metrics. We assess the reliance on simple features using the fol- lowing metrics: (1) Randomized Accuracy: the accuracy on the variant where samples contain random pairings between simple and complex features; (2) Correlated Accuracy: accuracy when pairings between simple and complex features remain correlated. Models that are susceptible to simplicity bias will have high Correlated Accu- racy and low Randomized Accuracy. Likewise, models that are not susceptible to simplicity bias will have relatively lower correlated accuracy and higher randomized accuracy.
Figure 4: Synthetic Data with Simple and Complex Features. Using synthetic dominoes dataset (Shah et al., 2020), study the effect of simplicity bias on safety and OOD generalization.
Training Details. A MoCo-V V2 ResNet-50 (He et al., 2020) pretrained on ImageNet- IK is the cuse-fcature extractor. See Supp. B.2 for additional details. We performed grid-search to find the best parameters. Results are over 3 seeds and correlation strengths.
Correlation = 0.05
Correlation = 0.90
Corvelation = 1.00
Protnool
0000
1P
0.9728
0:9156
0 7010
0.9409
0.9336
0.7862
0.9505
0.7696
FT
0.966
0,9614
0.6021
0.9923
0,9928
0.3629
0,9855
0,9789
012647
LP152
0,9902
0,9793
0S442
0.9644
0,9484
0.7513
0.9874
0.8594
0,2548
(Senad)
0.0123
0.1820
0.9905
0.1201
0,9952
0.9444
0.1055
Table 2: Simplicity Bias and Performance of Adaptation Protocols. Using the synthetic dominoes dataset, we measure the propensity of different models to simplicity bias by measuring the Corr. OOD and Rand. OOD accuracy With highest Corr. OOD accuracy and lowest Rand. OOD accuracy, we see that FT is particularly susceptible to inducing simplicity bias.
Results. Given the above experimental setup, we report the performance of different adaptation protocols in Table. 2. Across all correlation strengths, FT has the lowest Rand. OOD accuracy and high Corr. OOD accuracy. This clearly indicates that FT has learned to rely upon simple features, effectively disregarding the expressive features of the pretrained model, and is easily susceptible to simplicity bias. In contrast, by preserving the expressive features of the underlying feature encoder, LP best mitigates simplicity bias in high correlation (0.99.1.0 1.0) settings as evidenced by the highest Rand OOD accuracy (though Corr. ID/OOD accuracy does slightly suffer). However, in moderate correlation (0.95), LP+FT improves upon LP to achieve better Corr: OOD accuracy than LP and the best Rand. OOD accuracy across protocols. This suggests that when the correlation is not extreme, that moderate distortion, given a suitable LP, is in fact beneficial to mitigating simplicity bias. At higher correlation strengths (0.99,1.0), however, LP+FT has lower Rand OOD accuracy, while improving the Corr. OOD accuracy relative to LP, indicating in such extreme settings the distortion incurred by subsequent FT is not beneficial and has increased reliance upon simple features.

---

Published as a conference paper at ICLR 2023
4.1 IMPROVED LINEAR PROBES FOR MITIGATING SIMPLICITY BIAS
As discussed earlier, adaptation protocols have varying bias, and mitigating this susceptibility can help improve generalization and safety. In particular, we observe that LP+FT and LP are effective protocols for reducing reliance upon simple features on both synthetic datasets, and on low-distortion real datasets (Sec. 3). How- ever, some level of distortion is typically quired when adapting to downstream tasks to obtain sufficient ID task performance, pose new variants of the LP+FT protocol that at- tempt enable the subsequent step to distort features without compromising generalization or increasing simplicity bias.
susceptibility to simplicity
Corriation
Camiition
ca0 0220
0.844 5
pro-
0:733
paacs 0.822
0.12
We note that, while it is possible to modify the step as well, modifications to LP are inex- pensive as the feature-encoder is not updated. Moreover, as discussed Sec. 3, fine-tuned so- lutions remain in close vicinity of initial LP ini- tializations, further motivating strong starting solutions. To this end, we introduce the follow- ing modifications to the LP step of LP+FT be- low, where h are the hidden representations, model parameters, y labels, ú predicted classes, C' the number of classes, g the classifier, and the perturbation. See Supp. B. for additional discussion on the choice of these mitigation strategies and Supp. B.2 for discussion on the importance of applying mitigations during LP.
Figure 5: Hardness Promoting Augmentations help Mitigate Simplicity Bias. We evaluate the modified LP+FT protocols on the dominoes dataset, and find they improve the Rand. OOD Accuracy over vanilla FT and LP LFI This sug- gests that modified protocols can rely less upon shorteuts simple features.
LP (VAT): Virtual adversarial training (VAT) (Miyato et al., 2017) enforces local distribu- tion smoothness by minimizing the KL-divergence between the predictions of perturbed pairs of examples. Since we are using expressive pretrained models, such perturbations may be meaningful in the inverted latent space as well, and resulting classifiers become robust in some e-neighborhood around each latent-space input. Formally, let be some perturba- tion budget, and a hyper-parameter weighting distributional label smoothness, we mini- mize the following loss: ming (ce(ge(h)) y) aKL ip (y go(h)) .P (y ge (h + 8))] where ò == arg max KL jp (y ge(h)) P (y (h + 8))]
[óllaSe
LP (UDP): Instead of maximizing the loss, uncertainty-drrin perturbations (UDP) (Pagliardini al., 2022) adversarially maximize model's estimated uncertainty. UDPs have been shown to be effective in decreasing simplicity bias and improving generalization in non-adaptation settings. Formally, they can be defined & arg maxH(ggo 5), where H(go (h))
-cec Dc log Je, (e.g., entropy of predictions).
LP (Soup): Inspired by Wortsman et al. (2022), we train multiple, sparse, linear probes jointly and then take the average of their weights (aka soup) as the learned LP for subsequent FT. While soups of large models improve generalization by combining models from the same low-error basin, we consider sparse classifiers soups as an alternative strategy which seeks to average diverse decision rules, to avoid relying upon a single set of simple features. Formally, given classifiers, we minimize ming,... * Leu(go, (h). y) and let gø E fg for the FT step.
Empirical Evaluation of Hardness Promoting Augmentations. We evaluate the effectiveness of the above LP variants, which we collectively refer to Shardness-poomoting in mitigating the simplicity bias of LP+FT. We make the following observations (see Fig. 5).
Across all correlation strengths, find that using the modified hardness promoting LPs during LP+FT (aka hp-LP+FT) improves the Rand. OOD Accuracy over vanilla LP+FT(2 2%) and FT(> 20%). This elearly indicates that hp-LP+FT is indeed effective in decreasing reliance
7

---

Published as a conference paper at ICLR 2023
Generalization
Robustness
Calibration
Anomaly Det.
Rep. Similarity
ID
OOD
C
Adv.
ID
C
OOD.
Out-of -Caass
ID
Protocol
Acc.
Acc.
Acc.
Acc.
Acc.
1-RMS
1-RMS
1-RMS
1-RMS
AUROC
CKA
LP
0.9521
0.8124
0.7010
0,7378
0.2350
0,9313
0.3693
0.8802
0.9117
0,9907
1,0000
FT
0,9518
0.7168
0.7011
0.7164
0.1563
0,8873
0,9019
0,8604
0.9295
0,9794
0.7847
MP+FT
0.9643
0.8261
0.7426
0.7671
0.2135
0.9782
0,9472
0,9451
0,8742
0,9924
0,9887
(UPP)
0.9524
0.8110
0.7017
0.7382
0.2353
0.9308
0.8691
0.8801
0.9118
0,9908
1,0000
LP (VWT)
0,9524
0.8122
0,7010
0.7379
0.2345
0.9299
0.8682
0.8791
0.9103
0.9907
1,0000
TP (Soop)
0,9439
0.7996
0.6874
0.7290
0,2451
0.8806
0,7868
0.8094
0.9064
0,9897
1,0000
LP (UDP)+#2 FT
0.9637
0.8265
0.7448
0.7681
0.2157
0.9768
0,9464
0.9467
0.8757
0,9927
0.98927
LP (VAT) + FT
0.9647
0.8247
0.7425
0,7650
0.2224
0,9727
0,9521
0,9463
0,8775
0,9925
0,9893
LF (Soup) F2
0,9608
0.8163
0.7456
0.7684
0.1855
0.9760
0.9498
0.9492
0.8678
0,9936
0.98540
Table 3: Living17: Hardness Promoting Augmentation and Adaptation. In this low- dow-iistoniion adaptation setting, we see that vanilla LP+FT is an effective baseline and that hardness promoting variants of +P+FT tend to perform comparably.
simple features, potentially also leading to improved safety. Furthermore, with the exception of LP (Soup) +FT, hp-LP+FT also performs better than vanilla LP+FT on Corr. OOD accuracy. Vanilla FT does outperform all LF+FT protocols in this setting, but this is due to reliance upon simple features. Lastly, we observe that with respect to Corr. II) Accuracy that hp-I.P+FT improves performance at low correlation strength, but slightly loses performance at higher correlation strengths. This is not entirely unexpected as FT's reliance upon simple features will be useful in the correlated setting. Given that hp-LP +TT T is able to reduce reliance upon simple features in this controlled setting, we next evaluate whether these modified protocols are beneficial in improving the performance of LP +P+ real datasets.
5 EVALUATING GENERALIZATION AND SAFETY OF THE LP+FT FAMILY
Given the effectiveness of incorporating hardness promoting (hp) augmentations with the family of LP+FT protocols (hp-LP+ED FFT) in mitigating simplicity bias in synthetic setting, we further evaluate the modified protocols on the three real-world datasets (Living17. DomainNet, and CIFARIO) with respect to the generalization and safety metrics introduced in Sec. 3. We present our results in Tables 3,4, and 5) our observations are summarized below. Any method-specifie hyperparameters (e.g.. epsilon) are tuned using ID validation data and all results are reported over three seeds. We provide additional results Supp.
Results. As discussed in Sec. 3, these three datasets represent scenarios where different levels of distortion are necessary when adapting the pretrained model. On Living17, setting which requires minimal distortion during adaptation, we see that vanilla LP+FT is quite effective with respect to both generalization and safety metrics and is a difficult baseline to surpass. Indeed, while hp-LP+FT variants do not lead to significant benefits, they generally perform comparably to vanilla LP+FT. On DomainNet, a setting where fairly low distortion is required for LP FTT but FT struggles to find a good solution, to ID/OOD generalization and robustness, though vanilla LP and hp-LP have better calibration performance. In contrast on CIFARIO, which requires more distortion obtain an acceptable solution, we see that hp-LP +FT variants lead to improved generalization and a noticeable boost in corruption robustness. LP (VAT) + and LP (VAT) are particularly effective in this regard. Lastly, across all datasets, we observe that hp-LP+FT protocols lead to similar distortion to vanilla LP +FT, which suggests that any additional benefits of hp-LP+ES +FT should not be attributed to only reducing feature distortion.
that hp-LP+FT variants induce some slight benefits with respect
Discussion. We find that while vanilla LP+FT already an effective protocol, especially settings where low distortion is required, hp-LP+FT can provide some benefits and performs competitively. We suspect that the performance of these modified protocols can further be improved more sophisticated simplicity bias mitigation strategies are used. Indeed, our central claim, that adaptation protocols should mitigate feature distortion and simplicity, is not dependent on a specific strategy. We additionally note that while such mitigation strategies may optionally also be used during FT, they cannot solety be used in FT. Indeed, in the case of extreme simplicity. if the LP classifier relies upon simple features to find low-loss solution, during the subsequent FT step, gradients may not be
8

---

Published as a conference paper at ICLR 2023
Generalization
Robustness
Calibration
Anomaly Det.
Rep. Similarity
0OD
C
Adv.
ID
OOD.
Out-of-Class AUROC
Protocol
Acc.
Acc.
Acc.
Acc.
Acc.
1-RMS
1-RMS
1-RMS
1-RMS
CKA
LP
0.8913
0.8013
0.6019
0.6020
0.1768
0.9638
0.9264
0.9045
0.9014
0.8679
1,0000
FT
0.7613
0,4522
0,5186
0,2744
0.4164
0.8368
0.7234
0.7234
0.6379
0.8841
0.6092
LP+ PT
0.8985
0,7990
0.6343
0.5979
0.1927
0,9566
0.8445
0.8445
0.8899
0,9022
0.9222
(UD2)
0.8919
0.8021
0.6022
0.6101
0.9635
0.9250
0.9047
0.8619
0.8714
1.0000
T/P (VAT)
0,8836
0.7914
0.5893
0.5963
0.1687
0.8897
0.9552
0.8905
0.9178
0.8735
1,0000
SP (Soup)
0,8787
0.7977
0.5951
0.6048
0.1731
0.8844
0.9479
0.8861
0.9176
0.8661
1,0000
LP ( PT
0,9083
0,7965
0.6414
0.6178
0.1778
0.9436
0.8533
0.79415
0,752
0.8857
0,9662
LP (VAT)+FT FT
0.9048
0,8009
0.6466
0.6131
0.1942
0.9686
0.8911
0.8428
0.7965
0.9204
0.9370
LP (Soup)+ FT
0.9051
0,8013
0.6393
0.6091
0.1954
0.9670
0.9042
0.8692
0.8246
0.9097
0.9281
Table 4: DomainNet: Hardness Promoting Augmentations and Adaptation. While relatively low distortion is induced by LP+FT, FT FT struggles to find viable solution. Here, hardness- harddesss-roomoting LP+FT variants, particularly LP (VAT) +FT do slightly improve ID and OOD generalization as well as robustness corruptions.
Generalization
Robustness
Calibration
Anomaly Det.
Rep. Similarity
ID
OOD
C
C
Adv.
ID
C
C
OOD.
Out-of-Class AUROC
ID
Protocol
Acc.
Acc.
Acc.
Aoc.
Acc.
1-RMS
1-RMS
1-RMS
1-RMS
CKA
LP
0.9138
0,8190
0.6912
0.6553
0.0003
0,9595
0,8303
0,8142
0.8696
0.6206
1,0000
FT
0,9539
0.8754
0.7434
0,7883
0.0231
0,9668
0.8364
0,8453
0.9232
1,0000
0.6831
LP48T
0,9442
0.8678
0.6921
0.6790
0.0018
0.9521
0.7849
0.7721
0.8864
0.6511
0.7853
LP (UDP)
0,9033
0.8356
0.6948
0.6643
0.0003
0.9689
0.9111
0,9023
0.9277
0,9053
1,0000
TP (VAT)
0.8977
0.8251
0.6742
0.6483
0.0002
0.9265
0.9255
0.9139
0.9375
0.7200
1.0000
1P (Soup)
0.9052
0.8353
0.6917
0.6588
0.0003
0.9605
0.9205
0.9037
0,9364
0.8859
1.0000
LP (UDP)+ FT
0.944
0.8848
0,7028
0.6986
0.0004
0,9670
0,8472
0.8476
0.9237
0,9559
0,7764
(VAT)+
0,9611
0,8900
0.7442
0.7321
0.0027
0,9294
0.8355
0,8281
0.9178
0.8276
0,7839
LP (S
0,9466
0,8892
0.7031
0.6931
0.0001
0.9678
0.8390
0.8287
0.9216
0,9265
0.7806
Table 5: CIFAR10: Hardness Promoting Augmentations and Adaptation. In contrast to Living 17 and DomainNet, FT more effective than LP+FT in the safety metrics and performs comparably on ID/OOD generalization. However, hardness-prone promoting variants, particularly LP (VAT), see noticeable improvements with respect to generalization & corruptions, performing comparably to FT.
back propagated in directions that contain complex features. This entails that the decision boundary continues to rely upon simple features and is at risk of reduced safety performance. We provide further discussion in S.pp.B.2. To this end, we recommend incorporating hardness-promoting augmentations during LP as a potential safe-guard to simplicity bias.
CONCLUSION
In this paper, we took a closer look at the behavior of protocols designed for adapting large- scale pretrained models to downstream datasets. While it is argued that adaptation protocols should be designed to mitigate feature distortion (c.g.. LP+FT) in order to improve ID and OOD generalization, we found that when additional aspects of safe generalization are evaluated (c.g.. prediction calibration error, adversarial robustness etc.), mitigating feature distortion alone is not sufficient. We then considered the complementary perspective, that adaptation protocols should also mitigate simplicity bias. Using a synthetic dominoes dataset that allows for control over the correlation between simple and complex features, found that protocols have varying levels of effectiveness in reducing reliance upon simple features. While, as expected, FT, is most susceptible to simplicity bias, we see that LP+FT is able to balance both distortion and simplicity bias in settings where the correlation between simple and complex features is not too extreme. Motivated by the benefits of LP +P+ and given the known relationship between simplicity bias and sub-optimal generalization, we used "hardness- promoting" LP initializations (virtual adversarial, further improve LP +F''s performance. These modifications helped reduce LF+FT's reliance upon simple features on the synthetic dataset. On three real-world datasets, these modified protocols led to some improvements in safety and generalization performance, further validating the need to consider both distortion and simplicity bias when designing adaptation protocols.
eneertaintydriiven perturbations, sparse soups) to

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
We thank Ekdeep Singh Lubana for several helpful discussions during the course of this project. This work was performed under the auspices of the U.S. Department of Energy by the Lawrence Livermore National Laboratory under Contract No. DE-ACS2-007N22734, Lawrence Livermore National Security, LLC.and was supported by the LLNL-LDRD Program under Project No. 21-ERD-012. was also partially supported by the National Science Foundation under CAREER Grant No. IIS 1845491. PT began this work an intern at Lawrence Livermore National Laboratory.
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Feature purification: How adversarial training performs robust deep learning. In Proc. Symposium Foundations of Computer Science, FOCS, 2021.
Anders Andreassen, Yasaman Bahri, Behnam Neyshabur, and Rebecca Roelofs. The evolution of out-of-distribution robustness throughout fine-tuning, 2021.
Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Atsuto Maki, and Stefan Carlsson. Factors of transferability for a generic convnet representation. IEEE Trans. Pattern Anal. Mach. 2016.
Intell., 38(9))1790-111)
Alon Brutzkus, Amir Globerson, Eran Malach, and Shai Shalev-Shwartz. SGD learns over- parameterized networks that provably generalize on linearly separable data. In Proc. Int. Conf. on Learning Reprexentations (ICLR), 2017.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Proc. Ady. in Neural Information Processing Systems (NeurIPS), 2020.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proc. Int. Conf. on Computer Vision (ICCV), 2021.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In Proc. Int. Conf. on Machine Learning (ICML), 2020.
Xiangning Chen, Cho-Jui Hsieh, and Boqing Gong. When vision transformers outperform resnets without pretraining or strong data augmentations. In Proc. Int. Conf. on Learning Representations (ICLR), 2022.
Ekin Dogus Cubuk, Barret Zoph, Dandelion Mané, Vijay Vasudevan, and Quoc V. Le, Autoaugment: Learning augmentation policies from data. In Proc. Int. Conf. Computer Vision and Pattern Recognition (CVPR), 2019.
Ekin Dogus Cubuk, Barret Zoph, Jon Shlens, and Quoe Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2020.
Mohammad Reza Davari, Stefan Horoi, Amine Natik, Guillaume Lajoie, Guy Wolf, and Eugene Belilovsky. Reliability of CKA as a similarity measure in deep learning. Proc. Int. Conf. on Learning Representations (ICLR), 2023.
Terrance Devries and Graham Taylor. Improved regularization convolutional neural networks with cutout. CoRR, 00211108..4555.. 2017.
Frances Ding, Stan-Stanislas Denain, and Jacob Steinhardt. Grounding representation similarity with statistical testing. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2021.
Linus Ericsson, Henry Gouk, and Timothy M. Hospedales. How well do self-supervised models transfer? In Proc. Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
10

---

Published as a conference paper at ICLR 2023
Utku Evci, Vincent Dumoulin, Hugo Larochelle, and Michael C Mozer. Head2Toe: Utilizing intermediate representations for better transfer learning. In Proc. Int. Conf. on Machine Learning (ICML), 2022.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In Proc. Int. Conf. on Learning Representations (ICLR). 2019.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent- new approach to self-supervisee learning. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2020.
Suriya Gunasekar, Jason Lee, Daniel Soudry, and Nati Srebro. Implicit bias of gradient descent on linear convolutional networks. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2018.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proc. of the Int. Conf. on Machine Learning, (ICML), 2017.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. Momentum contrast for unsupervised visual representation learning. In Proc. Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2020.
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In Proc. Int. Conf. on Learning Representations, (ICLR), 2019.
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2019.
Dan Hendryeks, Norman Mu, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: simple data processing method to improve robustness and uncertainty. In Proc. Int. Conf. on Leurning Representations (ICLR), 2020.
Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ML safety. CoRR, abs/2109
2021.
Dan Hendrycks, Andy Zou, Mantas Mazcika, Leonard Tang, Bo Li, Dawn Song, and Jacob Steinhardt. Pixmix: Dreamlike pictures comprehensively improve safety measures. In Proc. lnt. Conf. Computer Vision and Pattern Recognition (CVPR), 2022.
Katherine L. Hermann, Ting Chen, and Simon Kornblith. The origins and prevalence of texture bias in convolutional neural networks. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2020.
Mi-Young Huh, Pulkit Agrawal, and Alexei A. Efros. What makes imagenet good for transfer learning? CoRR, abs/1608.0866 08.144.884..... 2016.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2019.
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew Gordon Wilson. On feature learning in the presence of spurious conrelations. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2022.
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. SMART: robust and efficient fine-tuning for pre-trained natural language models through principled regular- ized optimization. In Proc. Assn. for Computational Linguistics, ACL, 2020.
Simran Kaur, Jeremy M. Cohen, and Zachary C. Lipton. Are pereptuulll--aigneee gradients a general property of robust classifiers? CoRR, abs/1 91 910.08640, 2019.

---

Published as a conference paper at ICLR 2023
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. CoRR, abs/2204.02937, 2022.
Simon Kornblith, Jonathon Shlens, and Quoe V. Le. Do better imagenet models transfer better? In Proc. Int. Conf. on Computer Vision and Puttern Recognition (CVPR), 2019.
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang, Fine-tuning can distort pretrained features and underperform out-of-distriuution. In Proc. Int. Conf. on Learning Representations (ICLR), 2022.
Yoonho Lee, Annie Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. In Proc. Int. Conf. Learning Representations (ICLR), 2023a.
Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Out-of-distiibution robustness via disagreement. In Proc. Int. Conf. Learning Representations (ICLR), 2023b.
Hong Liu, Jeff HaoChen, Adrien Gaidon, and Tengyu Ma. Self-supervised learning more robust to dataset imbalance. CoRR, abs/21 10.05025, 2021.
Ekdeep Singh Lubana, Eric J. Bigelow, Robert P. Dick, David Krueger, and Hidenori Tanaka. Mechanistic mode connectivity, 2023.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proc. Int. Conf. on Learning Representations (ICLR), 2018.
Eric Mintun, Alexander Kirillov, and Saining Xie. On interaction between augmentations and corruptions in natural corruption robustness. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2021.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii Virtual adversarial training: regularization method for supervised and supervisd learning. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do wide and deep networks learn the same things? uncovering how neural network representations vary with width and depth. Proc. Int. Conf. on Learning Representations (ICLR), 2021.
Matteo Pagliardini, Gilberto Manunza, Martin Jaggi, Michael I. Jordan, and Tatjana Chavdarova. Improving generalization via uncertainty driven perturbations, 2022.
Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Trans. Knowl, Data Eng., 22 (10):1345-1359, 2010.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. In Proc. Int. Conf. on Machine Learning (ICML), 2021.
Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, patrick gallinari, and Matthieu Cord. Diverse weight averaging for out-of- -i-diiitintttonn generalization. In Proc. Adv. in Neural Information Processing Systems (NeurIPS). 2022.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adver- sarially robust imagenet models transfer better? In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2020.
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. In Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2020.
Daniel Soudry. Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro. The implicit bias of gradient descent separable data. J. Mach. Learn. Res., 19:70:1-70:57, 2018.
12

---

Published as a conference paper at ICLR 2023
Andreas Peter Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers. Transactions Machine Learning Research, 2022.
Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton van den Hengel. Evading the simplicity bias: Training diverse set of models discovers solutions with superior OOD generalization. In Proc. Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp-mixer: An all-mlp architecture for vision. Proc. Adv. in Neural Information Processing Systems (NeurIPS), 2021.
Francisco Utrera, Evan Kravitz, N. Benjamin Erichson, Rajiv Khanna, and Michael W. Mahoney. Adversarially--rainee deep nets transfer better: Illustration on image classification. In Proc. Int. Conf. on Learning Representations (ICLR), 2021.
Ross Wightman. Pytorch image models.
sytorch-immge-model 2019.
Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roclofs, Raphael Gontijo-Lppes Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proc. Int. Conf. on Machine Learning (ICML), 2022.
I. Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale semi- supervised learning for image classification. CoRR, tbss/995.00046, 2019.
Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh, Youngjoon Yoo, and Junsuk Choe. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proc. of Int. Conf. on Computer Vision, ICCV, 2019.
Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. Lit: Zero-shot transfer with locked-image text tuning. In Proc. Int. Conf. Computer Vision and Pattern Recognition (CVPR), 2022.
Jeffrey O. Zhang, Alexander Sax, Amir Zamir, Leonidas J. Guibas, and Jitendra Malik. Side Side-tuning: Network adaptation via additive side networks. In Proc. Euro. Conf. on Computer Vision (ECCV). 2019.
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning. Proc. IEEE, 109(1):43-76 2021.
13

---

Published as a conference paper at ICLR 2023
APPENDIX
A ADDITIONAL RELATED WORK
For comprehensive overview of transfer learning, please the surveys of Zhuang al. and Pan & Yang. Here, we discuss a few directly works directly relevant to our own.
Recently, Kumar et al. demonstrated that learning probing prior to fine-tuning (e.g.. LP+FT) can improve both in-distribution and out-of-distibbution performance when transferring to a downstream task given a highly expressive, pretrained model. They demonstrated that FT only modifies features in the ID representation subspace and not in other directions, which can lead higher OOD error as direction outside the ID subspace are necessary for OOD generalization. However, by initializing FTwith trained linear probe, feature distortion can be decreased since this initialization is closer to optimal model, and thus requires less distortion in ID subspace, preserving the expressiveness of the original model. Concurrently, Kirichenko et al. demonstrated that models are able to learn both core features and spurious features. However, classifiers can rely upon spurious features, harming performance on minority groups. To reduce the reliance on spurious features, they propose to retrain the classifier on a small amount of "re-weighting" data, that allows the model to leverage the core features instead of the spurious features.
Other modifications and heuristics have also been proposed to improve fine-tuning, including side- tuning (Zhang et al., 2019), which tunes small secondary network that is then combined with the original model, using larger/smaller learning rates for the classifier, as well as regularization based methods (Jiang al., 2020). We focus the LP+FT protocol, is principled and achieves strong OOD performance.
Additionally, several works have studied properties of the model that influence the effectiveness of transfer learning (Azizpour et al., 2016; Huh et al., 2016; Kornblith et al., 2019; Lee et al., 2023a;
Evci al., 2022: Lee et al., 2023b; Izmailov et al., 2022: Lubana al., 2023; Rame al., 2022), including the robustness of pretrained features (Salman et al., 2020; Utrera et al., 2021). While the connection between adversarial training and improved feature representations (Allen-Zhu & Li, 2021; Kaur et al., 2019) has been studied, we use virtual adversarial training during LP to learn better classifier that is less reliant upon simple features, and we do not use an adversarially trained feature extractor. Finally, we note that while we are, to the best of our knowledge, the first to consider this holistic evaluation of safcty and generalization in the context of transfer learning with highly expressive pretrained models, Hendrycks et al. have considered the trade-off's induced by different data augmentation strategies (Yun et al., 2019; Devries & Taylor, 2017; Hendrycks et al., 2020; Cubuk et al., 2019; 2020) on safety metrics in supervised learning. We emphasize that while our evaluation is similar, that our work focuses on a different context and contains an additional layer of complexity as we consider the interaction between adaptation protocols, generalization behavior and safety performance.
B EXPERIMENTAL DETAILS
for training details. In brief, we performed grid-search to find the best parameters, which are as follows. For CIFAR-10 and CIFAR-100, train only the classifier for 200 epochs with LR=30 during LP. For FT, the entire model is trained for 20 epochs with LR=le-5. For LP+FT, the model's 's classifier is initialized with the solution found by LP, and then it is fine-tuned for 20 epochs. A grid-search was conducted to determine the LR for LP and FT. For Domain-Net Experiments, we use 200 epochs with LR=30 during LP. For FT, the entire model is trained for 20 epochs with LR=3e-4. For LP+FTT, T, the model's classifier is initialized with the solution found by LP, and then it is fine-tuned for 20 epochs, using LR=3e-7. Furthermore, following Kumar et al., we freeze the batchnorm layers during LP+FT A CLIP (Radford et al., 2021) pretrained ResNet-50 is used for the DomainNet experiments, while MoCoV2 (He et al., 2020) is used for all CIFAR experiments. We use augmentation functions from timm (Wightman, 2019) and compute CKA scores using the packaged provided by torch-cka. When using augmented protocols, the same LRs are used. Note, all results were obtained by averaging over 3 seeds. We consider model soups of sizes 5, 10,20, tune in 0.005, 0.01, 0.02 and 0. for UDP, and
Please the https://github.com https://gttub..om/ppjaaccommpppp comp:/pjccoompets////..Aaaaaaaap
14

---

Published as a conference paper at ICLR 2023
in 0.001, 0.01, 0.1 for VAT. For CIFAR-MNIST results, LP is done for 100 epochs, and FT is done for 20 epochs.
B.1 MOTIVATION FOR WHRRDNESS-PRMMOTEE VARIANTS
We selected UDP (Pagliardini et al., 2022), VAT (Miyato al., 2017), and model- soups (Wortsman et al., 2022) simplicity bias mitigation strategies due to their effectiveness and ease of use. We emphasize, however, that our findings are not specific to the choice a given mitigation strategy and expect that advancements in such strategies will further improve the effectiveness of our proposed T.++avarssnts. At present, the selected strategies are strong, representative mitigations that we have confirmed are effective at mitigating simplicity bias in the adaptation context using the synthetic dominoes dataset in Sec. 4.
We conceptually justify each strategy here:
UDP is designed to help mitigate simplicity bias by learning by large margin classifier, opposed a narrow margin classifier that relies upon simple features. As noted by Shah et al. (2020), such narrow margin classifiers are sensitive to small perturbations and the simple features supporting the decision boundary may not be discriminative under distribution shifts. By maximizing uncertainty (instead of loss) to create adversarial perturbations, UDP is able to learn a maximum-marg classifier that is better able to handle such shifts. Notably, to create such maximum-mare margin classifier, the model will necessarily learn complex features;
We use virtual adversarial training (VAT) to help avoid reliance upon simple features, as VAT enforces distribution smoothness that classifiers become robust in some epsilon neighborhood around the input. We note that we are performing this training in the hidden representation space, so perturbations correspond may be altering high-level semantics. To maintain strong performance under such high-level perturbations, the model should learn to rely upon more complex features, and learn better margin classifier;
We model-soups that we may learn set of classifiers that rely upon disjoint sets of features. By learning a set of diverse classifiers, we are able to average classifiers that have learned to rely upon different features, instead of becoming overly reliant upon single simple feature. In future work, we intend to build a theoretical framework that helps us better justify these interventions and create new ones.
B.2 APPLYING SIMPLICITY BIAS MITIGATION STRATEGIES TO
STEP.
To demonstrate that simplicity bias mitigation
strategies must be applied during the LP step of for maximum effectiveness, conduct the following additional experiment.
1.55
0.4
0.00
Setup. We evaluate two additional protocols where V&T and UDP are applied only during the FT step, (LP+ (LP+T) and LP+FT LL+FT(UD?)), on the synthetic dominoes dataset. We plot the results for Randomized OOD Accuracy in Fig. 6.
0.78
0114
0.75
0.21
LP F7
LAUDO FT
LP O
LP- FROWNI
Figure 6: Applying Mitigation Strategies to FT. We create FT variants of our LP mitigation strate- gies and evaluate them on the synthetic dominoes dataset. We see that FT variants lose performance with respect to LP variants, indicating that inter- ventions must be undertaken during the LP step as originally proposed.
Results. Here, we see that, across three differ- ent correlation ratios, FT variants lose perfor- mance with respect the LP mitigation variants. Notably, LP+ FT (UDP) loses up to 4% perfor- mance with respect to LPDD)) FT. While per- formance drops
not large for VAT, nonetheless see that LP+ FT(VAT) loses performance with respect to LP(VAT)+ FT.
Our results in Fig. 6 support our conceptual argument that mitigation strategies must be undertaken during the LP step to ensure that subsequent FT is in a direction that preserves complex features; applying mitigation strategies during FT may be too late to avoid simplicity bias. We note that
15

---

Published as a conference paper at ICLR 2023
applying mitigation strategies during FT, in addition to LP, may further improve performance, and we will add these variants in the final version. We did not include a FT soup variant as it would be prohibitively expensive to train and average large soups of entire models (instead of classifiers). This highlights the computational efficiency of implementing mitigation strategies in the L. step itself.
C ADDITIONAL RESULTS
Below, we include results corresponding to different hyperparameters (number of souped classifiers, for vat, and 5 for udp).
Generalization
Robustness
Colibration
Anomaly Detection
Rep. Similarity
Protocol
OOD.
Out-of- Classs AUROC
Acc.
Acc.
Avc.
Acc.
Acc.
1-RMS
L-RMS
I-RMS
I-RMS
CKA
LP
0.9138
0.8190
0.6912
0.6553
0.0003
0.9595
0.8303
0.8142
0.8696
0.6206
1,0000
soup-5
0.0002
0.9788
0,8835
09108
D.8463
s0up-10
0.9199
03339
0.0085
0.6522
00003
019669
0.904
08956
0.9205
B8713
1.0000
LF+ soup-20
0.9052
0.8353
0.6917
0.6588
0.0003
0.9605
0.9205
0.9037
0.9361
0.88.5
1,0000
udp-0.005
0,6702
0.9729
0.8579
0.9017
0,8708
2F+ adp-0.01
0.9033
0.8356
0.6948
0.6643
0.0003
0.9689
0.9111
0.9023
0.9277
0,9033
1,00000
1PP4 adp-0.02
(18885
0.8281
0.6796
0.6922
0.0014
0.9655
0.9259
0.9142
0,9473
0.9217
1,0000
udp-0.1
0.9245
0.9531
0,8570
2F+ va:-0.001
0.9189
0.8276
0.6945
0.6606
0.0006
0,9714
0.8564
0.8442
0.8927
0.7159
1,00000
LF+ vat-0.01
0.8977
0.8251
0,6742
0.6483
0.0002
0.9265
0.9255
0.9139
0,9375
0,7200
1,0000
FT
0,7434
0,7553
0.0231
0,9668
0.8364
0,8453
0,9232
L,0000
0.6831
LP+IT
0,9442
0,8678
0.6921
0.6790
0.0018
0.9521
0.7849
0.7721
0.8864
0.6511
0,7853
(LP+ +1T
0,9639
0.899
0,9155
0,9020
0,7603
(LP+soup-10) +IT
0.9467
0.8357
0.7022
0.6907
0.0001
0,9660
0.8307
0.8182
0.9184
0.9161
0.7671
(LP (.Prsoup-20) IPT
0.9466
0.8892
0.7031
0.6931
0.0001
0.9678
D.8390
0.8287
0,9216
0.9265
0,7806
(LL.udp-0.008))
0.9643
0.8127
0.9119
0.9180
(LPrudp-0.01) +ET
0.9450
0.8369
0.7048
0.6977
0.0004
0.9642
0.8335
0.8311
0.9209
0,9419
0.7746
(.P.udp--0.2)) #FT
(39444)
0.8848
0.7028
0.6986
0.0004
0.9670
0.8472
0.8476
0.9237
0.9559
0.7764
(LP+uUp-0.1))
0.9355
0.9651
0.7382
(LPevat)-FT
0.9611
0.8900
0,7442
0.7321
0.0027
0.9294
0.8355
0.8281
0.9178
0.8276
0.7839
Table 6: CIFAR10, Hardness- Augmentations.
Generalization
Robustness
Colibration
Anomaly Detection
Rep. Similarity
Protocol
OOD
Adv.
ID
OOD.
Out-of-Class AUROC
ID
Acc.
Acc:
Acc:
Acc:
Acc.
1-RMS
1-RMS
1-RMS
1-RMS
CKA
1P
0.9521
0.8124
0.7010
0.7378
0.2350
0.9313
0.8693
0.8802
0.9117
0.9907
L00000
LB+ adp-0.005
0.9304
0.8699
0.9108
0.9907
L,000
1.P adp-0.01
0.9524
0.8110
0.7017
0.7382
0.2353
0.9908
0.8691
0,8801
0.9118
0.9908
1.000
1F+ adp-0.02
0.9500
0.8126
0.7036
0.7387
0.2373
0,9343
0.8621
0.8763
0,1135
0.9915
1,000
adp-0.1
0.8165
0.7220
0.2339
0.9032
0.8427
0.8990
0.9582
L,000
1F+ soup-5
0.9439
0.7996
0.6874
0.7290
0.2451
0.5806
0.7868
0.8094
0.9064
0.9897
L,0000
AP1 snup-
0.9373
0.7904
0.6767
0.7220
0,2547
0,8496
0.7478
0.7709
0.8841
0.9887
1,0000
soup-20
0.7082
0.2575
0.8056
0.7084
0.2305
0.8274
0.9867
L.0000
1P+ v.000.001
0.9524
0.8122
0,7010
0.7379
0.2345
0.9299
0,8682
0.8791
0,9103
0.9907
L,0000
FT
0.7011
0.7164
0.1563
0.9019
0.8604
0.9295
0.9794
0.7847
LF+FT
0.9643
0.8261
0.7426
0.7671
0.2135
0.9782
0.9472
0.9451
0.8742
0.9924
0.9887
(LP+udp-0.000)
0.9627
0.8243
0.9456
0.8736
0.9922
0.98950
(CP (.Prudp-0.0)) FTT
0.9627
0.8253
0.7436
0.7669
0.2133
0.9812
0.9454
0.9447
0.8737
0.9923
0.98957
(LP44dp-0.00)) 4FT
(39633
0.8265
0.7448
0.7681
0.2155
0.9768
D.9464
0.9467
0.8757
0.9927
0.58927
(:Psucp-00.))4F7
0.9614
0.8349
0.7689
0.2168
0.9808
0.9441
0.9420
0.8711
0.9912
0.9861
(.Posopp-S) FT
0.9608
0.8163
0.7456
0.7684
0.1855
0.9760
0.9498
0.9492
0.8678
0.9936
0.98540
(LP+somp-10) 10) FT
0.9580
0.8114
0.7445
0.7678
0.1753
0,9838
0.9503
0.9488
0187748
0.9938
0.588360
(LP+soup-20)+ FT
0.8165
0.7450
0.7684
0.9693
0.9503
0.9190
0.8609
0.9936
0.98190
(L (LB++at-0.00)+477 +2'
0.9647
0.8247
0,7425
0.7650
0.2224
0.9727
0.9521
0.9163
0.8775
0.9925
0.9370
Table 7: Living17, Hardness-Promoting Augmentations
16

---

Published as a conference paper at ICLR 2023
Graevalization
Rubustaasy
Calidration
Anomally Dotection
Rep. Similarit
Protacel
ID>
0OD
Skesch-C
Real-C
SkachhC
Real-C
Out-of-Closs AUROC
ID
Acc.
Acc.
Acc.
Acc.
Avc.
1-RMS
1-RMS
1-RMS
1-RMS
CKA
0.8913
0.8013
0.6019
0.6020
0.1768
0.9638
0.9264
0.945
09014
0.8679
1.0000)
L.P-angmxx
0.8997
0.7998
0.6336
0.6104
0.1872
0.9718
0.2300
0.263
0.003
0.8818
1,0000
LP LP-autoong
0.5944
0.8057
0.6419
0.6257
0.1857
0.9614
0.9357
0,9309
0,0022
0.5849
1,0000
L.Pranduug
0.8971
0.8090
0,6392
0.6232
0.1877
0.9559
0.9321
0
0,006
0.8875
1,0000
0.8836
0,7914
0,5893
0.5963
0.8897
0.9552
0.8905
09178
0.7335
L.0000
FT
0.7613
0.4522
0.5186
0.2764
0.1164
0.8368
0.6378
0.7234
0.5397
0.8811
0.6092
FTTangmix
0.8246
0.5233
0.5911
0.3408
0,4802
0.9308
0.8042
0.3665
0.6761
0,9255
0.5272
PT-antoong
0.7786
0.5161
0.5561
0.3166
0.4315
0.9157
0.7485
0.8346
0.7334
0,9231
0.7025
F74mndaug
0,7823
0.5610
0.3298
0,7970
0.8682
0.7444
0,9318
0.6477
0,8985
0.7990
0.9566
0.8899
0.8455
0.622
0.9047
0.8081
0.6673
0,5980
0.2597
0.9768
0,9200
0.9067
0.3443
0.9155
0.8811
L2*(2
0,9023
0.8028
0.6571
0.5851
0.2354
0:9830
0.9244
0.8990
0.8484
0,9034
0.9096
0
0.8099
0,6703
0.6152
0.2489
0.9786
0,9191
0
0.859
0,9252
0,9000
(LP+wat) +FI
0.9018
0.8911
0.7965
0.9201
(LP+va)) +0F (tuggiss)
0.9032
0.8024
0.6389
0.5896
0.2525
0.9769
0.9169
0.8929
0.8344
0.9212
0.8673
0.9003
0.8049
0.6600
0.5862
0.2531
0.9783
0.9178
0.9000
0.8381
0.9149
0.9244
(LP44E)))( (7 (rrandaagg
0.5006
0.8060
0.6651
0,5894
0.2622
0.9762
0.9197
0,8993
0.8414
0.9238
0.8955
Table 8: DomainNet, Diversity Promoting Augmentations and Generalization Trade-ofls.
Generalization
Robustness
Calibration
Anomaly Det.
Rep. Similarity
00D
ID
Out-of-Class AUROC
Protocol
Acc.
Acc.
Asc.
Acc.
I-RMS
1-RMS
1-RMS
1-RMS
CKA
0.9297
0.983
0,8532
0.7491
0,7077
0.9794
0.9006
0.9007
09301
0.9623
0.0668
LP soup-5
0.9220
0.9151
0.8315
0.7432
0,7050
0.9598
0.9232
0.9279
0.9623
0.9665
0.1399
7. soup-10
0.9156
0.9135
0,8833
0.7344
0.6985
0.9476
0.9221
0.9211
0.732
0.9602
0.1778
1P soup-20
0,9069
0.9064
0.8065
0.7216
0.6885
0.9279
0.9129
0.191
09714
0.9484
0.2617
L24 udp-0.005
0.9299
0,9092
0.8533
0.7494
0,7079
0,9794
0.9009
0,9003
0,9312
0.9614
0.0822
LF4 udp-0001
0.9298
0,9097
0.8535
0.7495
0.7083
0,9795
0.9007
0,9006
0.9316
0.9616
0.0850
udp-0.02
0.9291
0,9108
0.8538
0.7497
0.7088
0,9789
0.9012
0,9014
0.9335
0.9631
0,1017
LF+ udp-0.1
0.9238
0,9218
0.8377
0.7488
0.7111
0.9801
0.9154
0,9216
0.9517
0.9645
0,1478
vat00000
0.9298
0.9091
0.7078
0.9801
0.9012
0,9614
0.0784
LF+
0.9295
0.9094
0.7494
0.7060
0,9800
0.9040
0.9342
0,9632
0.0837
0.9275
0.9106
0.7087
0,9581
0.9246
0.9589
0.1528
FT
0.9724
0.8761
0.9218
0.8131
0.8074
0.9577
0.8429
0.8418
0.8855
0.9138
0.9317
L3+8T
0,9692
0.9387
0.9195
0,8106
0.7736
0.9151
0,8034
0,7743
0.9026
0.8919
0,5349
(LP+400p-5) 4P2
0.9685
0,9417
0,9210
0.8136
0,7787
0,9385
0.8079
0,7765
0,9102
0.8974
0,3315
(LF++50p-10) 4PT
0,9681
0,9411
0.9220
0,8178
0.7824
0,9382
0.8119
0.7796
0.9072
0.8933
0,5521
(L8+s0up:20)-4 427
0,9677
0,9395
0.9213
0.8164
0.7837
0.9385
0.8107
0.7817
0.9070
0,8964
0,5411
(LP+udp-00005)
0.9677
0.9297
0.7710
0.9422
0.8024
0.7718
0.8916
0.6428
(LP+udp-0.00))+ +FT
0.9677
0.9359
0.8098
0.7721
0.9417
0.8029
0.7732
0.8999
0,4239
(LPrudp-0.02) +FT
0.9687
0.9349
0.8136
0.7724
0.9437
0.8067
0.7736
0.8994
0.8981
0.5015
(1Prudp-O.1) IFT
0.9688
0.9423
0.8174
0.7811
0.9408
0.8130
0.7815
0.9072
0.9064
0.4496
(1P+vat- 0.00))4
0.9681
0.9366
0.9180
0.8111
0,7727
0.9422
0.8033
0.7732
0.9013
0.8962
0.5904
(12+vat-0.00)wFT
0.9689
0.9366
0.9168
0.8121
0.7766
0.9455
0.8062
0.7791
0.9013
0.8918
0.5687
0.9692
0,9402
0.9207
0.8127
0.7743
0.9420
0.8068
0.7734
0.9083
0.8978
0.4398
Table 9: CIFARIO with Resnet101/SimCL. Pretrained Model. We see that with a larger model, and different pretraining method, our proposed variants still have some benefits. We note that the baseline performance is also improved as a result of a more larger pretrained model.
17


Reviews:
[{"review_id": "t4Kbu4EHO48", "reviewer": "Reviewer_AzrZ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work the authors investigate protocols for utilizing pre-trained models for downstream tasks. Motivated by the observation that different protocols perform differently according to several metrics under varying levels of distribution shift, the authors propose to analyse such protocols under the light of simplicity bias, i.e. they verify that certain protocols tend yield simpler features. In order to improve the use of pre-trained models in terms of the considered safety metrics, the authors then propose to leverage training with perturbed versions of the input data, as well as a combination of multiple linear probes by averaging their weights (models soup). Experiments showed, for example, that in-distribution performance and safety in terms of the reported metrics are not consistent across protocols.\n", "summary_of_the_review": "This work investigates out-of-distribution generalization and safety aspects of different protocols for applying pre-trained models to downstream tasks. The authors empirically found that different approaches yield models with different performance in terms of out-of-distribution generalization and safety in terms of selected metrics. Those findings are explained through the lens of simplicity bias and I found this connection insightful, albeit a little not very well-motivated. My major concern with this work are its limitations with respect to the experimental protocol, especially because all conclusions are based solely on empirical evidence. Unfortunately, the authors only considered one type of architecture, two self-supervised tasks, and the ImageNet-1k dataset for pre-training. I believe this renders the findings in this submission limited and not sound. Moreover, the current manuscript contains presentations issues that compromise the clarity of the work. All in all, I believe the weaknesses of this work currently outweigh the merits, and I thus believe it is marginally below the acceptance bar. ", "strength_and_weaknesses": "- Strength\n  - The work tackles a relevant problem for the community: how to best leverage large pre-trained models to downstream tasks where safety-related metrics are also important?\n  - The empirical analysis is extensive in the sense it considers three protocols for employing pre-trained models, across three datasets, and multiple metrics. The authors explained those findings through the lens of simplicity bias in neural networks.\n\n- Weaknesses\n  - The contributions of this work are mostly empirical, which *is not a weakness per se*. My concern regarding this aspect is that the experiments were performed considering a rather restrictive, and perhaps also outdated, setting, which makes the findings of the work less relevant for the community. More specifically, the only architecture used throughout the experiment was a ResNet-50 trained with MoCo-V2 / CLIP using ImageNet-1k. Besides being very limited from an experimental perspective, this setting also does not match current trends in the community. I believe it is to diversify the types of architectures and pre-training tasks in order to increase the strength of the conclusions in this work. For example, I suggest the authors include experiments with ResNets-101 / 152 and Vision transformers, pre-trained with tasks such as SimCLR, and datasets such as ImageNet-21k. \n\n  - The above mentioned concern gets increased relevance given that in several parts of the manuscript the authors claim to be studying large-scale models. For example, in page 8, the authors mentioned \"In this work, we take a closer look at the behavior of protocols designed for adapting large-scale pretrained models to downstream datasets\" ), however, given that currently the term large-scale suggests bigger models and datasets, I am not confident I can agree the studied setting can be deemed *large-scale*. \n\n  - The manuscript lacks clarity mostly due to an excess of acronyms and typos (see next section for details).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "Utb27is7Z2", "reviewer": "Reviewer_QKt7", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies model adaptation.\nIt identifies simplicity bias as a weakness of the \"linear probe\" (LP) approach, which learns a linear classifier on top of pre-trained features.\nFeature distortion was previously identified as a weakness of the alternative approach of fine-tuning all of the pretrained weights (FT).\nThis led previous work to propose doing LP followed by fine-tuning the other weights (LP+FT).\nThis work shows that, while LP+FT addresses the issue of feature distortions present in FT, it may fail to address the issue of simplicity bias present in LP.\n\nThis observation is supported by experiments on synthetic and real data.\nAnd this leads the authors to propose methods of “hardness promoting” changes to the LP step of LP+FT.\nFurther experiments on synthetic and real data demonstrate the effectiveness of these approaches.\nAlthough the improvements on real data are perhaps relatively minor, they help demonstrate the scientific claim of the paper.", "summary_of_the_review": "The paper makes a solid and significant contribution to our understanding of model adaptation, which is central to modern large-scale deep learning.  It highlights, explains, and takes some small steps towards mitigating safety issues with leading approaches to model adaptation.\n", "strength_and_weaknesses": "\nStrengths:\n- The topic is significant and timely.  Robustness is a central concern in deep learning, and OOD robustness is a hot topic.  This paper identifies safety limitations of LP/FT/LP+FT that could be easy to overlook, given the strong performance of these methods (and especially LP+FT) on OOD generalization.\n- The investigation is insightful and effectively supports the paper’s claims.  Besides identifying these weaknesses, the paper provides a convincing explanation for them.\n\n\nWeaknesses:\n- The improvements in Section 5 aren’t that big.\n- The proposed mitigations aren’t clearly justified.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "2JN968WkGCX", "reviewer": "Reviewer_79RY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper empirically studies model adaptation with additional metrics such as calibration, robustness in addition to ID/OOD generalization. The study is conducted through the lens of simplicity bias and feature distortion, via studying the characteristics of linear probing and fine-tuning. Finally, the authors propose strategies to use during linear-probing which can decrease the simplicity bias and improve on OOD generalization and the additional metrics studied in the paper. ", "summary_of_the_review": "I think that overall, the paper is a strong empirical paper, inspite of the weaknesses stated. The insights from the paper that mitigating feature distortion might not be enough for metrics such as calibration etc and there exists a need to reduce the simplicity bias, are important. ", "strength_and_weaknesses": "Strengths:\n\n\t- The paper looks at additional metrics than solely the OOD generalization accuracy which is critical for model deployment. Although LP + FT  leads to less feature distortion and is a sweet spot for strong ID and OOD accuracy, it is not the best method for achieving good performance on the other metrics such as calibration / robustness. \n\n\t- The authors conduct several experiments to understand the interplay between feature distortion and simplicity bias. In principle, the authors find that under slightly less correlation, a small amount of feature distortion is good for mitigating simplicity bias. There are similar empirical observations throughout the paper which are informative.\n\n\t- The proposed plugin modules for LP leads to good improvements over FT or LP+FT, for Rand. OOD accuracy, which shows that these strategies can be leveraged to mitigate simplicity bias. Also these modules lead to improvements in the other metrics such as robustness and calibration.\n\nWhile the paper does an extensive study on feature distortion and simplicity bias, I have certain comments on the weaknesses/comments./questions of the paper: \n\nWeakness:\n\t- The proposed method for improving LP or LP+FT,  though motivated are not completely new, but are already existing in the literature.\n \n\t- It would be beneficial if the authors can give more intuition on the specific choice of pre-trained models? The paper would be strengthened if a larger variety of pre-trained models are evaluated on. \n\n\t- Would the proposed strategies also help in improving performance on OOD derivatives of the Imagenet validation set (e.g., Imagenet-A, Imagenet-C) . For e.g., train with LP+FT using the strategies on the training set of Imagenet and compute downstream test performance on Imagenet-A, C,R and ObjectNet?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-VubrRM-ITf", "reviewer": "Reviewer_WDPP", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the role of feature distortion and simplicity bias in the context of fine-tuning pre-trained models. They evaluate fine-tuned models using multiple metrics for model robustness / reliability; these findings on CIFAR10, DomainNet, Living17 show that simplicity bias, in addition to feature distortion, is needed to understand whether models adapt reliably. The paper also considers semi-synthetic datasets (with known simple features) to showcase the role of simplicity bias in model adaptation. Then, the paper considers multiple variants of LP+FT in order to mitigate simplicity bias (to some extent). ", "summary_of_the_review": "Please see weaknesses listed in the previous section.", "strength_and_weaknesses": "Strengths:\n\n1. Thorough empirical evaluation. The paper considers multiple variants of fine-tuning / adaptation, multiple evaluation metrics to measure model reliability in multiple axes, and multiple real-world and semi-real datasets. \n\n2. Interesting perspective on model adaptation. The finding that simplicity bias (previously studied in trained-from-scratch) settings also plays an important role in the pretrain-and-finetune paradigm is insightful. \n\nWeaknesses:\n\n1. Hard to read. The paper (especially first few sections) lacks focus. I don't know what exactly the paper is trying to do (and what the main contributions are) even after re-reading the first 2-3 sections. \n\n2. Empirical sections need re-organization. There is a lot going on in Section 3. It looks into (a) feature distortion insufficient to explain reliable adaptation, (b) motivates simplicity bias and (c) discusses the effect of LP initialization to mitigate simplicity bias with just one table. I would break this up into multiple smaller experiments to clearly showcase the findings. \n\n3. Train from scratch baseline. An important but missing worst-case baseline for simplicity bias is if you train from scratch directly on the downstream task (i.e., without adaptation). \n\n4. Role of LP initialization in mitigating simplicity bias understudied. I think the paper lacks concrete experiments that show that LP initialization is a major source of simplicity bias, given that it motivates the hardness-promoting variants in the later section(s).\n\n5. (Minor) CKA unreliable as a metric. There are multiple papers that show that representation similarity metrics have failure modes (https://arxiv.org/abs/2108.01661, https://openreview.net/forum?id=8HRvyxc606). Using multiple evaluation metrics for representation similarity is one way to sidestep this issue.\n\n6. (Minor) LP+FT as good as LP+FT variants. The LP+FT variants do not consistently improve model reliability in practice even thought it improves Rand OOD accuracy on semi-synthetic datasets. Having a discussion on this discrepancy might be useful. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%