Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the role of feature distortion and simplicity bias in the context of fine-tuning pre-trained models. They evaluate fine-tuned models using multiple metrics for model robustness / reliability; these findings on CIFAR10, DomainNet, Living17 show that simplicity bias, in addition to feature distortion, is needed to understand whether models adapt reliably. The paper also considers semi-synthetic datasets (with known simple features) to showcase the role of simplicity bias in model adaptation. Then, the paper considers multiple variants of LP+FT in order to mitigate simplicity bias (to some extent). 
strength_and_weaknesses: Strengths:

1. Thorough empirical evaluation. The paper considers multiple variants of fine-tuning / adaptation, multiple evaluation metrics to measure model reliability in multiple axes, and multiple real-world and semi-real datasets. 

2. Interesting perspective on model adaptation. The finding that simplicity bias (previously studied in trained-from-scratch) settings also plays an important role in the pretrain-and-finetune paradigm is insightful. 

Weaknesses:

1. Hard to read. The paper (especially first few sections) lacks focus. I don't know what exactly the paper is trying to do (and what the main contributions are) even after re-reading the first 2-3 sections. 

2. Empirical sections need re-organization. There is a lot going on in Section 3. It looks into (a) feature distortion insufficient to explain reliable adaptation, (b) motivates simplicity bias and (c) discusses the effect of LP initialization to mitigate simplicity bias with just one table. I would break this up into multiple smaller experiments to clearly showcase the findings. 

3. Train from scratch baseline. An important but missing worst-case baseline for simplicity bias is if you train from scratch directly on the downstream task (i.e., without adaptation). 

4. Role of LP initialization in mitigating simplicity bias understudied. I think the paper lacks concrete experiments that show that LP initialization is a major source of simplicity bias, given that it motivates the hardness-promoting variants in the later section(s).

5. (Minor) CKA unreliable as a metric. There are multiple papers that show that representation similarity metrics have failure modes (https://arxiv.org/abs/2108.01661, https://openreview.net/forum?id=8HRvyxc606). Using multiple evaluation metrics for representation similarity is one way to sidestep this issue.

6. (Minor) LP+FT as good as LP+FT variants. The LP+FT variants do not consistently improve model reliability in practice even thought it improves Rand OOD accuracy on semi-synthetic datasets. Having a discussion on this discrepancy might be useful. 


clarity,_quality,_novelty_and_reproducibility: The paper in general is a bit hard to read. I'd recommend clarifying the contributions + goal in the first half and then re-organizing section 3. The results are novel as it considers the role of simplicity bias in the context of model adaptation. See other sections for additional details.
summary_of_the_review: Please see weaknesses listed in the previous section.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this work the authors investigate protocols for utilizing pre-trained models for downstream tasks. Motivated by the observation that different protocols perform differently according to several metrics under varying levels of distribution shift, the authors propose to analyse such protocols under the light of simplicity bias, i.e. they verify that certain protocols tend yield simpler features. In order to improve the use of pre-trained models in terms of the considered safety metrics, the authors then propose to leverage training with perturbed versions of the input data, as well as a combination of multiple linear probes by averaging their weights (models soup). Experiments showed, for example, that in-distribution performance and safety in terms of the reported metrics are not consistent across protocols.

strength_and_weaknesses: - Strength
  - The work tackles a relevant problem for the community: how to best leverage large pre-trained models to downstream tasks where safety-related metrics are also important?
  - The empirical analysis is extensive in the sense it considers three protocols for employing pre-trained models, across three datasets, and multiple metrics. The authors explained those findings through the lens of simplicity bias in neural networks.

- Weaknesses
  - The contributions of this work are mostly empirical, which *is not a weakness per se*. My concern regarding this aspect is that the experiments were performed considering a rather restrictive, and perhaps also outdated, setting, which makes the findings of the work less relevant for the community. More specifically, the only architecture used throughout the experiment was a ResNet-50 trained with MoCo-V2 / CLIP using ImageNet-1k. Besides being very limited from an experimental perspective, this setting also does not match current trends in the community. I believe it is to diversify the types of architectures and pre-training tasks in order to increase the strength of the conclusions in this work. For example, I suggest the authors include experiments with ResNets-101 / 152 and Vision transformers, pre-trained with tasks such as SimCLR, and datasets such as ImageNet-21k. 

  - The above mentioned concern gets increased relevance given that in several parts of the manuscript the authors claim to be studying large-scale models. For example, in page 8, the authors mentioned "In this work, we take a closer look at the behavior of protocols designed for adapting large-scale pretrained models to downstream datasets" ), however, given that currently the term large-scale suggests bigger models and datasets, I am not confident I can agree the studied setting can be deemed *large-scale*. 

  - The manuscript lacks clarity mostly due to an excess of acronyms and typos (see next section for details).
clarity,_quality,_novelty_and_reproducibility: - From the manuscript, it is not very clear to me what motivated connecting the empirical findings about the disparities in the safety metrics with the notion of simplicity bias. Moreover, would the same findings hold in case different metrics and / or datasets / architectures were considered?
- As mentioned in the previous section, I have concerns regarding the experimental setting currently adopted in the work as I found it too limited and not supporting some claims in the manuscript. It is important to emphasize here that I do not think performing experiments with the setting considered in this work in a weakness or issue per se, my concern is that the contributions are solely based on the experiments, which have a narrow scope, restricting the strength and generalization of the findings. Also, the authors claim to be studying large-scale settings, which does not seem to be the case to me.
- Even though Table 3 indicates the strategies to mitigate simplicity bias were helpful to improve the safety metrics, it is not clear whether those improvements are in fact a consequence of attenuating simplicity bias in the studies cases. 
  - Perhaps one way to show this empirically could be reporting the gap in classification performance between the hardest and easiest classes (in terms of accuracy) in each dataset for models trained with and without the bias mitigation strategies. I believe this could be seen as a proxy measure for how vulnerability to the simplicity bias issue changed in each case. 
- The manuscript also lacks clarity due to an excess of acronyms (some of them were not defined in the text prior to use, such as DNNs), tables font is too small, and the text presents several typos. For example: 
  - Page 2, figure 1: Anamolies -> Anomalies  
  - Page 4: comprising -> compromising?
  - Page 5: sensitivity -> sensitive?
  - Add spaces between acronyms and the following word: e.g. Page 2: FPand LP -> FP and LP

summary_of_the_review: This work investigates out-of-distribution generalization and safety aspects of different protocols for applying pre-trained models to downstream tasks. The authors empirically found that different approaches yield models with different performance in terms of out-of-distribution generalization and safety in terms of selected metrics. Those findings are explained through the lens of simplicity bias and I found this connection insightful, albeit a little not very well-motivated. My major concern with this work are its limitations with respect to the experimental protocol, especially because all conclusions are based solely on empirical evidence. Unfortunately, the authors only considered one type of architecture, two self-supervised tasks, and the ImageNet-1k dataset for pre-training. I believe this renders the findings in this submission limited and not sound. Moreover, the current manuscript contains presentations issues that compromise the clarity of the work. All in all, I believe the weaknesses of this work currently outweigh the merits, and I thus believe it is marginally below the acceptance bar.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: This paper studies model adaptation.
It identifies simplicity bias as a weakness of the "linear probe" (LP) approach, which learns a linear classifier on top of pre-trained features.
Feature distortion was previously identified as a weakness of the alternative approach of fine-tuning all of the pretrained weights (FT).
This led previous work to propose doing LP followed by fine-tuning the other weights (LP+FT).
This work shows that, while LP+FT addresses the issue of feature distortions present in FT, it may fail to address the issue of simplicity bias present in LP.

This observation is supported by experiments on synthetic and real data.
And this leads the authors to propose methods of “hardness promoting” changes to the LP step of LP+FT.
Further experiments on synthetic and real data demonstrate the effectiveness of these approaches.
Although the improvements on real data are perhaps relatively minor, they help demonstrate the scientific claim of the paper.
strength_and_weaknesses: 
Strengths:
- The topic is significant and timely.  Robustness is a central concern in deep learning, and OOD robustness is a hot topic.  This paper identifies safety limitations of LP/FT/LP+FT that could be easy to overlook, given the strong performance of these methods (and especially LP+FT) on OOD generalization.
- The investigation is insightful and effectively supports the paper’s claims.  Besides identifying these weaknesses, the paper provides a convincing explanation for them.


Weaknesses:
- The improvements in Section 5 aren’t that big.
- The proposed mitigations aren’t clearly justified.
clarity,_quality,_novelty_and_reproducibility: Clarity:
The paper was moderately clear, but there is significant room for improvement.  The discussion of the pros and cons of FT vs. LP are buried in the text, and not as easy to extract as they could be; It might help to outline the structure of the paper including the core claims (some of which would be novel and some of which were previously demonstrated), along the lines of the summary I provided.  Feature distortion could be explained more in the background section.  It wasn’t entirely clear why the “hardness promoting” methods are aiming to accomplish or why they will accomplish it.  There are a few typos I stumbled on.  What do bold/underline represent in the tables?

Quality:
The paper is high quality.  The experiments are informative and thorough, including synthetic and real datasets.  They effectively support the central claims, which are noteworthy.

Novelty:
The novelty is moderate.  The paper is an in-depth analysis of how simple model adaptation approaches work and how they might fail.  Nonetheless, this is a significant contribution given recent results suggesting these simple model adaptation approaches are highly effective.
The weaknesses of these approaches identified in this work are important for the community to know about.

summary_of_the_review: The paper makes a solid and significant contribution to our understanding of model adaptation, which is central to modern large-scale deep learning.  It highlights, explains, and takes some small steps towards mitigating safety issues with leading approaches to model adaptation.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper empirically studies model adaptation with additional metrics such as calibration, robustness in addition to ID/OOD generalization. The study is conducted through the lens of simplicity bias and feature distortion, via studying the characteristics of linear probing and fine-tuning. Finally, the authors propose strategies to use during linear-probing which can decrease the simplicity bias and improve on OOD generalization and the additional metrics studied in the paper. 
strength_and_weaknesses: Strengths:

	- The paper looks at additional metrics than solely the OOD generalization accuracy which is critical for model deployment. Although LP + FT  leads to less feature distortion and is a sweet spot for strong ID and OOD accuracy, it is not the best method for achieving good performance on the other metrics such as calibration / robustness. 

	- The authors conduct several experiments to understand the interplay between feature distortion and simplicity bias. In principle, the authors find that under slightly less correlation, a small amount of feature distortion is good for mitigating simplicity bias. There are similar empirical observations throughout the paper which are informative.

	- The proposed plugin modules for LP leads to good improvements over FT or LP+FT, for Rand. OOD accuracy, which shows that these strategies can be leveraged to mitigate simplicity bias. Also these modules lead to improvements in the other metrics such as robustness and calibration.

While the paper does an extensive study on feature distortion and simplicity bias, I have certain comments on the weaknesses/comments./questions of the paper: 

Weakness:
	- The proposed method for improving LP or LP+FT,  though motivated are not completely new, but are already existing in the literature.
 
	- It would be beneficial if the authors can give more intuition on the specific choice of pre-trained models? The paper would be strengthened if a larger variety of pre-trained models are evaluated on. 

	- Would the proposed strategies also help in improving performance on OOD derivatives of the Imagenet validation set (e.g., Imagenet-A, Imagenet-C) . For e.g., train with LP+FT using the strategies on the training set of Imagenet and compute downstream test performance on Imagenet-A, C,R and ObjectNet?

clarity,_quality,_novelty_and_reproducibility: The clarity of the paper is good, however the paper lacks in novelty in terms of techniques proposed. Though the empirical analysis is strong and can be a good addition in understanding OOD generalization. 
summary_of_the_review: I think that overall, the paper is a strong empirical paper, inspite of the weaknesses stated. The insights from the paper that mitigating feature distortion might not be enough for metrics such as calibration etc and there exists a need to reduce the simplicity bias, are important.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: 
The paper studies the role of feature distortion and simplicity bias in model adaptation with additional safety metrics (calibration, robustness, ID/OOD generalization). The proposed modified LP+FT protocols decrease the simplicity bias, and improve on OOD generalization and the additional metrics. 

Strength: 
1. The research problem on model adaptation is important for the community.
2. The study is insightful. The evaluation is thorough and effectively supports the paper’s claims.

Weakness:
1. There are few concerns on the clarity of the work.
2. [Minor] "safety" is an overloaded/confusing word. I wonder whether robustness or others will be more appropriate. 


justification_for_why_not_higher_score: There are some concerns on the clarity of the work. 
justification_for_why_not_lower_score: 1. The research problem on model adaptation is important for the community.
2. The study is insightful. The evaluation is thorough and effectively supports the paper’s claims.
Decision: Accept: notable-top-25%