Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the reliance of deep learning models on spurious correlations. In particular, the authors look at the quality of feature representations learned by models trained via ERM versus models trained using group robustness methods. They evaluate these feature representations by utilizing the Deep Feature Retraining (DFR) procedure: retraining the last layer of the model on a held-out set which would likely not contain spurious correlations present in the training set. This procedure helps reveal how much information about causal factors is present in the learned representations. The authors further explore how these feature representations are influenced by the model architecture, pre-training task and strategy, regularization (via weight decay, data augmentation), training length, and whether or not model has been trained on the target data. They find that the quality of these representations often depends heavily on choice of data augmentation, model architecture, and the pre-training strategy involved, while regularization and training time may not be as helpful in improving the quality of said representations. The authors share results on CelebA, Waterbirds, WILDS-FMOW, CXR, MultiNLI, and CivilComments datasets.

Overall, the paper presents interesting results and insights. I have some minor comments which I hope the authors will address during the response period.
strengths_and_weaknesses: ### Strengths:
- This work looks at an interesting and well motivated problem.
- The experimental setup is well designed and results offer insights that would be useful to future researchers.
- The paper is well written and organized.

### Weaknesses:
- The experiments on NLP datasets are based on a BERT model. While I understand that the goal here is not to create a state of the art model but to analyze representations learned by a model, significantly better models (DeBERTa, ERNIE, T5, etc) are out there that the authors could have used. 
- There are several works in NLP that have looked at the problem of spurious correlations ([1,2,3,4] are just a few examples), addressing them and understanding when models weigh causal features vs non causal features. The paper currently does not position itself well in that literature.

### Additional comments:
- Section 3, Preliminaries (Lines 99-102): This appears to be incorrect in light of [1]. In fact, as most Machine Learning tasks are anticausal, models will rely on spurious correlations regardless. As [1] show in their anticausal setup, a model will rely on spurious factors most of the time (unless the spurious features observe higher noise compared to the causal features).

[1] Kaushik, D., Setlur, A., Hovy, E. H., & Lipton, Z. C. Explaining the Efficacy of Counterfactually Augmented Data. ICLR 2021.
[2] Eisenstein, J. (2022). Uninformative Input Features and Counterfactual Invariance: Two Perspectives on Spurious Correlations in Natural Language. arXiv preprint arXiv:2204.04487.
[3] Veitch, V., D'Amour, A., Yadlowsky, S., & Eisenstein, J. Counterfactual invariance to spurious correlations in text classification. NeurIPS 2021.
[4] Kaushik, D., Hovy, E., & Lipton, Z. Learning The Difference That Makes A Difference With Counterfactually-Augmented Data. ICLR 2020.
questions: - Line 364: What changes in each of these three runs?
- Regarding results on CXR: You suggest that the lack of improvement is because the model already learns the features correctly and DFR is unable to improve much upon it. Could it also be the case that this is one of the cases of failure modes for DFR?
- Regarding effect of regularization on FMOW: Is it plausible that the trends are this way because models trained on FMOW are already regularized quite significantly (given the amount of noise in the images), and increasing regularization (through weight decay or MixUp) is weakening the signal further?
limitations: As this is an analysis paper, it is hard to understand the limitations and potential negative social impact, but I would urge the authors to think about potential negative impacts arising from misinterpretation of their analysis.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper considers deep learning in the common case where the training data contains spurious correlations. The main takeaway is that empirical risk minimization (alone) is sufficient to obtain state-of-the-art performance; specialized group robustness methods do not appear to provide a significant benefit. This is demonstrated on six datasets spanning both vision and text problems. The effect of the architecture, pretraining strategy, and regularization is also considered.
strengths_and_weaknesses: Spurious correlations are a concern when fitting neural networks, so this paper tackles an important problem. Overall, I found the presentation to be quite good and the experiments fairly convincing. My main issue with the work in its current form is that the scope (and therefore potential impact) of the work is more limited than the title and introduction suggest: the spurious correlations studied are *labeled* properties of the inputs, rather than latent spurious features. In most cases, one does not have access to labeled attributes (or even class labels!) when fitting a neural network, and therefore this work has a narrower scope than expected.

This being the case, I think it is notable that specialized group robustness methods appear to perform no better than ERM when it comes to learning the in the presence of spurious correlations. In addition, the empirical observations regarding regularization and other effects of the base model are interesting, although many of them rely on DFR, which AFAIK is not peer reviewed.

The DFR procedure is somewhat similar to what's done in contrastive learning, e.g. "Supervised Contrastive Learning," except there the second stage is performed on the original dataset. This often results in improved performance thanks to the contrastive objective. I'm curious how supervised contrastive learning would impact the results, both using the original dataset or the "reweighted" one.

I would have liked to see the analysis of pretraining to include additional experiments with text / BERT.

A smaller concern is that this paper leans heavily on Deep Feature Reweighting (DFR), which appears in a recent arxiv preprint (Kirichenko et al. [22]). Unfortunately, reading that pre-print is necessary to understand this work; reading the short description in S3 was not sufficient to follow along. It would be better to make this paper self-contained, especially given how simple the DFR idea is (e.g., define the reweighting dataset).


questions: For all datasets considered, I assume the labeled spurious attributes form an *incomplete* labeling (there always exists some latent spurious attributes). How do latent spurious features impact the conclusions of the work?
limitations: I would have liked to see the scope of the paper defined a bit more clearly. In addition, while both vision and text datasets are used, the majority of the experiments are only on the image datasets.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This is primarily an empirical paper about learning in the presence of spurious correlations. They run a lot of tests that show training a model with empirical risk minimization (ERM) followed by deep feature reweighting (DFR) (which is retraining last layer on some on a held-out set that doesn't have spurious correlations) yields results that are not too different from group robustness training (Group DRO)
strengths_and_weaknesses: Thanks to the authors for the hard work on this paper. I like it overall, and think it is a valuable contribution.

Strengths
------------
- A large amount of insightful experiments
- Clear writing
- Sensible comparisons and conclusions

Weaknesses
----------------
- The conclusion of this paper hinges on the idea that: "If two different methods get roughly the same held-out performance, then they must be learning the same kind of thing." I don't think that's necessarily true, and your experiments don't really prove it. The conclusion in lines 207-211 could be made stronger with other types of analyses. To have more certainty it's that "better weighting of the learned features rather than learning better representations of the core features" you could, for example, actually extract the representations learned from ERM+DFT and GDRO and see if they look similar. This experiment could be tricky, but if you make sure to have identical weight initialization, and run the experiment ~10 times, you could show that the feature spaces learned are really similar or not.
- Why is early stopping important for RWY, RWG and GDRO? Are there some simple experiments you could do to elucidate this?
questions: My questions are all integrated into "Strengths and Weaknesses"
limitations: Not required here.
review_title: null

Meta-review: metareview: The paper shows that empirical risk minimization is sufficient to obtain good worst-group accuracies and specialized group robustness methods do not appear to provide additional benefits.

The reviewers pointed out that the current work depends on DFR, which seems to require some additional data compared to group robustness methods. The reviewers also note that the NLP experiments did not use more recent models, and the authors addressed these issues. Generally the reviewers think this is a well-executed paper on an important problem, and are unanimous in accepting it. 
confidence: Certain
Decision: Accept