Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations
Abstract
Neural network classifiers can largely rely on simple spurious features, such as backgrounds, to make predictions.
However, even in these cases, we show that they still often learn core features associated with the desired attributes of the data, contrary to recent findings.
Inspired by this insight, we demonstrate that simple last layer retraining can match or outperform state-of-the-art approaches on spurious correlation benchmarks, but with profoundly lower complexity and computational expenses.
Moreover, we show that last layer retraining on large ImageNet-trained models can also significantly reduce reliance on background and texture information, improving robustness to covariate shift, after only minutes of training on a single GPU.
1. Introduction
Realistic datasets in machine learning are riddled with spurious correlations — patterns that are predictive of the target in the train data, but that are irrelevant to the true labeling function.
For example, most of the images labeled as butterfly on ImageNet also show flowers (Singla & Feizi, 2021), and most of the images labeled as tench show a fisherman holding the tench (Brendel & Bethge, 2019).
Deep neural networks rely on these spurious features, and consequently degrade in performance when tested on examples where the spurious correlations break, for example, on images with unusual background contexts (Geirhos et al., 2020; Rosenfeld et al., 2018; Beery et al., 2018).
In an especially alarming example, CNNs trained to recognize pneumonia were shown to rely on hospital-specific metal tokens in the chest
X-ray scans, instead of features relevant to pneumonia (Zech et al., 2018).
Furthermore, spurious correlations can disproportionately affect different population groups (each with their own data distribution), leading to poor performance on the minority groups and unfair decisions (Buolamwini &
> . Gebru, 2018; Hovy & Søgaard, 2015; Blodgett et al., 2016;
Tatman, 2017; Hashimoto et al., 2018; Datta et al., 2015;
Chouldechova, 2017; Rahmattalabi et al., 2019).
In this paper, we investigate what features are in fact learned on datasets with spurious correlations.
We find that even when neural networks appear to heavily rely on spurious features and perform poorly on minority groups where the spurious correlation is broken, they still learn the core features sufficiently well.
These core features, associated with the semantic structure of the problem, are learned even in cases when the spurious features are much simpler than the core features (Appendix B) and in some cases even when no minority group examples are present in the training data (Appendix D)!
While both the relevant and spurious features are learned, the spurious features can be highly weighted in the final classification layer of the model, leading to poor predictions on the minority groups.
Inspired by these observations, we propose Deep Feature
Reweighting (DFR), a simple and effective method for improving worst-group accuracy of neural network classifiers in the presence of spurious features.
In DFR, we simply retrain the last layer of a classification model trained with standard Empirical Risk Minimization (ERM), using a small set of reweighting data where the spurious correlation does not hold.
DFR achieves state-of-the-art performance on popular spurious correlation benchmarks by simply reweighting the features of a trained ERM classifier, with no need to retrain the feature extractor.
Moreover, we show that DFR can be used to reduce reliance on background and texture information and improve robustness to certain types of covariate shift in large-scale models trained on ImageNet, by simply retraining the last layer of these models.
We note that the reason DFR can be so successful is because standard neural networks are in fact learning core features, even if they do not primarily rely on these features to make predictions, contrary to recent findings (Hermann & Lampinen, 2020; Shah et al., 2020).
Since DFR only requires retraining a last layer, amounting to logistic regression, it is extremely simple, easy to tune and computationally inexpensive relative to the alternatives, yet can provide state-of-the-art performance.
The rest of this paper is structured is follows.
In Section 2 we discuss the problem setting for group robustness.
In
Section 3 we study how well ERM-trained models learn the core features on the Waterbirds dataset in the presence of spurious correlations.
In Section 4 we describe the proposed method.
We provide benchmark results in Section 5.
Additionally, in the appendix we discuss the related work (App.
A), results on simplicity bias (App. B), background and texture reliance on ImageNet (App. E), and other experiments.
2. Problem Setting
We consider classification problems, where we assume that the data consists of several groups G i , which are often defined by a combination of a label and spurious attribute.
Each group has its own data distribution p i ( x, y ) , and the training data distribution is a mixture of the group distributions p ( x, y )
= (cid:80)
i
α
i p i ( x, y ) , where α i is the proportion of group G i in the data.
We will consider the scenario when the groups are not equally represented in the data.
The larger groups are referred to as majority groups and the smaller groups are referred to as minority groups .
As a result of this imbalance, features that are predictive of the target on the majority groups become spurious , i.e. features that are correlated with the target on the train data, but not predictive of the target on the minority groups.
In the paper we will discuss multiple examples of spurious correlations in both natural and synthetic datasets.
Ideally, we want to find classifiers that rely on core features , i.e. features that are predictive of the target on all the groups.
3. Understanding Representation Learning with Spurious Correlations
In this section we investigate the solutions learned by standard ERM classifiers on datasets with spurious correlations.
We show that while these classifiers underperform on the minority groups, they still learn the core features that can be used to make correct predictions on the minority groups.
We first consider the Waterbirds dataset (Sagawa et al., 2019) which is generated synthetically by combining images of birds from the CUB dataset (Wah et al., 2011) and backgrounds from the Places dataset (Zhou et al., 2017)
(see
Sagawa et al.
for a detailed description of the data generation process).
For the experiments in this section, we generate several variations of the Waterbirds data following the procedure analogous to Sagawa et al.
The Original dataset is analogous to the standard Waterbirds data, but we vary the degree of spurious correlation between the background and the target: 50% ( Balanced dataset), 95% (as in Sagawa et al.
and 100% (no minority group examples in the train dataset).
The FG-Only dataset contains images of the birds on uniform grey background instead of the Places background, removing the spurious feature.
We show examples of datapoints from each variation of the dataset in Appendix Figure 1.
[Table]
We train a ResNet-50 model with ERM on each of the variations of the data and report the results in Table 1.
Following prior work (e.g., Sagawa et al., 2019; Idrissi et al., 2021), we initialize the model with weights pretrained on ImageNet.
For the models trained on the Original data, there is a large difference between the mean and worst group accuracy on the Original test data: the model heavily relies on the background information in its predictions, so the performance on minority groups is poor.
The model trained without minority groups is especially affected, only achieving 38 .
4% worst group accuracy.
However, surprisingly, we find that all the models trained on the Original data can make much better predictions on the FG-Only test data: if we remove the spurious feature (background) from the inputs at test time, the models make predictions based on the core feature (bird), and achieve worst-group 1 accuracy close to 94% , which is only slightly lower than the accuracy of a model trained directly on the FG-Only data and comparable to the accuracy of a model that was trained on balanced train data.
In Appendix B.1, we provide the full details of this experiment, and also consider the reverse Waterbirds problem, where instead of predicting the bird type, the task is to predict the background type.
Notably, the results between the standard and reverse Waterbirds problems are fairly symmetric, e.g. in the case when the spurious correlation strength is 100% , the worst group accuracy for both problems is below random guess.
On the Waterbirds data, neural networks do not appear to be particularly biased towards either the background or the foreground, and treat them equally.
In Appendix B.3, we try to understand how the features from the foreground and the background interact with each other in a Waterbirds-trained neural network.
We find a phenomenon that we call logit additivity : the class logits on a Waterbirds test image are well approximated as the sum of the logits for the corresponding background image and the logits for the corresponding foreground image.
The logit addivity suggests that the background and foreground are processed mostly independently by the network.
In Appendix B.2, we perform a similar experiment in the setting where the spurious feature is much simpler than the core feature, and show that standard ERM models often still learn a high quality representation of the core feature.
In summary, we conclude that while the models trained on the Original data make use of background information to make predictions, they still learn the features relevant to classifying the birds almost as well as the models trained on the data without spurious correlations .
Later, we will see that we can retrain just the last layer of the Original-trained models and dramatically improve the worst-group performance on the Original data, by emphasizing the features relevant to the bird instead of the background.
4. Deep Feature Reweighting
In Section 3 we have seen that neural networks trained with standard ERM learn multiple features relevant to the predictive task, such as features of both the background and the object represented in the image.
Inspired by these observations, we propose Deep Feature Reweighting (DFR) , a simple and practical method for improving robustness to spurious correlations and distribution shift.
Let us assume that we have access to a dataset D
= { x i , y i } which can exhibit spurious correlations.
Furthermore, we assume that we have access to a (typically much smaller) dataset ˆ D , where the groups are represented equally.
ˆ D can be a subset of the train dataset D , or a separate set of datapoints.
We will refer to ˆ D as reweighting dataset .
We start by training a neural network on all of the available data D with standard ERM without any group or class reweighting.
For this stage we do not need any information beyond the training data and labels.
Here, we assume that the network consists of a feature extractor (such as a a sequence of convolutional layers or a vision transformer), followed by a fully-connected classification layer mapping the features to class logits.
In the second stage of the procedure, we simply discard the classification head and train a new classification head from scratch on the available balanced data ˆ D .
We use the new classification head to make predictions on the test data.
We illustrate DFR in Appendix Figure 4.
We will use notation DFR ˆ DD to specify the datasets used in each stage of the DFR procedure: D is used to train the base feature extractor model and ˆ D is used to train the last linear layer.
We will focus on DFR ValTr in the main text, where we use a group-balanced subset of the validation data available for each of the problems as the reweighting dataset D .
We consider other DFR variations in Appendix C.
5. Feature Reweighting Improves Robustness
In this section, we evaluate DFR on benchmarks with spurious correlations.
Data.
Following Idrissi et al.
(2021), we consider two image classification and two text classification benchmarks:
Waterbirds, hair-color prediction on CelebA (Liu et al., 2015), MultiNLI (Williams et al., 2017; Sagawa et al., 2019) and CivilComments-Coarse (Borkan et al., 2019; Koh et al., 2021).
On CivilComments, following Idrissi et al.
we use a binary spurious attribute ( coarse version of the dataset), which corresponds to the mention of any one of the protected identities (male, female, LGBT, black, white,
Christian, Muslim, other religion) in the comment.
We provide a detailed description of all datasets in Appendix D.
See Figure Appendix 5 for a visualization of the data.
Baselines.
We consider 5 baseline methods that work under different assumptions on the information available at the training time.
Empirical Risk Minimization ( ERM ) represents conventional training without any procedures for improving worst-group accuracies.
Just Train Twice ( JTT ) (Liu et al., 2021) is a method that detects the minority group examples on train data, only using group labels on the validation set to tune hyper-parameters.
Group-DRO (Sagawa et al., 2019) is a state-of-the-art method, which uses group information on train and adaptively upweights worst-group examples during training.
SUBG is ERM applied to a random subset of the data where the groups are equally represented, which was recently shown to be a surprisingly strong baseline (Idrissi et al., 2021; Sagawa et al., 2020).
Finally, Spread Spurious Attribute ( SSA ) (Nam et al., 2022) is a method that attempts to fully exploit the group-labeled validation data with a semi-supervised approach that propagates the group labels to the the training data where group information is not available.
We discuss the assumptions on the data for each of these baselines in Appendix F.1.
Hyper-parameters.
For all DRF variations, the size of the reweighting set ˆ D is small relative to the number of features produced by the feature extractor (2048).
For this reason, we use ℓ 1 -regularizaion to allow the model to learn sparse solutions and drop irrelevant features.
For DFR ValTr we only tune a single hyper-parameter — the strength of the regularization term.
We tune all the hyper-parameters on the validation data provided with each of the datasets.
We note that the prior work methods including Group-DRO,
JTT, SUBG and others extensively tune hyper-parameters on the validation data.
For DFR ValTr we split the validation in half, and use one half to tune the regularization strength parameter; then, we retrain the logistic regression with the
[Table]
optimal regularization on the full validation set.
For more details on the hyper-parameter tuning, see Appendix D.
Base models.
Following prior work (e g. Sagawa et al., 2019; Liu et al., 2021; Idrissi et al., 2021), we use ResNet50 initialized with weights pretrained on ImageNet on the vision datasets and a BERT model (Devlin et al., 2018) pretrained on Book Corpus and English Wikipedia data on the NLP data.
We select the weight decay and initial learning rate for the base model by choosing the hyperparameters that produce the base model with the best worstgroup accuracy on the validation data.
Note that the base model is tuned independently from DFR.
5.1. Results
We report the results for DFR and the baselines in Table 2.
On all datasets with DFR ValTr we obtain results competitive with state-of-the-art Group-DRO results.
DFR ValTr achieves similar performance to SSA, a method designed to make optimal use of the group information on the validation data.
Both methods use the same exact setting and group information to train and tune the model.
In Appendix D we provide results for other DFR variations.
In particular, DFR TrTr-NM matches or outperforms the stateof-the-art Group-DRO by using the same data to train and tune the model .
Notably, DFR TrTr-NM achieves these results with the features extracted by the network trained without seeing any examples from the minority groups !
Even without minority groups, ERM models extract the core features sufficiently well to achieve state-of-the-art results.
To sum up, with DFR we can match the performance of the best available method when group information is available on just the validation set (with DFR ValTr ) or on both the train and validation sets (with DFR TrTr-NM ).
This state-of-the-art performance is achieved by simply reweighting the features learned by standard ERM, with no need for advanced regularization techniques.
6. Natural Spurious Correlations on ImageNet
In the previous section we focused on benchmark problems constructed to highlight the effect of spurious features.
However, neural networks are known to learn undesirable patterns and rely on spurious features in real-world problems (see e.g. Rosenfeld et al., 2018; Geirhos et al., 2020).
In Appendix E we explore two prominent shortcomings of ImageNet classifiers: background reliance (Xiao et al., 2020) and texture bias (Geirhos et al., 2018).
7. Conclusion
We have shown that neural networks simultaneously learn multiple different features that can achieve low training loss, including relevant semantic structure, even in the presence spurious correlations.
By retraining the last layer of the network with DFR, we can significantly reduce the impact of spurious features and improve worst-group-performance of the models.
In particular, DFR achieves state-of-the-art performance on spurious correlation benchmarks, and can reduce the reliance of ImageNet trained models on background and texture information.
We provide further discus
Appendix Outline
This appendix is organized as follows.
In Section A we discuss the related work.
In Section B we present details on the experiments on feature learning in the presence of spurious correlations.
In Section C we provide further discussion on the proposed deep feature reweighting method.
In Section D we provide details on the results for the benchmark datasets.
We provide details on the experiments on the reliance of ImageNet-trained models on the background in and on the texture-bias in Appendix E. Finally, we provide a detailed discussion and conclusions in Appendix F.
Code references.
Our code is available at github.com/PolinaKirichenko/deep feature reweighting .
In addition to the experiment-specific packages that we discuss throughout the appendix, we used the following libraries and tools in this work: NumPy (Harris et al., 2020), SciPy (Virtanen et al., 2020), PyTorch (Paszke et al., 2017), Jupyter notebooks (Kluyver et al., 2016), Matplotlib (Hunter, 2007), Pandas (McKinney, 2010).
A. Related Work
Spurious correlations are ubiquitous in real-world applications.
There is hence an active research effort towards understanding and reducing their effect on model performance.
Here, we review the key methods and related work in this area.
Feature learning in the presence of spurious correlations.
The poor performance of neural networks on datasets with spurious correlations inspired research in understanding when and how the spurious features are learned.
Geirhos et al.
(2020) provide a detailed survey of the results in this area.
Several works explore the behavior of maximum-margin classifiers,
SGD training dynamics and inductive biases of neural network models in the presence of spurious features (Nagarajan et al., 2020;
Pezeshki et al., 2021; Rahaman et al., 2019).
Shah et al.
(2020) show empirically that in certain scenarios neural networks can suffer from extreme simplicity bias and rely on simple spurious features, while ignoring the core features; in
Appendix B.2 we revisit these problems and provide further discussion.
Hermann & Lampinen (2020) and Jacobsen et al.
(2018) also show synthetic and natural examples, where neural networks ignore relevant features, and Scimeca et al.
(2021) explore which types of shortcuts are more likely to be learned.
Kolesnikov & Lampert (2016) on the other hand show that on realistic datasets core and spurious features can often be distinguished from the latent representations learned by a neural network in the context of object localization.
In an independent and concurrent work, Rosenfeld et al.
(2022) show that standard ERM learns high quality representations in the context of domain generalization, and by training the last layer on the target domain it is possible to achieve strong results.
The observations in our work are complimentary, as they focus on different settings.
Spurious correlations in natural image datasets.
Multiple works demonstrated that natural image datasets contain spurious correlations that hurt neural network models (Kolesnikov & Lampert, 2016; Xiao et al., 2020;
Shetty et al., 2019; Alcorn et al., 2019; Singla & Feizi, 2021;
Singla et al., 2021; Moayeri et al., 2022).
Notably, Geirhos et al.
(2018) demonstrated that ImageNet-trained CNNs are biased towards texture rather than shape of the objects.
Follow-up work explored this texture bias and showed that despite being texture-biased, CNNs still often represent information about the shape in their feature representations (Hermann et al., 2020; Islam et al., 2021).
In Section 6 we show that it is possible to reduce the reliance of ImageNet-trained models on background context and texture information by retraining just the last layer of the model.
Improving group robustness.
The methods achieving the best worst-group performance typically build on the distributionally robust optimization (DRO) framework, where the worst-case loss is minimized instead of the average loss (Ben-Tal et al., 2013; Hu et al., 2018; Sagawa et al., 2019; Oren et al., 2019; Zhang et al., 2020).
Notably, Group-DRO (Sagawa et al., 2019), which optimizes a soft version of the worst-group loss holds state-of-the-art results on multiple benchmarks with spurious correlations.
Several methods have been proposed for the scenario where group labels are not known, and need to be inferred from the data; these methods typically train a pair of networks, where the first model is used to identify the challenging minority examples and define a weighted loss for the second model (Liu et al., 2021;
Nam et al., 2020;
Yaghoobzadeh et al., 2019; Utama et al., 2020; Zhang et al., 2021; Creager et al., 2021; Dagaev et al., 2021).
Other works proposed semi-supervised methods for the scenario where the group labels are provided for a small fraction of the train datapoints (Sohoni et al., 2021;
Nam et al., 2022).
Idrissi et al.
(2021) recently showed that with careful tuning simple approaches such as data subsampling and reweighting can provide competitive worst-group performance.
We note that all of the methods described above use a validation set with a high representation of minority groups to tune the hyper-parameters and optimize worst-group performance.
[Figure]
Another group of papers proposes regularization techniques to learn diverse solutions on the train data, focusing on different groups of features (Teney et al., 2021;
Lee et al., 2022; Pagliardini et al., 2022; Pezeshki et al., 2021).
Xu et al.
(2022) show how to train orthogonal classifiers , i.e. classifiers invariant to given spurious features in the data.
Other papers proposed methods based on meta-learning the weights for a weighted loss (Ren et al., 2018) and group-agnostic adaptive regularization (Cao et al., 2019; 2020).
Related methods have been developed in several areas of machine learning, such as
ML Fairness (Dwork et al., 2012; Hardt et al., 2016; Kleinberg et al., 2016; Pleiss et al., 2017; Agarwal et al., 2018; Khani et al., 2019), domain adaptation (Ganin & Lempitsky, 2015; Ganin et al., 2016) and domain generalization (Blanchard et al., 2011; Muandet et al., 2013; Li et al., 2018; Gulrajani & Lopez-Paz, 2020;
Ruan et al., 2021) including works on Invariant
Risk Minimization and causality (Peters et al., 2016; Arjovsky et al., 2019; Krueger et al., 2021; Aubin et al., 2021).
Transfer learning.
Transfer learning (Pan & Yang, 2009; Sharif Razavian et al., 2014) is an extremely popular framework in modern machine learning.
Multiple works demonstrate its effectiveness in computer vision (e.g. Girshick, 2015;
Huh et al., 2016;
He et al., 2017; Sun et al., 2017; Mahajan et al., 2018; Kolesnikov et al., 2020; Maddox et al., 2021), and study when and why transfer learning can be effective (Zhai et al., 2019; Neyshabur et al., 2020; Abnar et al., 2021;
Kornblith et al., 2019; Kumar et al., 2022).
Bommasani et al.
(2021) provide a comprehensive discussion of modern transfer learning with large-scale models.
While algorithmically our proposed DFR method is related to transfer learning, it has a different motivation and works on different problems.
We discuss the relation between DFR and transfer learning in detail in
Appendix C.
Our work contributes to understanding how features are learned in the presence of spurious correlations.
We show that even in the cases when ERM-trained models rely heavily on spurious features and underperform on the minority groups, they often still learn high quality representations of the core features.
Based on this insight, we propose DFR — a simple and practical method that achieves state-of-the art on popular benchmark datasets by simply reweighting the features in a pretrained ERM classifier.
B. Details: Understanding Representation Learning with Spurious Correlations
Here we provide details on the experiments in Section 3.
In Section B.2 we provide additional experiments on datasets where the spurious feature is much simpler than the core feature, used in the simplicity bias literature (Shah et al., 2020).
B.1. Details: Feature learning on Waterbirds
Inverse problem.
In Table 3 we present the results on the inverted Waterbirds problem, where the goal is to predict the background type while the bird type serves as a spurious feature.
The results for the inverted problem are analogous
[Table]
to the results for the standard Waterbirds (Table 1): models trained on the Original data learn the background features sufficiently well to predict the background type with high accuracy when the spurious foreground feature is not present, but perform poorly when presented with conflicting background and foreground features.
While it is often suggested than neural networks are biased to learn the background (Xiao et al., 2020), we see that in fact the network relies on the spurious foreground feature (bird) when trained to predict the background.
Data.
We show examples of Original, FG-Only and BG-Only Waterbirds images in Figure 1.
To generate the data, we follow the instructions at github.com/kohpangwei/group_DRO#waterbirds , but in addition to the Original data we save the backgrounds and foregrounds separately.
Consequently, the FG-Only data contains the same exact birds images as the Original data, and the BG-Only data contains the same exact places images as the Original data.
For the 100% spurious correlation strength we simply discard all the minority groups data from the Original (95% spurious correlation) training dataset.
For the Balanced data, we start with the Original data with 95% spurious correlation and replaced the background in the smallest possible number of images (chosen randomly) to achieve a 50% spurious correlation strength.
Hyper-parameters.
For the experiments in this section we use a ResNet-50 model pretrained on ImageNet, imported from the torchvision package: torchvision.models.resnet50(pretrained=
True) (Marcel & Rodriguez, 2010).
We train the models for 50 epochs with SGD with a constant learning rate of 10 − 3 , momentum decay of 0 .
9 , batch size 32 and a weight decay of 10 − 2 .
We use random crops ( RandomResizedCrop(224, scale=(0.7, 1.0), ratio=(0.75, 4./3.), interpolation=2) ) and horizontal flips ( RandomHorizontalFlip() ) implemented in torchvision.transforms as data augmentation.
B.2. Simplicity bias
Shah et al.
(2020) showed that neural networks can suffer from extreme simplicity bias , a tendency to completely rely on the simple features, while ignoring similarly predictive (or even more predictive) complex features.
In this section, we explore whether the neural networks can still learn the core features in the extreme simplicity bias scenarios, where the spurious features are simple and highly correlated with the target.
Following Shah et al.
(2020) and Pagliardini et al.
, we consider Dominoes binary classification datasets, where the top half of the image shows MNIST digits (LeCun et al., 1998) from classes { 0 , 1 } , and the bottom half shows MNIST images from classes { 7, 9 } ( MNIST-MNIST ),
Fashion-MNIST (Xiao et al., 2017) images from classes { coat, dress } ( MNIST-Fashion ) or CIFAR-10 (Krizhevsky et al., 2009) images from classes { car, truck } ( MNIST-CIFAR ).
In all Dominoes datasets, the top half of the image (MNIST 0 − 1 images) presents a linearly separable feature; the bottom half of the image presents a harder to learn feature.
See Appendix B.2 for more details about the experimental set-up and datasets, and Appendix Figure 1 for image examples.
We use the simple feature (top half of the images) as a spurious feature and the complex feature (bottom half) as the core feature; we generate datasets with 100% , 99% and 95% correlations between the spurious feature and the target in train data, while the core feature is perfectly aligned with the target 2 .
We then define 4 groups G i based on the values of the spurious and core features, where the minority groups correspond to images with top and bottom halfs that do not match.
The groups on validation and test are balanced.
We show data examples from Dominoes datasets in Figure 1.
20% of the training data
0 99 9999 1000 72 7883 920  6282 85 90  spur.
corr.
strength w o r s tg r o up a cc .
MNIST - MNIST MNIST - Fashion MNIST - CIFAR Original Acc.
Core-Only Acc.
Decoded Acc.
Optimal Acc. was reserved for validation or reweighting dataset
(see Section 4) where each group is equally represented.
We train a ResNet-20 model on each variation of the dataset.
In Figure 2 we report the worst group performance for each of the datasets and each spurious correlation strength.
In addition to the worst-group accuracy on the Original test data, we report the Core-Only worst-group accuracy, where we evaluate the model on datapoints with the spurious top half of the image replaced with a black image.
Similarly to Shah et al.
(2020) and Pagliardini et al.
(2022), we observe that when the spurious features are perfectly correlated with the target on Dominoes datasets, the model relies just on the simple spurious feature to make predictions and achieves 0% worst-group accuracy.
However, with 99% and 95% spurious correlation levels on train, we observe that models learned the core features well, as indicated both by their performance on the Original test data and especially increased performance on Core-Only test data where spurious features are absent.
For reference, on each dataset we also report the Optimal accuracy , which is the accuracy of a model trained and evaluated on the Core-Only data.
The optimal accuracy provides an upper bound on the accuracy that we can expect on each of the datasets.
Decoding feature representations.
The performance on the Original and even Core-Only data might not be fully representative of whether or not the network learned a high-quality representation of the core features.
Indeed, even if we remove the MNIST digit from the top half of the image, the network can still primarily rely on the (empty) top half in its predictions: an empty image may be more likely to come from class 1, which typically has fewer white pixels than class 0.
To see how much information about the core feature is contained in the latent representation, we evaluate the decoded accuracy: for each problem we train a logistic regression classifier on top of features extracted by the final convolutional layer of the network.
We use a balanced 3 validation set to train the logistic regression model, and then report the worst-group accuracy on a test set.
In Figure 2, we observe that for MNIST-MNIST and MNIST-FashionMNIST even when the spurious correlation is 100% , reweighting the features leads to high test accuracy.
Moreover, on all Dominoes datasets for 99% and 95% spurious correlations level the core features can be decoded with high accuracy and almost match the optimal accuracy.
This decoding serves as a basis of our DFR method, which we describe in detail in Section 4.
In Appendix B.2 we report an additional baseline and verify that the model indeed learns a non-trivial representation of the core feature, especially for spurious correlation strengths 99% and 95%.
Relation to prior work.
With the same Dominoes datasets that we consider in this section, Shah et al.
(2020) showed that neural networks tend to rely entirely on the simple features.
However, they only considered the 100% spurious correlation strength and accuracy on the Original test data.
Our results do not contradict their findings but provide new insights: even in these most challenging cases, the networks still represent information about the complex core feature.
Moreover, this information can be decoded to achieve high accuracy on the mixed group examples.
Hermann & Lampinen (2020) considered a different set of synthetic datasets, showing that in some cases neural networks fail to represent information about some of the predictive features.
In particular, they also considered decoding the information about these features from the latent representations and different spurious correlation strengths.
Our results add to their observations and show that while it is possible to construct examples where predictive features are suppressed, in many challenging practical scenarios,
−
Original Logits −
F GO n l y
L og i t s −
Original Logits −
B GO n l y
L og i t s −
FG-Only Logits −
B GO n l y
L og i t s −
Original Logits −
F G +
B G
L og i t s neural networks learn a high quality representation of the core features relevant to the problem even if they rely on the spurious features.
Finally, in a concurrent work, Rosenfeld et al.
(2022) show that ERM can learn features sufficient for strong performance in the domain generalization setting, adding further support to the results presented in this section.
In summary, we find that, surprisingly, if (1) the strength of the spurious correlation is lower than 100% or (2) the difference in complexity between the core and spurious features is not as stark as on MNIST-CIFAR, the core feature can be decoded from the learned embeddings with high accuracy .
Hyper-parameters.
We used a randomly initialized ResNet-20 architecture for this set of experiments.
We trained the network for 500 epochs with SGD with batch size 32, weight decay 10 − 3 , initial learning rate value 10 − 2 and a cosine annealing learning rate schedule.
For the logistic regression model, we first extract the embeddings from the penultimate layer of the network, then use the logistic regression implementation ( sklearn.linear model.
LogisticRegression )
from the scikit-learn package (Pedregosa et al., 2011).
We use ℓ 1 regularization and tune the inverse regularization strength parameter C in the range { 0 . 1 , 10 , 100 , 1000 } .
For more details on Deep Feature Reweighting implementation and tuning, see section D.
Transfer from simple to complex features.
In addition to the results presented in Figure 2, we measure the decoded accuracy using the model trained just on the spurious simple features: the top half of the image showing an MNIST digit.
After training, we retrain the last layer of the model using a validation split of the corresponding Dominoes dataset which has both top and bottom parts.
In this case, we measure the transfer learning performance with the features learned on the binary
MNIST classification problem applied to the more complex bottom half of the image.
We obtain the following transfer accuracy results: 92.5% on MNIST-MNIST, 92.1% on MNIST-Fashion and 61.4% on MNIST-CIFAR.
On all datasets, the decoded accuracy reported in Figure 2 for the spurious correlation levels 99% and 95% is better than the transfer accuracy.
For the 100% spurious correlation, transfer achieves comparable results on MNIST-Fashion and MNIST-CIFAR, but on
MNIST-MNIST the decoded accuracy with a model trained on the data with the core feature is significantly higher (99% vs 92%).
These results confirm that for the spurious correlation strength below 100%, the model learns a high quality representation of the core features, which cannot be explained by transfer learning from the spurious feature.
B.3. Logit additivity
To better understand why the models trained on the Original Waterbirds data perform well on FG-Only images, in Figurewe inspect the logits of a trained model.
We show scatter plots of logits for the negative class (logits for the positive class behave analogously) on the Original, FG-Only and BG-Only test data.
Both logits on FG-Only and BG-Only data correlate with the logits on the Original images, and FG-Only show a higher correlation.
The FG-Only and BG-Only logits are not correlated with each other, as in test data the groups are balanced and the foreground and background are independent.
We find that the sum of the logits for the BG-Only and the logits for the FG-Only images provides a good approximation of the logits on the corresponding Original image (combining the foreground and the background).
We term this phenomenon logit additivity : on Waterbirds, logits for the different predictive features (both core and spurious) are computed close to independently and added together in the last classification layer.
[Figure]
C. Details: Deep Feature Reweighting
We illsutrate DFR in Figure 4.
DFR versions.
We consider three variations of DFR ˆ DD , which differ by the way we construct the training dataset D and the balanced set ˆ D .
In DFR TrTr , we use a random group-balanced subset of the train data as ˆ D , i.e. we keep all of the data from the smallest group, and subsample the data from the other groups to the same size.
In DFR ValTr , we use a balanced subset of the validation data available for each of the problems.
For both DFR TrTr and DFR ValTr we use the standard training dataset (with group imbalance) as D .
Finally, with DFR TrTr-NM (NM stands for “No Minority”)
we use a random group-balanced subset of the train data as ˆ D , but remove the minority groups ( G 2 , G 3 on Waterbirds and G 4 on CelebA) from the data D used to train the feature extractor.
In order to make use of more of the available data, we train logistic regression 10 times using different random balanced subsets of the data, and average the weights of the learned models.
Here we can safely average the weights of the learned models because the models are linear.
By averaging, we are able to incorporate more of the majority group data without under-weighting the minority groups.
Full details are available in Appendix D.
Relation to transfer learning Algorithmically, DFR is a special case of transfer learning, where D serves as the source data, and ˆ D is the target data (Sharif Razavian et al., 2014).
However, the motivation of DFR is different from that of standard transfer learning: in DFR we are trying to correct the behavior of a pretrained model, and reduce the effect of spurious features, while in transfer learning the goal is to learn general features that generalize well to diverse downstream tasks.
For example, we can use DFR to reduce the reliance of ImageNet-trained models on the background or texture and improve their robustness to covariate shift (see Section 6), while in standard transfer learning we would typically use a pretrained model as initialization or a feature extractor to learn a good solution on a new dataset .
In the context of spurious correlations, one would not expect DFR to work as well as it does: the only reason DFR is successful is because, contrary to conventional wisdom, neural network classifiers are in fact learning substantial information about core features, even when they seem to rely on spurious features to make predictions .
Moreover, in Appendix Table 5 we show that transfer learning with features learned ImageNet does not work nearly as well as DFR on the spurious correlation benchmarks.
D. Additional Benchmark Results
In Table 4 we provide the results for all the DFR variations introduced in Appendix C. We focus on the image classification datasets for these experiments.
D.1. Datasets
Waterbirds.
The Waterbirds dataset is described in Figure 5.
The dataset contains images of birds from the CUB dataset (Wah et al., 2011) pasted on the backgrounds from the Places dataset (Zhou et al., 2017).
The spurious attribute s describes the type of background (water or land) and the core feature associated with the target y is the type of the bird (waterbird or
Waterbirds dataset
G
G G
G
ImageExamples
Description landbirdonland landbirdonwater waterbirdonland waterbirdonwater
Class Label    # Train data 3498 (73%) 184 (4%) 56 (1%) 1057 (22%)
# Val data
[Figure]
[Table]
[Table] [Figure] [Table] type; Minority : G 2 , G
CelebA hair color dataset
G
G G
G
ImageExamples
Description
Non [Figure] blondwoman
Non-blondman
Blondwoman
Blondman
Class Label    # Train data 71629 (44%) 66874 (41%) 22880 (14%) 1387 (1%)
# Val data
Target [Figure]
Spurious [Figure]
Minority : G landbird).
For a detailed description of the data generating process, see Sagawa et al.
The background is spuriously correlated with the bird type in the training datasets: waterbirds are more likely to be placed on a water background, and landbirds are more likely to be placed on a land background.
There are 4 groups defined to the tuples ( y, s ) .
CelebA.
The CelebA hair color dataset is described in Figure 5.
The dataset contains photos of celebrities from the CelebA dataset Liu et al.
The core attribute associated with the target y is the hair color (blond vs non-blond).
The gender serves as a spurious feature s : the vast majority of blond people in CelebA are female.
There are 4 groups defined to the tuples ( y, s ) .
Civil Comments
The dataset was originally collected in Borkan et al.
and is a part of the WILDS benchmark (Koh et al., 2021).
This dataset contains comments that are classified as toxic or not toxic.
We use the coarse version of the dataset, following Idrissi et al.
The spurious attribute s determines whether or not the comment mentions one of the following protected attributes: male, female, LGBT, black, white, Christian, Muslim, other religion.
These protected attributes are mentioned more frequently in toxic comments compared to neutral comments, constituting a spurious correlation.
There are 4 groups corresponding to the pairs ( y, s ) .
MultiNLI
The MultiNLI dataset contains pairs of sentences, and the label y describes the relationship between the sentences: contradiction, entailment or neutral.
The spurious attribute s describes the presence of negation words, which appear much more frequently in the examples from the negation class.
There are 6 groups corresponding to the pairs ( y, s ) .
[Table]
D.2. Details: Spurious Correlation Benchmarks
Here we provide details on the experiments in Section 5 and Table 4.
Base model hyper-parameters.
For the experiments presented in Table 6 we use a ResNet-50 model pretrained on
ImageNet, imported from the torchvision package: torchvision.models.resnet50(pretrained=
True) .
We use random crops ( RandomResizedCrop(224, scale=(0.7, 1.0), ratio=(0.75, 4./3.), interpolation=2) ) and horizontal flips ( RandomHorizontalFlip() ) implemented in torchvision.transforms as data augmentation.
We train all models with SGD with momentum decay of 0 .
9 and a constant learning rate.
On Waterbirds, we train the models for 100 epochs with weight decay 10 − 3 , learning rate 10 − 3 and batch size 32 .
On CelebA, we train the models for 50 epochs with weight decay 10 − 4 , learning rate 10 − 3 and batch size 128 .
We use cosine annealing learning rate schedule.
We do not use early stopping.
DFR details.
For DFR, we first extract and save the embeddings (inputs to the classification layer of the base model) of the training, validation and testing data using the base model.
We then preprocess the embeddings to have zero mean and unit standard deviation using the standard scaler sklearn.preprocessing.
StandardScaler from the scikit-learn package.
In each case, we compute the preprocessing statistics on the reweighting data used to train the last layer.
To retrain the last layer, we use the logistic regression implementation ( sklearn.
linear model.LogisticRegression
) from the scikit-learn package.
We use ℓ 1 regularization.
For DFR ValTr and DFR TrTr-NM we only tune the inverse regularization strength parameter C : we consider the values in range { 1 ., 0 . 7 , 0 . 3 , 0 . 1 , 0 . 07 , 0 .
and select the value that leads to the best worst-group performance on the available validation data (as described in Section 5, for DFR ValTr we use half of the validation to train the logistic regression model and the other half to tune the parameters at the tuning stage).
For DFR TrTr we additionally tune the class weights: we set the weight for one of the classes to 1 and consider the weights for the other class in range { 1 , 2 , 3 , 10 , 100 , 300 , 1000 } ; we then switch the classes and repeat the procedure.
For the final evaluation, we use the best values of the hyper-parameters obtained during the tuning phase and train a logistic regression model on all of the available reweighting data.
We train the logistic regression model on the reweighting data 10 times with random subsets of the data (we take all of the data from the smallest group, and subsample the other groups randomly to have the same number of datapoints) and average the weights of the learned models.
We report the model with averaged weights.
[Table]
All methods in Table 2 use ImageNet-trained models as initialization
Do we need ImageNet pretraining?
DFR relies on the ability of the feature extractor to learn diverse features, which may suggest that ImageNet pretraining is crucial.
In Table 5, we report the results on the same problems, but training the feature extractor from scratch.
We find that on Waterbirds, ImageNet pretraining indeed has a dramatic effect on the performance of DFR as well as the base feature extractor model.
However, on CelebA DFR shows strong performance regardless of pretraining.
The difference between Waterbirds and CelebA is that Waterbirds contains only 4.8 k training points, making it difficult to learn a meaningful feature extractor from scratch.
Furthermore, on both datasets, finetuning the feature extractor on the target data is crucial: just using the features extracted by an ImageNet-trained model leads to poor results.
We note that all the baselines considered in Table 2 use ImageNet pretraining.
What if we just use ImageNet features?
As a baseline, we apply DFR to features extracted by a model pretrained on
ImageNet with no fine-tuning on CelebA and Waterbirds data.
We report the results in Table 5 (DFR TrIN and DFR ValIN lines).
The results suggest that both ImageNet pretraining and fine-tuning on the target data are needed to train the best feature extractor.
However, we note that on Waterbirds the ImageNet features provide fairly strong performance.
Robustness to base model hyper-parameters In Table 6 we report the results of DFR ValTr for a range of configurations of the baseline model hyper-parameters.
While the quality of the base model clearly has an effect on DFR performance, we achieve competitive results for all the hyper-parameter configurations that we consider, even when the base model performs poorly.
For example, on Waterbirds with data augmentation, learning rate 10 − 3 and weight decay 10 − 2 the base model achieves worst group accuracy of 24 .
1% , but by retraining the last layer of this model with DFR ValTr we still achieve 88% worst group accuracy.
[Table]
[Figure]
D.3. Why is DFR TrTr-NM better than DFR TrTr ?
Let us consider the second stage of DFR ˆ DD , where we fix the feature encoder f ( · ) , and train a logistic regression model L on the dataset f ( ˆ D ) , where by f ( ˆ D ) we denote the dataset with labels from the reweighting dataset ˆ D and features from ˆ D extracted by f .
We then evaluate the logistic regression model L on the features extracted from the test data, f ( D Test ) .
Let us use ˆ M to denote a minority group in the reweighting dataset ˆ D , and M Test to denote the same minority group in the test data.
For DFR TrTr-NM and DFR ValTr , the model f is trained without observing any data from ˆ M or M Test .
Assuming the datapoints in ˆ M and M Test are iid samples from the same distirbution, the distribution of features in f ( ˆ M ) and f ( M Test ) will also be identical.
On the other hand, with DFR TrTr , the minority group datapoints ˆ M are used to train the feature extractor f .
In this case, we can no longer assume that the distribution of features f ( ˆ M ) will be the same as the distribution of f ( M Test ) .
Consequently, in DFR TrTr , the logistic regression model L will be evaluated under distribution shift , which makes the problem much more challenging and leads to inferior performance of DFR TrTr on the Waterbirds data.
We verify this intuition in Figure 6, where we visualize the feature embeddings for the reweighting dataset ˆ D and the test data.
We see that as we predicted, the distribution of the minority group features coincides between ˆ D and test data for DFR TrTr-NM and DFR ValTr , while DFR TrTr shows significant distribution shift.
What ˆ D should be used in practice?
In Table 2, the best performance is achieved by DFR TrTr-NM and DFR ValTr , which retrain the last layer on data that was not used in training the feature extractor.
In practice, we recommend collecting a group-balanced validation set, which can be used both to tune the hyper-parameters and re-train the last layer of the model, as we do in DFR ValTr .
[Figure]
[-) IN-
-]IN-9 Original (Xiao et al., 2020)[-(b) IN-
-]
[Figure]
) IN-9 FG-Only (Xiao et al., 2020) (
[Figure]
) IN-9 Mixed-Rand (Xiao et al., 2020)
[Figure]
(d) Stylized ImageNet (Geirhos et al., 2018)
[Figure]
(e) ImageNet-R (Hendrycks et al., 2021)
[Figure]
E. Natural Spurious Correlations on ImageNet E.1. Background reliance
Prior work has demonstrated that computer vision models such as ImageNet classifiers can rely on background to make their predictions (Zhang et al., 2007;
Ribeiro et al., 2016; Shetty et al., 2019; Xiao et al., 2020; Singla & Feizi, 2021).
Here, we show that it is possible to reduce the background reliance of ImageNet-trained models by simply retraining their last layer with DFR.
Xiao et al.
(2020) proposed several datasets in the Backgrounds Challenge to study the effect of the backgrounds on predictions of ImageNet models.
The datasets in the Backgrounds Challenge are based on the ImageNet-9 dataset, a subset of ImageNet structured into 9 coarse-grain classes (see Xiao et al.
(2020) for details).
ImageNet-9 contains 45 k training images and 4050 validation images.
We consider three datasets from the Backgrounds Challenge: (1) Original contains the original images; (2) Mixed-Rand contains images with random combinations of backgrounds and foregrounds (objects); (3)
FG-Only contains images showing just the object with a black background.
We additionally consider Paintings-BG using paintings from Kaggle’s painter-by-numbers dataset 4 as background for the ImageNet-9 validation data.
Finally, we consider the ImageNet-R dataset (Hendrycks et al., 2021) restricted to the ImageNet-9 classes.
See below for details and
Appendix Figure 7 for example images.
We use an ImageNet-trained ResNet-50 as a feature extractor and train DFR with different reweighting datasets.
As a
Baseline , we train DFR on the Original 45 k training datapoints 5 .
We train DFR MR and DFR OG+MR on Mixed-Rand training data and a combination of Mixed-Rand and Original training data respectively.
In Figure 8, we report the predictive accuracy of these methods on different validation datasets as a function of the number of Mixed-Rand data observed.
We select the observed subset of the data randomly; for DFR OG+MR , in each case we use the same amount of Mixed-Rand and Original training datapoints.
We repeat this experiment with a VIT-B-16 model (Dosovitskiy et al., 2020) pretrained on ImageNet-21 k and report the results in the Appendix Figure 9.
First, we notice that the baseline model provides significantly better performance on FG-Only (92%) than on the Mixed-Rand (86%) validation set, suggesting that the feature extractor learned the features needed to classify the foreground, as well as the background features.
With access to Mixed-Rand data, we can reweight the foreground and background features with
DFR and significantly improve the performance on mixed-rand, FG-Only and Paintings-BG datasets.
At the same time,
DFR OG+MR is able to mostly maintain the performance on the Original ImageNet-9 data; the small drop in performance is because the background is relevant to predicting the class on this validation set.
Finally, on ImageNet-R, DFR provides a small improvement when we use all of 45 k datapoints; the covariate shift in ImageNet-R is not primarily background-based, so reducing background reliance does not provide a big improvement.
Results for VIT-B-16.
We report the results for the VIT-B-16 base model in Figure 9.
The results are generally analogous to the results for ResNet-50: it is possible to significantly reduce the reliance of the model on the background by retraining the last layer with DFR.
For the VIT, removing the background dependence hurts the performance on the Original data slightly, but greatly improves the performance on the images with unusual backgrounds (Mixed-Rand, FG-Only, Paintings-BG).
Removing the background dependence does not improve the performance on ImageNet-R.
Data.
We use the ImageNet-9 dataset (Xiao et al., 2020).
To test whether our models can generalize to unusual backgrounds, we additionally generate Paintings-BG data shown in Figure 7.
For the Paintings-BG data we use the Original images and segmentation masks for ImageNet-9 data provided by Xiao et al.
(2020), and combine them with random paintings from Kaggle’s painter-by-numbers dataset available at kaggle.com/c/painter-by-numbers/ as backgrounds.
For the
ImageNet-R dataset (Hendrycks et al., 2021), we only use the images that fall into one of the ImageNet-9 categories, and evaluate the accuracy with respect to these categories.
We show examples of images from different dataset variations in
Figure 7.
Base model hyper-parameters.
We use a ResNet-50 model pretrained on ImageNet and a VIT-B-16 model pretrained on ImageNet-21k and finetuned on ImageNet.
The ResNet-50 model is imported from the torchvision package: torchvision.models.resnet50(pretrained=
True) .
The VIT-B-16 model is imported from the lukemelas/PyTorch-Pretrained-ViT package available at github.com/lukemelas/PyTorchPretrained-ViT; we use the command: ViT(’B 16 imagenet1k’, pretrained=
True) to load the model.
[Figure] [Figure]
To extract the embeddings, we remove the last linear classification layer from each of the models.
We preprocess the data using the torchvision.transforms paclage Compose([ Resize(resize size),
CenterCrop(crop size), ToTensor(), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]) , where (resize size, crop size) are equal to (256, 224) for the ResNet-50 and (384, 384) for the
VIT-B-16.
We do not apply any data augmentation.
DFR details.
We Train DFR on random subsets of the Mixed-Rand train data of different sizes (DFR MR ) or combinations of the Mixed-Rand and Original data (DFR OG+MR ).
For DFR OG+MR we use the same number of Original and MixedRand datapoints in all experiments.
As the full ImageNet-9 contains 50 k datapoints, we train the logistic regression on
GPU with a simple implementation in PyTorch (Paszke et al., 2017).
We then preprocess the embeddings to have zero mean and unit standard deviation using the standard scaler sklearn.preprocessing.
StandardScaler from the scikit-learn package.
In each case, we compute the preprocessing statistics on the reweighting data used to train the last layer.
For the experiments in this section we use ℓ 2 regularization (as the number of datapoints is large relative to the number of observations, we do not have to use ℓ 1 ).
We set the regularization coefficient λ to be 100 and train the logistic regression model with the loss (cid:80) x,y ∈D ′
L ( y, wx + b ) + λ 2 ∥
w ∥ 2 , where L ( · , · ) is the cross-entropy loss.
We use full-batch
SGD with learning rate 1 and no momentum to train the model for 1000 epochs.
We did not tune the λ parameter or SGD hyper-parameters.
E.2. Texture-vs-shape bias
Geirhos et al.
(2018) showed that when presented with images with conflicting texture and shape, ImageNet-trained CNNs tend to make predictions based on the texture, while humans usually predict based on the shape of the object.
The authors
[Table]
designed the GST dataset with conflicting cues
[Table]
and proposed the term texture bias to refer to the fraction of datapoints on which a model (or a human) makes predictions based on texture; conversely, shape bias is the fraction of the datapoints on which prediction is made based on the shape of the object.
Geirhos et al.
(2018) showed that it is possible to increase the shape bias of CNNs by training on Stylized ImageNet ( SIN ), a dataset obtained from ImageNet by removing the texture information via style transfer (see Appendix Figure 7 for example images).
Using SIN in combination with ImageNet ( SIN+IN ), they also obtained improved robustness to corruptions.
Finally, they proposed the Shape-RN-50 model, a ResNet50 (RN-50) trained on SIN+IN and finetuned on IN, which outperforms the ImageNet-trained ResNet-50 on in-distribution data and out-of-distribution robustness.
In this section, we explore whether it is possible to change the shape bias of ImageNet-trained models by simply retraining the last layer with DFR.
Intuitively, we expect that the standard ImageNet-trained model already learns both the shape and texture features.
Indeed, Hermann et al.
(2020) showed that shape information is decodable from the features of
ImageNet-trained models on the GST dataset to a certain extent.
Here, instead of directly targeting the GST dataset, we apply DFR to the large-scale SIN dataset, and explore both the shape bias and the predictive performance of the resulting models.
See below for details.
In Table 7, we report the shape bias, as well as predictive accuracy of ResNet-50 models trained on ImageNet ( IN ), SIN,
IN+SIN and the Shape-RN-50 model, and the DFR models trained on IN, SIN and IN+SIN.
The DFR models use an
IN-trained ResNet-50 model as a feature extractor.
First, we observe that while the SIN-trained RN-50 achieves a shape-bias of 81.4%, as reported by Geirhos et al.
, the models trained on combinations of IN and SIN are still biased towards texture.
Curiously, the Shape-RN-50 model proposed by Geirhos et al.
(2018) has almost identical shape bias to a standard IN-trained RN-50!
At the same time, Shape-RNoutperforms IN-trained RN-50 on all the datasets that we consider, and Geirhos et al.
(2018) showed that Shape-RNsignificantly outperforms IN-trained RN-50 in transfer learning to a segmentation problem, suggesting that it learned better shape-based features.
The fact that the shape bias of this model is lower than that of an IN-trained RN-50, suggests that the shape bias is largely affected by the last linear layer of the model: even if the model extracted high-quality features capturing the shape information, the last layer can still assign a higher weight to the texture information.
Next, DFR can significantly increase the shape bias of an IN-trained model.
DFR SININ achieves a comparable shape-bias to that of a model trained from scratch on a combination of IN and SIN datasets.
Finally, DFR IN+SININ improves the performance on both ImageNet-R and ImageNet-C (Hendrycks & Dietterich, 2019) datasets compared to the base RN-50 model.
In the
Appendix Table 8 we show similar results for a VIT-B-16 model pretrained on ImageNet-21 k and finetuned on ImageNet; there, DFR IN+SININ can also improve the shape-bias and performance on ImageNet-C, but does not help on ImageNet-R.
To sum up, by reweighting the features learned by an ImageNet-trained model, we can significantly increase its shape bias and improve robustness to certain corruptions.
However, to achieve the highest possible shape bias, it is still preferable to re-train the model from scratch, as RN-50(SIN) achieves a much higher shape bias compared to all other methods.
Results for VIT-B-16.
We report the results for the VIT-B-16 base model in Table 8.
For this model, baselines trained from
[Table]
scratch on IN+SIN and SIN are not available so we only report the results for the standard model trained on ImageNet21k and finetuned on ImageNet; for DFR we report the results using IN+SIN as the reweighting dataset.
Despite the large-scale pretraining on ImageNet21k, we find that we can still improve the shape bias ( 36% → 39 .
9% ) as well as robustness to
ImageNet-C corruptions ( 49 .
7% → 52% Top-1 accuracy).
On the original ImageNet and ImageNet-R the performance of
DFR is similar to that of the baseline model.
Data.
We generate the stylized ImageNet (SIN) data following the instructions at github.com/rgeirhos/Stylized-ImageNet (Geirhos et al., 2018).
For evaluation, we use ImageNet-C (Hendrycks & Dietterich, 2019) and ImageNet-R datasets (Hendrycks et al., 2021).
For ImageNet-C we report the average performance across all 19 corruption types and 5 corruption intensities.
We show examples of stylized ImageNet images in Figure 7.
Base model hyper-parameters.
We use the same ResNet-50 and VIT-B-16 models as described in the previous subsection.
DFR details.
We train DFR using the embeddings of the original ImageNet (IN), stylized ImageNet (SIN) and their combination (IN+SIN) as the reweighting dataset.
We preprocess the base model embeddings by manually subtracting the mean and dividing by standard deviation computed on the reweighting data used to train DFR; we did not use sklearn.preprocessing.
StandardScaler due to the large size of the datasets ( 1 . 2 M datapoints for IN and
SIN; 2 . 4 M datapoints for IN+SIN).
We train the logistic regression for the last layer on a single GPU with a simple implementation in PyTorch.
We use SGD with learning rate 1 , no momentum, no regularization and batch size of 10 4 to train the model for 100 epochs.
We tuned the batch size (in the range { 10 3 , 10 4 , 10 5 } ) and picked the batch size that leads to the best performance on the ImageNet validation set ( 10 4 ).
We did not tune the other hyper-parameters.
Detailed texture bias evaluation.
To provide further insight into the texture bias of the models trained with DFR, in
Figure 10 we report the fraction of shape and texture decisions for different classes following Geirhos et al.
We produce the figure using the modelvshuman codebase available at github.com/bethgelab/model-vs-human (Geirhos et al., 2021).
We report the results for both DFR models and models trained from scratch on IN, SIN and IN+SIN as well as the
ShapeResNet-50 model and humans (results from Geirhos et al.
When trained on the same data, models trained from scratch achieve a higher shape bias than DFR models, but DFR can still significantly improve the shape bias compared to the base model trained on IN.
Detailed ImageNet-C results.
In Figure 11, we report the Top-1 accuracy for DFR models and models trained from scratch on IN, SIN and IN+SIN on individual ImageNet-C datasets.
We use the ResNet-50 base model.
The model trained from scratch on IN+SIN provides the best robustness across the board, but DFR trained on IN+SIN also provides an improvement over the baseline RN50(IN) model on many corruptions.
[Figure]
brightness    5 0.
defocus blur     0. fog     0. gaussian blur     0. glass blur     0.
jpeg compression    5 0.
pixelate     0. shot noise    5 0. spatter     0. zoom blur     0.
contrast     0.
elastic transform     0.
frost     0.
gaussian noise     0.
impulse noise    5 0.
motion blur     0.
saturate     0.
snow     0.
speckle noise
[Figure]
F. Discussion
Spurious correlations and representation learning.
Prior work has often associated poor robustness to spurious correlations with the quality of representations learned by the model (Arjovsky et al., 2019; Bahng et al., 2020; Ruan et al., 2021) and suggested that the entire model needs to be carefully trained to avoid relying on spurious features (e.g. Sagawa et al., 2019; Idrissi et al., 2021; Liu et al., 2021; Zhang et al., 2021;
Sohoni et al., 2021; Pezeshki et al., 2021; Lee et al., 2022).
Our work presents a different view: representations learned with standard ERM even without seeing any minority group examples are sufficient to achieve state-of-the-art performance on popular spurious correlation benchmarks .
The issue of spurious correlations is not in the features extracted by the models (though the representations learned by ERM can be improved (e.g., Hermann & Lampinen, 2020; Pezeshki et al., 2021)), but in the weights assigned to these features.
Thus we can simply re-weight these features for substantially improved robustness.
Practical advantages of DFR.
DFR is extremely simple, cheap and effective.
In particular, DFR has only one tunable hyper-parameter — the strength of regularization of the logistic regression.
Furthermore, DFR is highly robust to the choice of base model, as we demonstrate in Appendix Table 6, and does not require early stopping or other highly problem-specific tuning such as in Idrissi et al.
(2021) and other prior works.
Moreover, as DFR only requires re-training the last linear layer of the model, it is also extremely fast and easy to run.
For example, we can train DFR on the 1 . 2 M -datapoint
Stylized ImageNet dataset in Section 6 on a single GPU in a matter of minutes, after extracting the embeddings on all of the datapoints, which only needs to be done once.
On the other hand, existing methods such as Group-DRO (Sagawa et al., 2019) require training the model from scratch multiple times to select the best hyper-parameters, which may be impractical for large-scale problems.
On the need for reweighting data in DFR.
Virtually all methods in the spurious correlation literature, even the ones that do not explicitly use group information to train the model, use group-labeled data to tune the hyper-parameters (we discuss the assumptions of different methods in prior work in Appendix F.1).
If a practitioner has access to group-labeled data, we believe that they should leverage this data to find a better model instead of just tuning the hyper-parameters.
Finally, we note that DFR can be easily combined with methods that automatically estimate group labels (e.g. Liu et al., 2021; Creager et al., 2021; Sohoni et al., 2021): we can retrain the last layer using these estimated labels instead of ground truth.
Future work.
There are many exciting directions for future research.
DFR largely reduces the issue of spurious correlations to a linear problem: how do we train an optimal linear classifier on given features to avoid spurious correlations?
In particular, we can try to avoid the need for a balanced reweighting dataset by carefully studying this linear problem and only using the features that are robustly predictive across all of the training data.
We can also consider other types of supervision, such as saliency maps (Singla & Feizi, 2021) or segmentation masks to tell the last layer of the model what to pay attention to in the data.
Finally, we can leverage better representation learning methods (Pezeshki et al., 2021), including self-supervised learning methods (e.g. Chen et al., 2020; He et al., 2021), to further improve the performance of DFR.
F.1. Prior work assumptions
In Section 5 we compare DFR to ERM, Group-DRO (Sagawa et al., 2019), JTT (Liu et al., 2021), SUBG (Idrissi et al., 2021) and SSA (Nam et al., 2022).
These methods differ in assumptions about the amount of group information available.
Group-DRO and SUBG assume that both train and validation data have group labels, and the hyper-parameters are tuned using worst-group validation accuracy.
DFR TrTr and DFR TrTr-NM match the setting of these methods and use the same data and group information; in particular, these methods only use the validation set with group labels to tune the hyper-parameters.
A number of prior works (e.g. Liu et al., 2021; Creager et al., 2021) sidestep the assumption of knowing the group labels on train data, but still rely on tuning hyper-parameters using worst-group accuracy on validation, and thus, having group labels on validation data.
In fact, Idrissi et al.
(2021) showed that ERM is a strong baseline when tuned with worst-group accuracy on validation.
Lee et al.
(2022) explore the setting where they do not necessarily require group labels on validation data, but their method implicitly relies on the presence of sufficiently many minority examples in the validation set such that different prediction heads would disagree on those examples to choose the most reliable classifier.
Recent works Nam et al.
(2022) and Sohoni et al.
(2021) explore the setting where the group information is available on a small subset of the data (e.g. on the validation set), and the goal is to use the available data optimally, both to train the model and to tune the hyper-parameters.
These methods use semi-supervised learning to extrapolate the available group labels to the rest of the training data.
We consider this same setting with DFR ValTr , where we use the validation data to retrain the last layer of the model.