Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness
Authors: Bhavya Vasudeva
Kameron Shahabi
Vatsal Sharan
Venue: Accepted by TMLR ()

Abstract: Neural networks (NNs) are known to exhibit simplicity bias where they tend to prefer learning 'simple' features over more 'complex' ones, even when the latter may be more informative. Simplicity bias can lead to the model making biased predictions which have poor out-of-distribution (OOD) generalization and subgroup robustness. To address this, we propose a hypothesis about spurious features that directly connects to simplicity bias: we hypothesize that spurious features on many datasets are simple features that are still predictive of the label. We empirically validate this hypothesis, and subsequently develop a framework which leverages this hypothesis to learn more robust models. In our proposed framework, we first train a simple model, and then regularize the conditional mutual information with respect to it to obtain the final model. We theoretically study the effect of this regularization and show that it provably reduces reliance on spurious features in certain settings. We also empirically demonstrate the effectiveness of this framework in various problem settings and real-world applications, showing that it effectively addresses simplicity bias and leads to more features being used, enhances OOD generalization, and improves subgroup robustness and fairness.

Published in Transactions on Machine Learning Research (08/2024)
Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustness
Bhavya Vasudeva University of Southern California
beasudeu@uusc.edv
Kameron Shahabi
kyshahab &lusc. edu
University of Southern California
Vatsal Sharan University of Southern California
esharon @usc.edu
Reviewed on OpenReview:
Abstract
Neural networks (NNs) are known to exhibit simplicity bias where they tend to prefer learning 'simple' features over more 'complex' ones, even when the latter may be more informative. Simplicily bias can lead to the model making biased predictions which have poor out-of- distribution (OOD) generalization and subgroup robustness. To address this, we propose hypothosis about spurious features that directly connects to simplicity bias: we hypothesize that spurious features on many datasets are simple features that are still predictive of the label. We empirically validate this hypothesis, and subsequently develop a framework which leverages this hypothesis to learn more robust models. In our proposed framework, we first train a simple model, and then regularize the conditional mutual information with respect to to obtain the final model. We theoretically study the effect of this regularization and show that it provably reduces reliance spurious features in certain scttings. We also empirically demonstrate the effectiveness of this framework in various problem settings and real-world applications, showing that it effectively addresses simplicity bias and leads to more features being used, enhances OOD generalization, and improves subgroup robustness and fairness.
1 Introduction
Motivated by considerations of understanding generalization in deep learning, there has been a series of interesting studies (Zhang al.] 22117a] Frankle & Carbin 2019 Nakkiran et al., 2020) on understanding function classes favored by current techniques for training large neural networks. An emerging hypothesis is that deep learning techniques prefer to learn simple functions over the data. While this inductive bias has benefits in terms of preventing overfitting and improving (in-distribution) generalization in several cases, it is not effective in all scenarios. Specifically, it has been found that in the presence of multiple predictive features of varying complexity, neural networks tend to be overly reliant on simpler features while ignoring more complex features that may be equally or more informative of the target (Shah et al. 2020 Nakkiran et al., 2019 Morwani et al. 2023). This phenomenon has boen termed simplicity bias and has several undosirable implications for robustness and out-of-distribution (OOD) generalization.
As an illustrative example, consider the Waterbirds dataset (Sagawa** et al. 2020 The objective here is to predict a bird's type (landbird vs. waterbird) based on its image (sce Fig. for an example). While features such as the background (land vs. waler) are easier to learn, and can have significant correlation with the bird's type, more complex features like the bird's shape are more predictive of its type. However, simplicity bias can cause the model to be highly dependent on simpler yet predictive features, such as the background in this case. A model which puts high emphasis on the background for this task is not desirable, since its performance may not transfer across different environments. similar story arises in

---

Published in Transactions on Machine Learning Research (08/2024)
M
min L(M) MEM
Full model class M.CM Simple model class
Simple features Complex features
Train Data
Test Data
Predictions
ERM
min L(M) M
ERM: Waterbird
X
CMID: Landbird
Lando
Landbird
CMID
min Ż(M) + 21(M, My (Y) ME.M
ERM: Landbird
x
CMID: Waterbird
Waterourd
Wateroord
on
Figure Summary of our approach. Models trained with ERM tend focus on simple features (such as background) that may not generalize OOD, whereas encouraging conditional independence with respect to a simple model increases reliance on complex features (such as shape) that generalize.
Dataset
Task-relevant/ arreleeannt/ivvariantt /invarianttttttaaa festure
parrogate/spprrionn feature
Waterbirds
Bird type (waterbird/landdil
Background (water/land)
CelebA
Hair colour (bloude/othe)
Gender (female/male
MultiNLI
Reasoning (entailment/ /neuteattaaaaaaeeeee cotnutral//oorrditttiiiiin contradiction)
Negation words ('not' etc.)
CivilComments- WILDS
Sentiment (toxie/nou-tooxe) non-toxic)
Demographic attributes (race, gender, religion)
Colored-MNIST
Digit (< 5 or 2 5)
Color (red/green)
Camelyon1-WWILDS
Diagnosos (tumor/nu no tumor)
Hospital
Adult-Confoundol
Income (< $50k 2 $50k)
Demographic attributes (race, gender)
Table 1: Summary of the datasets we consider. Spurious features seem simpler than invariant features.
many different tasks- Table summarizes various datascts, where the target or task-relevant features (also known as invariant features), are more complex than surrogate features that are superficially correlated with the label (also known as spurious features). Simplicity bias causes NNs heavily rely on these surrogate features.
Several methods (Arjovsky et al. 2020 Creager et aL. 2021 Bae et al. 2022 Gao al. 2022) have been proposed to address this problem of OOD generalization. However, some knowledge of the different environments of interests or the underlying causal graph is in general necessary to decide whether feature is spurious or invariant (due to the No Free Lunch Theorem, see Appendix D for more discussion). Therefore, most techniques in the literature require this information about whether feature spurious or invariant. to be available beforehand. To develop a principled framework for OOD generalization that does not use such prior knowledge, we believe it useful to first start with suitable assumption of what features will be regarded as being spurious. In this work, use simplicity bias and observations from Table to propose such candidate hypothesis about spurious features: Features typically regarded as being spurious are relatively simple while still being predictive of the label. If this hypothesis is true, alleviating simplicity bias can lead to better robustness and OOD generalization.
We empirically evaluate this hypothesis and find to be valid on considered datasots. In light of these results, we develop new framework to mitigate simplicity bias and encourage the model utilize a more diverse set of features for its predictions, and hence improve robustness and OOD generalization. The high-level goal of our framework is to ensure that the predictions of the trained model M have minimal conditional mutual information (|Y:) with any simple, predictive feature S, conditioned on the label Y. To achieve this, we first. train a simple model MC on the task, with the idea that this model captures the information in simple, predictive features. Subsoquently, to train the final model M we add its conditional mutual information [(M; YY) with the model MC as a regularizer to the usual empirical risk minimization (ERM) objective. With this regularization term, we incentivize the model to leverage additional, task-relevant features that
With slight notation abuse, M denotes both the model and the random variable associated with predictions.
2

---

Published in Transactions on Machine Learning Research (08/2024)
may be more complex. We refer to our approach Conditional Mutual Information Debiasing CMID (see Fig.
We show that our approach leads to models that use more diverse features on tasks that have been previously proposed to measure simplicity bias. The method achieves improved subgroup robustness and OOD generalization on several benchmark tasks, including in the presence of multiple spurious features. Tt. also leads to improved predictions from the perspective of fairness since the predictions are less dependent protected attributes such race gender. We also prove theorctical results to better understand and characterize our approach. Our contributions are:
We empirically evaluate and validate the hypothesis that spurious features are simpler than invariant features, with respect to representative datasets from Table (Section 2). Based on our findings, we propose a novel framework (CMID) to mitigate a model's simplicity bias and use a diverse set of features for prediction (Section 3
We theorctically analyze the effect of our regularization and demonstrate its capability to reduce reliance on spurious features using a Gaussian mixture model setup (Section Additionally, we establish an OOD generalization guarantee in a causal learning framework (Section Most approaches developed for subgroup robustness or OOD generalization cither lack thcorctical guarantoes or show results tailored to one of these two settings (discussed further in Appendix Ch.
Empirically, we demonstrate that our framework effectively mitigates simplicity bias, and achieves improved OOD generalization, sub-group robustness and fairness across 10 benchmark datasets (Section 5 These include different modalities such as image, text, and tabular data and application domains such healtheare and fairness. While several approaches have been proposed for each of the goals we consider, prior work typically focuses one two of these applications, while our approach proves effective across all cases.
1.1 Related Work
We discuss related work on simplicity bias, OOD generalization and subgroup robustness in this section, and related work on fairness, feature diversification and debiasing methods in Appendix B A detailed comparison with prior work appears in Appendix [ here we summarize some key differences. A conceptual difference from prior algorithms for these tasks is that we take a more data-driven approach to define spurious features. Hence our approach is more explicit in terms of its assumptions on the data which could make it easier to evaluate and use, and also allows us to prove theoretical guarantees. Various datasets seem to satisfy our assumptions and hence our approach proves effective across variod applications. Morcover, in contrast other methods, does not involve training multiple complex models, access to group labels, or unlabelled data from the target distribution.
Simplicity Bias in NNs. Several works (Apit et al. 2017 Valle-Perez et al. 2019 Geirhos et al. 2020
Pezeshki et al. 2021) show that NNs trained with gradient-basey methods prefer learning solutions which are 'simple'. Nakkiran et al. (2019) show that the predictions of NNs trained by SGD can be approximated well by linear models during the early stages of training. Morwani et al. (2023) show that 1-hidden layer
NNs are biased towards using low-dimensional projections of the data for predictions. Geirhos et al. (2022)
show that CNNs make predictions that depend more on image texture than image shape. (Shah al. (0200)
create synthetic datasets and show that in the presence of simple and complex features, NNs rely heavily on
simple features even when both have equal predictive power. In our work, experiment with datasets in both these papers (in addition to the datasets in Table I to investigate the effectiveness of our method to mitigate simplicity bias.
Towards developing models that perform better in the real world, OOD generaliza- tion requires generalization to data from new enwironents. Environments are usually defined based on the correlation between some spurious feature and the label. Various methods aim to recover a predictor that is invariant across a set of environments. AArovsky et al. (2020) develop the invariant risk minimization (IRM) framework where environments are known, while Creager et al.) (2021) propose environment inference for invariant learning (FII..) to recover the invariant predictor, when the environments are not known. Predict.
OOD Gencralization.
3

---

Published in Transactions on Machine Learning Research (08/2024)
then interpolate (PI) [Bao et al. 2021 and BLOOD (Bae et al. 2022) use environment-specffe ERMs to infer groups based on the correctness of predictions. We compare our approach with all these methods across several datasets to showcase its effectiveness in improving OOD generalization.
Subgroup Robustness. In many applications, models should do well not just on average but also on subgroups within the data. Several methods (Sagawa* * et al. 2020 Nam et al. 2020 Kirichenko et al. 2022.
Sohoni et al. 2022 Qiu et al. 2023 Zhang al., 2022b) have been developed to improve the worst-group
performance of model. One widely used approach to optimize the worst-case risk over set of subgroups in the data (Duchi et al.) 2019; Sagawa" et al.) 2020; Setlur et al. 2023). CVaRDRO (Duchi et al.) 2019)
optimizes over all subgroups in the data, which is somewhat possimistic, whereas GDRO (Sagawa* et al. 2020) does this over a set of predefined groups. [Dnng et al. (2023) begins with a group-balanced subset of training data and progressively expands it to encourage the learning of the task-relevant features. However,
group knowledge may not always be available, various mothods try to identify or infer groups and reweight
minority groups in some way; when group labels are not available (Nam al. 2020 Liu et al. 2021a] Sohoni
et al. 2020) or partially available (Sohoni et. al., 2022h. Just train twice (JTT) (Liu et al. 2021a) uses ERM
to identify the groups based the correctness of predictions. Learning from failure (LfF)) (Nam et al. 2020
simultancously trains two NNs, encouraging one model to make biased predictions, and reweighting the
samples it finds harder to learn (larger loss) to train the ht model. We compare our approach with these methods on some benchmark datasets for this task from Table !
2 Spurious Features are Simple and Predictive
In this section, we conduct an experiment to validate our hypothesis that surrogate features are generally 'simpler' than invariant features (see Table for examples). First, we define simple models/featuree for task follows:
Definition 1 (Simple models and features). Consider lask on which benchmark models that can allain near-zero training loss have a certain complerity (in terms of the number of parameters, layers, etc.). We consider models that have significantly lower complexity than benchmark models and cannot attain small loss on the braining dala as simple models. Similurly, fealures that can be effeclively learned using simple models are considered as simple features.
Def. 1 proposes a metric of simplicity that is task-dependent since it depends the complexity of the models which get high accuracy on the task. We choose to not define quantitative measures in Def. [ since those would be problem-dependent. As an example, for colOred-MNIST (CMNIST), NNs with non- non-lincasities are necessary to achieve high accuracy, and for Waterbirds, deeper networks such as ResNet-50 (He et al. 2016) achieve the best results. Thercfore, a lincar model and a shallow CNN can be considered as simple models for these two datasets, respectively. We discuss simple model selection in more detail in Section
Importantly, we observe that these simple models can still achieve good performance when the task is to distinguish between surrogate features even though they are not accurate in prodicting the invariant feature. Specifically, for CMNIST, we compare the performance of a linear model on color classification and digit classification on clean MNIST data. Similarly, for Waterbirds, we consider a shallow CNN (specifically, the 2DConvNet1 architecture in Appendix Fig. 110 and compare its performance on background classification (using images from the Places dataset (Zhou et al. 2017)) and bird type classification (using segmented images of birds from the CUB dataset (Wah et al. (011)). The results in Table 2 corroborate the hypothesis that surrogate features for these datasets are simple features, as they can be predicted much more accurately by simpler models than invariant features. Unsurprisingly, the respective benchmark models can capture the rask-relevant features much more accurately as they are more complex than the simple models used in each case.
Motivated by these observations, we define spurious features as follows. Operationally, our assumption on spurious features has the advantage that it does not require knowledge of some underlying cansal graph or data from multiple environments to determine if a feature is spurious.
Assumption (Spurious features). Spurious features are simple features that are still reasonably correlated with the larget label.
4

---

Published in Transactions on Machine Learning Research (08/2024)
Simple Model
Henchmark Model
Duteset
Train
Test
Dataset
Predict surrogate feature
Predict invariant. feature
Prediet invariant feature
CMNIST
=4.9
10.7 + 0.3
Train
Test.
Train
Test
Train
Test
Waterbirds
93.3 + 0.5
54.9 + 1.1
CMNIST
100
100
86.2 + 0.2
86.6 + 0.3
97.3 0.1
96.7 + 0.1
Waterbirds
79.6 + 0.6
78.4 + 0.6
60.5 4 2.5
60.4 + 2.4
98.8 + 1.2
96.2 + 1.1
Table 3: Train and test perfor- mance of the simple model on the target task.
Table 2: Comparison between performance for predicting the simple feature and the complex feature.
In the presence of such features, simplicity bias leads the model to prefer such features over invariant features that are more complex. We also verify that when simple models are trained the target tasks these datascts, they tend to rely on these spurious features to make accurate predictions. Specifically, we consider the digit classification task using CMNIST data, where the correlation between the color and the label in the test set is 10,, and bird type classification using Waterbirds data, where the test set consists of balanced groups (50% correlation). Table 3 shows that the test accuracy is close to the correlation between the spurious feature-based group label and the target label. This indicates that simple models trained on the target task utilize spurious features to make predictions.
What Assumption 1 is not true? We briefly note here that spurious features as defined by us may not always be irrelevant for the task, and mitigating simplicity bias may not always be desirable (further discussed with other limitations in Appendix D). Indeed, through similar experiments, we found that for the CelebA dataset, the spurious and task-relevant features not significantly different in terms of complexity (details in Section 5.5) As a result, our approach does not lead to much improvement in the worst-group accuracy on this dataset (Section 5.3 However, it is impossible for any learning rule to generalize across all types of distribution shifts (see Appendix D for more discussion); one can only aim to generalize under specific types of distribution shifts by employing the appropriate inductive bias. Reducing reliance on simple, predictive features represents one such assumption, which seems reasonable based on the results presented above and proves effective in several cases, shown in Section
3 CMID: Learning in the Presence of Spurious Features
In this section, we outline our approach to mitigate simplicity bias. Our approach leverages the fact that simple models can capture surrogate features much better than invariant features (Table D. and rely on the spurious features make predictions, even when trained on the target task (Table 3). Thus, by ensuring that the final model has low conditional mutual information with respect to such simple model, encourage it to utilize more diverse set of features.
Let Z= (X denote an input-label pair, where X X E (0,1), sampled from some distribution D, D denote a dataset with n samples, M(0): : x [o, 1] denote a model, paramctorized by 0 (shorthand M). Subscripts ()), and (-)e denote simple and complex, respectively. Let M denote the class of all models, (Z) X' x 10,1] + denote a loss function. I(:-) measures Shannon mutual information between two random variables. With slight abuse of notation, let M and Y also denote the (binary) random variables associated with the predictions of model M on datapoints X,'s and the labels Y's, across all i E [m]. respectively. The conditional mutual information (CMI) between the outputs of two models given the label is denoted by (M)MM)))) The empirical risk minimizer for class M is denoted as
M : ERM(M) arg min
Σ (M REIM]
MEM
72
We consider the class of simple models M, C M and complex models M. = M Ms. We now describe our proposed two-stage approach CMI Debiasing (CMID):
First, learn simple model M., which minimizes risk on the training data: M ERM(M.).
Next, learn a complex model Me by regularizing its CMT with M,:
M. arg min 1 MEM n sc(m]
Σ (M(Z) (MM
5

---

Published in Transactions on Machine Learning Research (08/2024)
where (M; MYI denotes the estimated CMI over D, and X is the regularization parameter.
A few remarks are in order about the choice of our regularizer and the methodology to select the simple model class. We penalize the CMI instead of MI. This is because both Mº and M are expected to have information about Y (for e.g. in the Waterbirds dataset both bird type and background are correlated with the label). Hence they will not be independent each other, but they are closer to being independent when conditioned on the label. We note that we use CMI to measure dependence, instead of other measures such as enforcing orthogonality of the predictions. This is for the simple reason that MI measures all potentially between the random variables.
non-linarr
We note that, while several models can be considered simple based on Def. []] the choice of the simple model class M, used for our approach is task dependent. Intuitively, we want to use the simplest model that we expect to do reasonably well on the given task. Models that are too simple may not be able to capture surrogate features effectively, whereas models that are very complex may rely task-relevant features, even though such reliance may be weak due to simplicity bias. As an example, Fig. 2 considers the problem of subgroup robustness on the Waterbirds dataset (where typically the hardest groups are images of waterbirds on land backgrounds and vice versa) and shows a comparison of the average and worst-group accuracy of the model learned by CMID using various simple model architectures (see Section 5.3 for more details about the task). We consider four models: linear model, the shallow CNN (2DConvNet1) used in Table ResNet-18 pre-trained on ImageNet, and the ResNet-50 pre-trained on ImageNet, which is also the architecture of the benchmark model. We observe that using a shallow CNN as the simple model is most effective for CMID. This is because a linear model is loo simple to capture the surrogate feature, i.e. the background in this case, whereas deep models like ResNet are complex enough to learn both types of features while relying more strongly on the surrogate feature. As a result, regularizing the CMI with respect such models may not be as effective reducing reliance on spurious features. However, we note that doing so still leads to significantly better worst-group accuracy values compared to ERM.
100
Average
CMID
Test Accuracy
grosp
BBM
50
80
10-
Linerr
Shallow CAE
MesNett0
Simple Model
Figure 2: Performance of CMID using dif- ferent simple model architectures for the Waterbirds dataset.
Although the choice of M, may impact the performance of CMID, we use fairly simple selection rule throughout our experiments, which works well. In general, for datasets where the final model is a shallow NN, we used a linear model as the simple model. For image datasets where the final model is a ResNet or DenseNet- BERT- based models are the final models, we consider shallow MLPs or 1D CNNs as simple models. Details about the model architectures can be found in Appendix G.3.2
model, we consider shallow 2D CNNs as simple models. For language datasets where
We also note that the CMI is not differentiable in its original form since Y is discrete. Therefore, we employ an estimator that uses the predicted probabilities instead of the thresholded predictions to make the regularization differentiable and hence suitable for gradient- gradient-based methods. We include details for this estimator and also an empirical comparison with the original CMI in Appendix A
4 Theoretical Results
In this section, we analyze the effect of CMT regularization and show it leads to reduced dependence on spurious features in a Gaussian mixture model. We also obtain a simple OOD generalization guarantee for CMID in a causal learning framework.
4.1 Effect of CMI Regularization
We consider data generated from the following Gaussian mixture model (see the left-most panel in Fig. 3 for an example). We note that Sagawa et al. (2020); Rosenfeld et al. (2021) consider a similar Gaussian mixture-based data model for invariant and spurious features.
6

---

Published in Transactions on Machine Learning Research (08/2024)
Assumption 2. Let the label y R(0.5), where R(p) is {=1} random variable which is 1 with probobility p- Consider d invariant features XI...., Na and spurious feature Xa+1 such that X; 1 Xj for all
i,j E + [d+ j, with distributions,
X; NWM of) Vi E [d], and Xat1 ~ (N(p22 o2),
where a ~ yR(n) is spurious attribute, with an unstable correlation with y, and M1, > 0, 0.5. Let 112 = (2n 1),,, of = o + 1 -
We consider linear models to predict y from the features X Xa-1. Let M fw = (w1,.... Wd41) : e [d+ be all possible linear models and M, {w (wm,... wd++1 E Rd+1 ||/o|lo 1} be simpler model class which only uses of the d features. We consider the mean squared error (MSE) loss, and the ERM solution is given by:
d+1 Σ x
ERM(M) = arg min E WEM
Proposition 1. For all i [d], ERM(M) sotisfies (upto scaling).
My wa-1
H1 When
3 <1, ERM(M.)) 0 Hy
0,
of
We now consider the effect of CMI regularization. We consider the case when in d² #1 of < 1, so the spurious feature has a higher signal-to-noise ratio than any individual invariant feature. Then, as per Proposition [ in the first step we learn a simple model w2+1 Xa+1 which uses only Xa+1- We now consider the ERM problem but with a constraint on the CMI:
d+1 w;
2
ERMc(M)
arg min E WEM
Σ
-V y
1=1
d+1
s.l. I
Σ
wiXi; w
Su
(1)
We show the following guarantee on the learned model.
Theorem 1. Let data be generoted as per Assumption
For v = 0.5 log(1 + e²) for some c:
1. When of 1 24 0100 ova the solution
the same as FRM(MM, so for all i E [d],
wi o
wa+1
14 of
2. Otherwise, for all E [d], w; is upweighted and the solution to satisfies |w241|
|usl
0z cv
01
Theorem suggests that for an appropriately small c, regularizing CMT with the simple model leads to predictor which mainly uses the invariant features. This is supported by experimental results data drawn according to Assumption / with d = 1, shown in Fig, 3 We note that a lower value of promotes conditional independence with Xa+1 and upweighs w; more strongly. When c + O, wa+1 + 0. We visualize this in Fig, g in the Appendix for 1. In Appendix E. we show a similar theoretical result when multiple spurious features are present.
4.2 OOD Generalization in Causal Learning Framework
Following the setting in Arjovsky al. (220)); Liu al. (2021b), consider dataset D (D*) JoEE, which is composed of data D ~ D gathered from different training environments Eyr, where denotes environment label, represents the number of samples in e. Esr denotes the set of training environments.
The problem of finding a predictor with good OOD generalization performance can be formalized as:
arg min max ED E (Z))e). MEM eEε
7

---

Published in Transactions on Machine Learning Research (08/2024)
ERM (Full)
ERM ISimple)
CMID
Spurious
Spurious
Spurious
Spurious
Invariant
Invariant
Invariant
Invariant
Figure 3: Results on synthetic Gaussian data generated as per Assumption 2 with 2000 samples, d= 1, = 12= 01 m==1. 1.5, 02 = 0.5. Left to right: Ground truth (GT) predictor, ERM with M as the class of linear models, ERM over the class of threshold functions, ERM with CMI constraint, c=0.01.
i.e., optimizing over the worst-case risk on all environments in set E. Usually, & Etr, and hence, the data and label distribution can differ significantly for E Eer and e E E Er. This makes the OOD generalization problem hard to solve.
The invariant learning literalure assumes the existence of invariant and variant features. In this section, we assume that the model of interest, say (X) is composed i.e. (X) = w (XX). For simplicity, we omit the argument X and assume that learning a fcaturization includes learning the corresponding classifier, so we can write M ①. Let be a random variable sampled from distribution on E.
featurizer and a classifier w on top of it,
Definition 2 (Invariant and Variant Predictors). A feature map ① is called invariant and is denoted by & if Y 1 E wherens is called variant and is denoted by v Y y. EVV.
Several works (Arjovsky al. 2020 Liu et al, 2021b Creager et al. 2021 attempt to recover the invariant feature map by proposing different ways to find the maximally invariant predictor (Chang al. 2020 defined as:
Definition 3 (Invariance Set and Maximal Invariant Predictor). The invariance set I with respect to environment set & and hypothesis class M is defined as:
Ie(M) = (0: Y 1 H[ = H|
The corresponding marimal invariant predictor (MIP) of Is(M) is क0" arg max [(Y; ().
(ETe(M)
The MIP is an invariant predictor that captures the most information about Y. Invariant predictors guarantee OOD generalization, making MIP the optimal invariant predictor (Theorem 2.1 in Liu et al. (2021b)) ).
discussed in Section [ most current work assumees that the environment labels e for the datapoints are known. However, environment labels typically are not provided in real-world scenarios. In this work, we don't assume access to environment labels and instead, rely another aspect of these features: are they simple complex? We formalize this below:
Assumption 3 (Simple and Complex Predictors). The invariont feature comprises of complex features, i.e. Φ = [ where &. E Me. The voriant feature comprises of simple and compler features, i.e., T = [ W.), where V. E Me, V. E M, and I(Y; > 0.
We consider the underlying causal model in Rosenfeld et al. 2021 which makes the following assumption.
Assumption 4 (Underlying Causal Model). Given the model in Fig. [ v/ Y =0 and I [ YY))>> T(W,, VY
The following simple result shows that method finds the maximal invariant predictor, and thus generalizes OOD.

---

Published in Transactions on Machine Learning Research (08/2024)
Proposition 2. Let M))(MM) = Mÿ. Under Assumptions and the solution to the problem:
E
Zyar
X
arg min E Ead a (Z) s.t. M(MY = MEM
(2)
Zin
is M db, the marimal invariant predictor.
Figure 4: Our causal model. La- tent variables Zav and Zvar cor- respond to invariant and variant. features o and u respectively.
Here, we note that Assumption 3 is key to our result. Although this assumption may not always hold directly in practice, it characterizes the condition under which our approach can recover the invariant predictor even though it does not have an explicit bousal/invariant learning motivation or access to environment labels.
5 Experiments
We show that CMID reduces simplicity bias, and yields improvements across various robustness, OOD generalization and fairness metrics. We first show that CMID mitigates simplicity bias in Slab and ImageNet- 9 datasets. We then evaluate CMID on various OOD generalization tasks. These include data with multiple spurious features, world medical data, and a real-world fairness application. Finally, we test CMID on some benchmark datasets for subgroup robustness and a fairness application. P We note that past approaches usually target one or two of these problem settings. Thus, in each section, we generally choose the most task-relevant methods to compare with, as established in prior work that task. For consistency, in addition to ERM, compare with JTT on most of the lasks, since it is the most similar method to CMID in terms of its requirements. Throughout, we observe that CMID improves considerably on ERM and compares favorably with JTT and domain-specific approachos on most data.
5.1 Mitigating Simplicity Bias
Slab Data. Slab data was proposed in Shah et al. (2020 to model simplicity bias. Each feature is composed of k data blocks or slabs. We consider two configurations of the slab data, namely 3-Slab and 5-Slab, as shown in Fig. 5 In both cases, the first feature linearly separable. The second feature has slabs in the 3-Slab data and 5 slabs in the 5-Slab data. The first feature is simple since it is linearly separable, while the features with more slabs involve piece-wise lincar model and are thus complex. The linear model is perfectly predictive, but the predictor using both types of features attains a much better margin, and generalizes better under fixed f--ormm perturba- tions to the features. Fig. 3 shows the decision boundary using ERM/JTT and the proposed approach. We see that CMTD encourages the model to use both features and attain better margin. We note that since both the features are fully predictive in this sctting, approaches like JTT are ineffective they rely on incorrect predictions of one model to train another model by upweighting such samples.
Linear:
ERMUTTT
NN:
Figure 5: Results on the slab dataset, when training a lincar model and a 1-hidden-layer NN using ERM, JTT, and CMID.
Texture vs Shape Bias on ImageNet-9. Geirhos et al. 20222 showed that CNNs trained on ImageNet. tend to make predictions based on image texture rather than image shape. To quantify this phenomenon, the authors designed the GST dataset, which contains synthetic images with conflicting shapes and textures (e.g., an image of cat modified with elephant skin texture a conflicting cue). The shape bias of a model on the GST dataset is defined as the number of samples for which the model correctly identifies shape compared to the total number of samples for which the model correctly identifies either shape texture. A shape bias of 100 indicates that the model always uses shape when shape conflicts with texture, whereas 0 indicates that the model al always uses texlure instead of shape.
"Qur code available https: //gi thub httpg//hc..cccct/////////ca

---

Published in Transactions on Machine Learning Research (08/2024)
Method
ERM
JTT
CMID
Shape Blas
31.011.
3.29. 30.7=26
40.2 35.5+2
IetgeNet-A Acc
32.6+0.6
Table 4: Shape bias and OOD accuracy on ImageNet-9 datasct.
Bias: Color
Bias: Color & Patch
Group labels
Method
Test (1.i.d)
Test (OOD)
5 gay
Test (i.i.d)
Test (OOD)
6gup
Pr 0.1
Pe 0.9
Pe 0.1
Pe 0.9
ERM
No
88.6+0.3
16.4
- -22.2
93.7
14.0
79.7
EIIL
Creager et al.
2021
No
71.7+1.6
62.8+5.0
-8.9
65.3_8.4
53.0+5.6
-2.3
JTT
Liu et al. 2021a
No
72.2
64.6.20.66
-7.6
64.0_2.7
56.2+2.7
-7.8
CMID
No
69.2 49.2
68.9
-0.3
60.3
59.4
-0.9
IRM
Arjovsky et al.
2020
Yes
71.4
66.942.5
-4.5
93.520.2
13.420.3
-00.1
GDRO
(Sagawa" x et
2020
Yes
89. 39.2
13.643.8
-75.6
92.3=0.3
14.1
78..2
P Bao al. 2021
Yes
70.3
70.240.9
-0.1
85. 85.4
15.3
70.1
BLOOD [Bae al.
2022
Yes 70.5+1.1
70.7+1.4
0.2
68.3_2.3
62.3+3.
-6.0
Optimal
75
75
Table Comparison of average test accuracies on i.i.d and OOD data and their difference (ógap) on Colored MNIST and Cotcr+Patch MNIST datasets. Bold and underlined numbers indicate the best and second-bess OOD performance among the methods that don't use group labels.
We consider ImageNet-9, subset of ImageNet organized into nine classes by Xiao al. (2020) (details in
Appendix G.1.2 and train ResNet18 model. We test OOD model performance on ImageNet-A (Hendrycks) et al. 2021 a set of natural adversarial examples on which existing ImageNet. classifiers perform poorly.
Table shows that CMID mitigates texture bias, resulting in improved performance on adversarial examples.
5.2 Better OOD Generalization
Synthetic: CMNIST and Patch MNIST. We present results on two variants of the MNIST dataset (Deng) 2012), which contains images of handwritten digits, using a binary digit classification task (< 5 or not). The volored-MNIST data was proposed in Arjovsky et al. (2020), where color (red/ (red/green) /green) is injected as spurious fcature, with unstable correlation - Pe with the label across environments. The train data has two environments with Pe 0.1, while the test data has Pe 0.9, to OOD performance. Further, contains 25% label noise reduce the predictive power of the task-relevant. feature: digit shape. We also consider the color+patch MNIST data proposed in [ae et al. (2022), where additional spurious feature is injected into the data, in the form of a 3 x 3 patch. The position of the patch (top left/ /otttomt right) is correlated with the label, with the same Pe, but independent of the color.
Following Bae et al. 2022), , we evaluate on i.idd test data with Pe = 0.1 and OOD data with P. = 0.9 for both the cases (details in Appendix G.2.1). Table 5 shows that CMID gets competitive OOD performance with methods that require group knowledge, and has the lowest gap 8gap between test performance on i.i. and OOD samples, even in the presence of multiple spurious features.
10nelyoon1-WILDS is real-world medical image dataset of data collected from five hospitals (Bándi et al. 2019 Koh et al.
Medical: Camelyon17-WILDS
2021 Three hospitals comprise the training set, one is the validation set and the third is the OOD test set. Images from different hospitals vary visually. The
task is to predict whether or not the image contains tumor tissue, and the dataset is a well-known OOD generalization benchmark (Bae et al.
2022
Koh et al.
2021 Table shows that CMID leads to higher
Method
FRM
IRM
GDRO
PI
BLOOD
JTT
CMID
Train Acc
97.310.1
97.110.1
96.5114
93.210.2
93.011s
88.4113
94.012.0
0OD Test Acc
66.5+42
59.4+37
70.2
71.7+7.6
74.9
78.046.3
77.9
Table 6: Comparison of average train and OOD test accuracies CamelyonI7-WILDD dataset.
10

---

Published in Transactions on Machine Learning Research (08/2024)
Method
ERM
ARL
JTT
EIIL
CMID
Train Ace
92.7
72.1
80.2=1.7
69.7.1.6
76.2+2.2
OOD Test Ace
31.1144
61.3117
71.825.
15.8
78.810.7
6gep
61.6
10.8
8,4
9.1
2.6
Table 7: Comparison of average train and OOD test accuracies and their difference (ógap) on Adult-Confounded dataset.
Group labels
Waterbirds
CelebA
MultiNLI
CivilComments- WIL.DS
Method
Average
Worst-group
Average
Worst-group
Average
Worst-group
Average
Worst-group
ERM
No
97.3
72.6
95.6
47.2
82.4
67.9
92.6
57.4
CVaRDRO
No
96.0
76.9
82.5
64.4
82.0
68.0
92.5
60.5
LFF
No
91.2
78.0
85.1
77.2
80.8
70.2
92.5
58.8
JTT
93.3
86.7
88.0
78.6
72.6
91.1
69.3
CMID
88.6
84.3
84.5
75.3
81.4
71.5
74.8
GDRO
Yes
93.5
91.4
92.9
88.9
81.4
77.7
88.9
69.9
Table 8: Average and worst-group test accuracies on benchmark datasets for subgroup robustness. Bold and
underlined numbers indicate the best and second-best worst-group accuracy among the methods that don't. use group labels.
average accuracies than existing group-based methods when evaluated on images from the test hospital. While JTT attains comparable test accuracy, it exhibits much larger gap between ID and OOD accuracy compared to CMID.
dataset is semi-synthetic variant of the UCI Adult datasel (Newman el al. 1998: Leisch & Dimitriadou 2021), developed by Creager et al. (2021). It consists of four sensitive subgroups based on binarized race and sex labels, and has confounded data where subgroup membership predictive of the label on the training data but the correlation is reversed at test time. Therefore, it tests whether the classifier makes biased predictions based on subgroup membership.
Fairness:
Adult-Con Colnt-Cooffonded. The
Table shows that compared other methods CMID achieves superior OOD) test performance with the least gap between train and test performance, indicating its low reliance on sensitive subgroup information.
5.3 Subgroup Robustness
We evaluate our approach on four benchmark classification tasks for robustness to spurious correlations, namely on Waterbirds (Sagawa* et al. 2020), CelebA (Liu et al. 2015), MultiNLI (Williams et al. 2018 and
CivilComments- WILDS [Brkkan et al. 2019 datasets (Table details Appendix G.3).
Although CMID does not require group knowledge for training, following Liu (2021a). we use validation set with group labels for model selection. Table [ shows the average and worst-group accuracies for CMTD and comparison with other methods (Duchi et al. 2019 Nam et al. 2020 Liu et al. 2021a which do not use group information. GDRO (Sagawa* el. al. 2020 which uses group information, acts benchmark. We see that on three of these datasets, CMID competes with satt--of-the-ar algorithms that improve subgroup robustness. Interestingly. CMID seems particularly effective on the two language-based datasets. We also note that CMID not very effective on CelebA images. We believe that this is because both the spurious feature (gender) and the invariant feature (hair color) for CelebA are of similar complexity. In Section 5.5 we explore this further with a similar experiment in Section for CelebA.
5.4 Fairness Application: Bias in Occupation Prediction
The Bios dataset (De-Artcaga et al. 2019 Chong et al. 2023) is a large-scale dataset of more than 300k biographies scrapped from the internel. The goal is to predict a person's occupation based on their bio. Based on this task, Cheng et al. (2023) formalizes a notion social norm bias (SNoB). SNoB captures the extent which predictions align with gender norms associated with specific occupations. In addition gender-specific pronouns, these norms encompass other characteristies mentioned in the bios. They represent implicit expectations of how specific groups are expected to behave. Cheng et. al. (2023) characterizes SNoB
11

---

Published in Transactions on Machine Learning Research (08/2024)
Method
ERM
Decoupled 0.94
CMID
Predict invariant feature
Predict surrogate feature
Accuracy
0.95
0.96
Train
Test
Train
Test
((Pe.Te) 0.66
0.60
0.38
89.1 1 1.5
84.3 1 0.6
92.4 22.41.44 2,4
88.3 1 1.6
Table 9: Comparison of accuracy in occupation prediction and the correlation between gender and occupation prediction on Bios data.
Table 10: Comparison between performance for pre- dicting the simple feature and the complex feature on CclebA datasct.
as a form of algorithmic unfairness arising from the associations between an algorithm's predictions and individuals' adherence to inferred social norms, They also show that adherence to or deviations from social norms can result in harm many contexts and that SNoB can persist even after the application of some fairness interventions. We note that SNoB is distinct type of bias, and existing de-biasing methods have not been evaluated on this task.
To quantify SNoB, the authors utilize the Spearman rank correlation coefficient p(pe, re), where Pe represents the fraction of bios associated with occupation e that mention the pronoun 'she', and re measures the correlation between occupation predictions and gender predictions. The authors employ separate one-vs-all classifiers for each occupation and oblain the occupation prediction for a given bio using these classifiers For gender predictions, they train occupation-specife models to determine the gender-based group membership (femalc or not) based a person's bio, and use the predictions from these models. A higher value of p(pe; re) represents larger social norm bias, which indicates that in male-dominated occupations, the algorithm achieves higher accuracy on bios that align with inferred masculine norms, and vice-versa. Table 9 shows results on the Bios data. We compare with a group fairness approach, Decoupled (Dwork et al. 2018) that trains separate models for each gender, in order to mitigate gender bias. We see that CMID addresses SNoB bias better than ERM and Decoupled, achieving a lower p(pe, re) and improved accuracy.
5.5 Additional Experiment: Invariant and Spurious Features have Similar Complexity for CelebA
In subgroup robustness experiment for CelebA (Table we found that our method did not yield significant improvement in worst-group accuracy. We investigate this further in this section. We show that for the CelebA dataset, the complexity of the invariant and surrogate features are actually quite similar. The experiment is similar to the experiments we did for CMNIST and Waterbirds in Section 2 We create a subset of the CclebA dataset by sampling an equal number of samples from all four subgroups. Table I0 presents comparison of the results when predicting the invariant feature (hair color) and the surrogate feature (gender) using the simple model (2DConvNet2). We observe that the performance for both tasks is comparable, suggesting that the features exhibit similar complexity.
We contrast these results with those for CMNIST and Waterbirds in Table 2 For CMNIST and Waterbirds, there was significant difference in the accuracy with which the simple model could predict the invariant and surrogate features. For CelebA, the difference is much smaller which suggests that spurious features are not simpler than invariant features for this caaassttexppaiingng why our method is not as effective for it.
6 Conclusion
Our work presented a hypothesis about spurious features connected to simplicity bias: spurious features are simple features that are still predictive of the label, which proved to be valid across several datascts. We leveraged this hypothesis to propose a new framework (CMID) to mitigate simplicity bias and showed that it yields improvements over ERM and many other previous approaches across number of OOD generalization, robustness and fairness benchmarks. It would be interesting to consider other natural definitions of spurious features (some of which may apply even when our definition is not suitable), and to build comprchensive and principled framework for distributional robustness under suitable assumptions. On the algorithmic side, our work suggests that using simple models to audit and refine much larger models can lead to improved properties of the larger model along certain robustness and fairness axes. It would be interesting explore the power of similar approaches for other desiderata and requirements that arise in trustworthy machine learning, and to understand its capabilities for fine-tuning large pre-trained models.
12

---

Published in Transactions on Machine Learning Research (08/2024)
Acknowledgements
This work was supported by NSF CAREER Award CCF-222 -233222 and an Amazon Rescarch Award. The authors acknowledge use of the Amazon Elastic Compute Cloud and USC Advanced Research Computing's Discovery cluster. BV thanks Puneesh Deora for helpful discussions and feedback, and Deqing Fu for discussion on simplicity bias.
References
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez- Paz. Invariant risk minimization, 2020.
Devansh Arpit, Stanistaw Jastrzebski, Nicolas Ballas, David Krueger, Enmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj. Asja Fischer, Aaron Courville, Yoshua Bengio, et al. A closer look at memorization in deep networks. In International conference machine learning, pp. 233 242. PMLR, 2017. 3
Jun-Hyun Bae, Inchul Choi, and Minho Lee. BLOOD: Bi-level learning framework for out-of--distribute generalization, 2022. URL https://openreeiies net wttp:/oommmdddcmmmmeeeeepoop
10
24
32
Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 97th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research,
pp. 528 539. PMLR, 13 Jul 2020. URL https://prroceeding.. .mlr wtwpps//119//aaag220ee... 26
22
Rachit Bansal, Danish Pruthi, and Yonatan Belinkov. Measures of information reflect memorization patterns. Advances in Neural Information Processing Systems, 35, 2022. 24
Yujia Bao, Shiyu Chang, and Regina Barzilay. Predict then interpolate: A simple algorithm to learn stable
classifiers. In International Conference on Machine Learning. PMLR, 2021. T0 23 24
Carlo Alberto Barbano, Enzo Tartaglione, and Marco Grangetto. Unsupervised learning of unbiased visual
representations, 2022. 24
Nourhan Bayasi, Ghassan Hamarnch, and Rafeef Garbi. Boosternet: Improving domain generalization of deep neural nets using culpability-ranked features. In 2022 IEFE/CVF Conference on Compuler Vision and Pattern Recognition (CVPR), pp. 528-538, 2022. doi:
CCVP55268820222000006 24
Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for lext classification. In Companion Proceedings of The 2019
World Wide Web Conference, 2019. 11 34
Cristian Bucilua, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '06, pp. 531-541, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595933395. doi:
10.1145/1150402.11550 URL https://doi.orr/// 111p:////....111111000000000.... 1150444
23
Péter Bándi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, Quanzheng Li, Farhad Ghazvinian Zanjani, Svitlana Zinger, Keisuke Fukuta, Daisuke Komura, Vlado Ovtcharov, Shenghua Cheng, Shaoqun Zeng, Jeppe Thagaard, Anders B. Dahl, Huangjing Lin, Hao Chen, Ludwig Jacobsson, Martin Hedlund, Melih Çetin, Eren Halicı, Hunter Jackson, Richard Chen, Fabian Both, Jörg Franke, Heidi Küsters- Vandevelde, Willem Vreuls, Peter Bult, Bram van Ginneken, Jeroen van der Laak, and Geert Litjens. From detection individual metastascs to classification of lymph node status at the patient level: The camelyon17 challenge. IEEE Transactions on Medical Imaging. 38(2):550055) 560, 2019. doi: 10. N0.1109/TMT.200 2867350.. 10
13

---

Published in Transactions on Machine Learning Research (08/2024)
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. Invariant rationalization. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1448 1458. PMLR, 13 18 Jul 2020. URL https: //proceedings.oo www.mml.pp.... wwl..ppees/vil. wttsss//111//hc222..htat
g
Yonggiang Chen, Wei Huang, Kaiwen Zhou, Yalao Bian, Bo Han, and James Cheng. Understanding and improving feature learning for out-of-distribution generalization. In Thirty--seennth Conference on Neural Information Processing Systems, 2023. URL https::/oppnrewiew. httpe/ewwiiwweeeeeeeeeeeee httt//forumidd=ezoooo0
23 24
Myra Cheng, Maria De-Artcaga, Lester Mackey, and Adam Tauman Kalai. Social norm bias: residual harms of fairness-aware algorithms. Dala Mining and Knowledge Discovery, pp. 1 1-27, 2023.
35
Elliot Creager, -eerik Jacobsen, and Richard Zemel. Enviromment inference for invariant learning.
In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 2189-2200. PMLR, 18-24 Jul 2021.
URL https://proceeddnng. .w....r.... httppp/sss//iieerraareeeee
html
32 33
Nikolay Dagaev, Brett D. Roads, Xiaoliang Luo, Daniel N. Barry, Kaustubh R. Patil, and Bradley C. Love. A
too-good-to-be-true prior to reduce shorteut reliance. Pattern Recogn. Lett., 166(C))164-171, feb 2023. ISSN 0167-8655. doi: pttp:/j.paarre.222...2222
URL
patrec. 2022. 12.2.122.00 010
23
Maria De Arteagg, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Choulde- chova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of seman- tic representation bias in a high-stakes setting. In Proceedings of the Conference on Pairness, Accountabilityy and Thansporency, FAT* '19, pp. 120 128, New York, NY, USA, 2019. Association for Computing Machinery. 3287777. URL htt.://ps./oodoorr/////11111/200
ISBN 9781450361255, doi:
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142, 2012. 10
Yihe Deng, Yu Yang, Baharan Mirzasoleiman, and Quanquan Gu. Robust learning with progressive data expansion against spurious correlation. In shirth-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreeiiet wttp:rrriiiiieee//ounuuuuunn
24
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. Tn Proceedings of the 2019 Conference of the North Americon Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: nt.18633/v//NI/ 1423. URL
http://cccantollooyor/r//-/-- org/N19-1423
35
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. 2018 34
Alexcy Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL http::///ppeeeeeeeee...e////oaap
26
John C. Duchi, Tatsunori Hashimoto, and Hongscok Namkoong. Distribntionally robust losses against mixture covariate shifts, 2019.
24
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS '12 pp. 214-226, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 9781450811151. doi: 10. 10.1145/2090230 22200206.29922220 URL http://00...r9//10.114000//99000 2090255
22
14

---

het/fforumidddzr1--BRRFFFF Published in Transactions on Machine Learning Research (08/2024)
Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. Decoupled classifiers for group-fair and efficient machine learning. In Sorelle A. Friedler and Christo Wilson (eds.), Proceedings of the 1st Conference on Fairness, Accountability and Transparency, volume 81 of Proceedings of Machine Learning Research, pp. 119-133. PMLR, 23-24 Feb 2018. URL https::/pproeeedings.. mlr.ggg.....ppp..... /
dwork18a htmll
12 22
35
Jonathan Frankle and Michael Carbin.
The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019. URL https://openrevieew
1
Irena Gao, Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Ont-of-distribbttion robustness via targeted augmentations. In NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications, 2022. URL bttps://peeneview..nee// http:/ttw//ooommmdddooollllilll
Robert Geirhos, Jörn-Heurik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):
665 673, 2020. g
Robert Gcirhos, Patricia Rubisch, Clandio Michaclis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. sragenet-traince enns are biased towards texture; increasing shape bias improves accuracy and
robustness, 2022. 3 22 31
Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision 129 1819, March 2021. ISSN 1573-1405. doi: 10.1007/ sl -11263-021-01453-z URL
23
Suchin Gururanga, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 107-112, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: (18865//11/IB/ 2017. URL htttps://allatthooogg org/N18- 2017
22
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Newral Information Processing Systems, NIPS'16, 16, pp. 3323 3331, Red Hook, NY, USA, 2016. Curran Associates Tne. ISBN 9781510838819 22
Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, 2018. 22
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 TEEE Conference on Compuler Vision and Pallern Recogrétion (CVPR), pp. 770- 2016. doi: 10.1109/CVVI /CVPR.201....00
34
Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the Coomputaaionall---eeddffaabb masses. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1939 1948. PMLR, 10-15 Jul 2018. URL http:///r//ooeeeoss...psspss///p ttwpapppeee/ee//eeee-eeiooio html
22
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. CVPR, 2021.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015.
23
Myeongho Jcon, Dackyung Kim, Woochul Lee, Myungjoo Kang, and Joonscok Lec. A conservative approach for unbiased learning on unknown biases. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16731 2022. doi: 21110//CVVP522882222222002 24
15

---

Published in Transactions on Machine Learning Research (08/2024)
Shengyu Jia, Tao Meng, Jieyu Zhao, and Kai-Wei Chang. Mitigating gender bias amplification in distribution by posterior regularization. In Proceedings of the 58th Annual Meeling of the Association for Compulational Linguistics, pp. 2936 2942, Online, July 2020, Association for Computational Linguisties. doi: 10.18653/
v1/2020.acl- /a/2020.aclmmiin.22000 main. 264. URL
acl -aain.annnm 264
23
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst., (3 oct 2012. ISSN 0219-1377. doi: 10.1007/s10115-011--040 URL https:
10100000000011001100555555 8
22
Nayeong Kim, Sehyun Hwang, Sungsoo Ahn, Jaesik Park, and Suha Kwak. Learning debiased classifier with biased committee. CoRR, abs/2206.108433 2022. URL
h1t.:////000111.15//////1 200 10843
2
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson Last layer ro-training is sufficient for robustness to spurious correlations. In International Conference on Machine Learning, 2022, 4
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang. Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664 PMLR, 2021.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed H. Chi. Fairness without demographics through adversarially reweighted learning. In Proceedings of the 34th International Conference Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020.
Curran Associates Inc. ISBN 9781713829546.
22 33
Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning debiased representation via disentangled feature augmentation. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Newral Information Processing Systems, 2021. URL https:://openreview.rt
id= JLWHH 26
Jungsoo Lee, Jeonghoon Park, Dacyoung Kim, Juyoung Lee, Edward Choi, and Jacgul Choo. Revisiting the importance of amplifying bias for debiasing, 2022a. 24
Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Learning from underspecified
data. In ICML 2022: Workshop on Spurious Corvelations, Invariance and Slability, 2022b. URL https
7/openreview www.te/ororiiddleell.....c.
23
24
Friedrich Leisch and Evgenia Dimitriadou. mibench: Machine Learning Benchmark Problems, 2021. R
package version 2.1-3.1.
33
Zhiheng Li, Anthony Hoogs, and Chenliang Xu. Discover and mitigate unknown biases with debiasing alternate
networks. In Computer Vision ECCV 2022: 17th Europeun Conference,, Tel Aviv, Isruel, October 23 27, 2022, Proceedings, Part XIII, pp. 270-288, Berlin, Heidelberg, 2022. Springer- Verlag, ISBN 978-3-031- 19777-2. doi: 10.1007 1110077978-3-031-19000
16. URI.httss: http::///oo..org///000////////// 1007/997 --1778---03111177--9998
23
24
Jongin Lim, Youngdong Kim, Byungjai Kim, Chanho Ahn, Jinwoo Shin, Eunho Yang, and Seungju Han. Biasadv: Bias-adversarial augmentation for model debiasing. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pallern Recognition (CVPR), pp. 3832 3841, June 2023. 24
Evan Z Liu, Behzad Haghgoo, Annie Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine
Learning, volume 139 of Proceedings of Machine Learning Research, pp. 6781 6792. 2021a. CRL https://proceediiiii
PMLR 18
18-24 Jul
mlr. wwwpss///19//li221.... html
10
24
32
Jiashuo Liu, Zheyuan Hu, Peng Cui, B. Li, and Zheyan Shen. International Conference on Machine Learning, 2021b.
Heterogeneous risk minimization. In
16

---

Published in Transactions on Machine Learning Research (08/2024)
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In
Proceedings of International Conference on Compuler Vision (ICCV), December 2015. 11 33
Seyed-Iman Mirzadeh; Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant, 2019. 23
Depen Morwani, Jatin Batra, Prateek Jain, and Praneeth Netrapalli. Simplicity bias in 1-hidden layer neural networks, 2023.
Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L. Edelman, Fred Zhang, and Boaz Barak. SGD on Neural Networks Learns Functions of Increasing Complexity. Curran Associates Inc., Red Hook, NY, USA, 2019.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Nyy Sulskever. Deep double descent: Where bigger models and more data hurt. In International Conference on Learning Representations, 2020. URL https://openreviee.. httpr//wwww...etfoooooooor//lgaa
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Newral Information Processing Systems, 2020.
26
D.J. Newman, S. Hettich, C.L. Blake, and C.J. Merz. Uci repository of machine learning databases, 1998.
URL http://www ics. uci cedl://mlearm//Leposssiie html
33
Hongjing Niu, Hanting Lú, Feng Zhao, and Bin Li. Roadblocks for temporarily disabling shortcuts and learning new knowledge. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and
A. Oh (eds.), Advances in Newral Information Processing Systems, volume 35, pp. 29064-29075. Cur- ran Associates, Inc., 2022. URT, https:/ppooceeiigg...n neurips...n.iipi....... httss///pppe..ille//aarr/////// 2aa27556bbaaaca000eea..0093553
conpereroonfeeeceee.po pdf
24
Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Pranceth Karimireddy. Agree to disagree:
Diversity through disagreement for better transferability. In The Eleventh International Conference on Learning Representations, 2023. URL
23
24
Geon Yeong Park, Chanyong Jung, Sangmin Lee, Jong Chul Ye, and Sang Wan Lee. Self-supervised debiasing using low rank regularization, 2023. 24
Ji Ho Park, Jamin Shin, and Pascale Fung. Reducing gender bias in abusive language detection, 2018 34
Mohammad Pezeshki, Sékou- Oumar Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, and Guillaume Lajoie. Gradient starvation: learning proclivity neural networks. In A. Beygekzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Newral Information Processing Systems, 2021. URL
Shikai Qiu, Andres Polapczynski, Pavel Ixmailov, and Andrew Gordon Wilson. Simple and Fast. Group Robustness by Automatic Feature Reweighting. International Conference on Machine Learning (ICML), 2023.
Elan Rosenfeld, Pradeep Kumar Ravikumar, and Andrej Risteski. The risks of invariant risk minimization. In International Conference on Learning Representations, 2021. URL https://openrevieew http://ppppneeeeeeeeeeeeeeeeeeee
id-BbNIbUPJ-42
24
Shiori Sagawa* Pang Wei Koh* Tatsunori B. Hashimotoo, and Percy Liang. Distributionally robust neural networks. In International Conference Learning Representations, 2020. URL https://openrevieew
35
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparamotori- zation exacerbates spurious corrclations. In Proceedings of the 37th International Conference on Machine Learning, ICML/20. JMr.Rorg, 2020.
17

---

Published in Transactions on Machine Learning Research (08/2024)
Amrith Sctlur, Don Dennis, Benjamin Eysenbach, Aditi Raghunathan, Chelsea Finn, Virginia Smith, and Sergey Levine. Bitrate-constrained dro: Beyond worst case robustness to unknown group shifts, 2023. 4
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. In Proceedings of the 34th International Conference on Newral Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Shai Shalev-Shwarzz and Shai Ben-David, Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, USA, 2014. ISBN 1107057132. 25
Nimit. Sohoni, Jared A. Dunnmon, GeofTrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran
Associates Inc. ISBN 9781713829546. 4
Nimit S. Sohoni, Maziar Sanjabi, Nicolas Ballas, Aditya Grover, Shaoliang Nie, Hamed Firooz, and Christopher Ré. Barack: Partially supervised group robustness with guarantees, 2022. 4
Saeid Asgari Taghanaki, Aliasghar Khani, Fereshte Khani, Ali Gholami, Linh Tran, Ali Mahdavi-Amiri, and Ghassan Hamarnch. Masktune: Mitigating spurious corrclations by forcing explore, 2022. 24
Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEFF/CVP Conference on Computer Vision and 26
Pottern Recognition (CVPR), pp. 13508 13517, June 2021.
Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton van den Hengel. Evading the simplicity bias: Training diverse set of models discovers solutions with superior ood generalization. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 16761-16772, June 2022.
23 24
Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. Mind the trade-off: Debiasing NLU models without degrading the in-distribution performance. In Proceedings of the 58th Annuol Meeting of the Association for Computational Linguisties, pp. 1717-8729, Online, July 2020a. Association for Computational Linguisties. doi:
URL https://aclatthooooyyy.oo orgg220
acl main. 770
23
Prasetya Ajie Utama, Nafise Sadat, Moosavi, and Tryna Gurevych. Towards debiasing NLU models from unknown biases. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7597-7610, Online, November 2020b. Association for Computational Linguistics. doi: 10.18653 10118653/1110000eemmm/// 0200000000mllmmmmin..f... URL bttps://ac..nnthoooo..oog/////
wennp-mmiin 613. 24
Guillermo Valle-Perez, Chico Q Camargo, and Ard A Louis. Deep learning generalizes because the parameter- function map is biased towards simple functions. International Conference on Learning Representations, 2019. g
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report
California Institute of Technology, 2011.
Haohan Wang, Zexue He, Zachary L. Lipton, and Eric P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019a. URL httpps://opereevie..t www.:neef/oommmiddreeeeoe.
26
Tianlu Wang, Jicyu Zhao, Mark Yatskar, Kai- Wei Chang, and Vicente Ordoncz. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In International Conference
on Computer Vision (ICCV), October 2019b. 23
18

---

Published in Transactions on Machine Learning Research (08/2024)
Alex Wilf, Alex Tianyi Xu, Paul Pu Liang, Alexander Obolenskiy, Daniel Fried, and Louis-Philippe Morency. Comparative knowledge distillation, 2023.
23
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long Papers), pp. 1112 1122. Associalion for Compulational Linguistics, 2018. URL
org/anthologyy
N18 1101
II
30
David H. Wolpert. The lack of priori distinctions between learning algorithms. Newral Compulation, 8:
1341-1390, 1996. URL
207669933
25
Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise or signal: The role of image backgrounds in object rocognition, 2020. 10
31
Wanqian Yang, Polina Kirichenko, Micah Goldblum, and Andrew Gordon Wilson. Chroma-vae: Mitigating
shortcut learning with generative classifiers, 2022. 24
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017a. URL https://openrrviiet www.eeoorunidd-yyyddaaeeee
Jianyu Zhang, David Lopez-Paz, and Leon Bottou. Rich feature construction for the optimization- generalization dilemma. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba. Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 26397-26411, PMLR, 17-23 Jul 2022a. URL
https://proceeding mlr. wwtpp//eeeeeeeeeeeeeeppeepee http://////aaan2..h.hm html
24
Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Re. Correct-n-contrast. contrastive approach for improving robustness to spurious correlations. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 26484-265 PMLR, 17-23 Jul 2022b. URL htttps:/ppoceedding: mlr. pptps//v16//2aang22... html
Ying Zhang, Tao Xiang, Timothy M. Hospedales, and Huchuan Lu. Deep mutual learning, 2017b. 23
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. JEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
19

---

Published in Transactions on Machine Learning Research (08/2024)
Appendix
A Estimating CMI
20
A.1 Evaluating the Estimated CMI
21
A.2 Eflect the Simple Model Architecture
B Further Related Work
22
C Connections with Prior Work
23
D Limitations of our Approach
24
E
Additional Results
25
E.1
Effect of CMI Regularization
25
E.2 Results the 10-class Colored MNIST Datasel
................................
26
E.3 Results using ViT
26
E,
E.4
Comparison with CLIP
26
Proofs for Thcoretical Results F
27
1 Prool of Proposition
27
2 Proof of Theorem
28
3 Prool of Theorem 2
29
Proof of Proposition
30
G
Experimental Settings
30
Mitigating Simplicity Bias Experiments
30
G.2
Better OOD Generalization Experiments
G.3
Subgroup Robustness Experiments
33
G.4 Bias in Occupation Prediction Experiment,
35
A
Estimating CMI
In this section, describe technique compute the CMI regularization for our approach, mentioned in Section We begin with the relevant notation.
Let Z = (X,Y) denote an input-label pair, where X X E (0,1), sampled from some distribution D, D denote a datasel with n samples, M(0) X + [0, 1] denote a model, parameterized by (shorthand M). The predictions of the model are given by (X) > 0.5]. Subscripts (-), and (-)e denote simple and complex, respectively. Let M denote the class of all models M, LM (Z) Let. H() denote the Shannon entropy of random variable. I(:-) measures Shannon mutual information between two random variables. With slight abuse of notation, let M and Y also denote the (binary) random variables associated with the predictions of model M datapoints X,'s and the labels Yis, across all E [n], 1
x 10.1) - l denote loss function.
respectively. Let (zx T') 1 T denote the sigmoid function, with temperature parameter T.
20

---

Published in Transactions on Machine Learning Research (08/2024)
To estimate CMI, first consider the conditional (joint) distributions over samples:
p(M=m Eie(1] [Y, -] ((M)
Σig(n] 21 y
= Diein 1[, - C(M (X:), ((M.
Mieln] -[
where m, m', E 10,
and
Note that the CMI computed using these would not be differentiable as these densities are computed by thresholding the outputs of the model. Since we want to add a CMI penalty as a regularizer to the ERM objective and optimize the proposed objective using standard gradient-based methods, we need a differentiable version of CMT. Thus, for practical purposes, we use an approximation of the indicator function [ > 0.5), given by oxx - 0.5, T), where I' determines the degree of smoothness sharpness in the approximation.
This can be easily generalized for multi-class classification with C' classes. In that case, m', y (0...., C }. M(X) is C-dimensional vector with the m th entry indicating the probability of predicting class m, and (X,,, m) = 1 [arg max Mx( m]. To make this differentiable, we use the softmax function with jE[C]
temperature parameter T to approximate the indicator function.
Using these densities, the estimated CMI is:
i.,M, Y E((Y=)) Σ M log ppM m[M==mY+ Y =>)p(M, = m = m.m!
plM==, M. MM==:1 mm'ly =v)
(3)
This estimate differentiable, making it compatible with gradient-basod methods. Therefore, we utilize it as a regularizer for the proposed approach.
A.1 Evaluating the Estimated CMI
In this section, we evaluate the reliability and scalability of the CMI estimate in (3) compared to the original CMI, which is computed with discretized model outputs. We consider CMNIST data with the hyperparameter values as mentioned in Section G.2.1 for the results in this section. Table I1 compares the times (in milliseconds) to compute the original CMI and the estimated CMI, using batch size 64, for 10 classes and 200 classes. We see that the computation time for the estimated CMI does not increase significantly as the number of classes
Method I(M, M.IY)
10 classes
200 classes
14.1 3.3
23.3 + 2.7
M.
20.17 4.2
24.3 3.3
Table 11:
Comparison of computation times (in milliseconds) for the original CMI and the estimated CMI in
increases.
1.0
0.08
[[x=0]
NM, MMYY
0.8
dxx 125)
0.06
1,M, MyJY) T-125
[MM (MMM
0.6
0.04
7,M. M.lr) T=25
ERM
0.4
12.5)
7. MJII), T-12.5
0.2
CMID
0.2
0.02
0.0
0.0
3.0
0.00
5
10
15
Epochs
Epochs
Figure 6: (Left) Sigmoid functions for different values of the temper- ature parameter T. (Right) Comparison of the original CMI with the estimated CMT using different values of T.
Comparison of the original CMI for models trained with ERM and CMID as function of training time.
Figure 7:
Fig. compares the estimated CMI for different values of the temperature parameter T with the original CMI. The values shown in the figure computed with batch size 1000, while for training, the estimated
21

---

Published in Transactions on Machine Learning Research (08/2024)
CMI with T = 12.5 and batch size 64. We see that as T increases, the sigmoid approximates the indicator function more closely and the estimated CMT tends closer to the original CMI. Fig. T compares the CMT computed with discretized outputs using batch size 1000, when the model is trained with ERM or CMID. We see that regularizing the estimated CMI causes the original CMI to decrease over time, whereas training with ERM leads to an increase in the CMI. This shows that the solution learned by ERM makes predictions aligned with those of the simple model, whereas our approach CMID learns a model that attains low CMT with the simple model.
A.2 Effect of the Simple Model Architecture
In Fig. 2 [Section] we compared the effect of using various simple model architectures on the performance of the model learned by CMID on the Waterbirds datasct. In this soction, we conduct a similar experiment to analyze the effect of using various simple model architectures on the CMNIST dataset. Fig. [ compares the performance of CMTD when using a linear model and MLP as the simple model. We observe that using a linear model is more effective in this case since the benchmark model for this task is an MLP. As a result, the MLP may capture both features, and regularizing the CMI with respect to this model may not be very effective in reducing the reliance on just the spurious features.
Train OOD Test
Accuracy
70
60
MLP
Simple Model
Waterbirds
CMNIST
Figure 8: Performance of CMTD using different simple model archi- tectures for CMNIST dataset.
Linear
Shallow CNN ResNec18
ResNet50
Linear MLP
LR. 5x 10
10
x 10
2 10
0.01
0.01
Table 12: Training details for the simple models.
We note that in these experiments (for both Waterbirds and CMNIST datasets), we only tune the learning rate (LR) and weight decay (22) for training the simple model (as listed in Table 12p. The rest of the hyperparameter values for training the final model are kept consistent across each dataset (as listed in Table 17 for CMNIST and Table 19 for Waterbirds).
B Further Related Work
Fairness. Several works aim to ensure fair predictions of models across subgroups in the data, which are defined based on sensitive demographic information. Commonly used notions of group fairness include demographic parity (Dwork et al. 2012). which aims to ensure that the representation of different demographic groups in the outcomes of model is similar to their overall representation, and equalized odds (Hardt et al.) 2016), which aims for equal predictive performance across groups using metrics such true positive rate. Fairness interventions used when group information is available include data reweighting to balance groups before training (Kamiran & Calders 2012), learning separate models for different subgroups (Dwork et al.) 2018), and post- posst-prccssing of trained models, such as adjusting prediction thresholds based on fairness-basod metrics (Hardt et al. 2016).
Various approaches have been proposed to achieve fairness when demographic information may not be available. Multicalibration (Hebert-Johnson et al. 2018)) aims to learn model whose predictions are calibrated for all subpopulations that can be identified in computationally efficient way. Hashimoto et al. (2018) proposes distributionally robust optimization (DRO) close to the empirical distribution to ensure fairness. Lahoti et al. (2020) proposes adversarial reweighted learning ( (ARL), where an auxiliary model identifies subgroups with inferior performance and the model of interest retrained by reweighting these subgroups to reduce bias.
approach to minimize the worst-casc risk over distributions
Feature Diversification and De-biasing Methods. Deep neural networks are known to exhibit unwanted biases. For instance, CNNs trained on image data may exhibit texture bias (Geirhos et al. 2022). and language models trained on certain datasets may exhibit annotation bias (Gururangan et al.) [2018). Several methods have been proposed to mitigate these biases. Bahng et al. (2020) introduce framework (ReBias) to
22

---

Published in Transactions on Machine Learning Research (08/2024)
learn de- de-biased representations by encouraging them to differ from reference set of biased representations.
et al. (2022) propose to train two models alternatively, using one to identify biases using an equal opportunity violation criterion and training the other with a reweighted cross-entropy loss to make unbiased predictions.
Dagaev et al. [023) reduce shortcut reliance by upweighting samples based on the misclassification probability
of a simpll model to train a complex model. [lama et al. (2020a) propose a confidence regularization approach to encourage models to learn from all samples. Recent works also show that deep neural networks tend to amplify the societal biases present in training data (Wang et al. 2019b: Jia et al. 2020) and they
propose domain-specific strategies to mitigate such amplification.
Several works aim to improve feature learning for better generalization. Zhang et al. (2022a) aims to learn a shared representation using a succession of training episodes, where they train new classifiers in each episode to do better on misclassified samples from previous episodes. Chen et al. (2023) propose a similar but more efficient approach: at each training step, the data is divided into two sets based on whether the predictions are correct or not, followed by applying DRO on the latter to learn new features with ERM on the former to retain the learned features. Teney et. al. (2022) also involves learning a shared representation and training multiple classifiers, where the alignment between their gradients is regularized to encourage feature diversity,
Pagliardini et al. (2023): Lee ct al. (2022b) do so by encouraging disagrocment between two models leveraging
unlabeled samples from the target domain.
Knowledge Distillation. One can view our approach, in some sense, as the reverse of knowledge distillation (Bucilua et al. 2006; Hinton et al. 2015). These methods aim to transfer knowledge behaviors from large, complex neural networks into smaller, simpler ones. This can be useful when employing deep learning models in montory-constrained systems such as mobile devices or embedded systems (Gou et al. 2021). To achieve Enowledge distillation, Hinton et al. (2015) propose training a small model using a transfer set consisting of samples labeled with the softmax probabilities given by a large pretrained model. Several works have aimed to build upon this approach. Mutual learning (Zhang el al. 2017b) teaches multiple simple models simultaneously, with a KL Divergence-basee mimicry loss that ensures the simple models match each others predictions. Teaching assistant (TA) learning (Mizzadeh et al.] 2019) first distills knowledge from a large model into an intermediate-sizeed TA network, then uses the TA network to teach a small model. Comparative knowledge distillation (Will et al. 2023). constructs a loss function that teaches small model to mimic the large model's comparison on two samples from the transfer set, rather than mimic its prediction on individual samples.
While these methods can effectively transfer knowledge without facing significant increases in loss, we do not suspect they would be effective in mitigating simplicity bias. In this paper, we observe that even complex models trained with ERM achieve poor OOD performance due to simplicity bias in deep learning, and using smaller models to mimic their behaviors could further increase the reliance on simple spurious urat due to their limited capacity.
C Connections with Prior Work
In general, prior work focuses on one or two of the applications that we consider, while our approach proves effective across several datasets for all cases.
Several methods that aim to improve subgroup robustness or OOD generalization, such as IRM (Arjovsky et al, 2020), GDRO (Sagawa* * et al. 2020), PI (Bao et al. 2021) and BLOOO (Bac et al., 2022), require knowledge of group or environment labels. This additional information not always available and consequently, these methods have limited applicability in such cases. Approaches like JTT (Liu et al. 2021a) and EIIL (Creager
et al.
2021) overcome this requirement. However, these methods rely on incorrect predictions (Liu et al.
2021a a Creager et al. 2021) of one model to train another model by upweighting such samples (Liu et al. 2021a) or learning features that generalize across such samples (Creager el al. 2021). As a result, they may not be effective in settings where the spurious feature is highly predictive. Such settings can be relevant when our goal is to mitigate simplicity bias. For example, in the slab dataset (Figure [), the simple model achieves perfect accuracy but has a much worse margin compared to the global max margin predictor. Our approach is still effective in such cases because it relies on prediction probabilities rather than thresholded predictions,
23

---

Published in Transactions on Machine Learning Research (08/2024)
which enables the differentiation between features that result in smaller margin or low confidence predictions and those that lead to better margin or high confidence predictions.
We note that Sesiasing/feature diversification methods (Nam et al. 2020 Bahng et al. 2020 Li et al. 2022:
Zhang et al.) 2022a Teney et al., 2022) can often be computationally expensive. Some of these involve training two complex models simultancously (Nam et al.) 2020: Bahnn et al.) 2020) or alternatively (Ti et al. (2022a); [Teney et al. (2022) involves training multiple models and returning the average or selecting the best one among them, respectively. In contrast, our results show that the simple approach to train single model with the CMI regularization can prove effective across several settings. While methods like Pagliardini et al. (2023); Lee ct al. (2022b) are simpler, they require access to additional data from a target domain to encourage disagreemeent between two models for feature diversification. Our approach does not have such a requirement and allows for diversification directly on the training data.
2022]. Zhang et al.
Several methods use a similar framework of first training a biased model and then training another model debiased or regularized with respect to the first one. We discuss the main differences with these methods
here. Many of these methods are designed (or evaluated) specifically for either vision (Jeon et al. 2022 Lee
et al. 2022a Bayasi et al. 2022
Yang et al. 2022 Kim et al. 2022 Taghanaki et al. 2022 Niu et al. 2022
Park et al.
2023 Barbano et al.
2022 Lim et al. 2023) or language [Utama et al., 2020b) datasets, whereas
our approach is more broadly applicable to several modalities including vision, language and tabular data.
Utama et al. (2020b) is similar to JTT and can suffer from similar brawb [Jeon et al. [002]; Bayasi et al.
(2022) are designed specifically for CNN-based architectures. Several of these methods are computationally
intensive: [ee et al. (2022a): Kim al. (2022) involve training several biased models before training the final
debiased model, Yang et al. 2022 involves training VAE in the first stage, TTahhnaki et al. (2022) involves
using an explainability approach to create masked version of the dataset based on the first stage model to train the final model, [Bansal et al. 20022) proposes model selection strategy that requires training multiple models before selecting the best one, Niu et al. (2022) involves using three models: a pre-trained model, and an autoencoder and the final model that are trained simultancously, Barbano et al. (2022) involves a
three-stage approach, Lim el al. 20023 involves crealing adversarial samples for a biased model which are used to train the final model.
In addition, approach is theoretically grounded and explicit in its assumptions, which could make it easier for a practitioner to evaluate and use. In particular, our definition of spurious features gives an explicit characterization of features/biaser that we regard as undesirable and our approach seeks to reduce, which can allow the user to understand the situations when the approach might be effective. We note that most approaches developed to improve subgroup robustness or OOD generalization either lack theoretical guarantees (Bao et al.] 2021 Bae et al. 2022: Liu et al., (021a) or show results tailored to one of these two settings. (Sagawa* al. (022)); Duchi et al. (2019) seek to optimize the worst-case risk over a small set of groups and hence, guarantees on worst-group risk developed in prior work on DRO which considers all subsets of the data as the groups of interest, apply in these cases. Recent work by Deng et al. (2023) ; Chen et al. (2023) analyzes the learning dynamics of two-layer CNN in the presence of spurious features. Deng et al. (2023) show that imbalanced data groups and predictive spurious features can cause stronger reliance on these features. Similarly, Chen et al.) (2023) show that while the model learns both features, the spurious features are learned faster. However, these methods may not generalize to unseen environments, as is often times the case in the OOD generalization problem. On the other hand, Arjovsky et al. (2020) prove that IRM can recover the optimal invariant predictor that can generalize to any environment, under certain assumptions the training data. However, follow-up work (Rosenfeld et al. 2021 shows that IRM may fail to recover such a prodictor if the training set does not contain samples from several environments. In contrast, we show that our approach reduces reliance on the spurious feature in a Gaussian mixture-base data setting, and also obtain an OOD generalization guarantee in a causal learning framework, without requiring group or environment labels.
Limitations of our Approach
We note that like any other regularization or inductive bias, CMID may not be effective for every task. For certain tasks, spurious features as defined by us in Section may not actually be spurious, and we may not always want to reduce reliance features that are simple and highly predictive of the label. However, it is
24

---

Published in Transactions on Machine Learning Research (08/2024)
worth noting here that it is impossible for single algorithm to perform well in all cases. This is established by the No Free Lunch theorem: there is no inductive bias that is suitable for all tasks (Shalev-Swwartz & (Ben-Davidn 2014) and relatedly, it is impossible to generalize without certain assumptions on the train and test distributions (Wolpert 1996..
In most cases, we assume that the test data i.i.d. and indeed, simplicity bias is useful explaining in-distribution generalization of NNs in these cases. However, these observations may not necessarily extend to situations where the i..d. assumption significantly different and in such cases, simplicity bias prevent the model from learning more complex, rask-relevant features that may generalize better. Consequently, alleviating simplicity bias can be a useful inductive bias in such cases. This is consistent with our experimental evaluations on several datasets from various modalities and domains.
the test set is violated. Generalization under distribution shift is
Even in situations where mitigating simplicity bias is useful, there can be cases where the separation between the feature complexity of spurious and invariant features may not be very large. This can make the selection of appropriate simple model class for our approach challenging. Indeed, in our experiments, we found that for the CelebA dataset, the spurious and task-relevant features are not significantly different in terms of complexity (Appendix 5.5). Consequently, our approach does not lead to much improvement in the worst-group accuracy on this dataset.
We also note that in general, the use of methods designed to improve OOD generalization and subgroup robustness can lead This is seen in some of our experimental results as well (see Fig. 12 and related discussion for more details). This is because a model trained with the ERM objective relics strongly on the spurious fcature(s), which is predictive of the train set as well the iid. test set. In contrast, improvement on OOD test sets is achieved by leveraging other features, which may be less predictive on the train set but generalize better. Some ways to control this trade-off are to change the regularization strength for our approach, or the model selection rule to take both train and validation set accuracies into account. This can allow the user to ensure good training performance, while also attaining improvements in OOD generalization.
drop in the accuracy on the training set or the i.i.d test set, compared ERM.
E
Additional Results
E.1 Effect CMI Regularization
In this section, consider the same setup as in Section 4 and present some additional results.
ERM
CMID, C=0.3
CMID, c=0.1
CMID, c-0.03
0/
Invoriant
Figure 9: Effect of CMI regularivation (wrt X2) for different values of e (corresponding to regularization strength). Left: Lower values of c indicate stronger CMI regularization, resulting in more upweighting of w1 wrt W2. Insct shows zoomed-in region and markers compare the three solutions when 0 = 1/6. Right: Decision boundaries for predictors corresponding to the markers.
In Fig. 9 we visualize the relationship between We and predicted in Theorem (assuming /n 1 and 7=0.95 0.95.) We see that lower value of e promotes conditional independence with X and upweighs wi more strongly. When 0, we + 0.
25

---

Published in Transactions on Machine Learning Research (08/2024)
Next, consider the case where there are multiple spurions features, by adding another feature
X3 ~ Ni ua, og), where d' yR((/)
(4)
the setup Assumption 2 We show that regularizing CMI with respect to the optimal predictor that uses X2 and X3 also results in w1 being upweighted. We consider the constrained problem:
ERMc(M) = arg min E (w, + W2 Wyyx3 - y)² s.1. [w1 X1 + 22 + w'y W3x3; w3s + wg N3gaaly) Su, WEM
(5)
where (wg, wz) & (P2/ . p3//o5) (the weights for the optimal linear model which uses X2 and X3). We show the following result.
Theorem 2. Let the data generated as per Assumption and
Let 00. for some sufficiently
small c, and o² =
o
oz.
Then the solution to 5 satisfies:
bepak+wausl = c', where d' = 201 1/ [ws/o1
o
03 o3
Morcover, W2, wy dwi.
Similar to before, the result says that for a sufficiently small CMT constraint, we learn a model which mainly uses the invariant feature.
E.2 Results on the 10-class Colored MNIST Dataset
In this section, we consider the 10-class colored MNIST dataset (following the setup in (Lee et al. 2021 which is a commonly used dataset for evaluating debiasing methods. In the train set, the samples in every class are injected with color with 95% correlation, while the colors in the test set are uncorrelated with the label.
In Table 13 we compare the test accuracy of our approach with several debiasing methods, namely LIF (Nam et al.) 2020), HEX (Wang et al., 2019a), and recent methods such as EnD (Tartaglione et al. 2021), , DFA (Lee et al.) 2021 and ReBias (Bahng et al. 2020). We observe that
Method
HEX
EnD
DFA
ReBias
CMID 87.12
Accuracy
74.62
80.57
81.15
89.66
96.96
Table 13: Comparison of OOD test accuracy on 10-class colored MNIST dataset.
CMID significantly improves over HEX, LfF and EnD, and is comparable DFA, while ReBias significantly oulperforms all other methods. We note that ReBias focuses particularly on debiasing with respect to texture color bias, which is why it is more effective on this dataset.
E.3 Results using ViT
In this section, we seek to verify that our method remains effective when the complex model is a VIT instead of CNN. Specifically, we compare the performance of ERM and CMID on the Waterbirds dataset, when training ViT-B/16 model (Dosovitskiy et al. 2021 As shown in Table 14 our method attains better test performance compared to ERM.
In this experiment, for our approach, we use the same shallow NN as the simple model class, as used in the subgroup robustness experiments, Note that when considering our definition of model complexity (number of trainable paramcters, layers, etc.), deop ViTs and CNNs of the same scale. As result, the choice of the simple model class can remain the same when considering the two architectures.
Method Average Ace Worst-group Ace
ERM
83.8
67.3
CMID
84.8
70.1
Table 14: Comparison of average and worst-
group accuracy on Waterbirds dataset when the complex model ViT.
E.4 Comparison with CLIP
In Table we compare the test performance of CMID with CLIP VIT-B/16 model (Dosovitskiy al., 2021 on two datasets. To classify test image, first provide CLIP with text descriptions for each class.
26

---

Published in Transactions on Machine Learning Research (08/2024)
For waterbirds, we use I'a landbird', 'a waterbird]], and for CelebA, use l'a non-blonde', 'a blonde]]. We then feed the test image into CLIP and choose the class with the highest cosine similarity to its output the predicted label.
We note that these datasets are challenging due to imbalances in training data. For example, predicting "blonde" vs. "not-blonde" on the CelebA dataset is difficult because there are very few blonde males in the training data. The imbalances in CLIP training data, on the other hand, are unclear. While CLIP models may perform well on some benchmarks, it remains uncertain if they are effective all cases. The results above demonstrate that CLIP models can be susceptible natural spurious correlations, highlighting the need for grounded and consistent robustness techniques, such as the proposed method CMID.
Dataset Waterbirds
Method
Average Ave
Worst-group Ace
CLIP
78.3
29.3
CMID
84.3
CelebA
CLIP
88.7
40.4
CMID
84.5
75.3
Table 15:
Comparison of average and worst-group ac-
curacy on Waterbirds and CelebA datasets with CLIP.
F Proofs for Theoretical Results
In this section, we present the proofs for the theoretical results in Section 4 and Appendix E
F.1 Proof of Proposition
We first restate Proposition
Proposition 1. For all E [d], ERM(M) sotisfies (upto scaling).
wi 41 2. wa-1
When Hy
3.
<1, ERM(M.)= 00............
Proof. We have:
d+1
2 -
Σ
x
[oî + m1) Σ wi + (03 + (2)
Σ
2w,wy
Σ 2w0
j=++1
2p11
w/
2w + 1.
1=1
d+1 w;Xx -
2
Since arg min E WEM to to be O. Subsoquently, solve the resulting set of equations. By taking the gradient and setting it to 0, obtain the following set of equations:
Σ
a convex problem, to find the minimizer we set gradients with respect
2wr + + 2/11 + 2nr
Σ
Wy + 2w0+1/41hs for i e [d].
(6)
j=1,
d
2w 1 (0} (1) 2 2p 1ub
Σ /=1
wy 0.
(7)
From (6, have that w. wy for all pairs i, [d] such that i + j. Specifically w; #1 W.+ 11111 for
dpf + of
all i E d. Substituting this into 7 and solving for Wg+1, get:
Wd+1
duf
H
of
0
27

---

Published in Transactions on Machine Learning Research (08/2024)
where o2 = +12 p2z Using this, get the expression for WA when i E [d]:
duf
H +1
di
Thus, for ERM(M) and any E [d],
wy
M1 Mg of
Next, consider ERM over M.. If for some E [d], wi non-zero, we use (6) and get
0,.... of +
as
the solution, for which the loss value is oftrl
If w; for all i [d], we use
( and get 0..... 0,
as+r3
as
" o < 1, the latter has a smaller loss and thus, (.M(MM))
the solution, for which the loss is oftp3
When
is 0
F.2 Proof of Theorem
Theorem 1. Let data be generoted as per Assumption 2 For v 0.5 log(1 + (²) for some c:
1. When o²² > eva' the solution 0102
the same as ERM(M), so for E [d],
wi
Bi 0 44 07
wa+1
2. Otherwise, for i [d], w, is upweighted and the solution to satisfies
wy
1
wa+1
cvd
d+1 Σ
Proof. Consider the constraint
wi Xi; wd+1 Xatily S u. As we are working with continuous random
i=1
variables in this setup, we employ differential entropy for our entropy computations. The entropy of a Gaussian random variable X with variance o² is given by H(X) 0.5(log(2mo²) 1). Using this and the definitions of CMI and conditional entropy, have:
d+1
d+1
Σ wi X u Xe+1ly
Σ
H
Σ
wi X,,yy ly,, wās Xa+1
d+1
H
Σ
w.
11
Σ wy w.XK
= log
f1llw:1li+03u of mor
where wy = w.)) Using 0.5 + )²,, the constraint becomes: to solving:
wa 1 10
S c. Thus,
reduces
d+ EwiX: y/ v)
min E
s.t.
wy 2
If 1/2 0102 <evd, ERM(M) satisfies the constraint and serves as the solution of
Otherwise, since this convex optimization problem with an affine constraint, the constraint must be tight. Therefore, we determine The solution, for i E [d] is given by:
the solution by finding ERM(M) subject to wasilds |arllam
Wd+1 + eeddau
of
dui 1+
1v of
0201
dui of
ivdpinh
28

---

Published in Transactions on Machine Learning Research (08/2024)
F.3 Proof of Theorem 2
Theorem 2. Let the dala be generaled per Assumption 2 and
with d = 1. Let v 0.5 log(1 + 2
for some sufficiently small and _ o.. Then the solution to
salisfies:
Iwash wwspll 01
d', where
d' 201 c 2 3
Morcover, W2, wy x d'u.
Proof. Let Z1 = 11 + 22 + 2y and Zz = W2X² + Xz. Then,
I(21, Z2/v) H (Z) + (22 (21,,
Since all fcatures are Gaussian, we have:
H(Zily) = 0.5 fF + 1 + -g03) + log (2T) ) + 1), H(Z2 y) = 0.5(log((w}) ( + (m3)3oz) (2x) + 1), (21,, Z2 Z2(y) = 0.5 K| 1og(2 + 1,
where K is the covariance matrix of Zi and Z2 conditioned on y. We can calculate K follows. Since all features are conditionally independent, have:
B(Z1 - B(Z1))² = + + w20 + w.
B(Zi B(Z)))(22 B((2)) = w2wjo²z + wawzo3.
B(Z2 B(Zz))² ( (w Po
Thus,
(mz) (13) - (wrwioz + wzwzo3)² Using these, the constraint. |K]= = (w]o} + W202 + w (1m))
becomes:
log
(wfof auga ( y - (w)) (wza tosw3oo3 < logg( x²)
(wf wiat (0 (o3
i (p)))))))))) (w www
u c²²
(o 2
(iawy of hns do w < <0
(fo +
3 o2
Assume that e is sufficiently small, i.e., c S 2020g wyw wy wywil
Then, we get the condition:
(wyy 3 traw (10 Fag [g
S 4c
w503 warg
twark w503
Mg
H/S
< 20y (wz) + (wz)2az.
When (mn, wg] [P///02 ,///03]] where is some constant, we get::
wayz wapu
2oo3 0/
of +
22
w1/01
0
2 Since
of ag
we get:
d2
Waph twaug
200
1 +
m o
c'
11101 01
When +
satisfies the constraint, and thus is the solution to Otherwise, since 5 is convex problem with affine constraint, the constraint must be
d, ERM(M), d.e., [wi, W2, wa] x
o 0
29

---

Published in Transactions on Machine Learning Research (08/2024)
tight. Therefore, find (RM(M) subject to the equality constraint (wzhh + wastzs c'or (r// The solution is given by:
146 21
W1
B1
2e
+
2 t1
11
M3
12
W2 c'a, 11
2tg
og
W3
co w1
0g +
0
1/
7
F.4 Proof of Proposition 2
Proposition 2. Let ERM(M.) Mr. Under Assumptions and the solution to the problem:
arg min E (Zr s.l. MMI)
MEM
is M Ø², the marimal invoriant predictor.
Proof. Using Assumption the class of simple models only contains variant predictors, so ERM(M.) V.. Consequently, the constraint in 2 can be written as U 0.
Considering the set of candidate predictors for M. namely {0, P.. V. Ve), we examine the CMI constraint and T(V,, V. = HV).
for each. Using the definition of mutual information, we have 7(0, V. Y)) According to the definition of variant predictor, H(V, > H(W. YY, E) 2 0.
From Assumption which states that the invariant and variant predictors are conditionally independent, we can deduce that [ V. YY O. From Assumption we also have (.lY)
E) 2
Using these results, the feasible set is |0, Pe], which corresponds to the invariance set (MM).. Consequently, problem (2) is equivalent to finding arg max Ø) The solution to this problem is D., which represents $EIz(M)
the MIP "-
G Experimental Settings
We begin by describing some common details and notation that we use throughout this section. As in the main text, we use À to represent the regularization strength for CMI. To ensure effective regularization, we adopt an epoch-dependent approach by scaling the regularization strength using the parameter Specifically, we set A₂ (1 + 1/S) at epoch The temperature parameter T is set as 12.5 throughout the experiments. Additionally, we use LR denote the learning rate, BS to denote the batch size, and A2 to denote the weight decay parameter, which represents the strength of éy-regularization. When using the Adam optimizer, we employ the default values for momentum.
The experiments on Slab data, CMNIST and CPMNIST data and Stult-Confounded data were implemented on Google Colab. The ImageNet-9 experiments were run on an AWS G4dn instance with one NVIDIA T4 GPU. For experiments on the subgroup robustness datasets and the Camelyon 1o-oon177-WLDD data, we used two NVIDIA V100 GPUs with 32 GB memory cach. We only used CPU cores for the Bios data experiments.
G.1 Mitigating Simplicity Bias Experiments
This section includes the details for the experiments showing that CMID mitigates simplicity bias where we use the Slab data and the ImageNet-9 data.
30

---

Published in Transactions on Machine Learning Research (08/2024)
G.1.1 Slab Data
Dataset. All the features in the 3-Slab and 5-Slab data are in the range [-1,1]. The features are generated
by defining the range of the slabs along each direction and then sampling points in that range uniformly at random. The basc codc for data generation came from the official implementation of [Shhh et al. (2000) available at https://gittheii htt.b..tttht..../// cww http///araahassimimmse citiipppiityyyiipppppppp tybiaspiiiiiiei tfalls We consider 10 training samples
and 5 x 10a test samples. In both cases, the linear margin is set as 0.05. The 3-Slab data is 10-dimensional, where the remaining coordinates are standard Gaussians and are not predictive of the label. The slab margin is set as 0.075. The 5-slab data is only 2-dimensional, and the slab margin is set as 0.14.
Training. We consider a linear model for the simple model and following Shah et al. 2020 1-hidden layer NN with 100 hidden units as the final model (for both ERM and CMID). Throughout we use SGD with BS = 500, A2 = 5 x 10-2 for training. The linear model is trained with LR = 0.05, while the NN is trained with LR- = 0.005.
For the 3-Slab data, the models are trained for 300 epochs. We consider de e {100, 150, 2003 and choose A 150 for the final result. For the 5-Slab data, the models are trained for 200 epochs and we momentum in this case. We consider Åe E {1000, 2000, 2500, 3000} and choose A = 3000 for the final result. Note that we consider significantly high values of xe for this dataset compared to the rest because the simple model is perfectly predictive of the label in this case. This implies that its CMT with the final model is very small, and the regularization strength needs to be large in order for this term to contribute to the loss.
0.99
G.1.2 Texture vs Shape Bias on ImageNet-9
Datasct ImageNet-9 (iaoo et al. 2020 is subset of ImageNet with nine condensed classes that cach consist of images from multiple ImageNet classes. These include dog, bird, wheeled vehicle, reptile, carnivore, insect, musical instrument, primate, and fish. ImageNet-A (Hendrycks et al., 2021) is set of handpicked images that ResNet50 models trained on ImageNet classify incorrectly. We organize these images into the same nine classes as ImageNet-9.
Calculating shape bias The GST dataset (Geirhos et al.] 2022 consists of 16 shape classes. To interpret a prediction from model trained on ImageNet-9 as a GST dataset prediction, consider subset of classes from both and use the mapping listed in Table [6 Specifically, to determine whether model trained on ImageNet-9 predicts correctly on a GST image, we first determine which of the 5 Image classes from the Table has the highest probability based on the model's output, and then use the mapping to obtain the predicted GST class label.
ImageNeet9 Class dog bird wheeled vehicle
GST I Dateset Classes dog
bird
blcycle, car, truck bear, keyboard
carnivore musical instrument
Table 16: ImageNet-9 classes mapped to datasct classes.
corresponding GST
Thus, following the procedure detailed in https://github.co com/ Bestrhos/texture-vs--ree
the shape bias is calculated as:
number of correct shape predictions
shape bias number of correct shape predictions + number of correct texture predictions'
Training We use ResNet18 pre-trained on ImageNet data as the simple model and train on ImageNet-9 using Adam with LR 0.001, BS 32, A₂ 1 for 2 epochs. We do not consider a simpler architecture and training from scratch since this pre-trained model already exhibits texture bias. For the final model, consider the same model and batch size, but we use SGD with 0.9 momentum as the optimizer, LR 10-4, X2 = 0.001, and train for 10 epochs. We report the mean and standard deviation over 3 runs, where for each run we select the model with the highest shape bias. Values of CMID-specifie parameters were A = 30, S 10. For tuning, we consider LR E {10 10-4, 3) for both the models and BS (16,32), X2 E (0.0001, 0.001, 0.013 and Xe E (0.5, 15, 30, 50). We implement JTT to obtain the results. We consider the upsampling parameter (Liu) et al. 2021a) Aup E (5, 20, 50, 100) for tuning. The other hyperparameters are the same as our approach.
31

---

Published in Transactions on Machine Learning Research (08/2024)
G.2 Better OOD Generalization Experiments
This section includes the details of the experiments showing that CMID leads to better OOD generalization. For this, we used CMNIST and CPMNIST, CamelyonI7-WILDS and Adult-Confounee datasets.
G.2.1 CMNIST and CPMNIST
Datasct. Following Bac et al. 2022), we use 25, 000 MNIST images (from the official train split) for each of the training environments, and the remaining 10, 000 images to construct validation set. For both test sets, we use the 10, 000 images from the official test split.
Training. The details about the model architecture and parameters for training the simple model with ERM and the final model with CMID, for both datascts, are listed in Table 17 Following [ac et al. (2022), the MLP has one hidden layer with 390 units and ReLU activation function. In both cases, the simple model is trained for epochs, while the final model is trained for 20 epochs. We choose the model with the smallest accuracy gap between the training and validation sets. For tuning, we consider the following values for each parameter: for the simple model, LR E {0.005, 0.01, 0.05), X E {0.001, 0.005, 0.01J, and for the final model, LR E 1..001,00055) da E [3, 8] and S E [3, 6). where lower values of S were tried for higher values of X. and vice-versa.
For the final results, we report the mcan and standard deviation by averaging over runs. For comparison, we consider the results reported by Bae et al. (2022) for all methods, except EII.. (Creager et al. 2021 and JTT (Liu al. 2021a). Results for EILL are obtained by using their publicly available implementation for CMNIST data (available at https: thub. www.ttcwubee.eeeeeeeeeeeeeeeeeee com//crreger/eiii.......
and incorporating the CPMNIST data into their implementation. The hyperparaneeeer values in their implementation are kept the same. We implement JTT to obtain the results. We consider LR E [0.001, 0.005, 0.01} and the paramcter for upweighting minority groups (Liu et al.,
2021a] Aup E {5, 10, 15, 20, 25} for tuming.
Dataset CMNIST
Simple Model Optimizer
LR
BS
Aa
Final Model
Optimizer
LR
BS
A
S
Linear
SGD
0.01
64
0.005
MLP
SGD
0.001
64
4
CPMNIST
Linear
SCD
0.01
64
0.001
MLP
SCD
0.005
64
5
3
Table 17: Training details for CMNIST and CPMNIST.
G.2.2 Camelyon17-WILDS
Dataset. Cammlyon17-WTT.DD Kohh et al. 2021 contains 96 x 96 image patches which may or may not display tumor tissue in the central region. We use the same dataset as Bae et al. (2022). which includes 302, 436 training patches, 34, 904 OOD validation patches, and 85, 054 OOD test patches, where two data splits contain images from overlapping hospitals. We use the WILDS package, available at. for data-loading.
Training. For the simple model, we train 2DConvNet1 model (see Section G.3.2 2 for details) for 10 epochs. We use the Adam optimizer with LR = 10-1, 1, = 32, X2 = 10-4, For the final model, we train a NenseNet121 (randomly initialized, no pretraining) for 5 epochs using SGD with 0.9 momentum with BS = 32, A2 = 0.01 and A = 0.5, S = 10. We use the same BS and Ag values as Bac et al. 2022
LR = 14 for consistency. For tuning, we consider LR [10 10 10-3) for both the models and > E (0.5, 2, 5, 15) for CMID. While [ae al. (2022) and Koh et al. (2021) use learning rates 10-5 and 0.001, respectively, we found learning rate of 10 was most suited for our approach. We select the model with the highest average accuracy on the validation set to report the final results and report the mean and standard deviation by averaging over 3 runs. For comparison, we use the results reported by Bac et al. (2022) for all the methods,
except JTT. For the results for JTT, we implement the method and lune Aup E (20, 50, 100) and the number of epochs to train the first-stage model as [1,2]. The rest of the hyperparameters are kept the same as our approach. We noticod that JTT (Liu et al., (2021a) had a high variance test accuracy across multiple runs for different random weight initialization. To account for this, we ran JTT and CMID over the same 8 seeds and randomly chose 3 of them to report the average accuracies over.
32

---

Published in Transactions on Machine Learning Research (08/2024)
G.2.3 Adult-Confounded
Dataset. The UCI Adult dataset (Newman et al., 1998: Leisch & Dimitriadou 2021 p,, comprises 48, 842
census records collected from the USA in 1994. contains attributes based on demographics and employment. information and the target label is a binarized income indicator (thresholded at $50, 000). The task is commonly used as an algorithmic fairness benchmark. Lahoti et al. 22000g Creager al. (2021) define four sensitive subgroups based on binarized sex (Male/Female) and race (Black/non-Blaar
labels: Non-Black Males (G1), Non-Black Females (G2), Black Males (G3), and Black Females (G4). They observe that each subgroup has a different correlation strength with the target label (p(y - 1[G)), and thus, in some cases, subgroup membership alone can be used to achieve a low error rate in prediction.
Based on this observation, Creager et al. (2021) create a semi-synthetic variant of this data, known as Adult-Confounded, where they exaggerate the spurious correlations in the original data. As G1 and G3 have higher values of ply = 1/G) across both the splits, compared the other subgroups (see Creager et al. (2021) for exact values), these values are increased to 0.94, while they are set to 0.06 for the remaining two subgroups, to generate the Adult-Confounded dataset. In the test set, these are reversed, so that it serves as a worst-case audit to ensure that the model is not relying on subgroup membership alone in its predictions. Following Creager et al. (2021). we generate the samples for the Adult-Confounl using importance sampling. We use the original train test splits from UCI Adult as well as the same subgroup using importance weights based on the correlation
dataset by
sizes, but individual examples are between the original data and the desired correlation.
Training. We use a linear model (with bias term) as the simple model, and following Creager et al. (2021) use an Adagrad optimizer throughout. We use BS = 50. The simple model is trained for 50 epochs, with LR = 0.05, A2- = 0.001. Following Creager et al. (2021 Lahoti et al. (2020), we use a two-hidden layer MLP architccture for the final model, with 64 and 32 hidden units, respectivcly. It is trained with LR 0.04, A, = 4, S = 4 for 10 epochs. We also construct a small validation set from the train split by randomly selecting a small fraction of samples (5 ( 50) from each subgroup (depending
its size) and then upsampling these samples to get balanced subgroups of size 50. We choose the model with the lowest accuracy gap between the train and validation sets. For tuning, we consider LR E {0.01, 0.02, 0.03, 0.04, 0.05} and S E (3.4.5). For the final results, we report the mean and accuracy by averaging over 4 runs. For comparison, we reproduced the results for ERM from Creager et al. (002)), and thus, consider the values reported in Creager et al.) (2021) for ERM, ARL and EIIL. We implement JTT to obtain the results, using LR 0.05 and X2- 0.001 to train the final model. We tune the parameter for upweighting minority groups LLu al. 2021a) Xup over (10, 20, 25, 30). The remaining paramcters are kept the same for our approach.
G.3
Subgroup Robustness Experiments
This section includes the details of the experiments showing that CMlD enhances subgroup robustness, where we use four benchmark datascts: Waterbirds, CclebA, MultiNLI and CivilComments.
G.3.1 Datasets
We consider the following datasets for this task. We follow the sctup in Sagawa* et al. (020 for the first three and Koh el al. (2021) for WivilCoommenns-IILDD.
Waterbirds Waterbirds is a synthetic dataset created by Sagawa* et al. (2020) consisting of bird images over backgrounds. The task is to classify whether a bird is a lundbird or a waterbird. The background of the image land background or water background, acts a spurious correlation.
CelebA CelebA is a synthetic dataset created by Liu et al. 2015 containing images of celebrity faces. We classify the hair color as blonde or not blonde, which is spuriously correlated with the gender of the celebrity
male or female, as done in Sagawa* et al. (0200) Liu et al. (2021a).
MultiNLI MultiNLI (Williams et al. 2018 is a dataset of sentence pairs consisting of three classes: entailment, neutral, and contradiction. Pairs labeled based on whether the second sentence entails, is
33

---

Published in Transactions on Machine Learning Research (08/2024)
neutral with, contradicts the first sentence, which is correlated with the presence of negation words in the second sentence (Sagawa* et al. 2020: Liu et al. 2021a
WwilCommennt-WILLDS is dataset of online comments proposed by Borkan et al. (019)). The goal is to classify whether a comment is toric or non-tozic, which is spuriously correlated with the mention of one or more of the following demographic attributes: male, female, White, Black, LGBTQ, Muslim, Christian, and other religion (Park et al.) 2018 Dixon et al. 2018). Similar to previous work (Koh al., 2021 Liu al. 2021ap, we evaluate over 16 overlapping groups, one for each potential label-demographic pair.
Comimnometts WILDS
G.3.2 Model Architectures
In this section, we discuss the architectures we consider for the simple models for this task. For the two image datasets, a shallow 2D CNN is a natural choice for the simple model as 2D CNNs can capture local patterns and spatial dependencies grid-like data. On the other hand, for the two text datasets with tokenized representations, we consider a shallow MLP or 1D CNN for the simple model. MT.Ps can capture high-level relationships between tokens by treating each token a separate feature, while ID CNNs can capture local patterns and dependencies in sequential data.
Dan Rulll Pool
Curs
Avg Pool
FC Pulli
FC
Duw Rull
FC
Rull
PC
QMBLE
Onpet
Figure 10: Left: 2DConvNet1 and Right: 2DConvNet2 architectures.
Figure 11: Left: 2MLP and Right: 1DConvNet architectures.
Next, we describe the details for the model architectures. Tet F denote the filter size and C' denote the number of output channels (for convolutional layers) or the output dimension (for linear connected (FC) layers). Throughout, we use F=2 for the average pooling layers. Fig. shows the 2DConvNet1 and the 2DConvNet2 architecture, which were used as simple models for Waterbirds and CelebA respectively. These were the only two architectures we considered for the 2DCNN on these datascts. In 2DConvNet1, the 2D convolutional layers use F C 10 and 44 -0, respectively, while C = 2000 for the FC layer. 20, In the 2DConwNet2 architecture, F= 10 for the 2D convolutional layer and C = 500 for the FC layer. Fig. 17 shows the 2MLP and the 1DConvNet architecture, which were used as simple models for MultiNT. and respectively. For tuning, we considered these models as well as 1DCNN with one less ID convolutional layer than 1DConvNet for both datasets. In 2MLP, the FC layers use C= 100 and - respectively. In the 1DConvNet architecture, the ID convolutional layers use F=7, C'= 10, F= C 32 and F=5, 5, =464, respectively, while 500 for the FC layer.
GovilComments-Wlll
G.3.3 Training Details
We utilize the official implementation Sagawa et al. (2020) available at htttp::://githubb.omm//////o group_DRO as bascline code and integrate our approach into it. Most hyperparameter values are kept unchanged, and we list the important parameters along with model architectures for all the datasets in Table 18 For the simple models, we consider shallow 2D CNNs for the image datasets, and MLP and ID CNN for the text data, as discussed in the previous section. In all cases, the simple model is trained for 20 epochs. For the final model, following Sagawa* al. (2020), we use the Pytorch torchvisi in implementation of ResNet50 (He et al.] 2016) with pre-trained weights ImageNet data
34

---

Published in Transactions on Machine Learning Research (08/2024)
for the image datasets, and the Hugging Face pytorch-transformer implementation of the BERT bert-base-uncasser model, with pre-trained weights (Devlin et al. 2019) for the language- based datasets.
Datasnt
Simple Model
Optimizer
LR
BS
A2
Final Model
Optimizer
B8
Ag
A
S
# epochs
Waterbirds
2DConvKetl
Adam
10
32
10-1
ResNet50
SGD
5x10
128
10
20
4
100
CelebA
20ConvNet2
Adam
10-5
32
5x 10-
ResNet50
SGD
3
128
0.001
10
5
50
MulsiNLI
2MLP
Adam
0.005
16
10-4
BERT
Adham
5x10-
32
0
75
10
CivlCoomments
IDConvNet
Adam
10
16
10-1
BERT
AdamW
10-5
32
0.001
10
10
Table 18: Training details for subgroup robustness datasets.
Table 19 shows the values of LR, and S we consider for tuning for the final model. Following Sagawa lt al. (2020), we keep A = 0 for MultiNL. For the rest, we consider A2 E f0.0001, 0.0005, 0.0013. For re- sults, we choose the model with the best gorst-group accuracy on the validation set. For comparison, we consider the values reported in Liu et al. (2021a).
Dataset
LR
Aa
S
Waterbirds, CelebA
[1.G] x 10
{10. 15, 20, 25, 50, 75)
{4, (4,5. 6,8, 10)
MultiNLI, CivilConments
(1,2,5) 10-5
{10, 25, 50,
(10)
Table 19:
Values considered for tuning the hyperparameters for training the final model for the four subgroup robustness datasets.
G.3.4 Further Discussion Experimental Results
Waterbirds
CelebAA
MulU
CivilCumments
96
80
MEND
Accursyy
.90-
#11
6%%
76-
75
DMD
85
70
70
72
65
70
23
coomen
3M
70
66
55
05
95 Average Accuracy
90
100
75
80
85
90
95
100
78
10
80
85
90
Acti ago Accuracy
Avaraga Accuracy
Average Accuracy
Figure 12: Scatter plots comparing worst-group and average accuracy of various methods on four benchmark datascts for subgroup robustness. The dotted line is the x z line. The dashed line represents the trade-off between average and worst-group accuracy.
Fig. 12 shows scatter plots comparing the worst-group and average accuracy of various methods on four benchmark datasets for subgroup robustness. The dashed line represents the trade-off between average and worst-group accuracy. Points on or close this line correspond to methods that shift the balance between the unvarian//spuriou features so that the gains in the worst-group accuracy match the decrease in the average accuracy. We observe that GDRO in all cases as well CMID and JTT in several cases (except Waterbirds and MultiNLI, rospectively) lie far from this line. This suggests that in most cases, these methods allow for more gains in the worst-group accuracy compared to the decrease in the average accuracy.
G.4 Bias in Occupation Prediction Experiment
We use version of the Bios data shared by the authors of (De-Arleaga et al. 2019) We used the olficial to
implementation of (Cheng et al., 2023), available at httpp:://itthbb wtw.g.tttt..........oooeooete. wtt.:/uw....o..iiweeeeeee.eeeeee Settiveleet//noopaaaee obtain results and for comparison purposes. In this implementation, they consider 25 occupations and train separate one-vs-all linear classifiers for each occupation based on word embeddings to make predictions. We directly used their implementation to obtain results for FRM and Decoupled (Dwork et al., 2018) on the data.
For our approach, we employed linear models for both the simple model and the final model. We directly regularized the CMI with respect to the ERM from their implementation. The final model was trained for 5 epochs using SGD with LR 0.1, BS 128, > = 5. S = 5. We only tuned the LR for this case, considering values of 0.05 and 0.1.
35


Reviews:
[{"review_id": "5bjDEgW6gq", "reviewer": "Reviewer_nfY4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strenghts**:\n\n- The paper is well-written.\n\n- The problem considered in this work is of interest to a wide range of audiences in the machine learning community, including researchers focusing on simplicity bias in neural networks, the role of information theory in machine learning, and those working on mitigating the effect of spurious correlations in ML models.\n\n- The overall method is logical, natural, and appears novel. The idea is beautiful and holds significant potential for future development.\n\n- Experimentations and additional materials in the Appendix section are thorough and comprehensive. Also, the method has been empirically shown to be quite effective in at least some real-world applications.\n\n-------------------------\n\n**Weaknesses**:\n\n- The theoretical setting considered is very limited, but this might not be a weakness since the paper is primarily experimental.\n\n- (**Major issue 1**) As far as I have noticed, the authors have not empirically investigated in-domain generalization. Training a complex model to deviate from a simpler one is intuitively justified to improve OOD generalization, but it may degrade in-domain generalization, especially when data is scarce. As the data volume increases, this issue is expected to diminish, but at what point does this phenomenon shift? How many more samples does your method require to achieve comparable in-domain generalization? Please either justify your method based on this argument or conduct additional experiments to address this issue.\n\n- (**Major issue 2**) The simple model can learn \"some\" of the simple and spurious features, and through your CMI regularizer, these features do not transfer to the complex model. However, what if there are several spurious features? How large or small does the simple model need to be to i) only learn the simple and \"bad\" features, and ii) learn \"all\" such features? I believe this question is also left unanswered in the paper.\n\n- (**Question**) In Table 7, both CMID and EIIL have a higher OOD accuracy compared to training accuracy. Why?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the issue of out-of-distribution (OOD) generalization in neural networks by building on concepts from previous studies on simplicity bias. Neural networks, even when overparameterized, tend to learn \"simple\" features, an empirically proven phenomenon known as simplicity bias. While simplicity bias enhances in-distribution generalization (evidenced by phenomena such as the double-descent curve), it undermines OOD generalization because simple features are often spurious (i.e., correlated with irrelevant parts of the input, such as the background). Consequently, learning more complex features that achieve similar in-domain classification accuracy but significantly better OOD classification performance is desirable.\n\nThe authors correctly note that, due to the No Free Lunch (NFL) theorem, there are no universal solutions to this problem, and distinguishing between invariant and spurious features requires some assumptions. They propose that simple (and hence spurious) features are those that can be learned with a \"simple\" neural network. Their method involves first training a simple neural network to perform the task, then training a more complex neural network architecture such that it i) achieves low training/validation error, and ii) learns features with low conditional mutual information with those of the simple neural network. They refer to this method as CMID.\n\nThrough several experiments, the authors demonstrate that CMID achieves superior OOD generalization compared to competing methods on well-known datasets containing both relevant (invariant) and irrelevant (spurious) features. They also provide some theoretical guarantees using a toy example of Gaussian mixture model data (with one spurious feature/dimension) and within a causal learning framework.\n\nOverall, this paper is a strong experimental study (with some limited theoretical results), and its core idea is novel and beautiful. I have a number of questions and requests (see the Weaknesses section), and assuming these are addressed satisfactorily, I believe this paper is a good candidate for TMLR.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please adjust the tables at the bottom of each page instead of in the middle of the text.\n\n- Please see (weaknesses) section.", "broader_impact_concerns": "-", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hxB6D0ADBR", "reviewer": "Reviewer_2xRY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-----------------\n\n1. A simple method for improving robustness to spurious features with no group annotation required.\n2. Adequate theoretical and empirical support provided to justify the hypothesis.\n3. Better worst-group accuracy than prior methods.\n\nWeaknesses\n--------------------\n\n1. The choice of \"simpler\" models vs \"complex\" models seems to be very task-specific and undecidable without access to a validation set with worst-group labels. Since this is fundamental to the problem, a more detailed suggestion on this front could help.\n\n2. In some sense, this seems to be the reverse of the distillation problem - while in distillation, task-specific knowledge is \"distilled\" from a complex model to a simple model, here task-unrelated information is being avoided from a small model to a large model. On this note, more experimental results have to be presented comparing the accuracy using distillation from large models.\n\n3. All the image datasets use CNNs, but ViTs have been more commonplace for a while now. The experiments have to incorporate the equivalent choice of \"simpler\" vs complex models when using ViTs. Similarly, results using training-free, zero-shot methods like CLIP have to be included in the comparisons.\n\n4. While the worst-group accuracy is improved, this comes at the cost of average accuracy/dominant group accuracy - this trade off is more severe using CMID compared to prior approaches in table 8.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper first analyzes the hypothesis put forth that simple features are often spurious and lead to shortcut learning hurting the generalization to few subgroups. To counter this limitation, the paper proposes CMID, where they train a simple neural network first to learn the spurious features, and then train another (more complex?) network to \"stay away\" from learning these features using an MI minimization constraint. They show theoretical support for their hypothesis and better empirical evidence on many standard datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) More experimental results with advanced backbones.\n\n2) Comparisons with zero shot methods like CLIP using comparable backbones.\n\n3) Comparisons using distillation approaches.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "idGOZxMgil", "reviewer": "Reviewer_ojhn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1.\tThis paper proposes a universal framework that can effectively alleviate the simplicity bias of neural network, and the proposed framework can be simply inserted into existing methods and improve their generalization ability, subgroup robustness, and fairness.\n2.\tThis paper provides a comprehensive theoretical proof that its proposed framework can alleviate simplicity bias and enhance the generalization of neural networks on out of distribution data.\n3.\tThis paper conducts experiments on datasets with different focuses in multiple fields and verifies the competitiveness and superiority of its proposed framework. \nWeaknesses:\n1.\tThe two-stage training strategy proposed in this paper first trains a simple model on the training data, and then trains a complex model by adding conditional mutual information terms to the loss function. However, the complexity of the models used in both theoretical and empirical analyses is the same. This causes inconsistency in the contextual statements.\n2.\tThe paper proposes a universal framework for alleviating simplicity bias, but its experimental setup is relatively simple, and the experimental results do not reflect whether the framework is still effective when combined with existing state-of-the-art methods.\n3.\tThere are some problems in the usage of symbols, which reduces its readability.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an assumption about simplicity bias: surrogate features are simple features that are still predictive of the label. This paper proposes a new framework that utilizes conditional mutual information regularization to alleviate simplicity bias, and theoretically analyzes its effectiveness. Experiments verify that the proposed framework can effectively alleviate simplicity bias and improve the model's generalization ability, subgroup robustness, and fairness.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "see above", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I believe, there are several downsides to the paper:

1. The core intuition that the spurious features correspond to simple features that correlate with the target is not novel, and is in fact the default understanding of the spurious correlation issue. This point may be taken as subjective.

2. There are counter-examples to the above intuition, at least if taken literally. The authors discuss how the Celeb-A dataset doesn't follow this intuition. Moreover, [this paper in Appendix B.1](https://arxiv.org/abs/2204.02937) shows that when trained on the *inverse waterbirds problem*, i.e. when the label is associated with the background and the bird is a spurious feature, the models still perform poorly on minority groups. In other words, the poor worst group performance on waterbirds cannot be explained by the background being simpler than foreground.

3. The proposed method is qualitatively very closely related to multiple existing methods. [JTT](https://arxiv.org/abs/2107.09044) also trains  two models, where the first one is supposed to learn the spurious features, and the second one uses predictions from the first one. There are also many other very similar methods: [LfF](https://arxiv.org/abs/2007.02561), [AFR](https://arxiv.org/abs/2306.11074), etc.

4. The results reported are not outstanding, compared to other recent methods. For example, [AFR](https://arxiv.org/abs/2306.11074) achieves stronger results on waterbirds, CelebA and MultiNLI, as far as I can tell.

Despite these limitations, I believe this paper meets the standards of TMLR, and will be a valuable contribution to the journal. I think the paper should be accepted. 

Minor: I would recommend that the authors include stronger more recent baselines in Table 8: e.g. [AFR](https://arxiv.org/abs/2306.11074), [CnC](https://arxiv.org/abs/2203.01517).
Decision: Accept (poster)