Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 18:39:40

Abstract: Classifiers trained with Empirical Risk Minimization (ERM) tend to rely on attributes that have high spurious correlation with the target. This can degrade the performance on underrepresented (or 'minority') groups that lack these attributes, posing significant challenges for both out-of-distribution generalization and fairness objectives. Many studies aim to improve robustness to spurious correlation, yet nearly all require group annotation for training and/or model selection. This constrains their applicability in situations where the nature of the spurious correlation is not known, or when group labels for certain spurious attributes are either insufficient or completely absent. To meet the demand for effectively enhancing the model robustness under minimal assumptions about group annotation, we propose Environment-based Validation and Loss-based Sampling (EVaLS). It uses the losses from a trained model to construct a balanced dataset of high-loss and low-loss samples in which the training data group imbalance is mitigated. This results in a significant robustness to group shifts when equipped with a simple mechanism of last layer retraining. Furthermore, by utilizing environment inference methods for creating diverse environments with correlation shifts, EVaLS can potentially eliminate the need for group annotation in the validation data. In such a context, the worst environment accuracy acts as a reliable surrogate throughout the retraining process for tuning hyperparameters and finding a model that performs well across diverse group shifts. EVaLS effectively achieves group robustness, showing that group annotation is not necessary even for validation. It is a fast, straightforward, and effective approach that reaches near-optimal worst group accuracy without needing group annotations, marking a new chapter in the robustness of trained models against spurious correlation.

Main paper: 
 problem setting: 
 We assume a general setting of a supervised learning problem with distinct data partitions D tr for training, D val for validation, and D test for final evaluation. Each dataset comprises a set of paired samples (x, y), where x ∈ X represents the data and y ∈ Y denotes the corresponding labels.Conventionally, D tr , D val , and D test are assumed to be uniformly sampled from the same distribution.However, this idealized assumption does not hold in many real-world problems where distribution shift is inevitable. In this context, we consider the sub-population shift problem [4] . In a general form of this setting, it is assumed that data samples consist of different groups G i , where each group comprises samples that share a property. More specifically, the overall data distribution p(x, y) = i α i p i (x, y) is a composition of individual group distributions p i (x, y) weighted by their respective proportions α i , where i α i = 1. In this work, we assume that D tr , D val , and D test are composed of identical groups but with a different set of mixing coefficients {α i }. It is noteworthy that the validation set may have approximately identical coefficients to those of the training or testing sets, or it may have entirely different coefficients.Several kinds of subpopulation shifts are defined in the literature, including class imbalance, attribute imbalance, and spurious correlation [4] . Class imbalance refers to the cases where there is a difference between the proportion of samples from each class, while attribute imbalance occurs when instances with a certain attribute are underrepresented in the training data, even though this attribute may not necessarily be a reliable predictor of the label. On the other hand, spurious correlation occurs when various groups are differentiated by spurious attributes that are partially predictive and correlated with class labels but are causally irrelevant. More precisely, we can consider a set of spurious attributes S that partition the data into |S| × |Y| groups. When the concurrence of a spurious attribute with a label is significantly higher than its correlation with other labels, that spurious attribute could become predictive of the label, resulting in deep models relying on the spurious attributes as shortcuts instead of the core ones. This is followed by a decrease in the model's performance on groups that do not have this attribute.Given a class, the group containing samples with correlated spurious attributes is referred to as majority group of that class, while the other groups are called the minority groups. As an example, in the Waterbirds dataset [7] , for which the task is to classify images of birds into landbird and waterbird, there are spurious attributes {water background, land background}. Each background is spuriously correlated with its associated label, decompose the data into two majority groups waterbird on water background, and landbird on land background, and two minority groups waterbird on land background and landbird on water background. Our goal is to make the classifier robust to spurious attributes by increasing performance for all groups.In scenarios where group annotations are absent, traditional methods that depend on these annotations for training or model selection become infeasible. Moreover, as previously discussed by [14] , when data contains multiple spurious attributes and annotations are only available for some of them, such methods would make the model robust only to the known spurious attributes. To explore such complex scenarios, we introduce the Dominoes Colored-MNIST-FashionMNIST (Dominoes CMF) dataset (Figure 3(d) ). Drawing inspiration from Pagliardini et al. [17] and Arjovsky et al. [18] , Dominoes CMF merges an image from CIFAR10 [19] at the top with a colored (red or green) MNIST [20] or FashionMNIST [21] image at the bottom. The primary label is derived from the CIFAR10 image, while the bottom part introduces two independent spurious attributes: color and shape. Although is validated based on the worst accuracy of the inferred environments. Note that majority and minority groups are shown with dark and light colors for better visualization, but are not known in our setting.annotations for shape are provided for training and model selection, color remains an unknown variable until testing. For more details on the dataset refer to the Appendix.The illustrations in Figure 3 (a-c) depict the outlined scenario. A classifier trained using ERM is dependent on both spurious features (Figure 3(b) ). Yet, achieving robustness against one spurious correlation (Figure 3 (c)), does not ensure robustness against both (Figure 3 (a)). In Section 4 we show that our method, which does not rely on the group annotations of the identified group, achieves enhanced robustness against both spurious correlations, outperforming strategies that depend on the known group's information.The variance of L can be computed using the properties of covariance. Recall that if Z ∼ N (h, t 2 I d ),then the covariance matrix of Z is t 2 I d .The variance of the linear combination L = W T Z is given by:Var(L) = Var(W T Z) = W T Cov(Z)W.Given Cov(Z) = t 2 I d , we have:Var(L) = W T (t 2 I d )W = t 2 W T I d W = t 2 ∥W ∥ 2 ,where ∥W ∥ denotes the Euclidean norm of W .Combining the mean and variance results, we conclude that L is normally distributed with mean W h and variance t 2 ∥W ∥ 2 :L = ⟨W, Z⟩ ∼ N (W h, t 2 ∥W ∥ 2 ).Thus, we have proved that if Z ∼ N (h, t 2 I d ), then the logits L = ⟨W, Z⟩ follow the distributionN (W h, t 2 ∥W ∥ 2 ).From now on, we consider N (µ min , σ 2 min ) and N (µ maj , σ 2 maj ) as the distribution of minority and majority samples in logits space.Next, we prove the more formal version of the main proposition 3.1 which describes the existence of a balanced dataset, only after we define a key concept, proportional density difference (illustrated in figure 4 ) to outline our proof.Definition C.1 (Proportional Density Difference). For any interval I = (a, b] and a mixture distribution εP 1 (x) + (1 -ε)P 2 (x), proportional density difference is defined by the difference of accumulation of two component distributions in the interval I and is denoted by ∆ ε P mixture (I).∆ ε P mixture (I) ∆ = εP 1 x ∈ I -(1 -ε)P 2 x ∈ IDefinition C.2 (Tail Proportional Density Difference). For a mixture distribution εP 1 (x) + (1ε)P 2 (x), we define tail L (α) as ∆ ε P mixture (-∞, α] and tail R (β) as -∆ ε P mixture (β, +∞) .Corollary C.1. tail L (α) = εF 1 (α) -(1 -ε)F 2 (β) tail R (α) = (1 -ε) 1 -F 2 (β) -ε 1 -F 1 (β)where F 1 and F 2 are CDF of two component distributions. Proposition C.1 (Feasiblity Of Loss-based Group Balancing). Suppose that L is derived from the mixture of two distributions N (µ min , σ 2 min ) and N (µ maj , σ 2 maj ) with proportion of ε and 1 -ε, respectively, where ε ≤ 1 2 . There exists α and β such that restricting L to the α-left and β-right tails of its distribution results in a group-balanced distribution if and only if σ min ≥ σ maj orEQUATIONandEQUATIONwhereA = 1 2σ 2 maj -1 2σ 2 min , B = µmin σ 2 min - µmaj σ 2 maj and ∆ = (µmin-µmaj) 2 σ 2 min σ 2 maj -4 log σmaj σmin + log ϵ 1-ϵ 1 2σ 2 maj -1 2σ 2 min .Proof outline Our proof proceeds with three steps. First, we reformulate the theorem as an equality of left-and right-tail proportional distribution differences. In other words, we show that the more mass the minority distribution has on one tail, the more mass the majority distribution must have on the other tail. Afterward, supposing µ min < µ maj WOLG, we propose a proper range for β values on the right tail. We show that when σ maj ≤ σ min , values for α trivially exist that can overcome the imbalance between the two distributions. In the last step, for the case in which the variance of the majority is higher than the minority, we discuss a necessary and sufficient condition for the existence of α and β based on the left-tail proportional density difference using the properties of its derivative with respect to α.Step 1 Reformulating the problem based on proportional distribution difference.We introduce a utility random variable Logit Value Tier as T , which is defined as a function of a random variable L.T α,β =    High if L ≥ β M id if α < L < β Low if L ≤ α (6)We can rewrite the problem in formal form as finding an α and β which satisfies the following equation:EQUATIONEquation 5 now can be rewritten to a more suitable form:EQUATIONEQUATION⇐⇒ P T α,β ̸ = M id g = min P g = min = P T α,β ̸ = M id g = maj P g = maj ( 10)EQUATIONEQUATION(1 -ε) P T α,β = Low g = maj + P T α,β = High g = maj (13)EQUATION(1 -ε) P L ≤ α g = maj + P L ≥ β g = maj (15)EQUATIONEQUATIONWe can see the left side of equation 17 is just a function of alpha. The same goes for the right side of the equation which is a function of β.Rewriting the left side of the equation as tail L (α) and right side as tail R (β), the problem is now reduced to finding an α and β that satisfiesEQUATIONwhich is shown in figure 4 .Before reaching out to step two we discuss the properties of tail L and tail R in Lemma C.2.Lemma C.2. tail L (α) and tail R (β) are continuous functions and lim α→-∞ tail L (α) = 0,lim α→+∞ tail L (α) = 2ε -1 < 0 , lim β→+∞ tail R (β) = 0 and lim β→-∞ tail R (β) = 1 -2ε > 0.Proof. Simply proved by the definition of tail functions and properties of CDF.Step 2 Solving the equation 18 for simple cases.EQUATIONCorollary C.2. Because tail R is continuous and lim β→+∞ tail R (β) = 0, based on the mean value theorem, any value between zero and (1-2ε) 2 is obtainable by selecting a β in [µ 2 , +∞).According to the previous corollary C.2 finding a positive tail L (α) will satisfy our need. to find a suitable point, we employ derivatives and properties of relative PDFs to maximize tail L (α) and find a positive value.EQUATIONThe term [ f min (α) f maj (α) -1-ε ε ] has the same sign with derivative of tail L (α), also it's roots are critical points of tail L , analyzing characteristics of log f min (α) f maj (α) is the key insight to find a proper α value.log f min (α) -log f maj (α) = log 1 -ϵ ϵ ⇒ log σ maj σ min -log 1 -ϵ ϵ - (α -µ min ) 2 2σ 2 min + (α -µ maj ) 2 2σ 2 maj = 0 ⇒ 1 2σ 2 maj - 1 2σ 2 min α 2 + µ min σ 2 min - µ maj σ 2 maj α + µ 2 maj 2σ 2 maj - µ 2 min 2σ 2 min + log σ maj σ min + log ϵ 1 -ϵ = 0Because lim α→-∞ tail L (α) = 0 and lim β→+∞ tail R (β) < 0 to have a positive tail L (α) we need to have an interval which dtail L (α) dα is positive, for a second degree polynomial like ax 2 + bx + c to have positive value, either a ≥ 0 or ∆ > 0, in our case a is 1σ 2 maj -1 σ 2 min . if σ min ≥ σ maj then a ≥ 0and the minority CDF function will dominate the majority CDF function in the left-side tail and by choosing a negative number with big enough absolute value for alpha and tail L (α) will be positive.Step 3 Solving equation 18 for special case σ min < σ maj In case of σ min ≤ σ maj , having ∆ > 0 is a necessary condition, also derivative of tail L (α) is only positive in( -b- √ ∆ 2a , -b+ √ ∆ 2a ) so the maximum of tail L is either in -∞ or in -b+ √ ∆ 2a . Having tail L ( -b+ √ ∆ 2a) > 0 next to ∆ > 0 condition, would be the necessary and also sufficient in this case. ∆ = (µ min -µ maj ) 2 σ 2 min σ 2 maj -4 log σ maj σ min + log ϵ 1 -ϵ 1 2σ 2 maj - 1 2σ 2 min ≥ 0 ⇐⇒ (µ min -µ maj ) 2 ≥ 2 log 1 -ϵ ϵ -log σ maj σ min σ 2 maj -σ 2 min ⇐⇒ ϵ ≥ sigmoid - (µ maj -µ min 2 2(σ 2 maj -σ 2 min ) -log σ maj σ minNext, we investigate properties of the conditions of the proposition C.1 in case of σ maj < σ min .Schematic interpretation of these conditions is presented in figure 6 .• As equation 5 indicates, the minority group is not allowed to be too underrepresented. This especially has a direct relation with the difference of means. The more mean values of groups are different, the more imbalance can be mitigated through loss-based sampling.Mean value difference is especially affected by the spurious correlation, it escalates as the model relies on spurious correlation and also when the spurious features between groups are too different.• On the other hand condition 4 is more complex and doesn't have a simple closed form, we analytically describe its behaviors by fixating the means and calculating the valid values for ε. As the results show in figure 6 , most of ε are feasible in for σ min < ∆µ as we can see the possible region declines with an increase of σ min and valid ε values cease to exist. The complete results on Waterbirds, CelebA, and UrbanCars, in addition to complete results on CivilComments and MultiNLI are reported in Tables 4 and 5 respectively. The results for all methods except Group DRO + EIIL on all datasets except UrbanCars are reported by Qiu et al. [9] . The results for Group DRO + EIIL are taken from Zhang et al. [24] . Also, the results of our method and DFR are shown in Table 6 D.2 Dominoes-Colored-MNIST-FashionMNIST Dominoes-Colored-MNIST-FashionMNIST (Dominoes-CMF) is a synthetic dataset. We adopt a similar approach to previous works [37, 38, 1] using a modified version of the Dominoes binary classification dataset. This dataset consists of images with the top half showing CIFAR-10 images [19] , divided into two meaningful classes: vehicles (airplane, car, ship, truck) and animals (cat, dog, horse, deer). The bottom half displays either MNIST [20] images from classes {0 -3} or Fashion-MNIST [21] images from classes {T-shirt, Dress, Coat, Shirt}. The complex feature (top half) serves as the core feature and the simple feature (bottom half) is linearly separable and correlated with the class label at 75%. Furthermore, inspired by the approaches in Zhang et al. [24] , Arjovsky et al. [18] , we intentionally introduce an additional spurious attribute by artificially coloring a subset of images in the following manner: 90% of the bottom half images in class c 1 are randomly assigned a red color, while 10% are assigned a green color, and vice versa for class c 2 . See Table 7 for more details about the dataset statistics. Waterbirds [7] The dataset comprises images of diverse bird species, classified into two categories:waterbirds and landbirds. Each image features a bird set against a backdrop of either water or land.Interestingly, the background scene acts as a spurious feature in this classification task. Waterbirds are primarily shown against water backgrounds, and landbirds against land backgrounds. Consequently, waterbirds on water and landbirds on land form the minority groups in the training data. It's important to note that the validation dataset for waterbirds is group-balanced, meaning birds from each class are equally represented against both water and land backgrounds. This dataset is mainly categorized as a spurious correlation dataset.CelebA [13] are women, while men with blond hair represent a minority group. In addition to spurious correlation in the class of blond hair, this dataset also exhibits class imbalance.MultiNLI [15] dataset involves a text classification task focused on determining the relationship between pairs of sentences: contradiction, entailment, or neutral. Sentences containing negation words such as "no" or "never" are under-represented in all three classes, inducing attribute imbalance in the dataset. Figure 7 illustrates the distinct behavior of this dataset compared to other datasets that contain spurious attributes.CivilComments [16] dataset, as part of the WILDS benchmark, involves a text classification task focused on labeling online comments as either "toxic" or "not toxic". Each comment is associated with 8 attributes, including gender (male, female), sexual orientation (LGBTQ), race (black, white), and religion (Christian, Muslim, or other), based on whether these characteristics are mentioned in the comment. While there is a small attribute imbalance in the dataset, it can categorized into datasets with class imbalance. In this paper, we use the implementation of the dataset by the WILDS package [39] .UrbanCars [14] is an image classification dataset with multiple shortcuts. Each image in the dataset consists of a car in the center of the image on a natural scene background, with another object to the right of the image. Images are labeled Urban or City according to the type of car present in the center. However, each of the backgrounds and the additional objects is highly correlated with the label. While the test set consists of 8 environments based on combinations of the core and two spurious patterns, the training and validation set consist of four groups, based on combinations of the label and only one of the shortcuts.ERM For Waterbirds and CelebA, we utilize the ResNet50 checkpoints available in the GitHub repository of Kirichenko et al. [1] as our base model. We use the ResNet-50 architecture provided by the torchvision package.In the case of Civil-Comments and MultiNLI, we adopt a similar approach to Kirichenko et al. [1] , usingBertForSequenceClassification.from_pretrained('bert-base-uncased', ...) from the transformers package. The model is trained using the AdamW optimizer with a learning rate of 10 -5 , weight decay of 10 -4 , and a batch size of 16 for a total of 5 epochs.For the UrbanCars dataset, we adhere to the settings described in Li et al. [14] , which involves training a ResNet-50 model pretrained on ImageNet using the SGD optimizer with a learning rate of 10 -3 , momentum of 0.9, weight decay of 10 -4 , and a batch size of 128 for 300 epochs. For the Dominoes-CMF dataset, we train a ResNet18 model pretrained on ImageNet for 20 epochs with a batch size of 128 and an SGD optimizer with a learning rate of 10 -3 , momentum of 0.9, and weight decay of 10 -4 .EVaLS and EVaLS-GL For every dataset, EIIL was utilized with a learning rate of 0.01, a total of 20000 steps, and a batch size of 128. The last layer of the model was trained on all datasets using the Adam optimizer. A batch size of 32 and a weight decay of 10 -4 were used for all datasets. Our method was evaluated on the validation sets of each dataset, considering both fine-tuning and retraining of the last layer. For all datasets, with the exception of MultiNLI, retraining provided superior validation results. The specifics regarding the number of epochs and the ranges for hyperparameter search (including learning rate, l 1 -regularization coefficient (λ), and the number of selected samples (k)) for each dataset are as follows:• Waterbirds.epochs = 100, lr = 5 × 10 We conducted an ablation study to investigate the impact of using environments inferred from EIIL on model selection. Specifically, we benchmarked the performance of DFR and AFR with EIIL-inferred groups. The results, presented in Table 9 , demonstrate the effectiveness of incorporating EIIL-inferred groups in model selection. The results show that while EIIL-inferred groups reduce the performance compared to ground-truth annotations for model selection, they still can be effective for robustness to an extent. Moreover, EVaLS outperforms these two methodw when using EIIL inferred environments. shown in Table 10 . One limitation of error splitting is that in datasets with noisy labels or corrupted images, samples that an ERM model misclassifies may not always belong to minority groups. In these situations, choosing models based on their accuracy on corrupted data could lead to the selection of models that are not robust to spurious correlations. This is demonstrated by the results of EVaLS-ES on the CelebA dataset.This shortcoming of error splitting can be alleviated by employing a random classifier instead of the ERM-trained one. Due to the feature-level similarity between minority and majority samples in datasets affected by spurious correlation [23, 1, 29] , it is expected that the classifier can differentiate between the groups to some extent. As shown in Table 10 , surprisingly, EVaLS-RC produces results that are generally comparable to EVaLS. However, the performance of this method may have high variance, depending on the different initializations of the classifier. Real-world datasets often encapsulate social biases that stem from entrenched stereotypes and historical discrimination, affecting various groups such as genders and races. Machine learning methods, which learn the correlation between patterns in input data and their targets (e.g., labels in a classification task) [40] , inadvertently absorb this bias. This unintended consequence leads to fairness issues in many applications. While strategies to mitigate such biases have been proposed(as discussed comprehensively in Section A), societal biases are not always known and determined.We believe that our work, as it addresses these unidentified biases, takes a significant step towards making machine learning fairer for our society.Each experiment was conducted on one of the following GPUs: NVIDIA A100 with 80G memory, NVIDIA Titan RTX with 24G memory, Nvidia GeForce RTX 3090 with 24G memory, and NVIDIA GeForce RTX 3080 Ti with 12G memory.NeurIPS Paper Checklist Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The limitations are discussed in the discussion section.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
environment-based validation and loss-based sampling: 
 Our method, EVaLS, is designed to improve the robustness of deep learning models to group shifts without the need for group annotation. In line with the DFR [1] approach, we utilize a classifier defined as f = h ϕ • g θ , where g θ represents a deep neural network serving as a feature extractor, and h ϕ denotes a linear classifier. The classifier is initially trained with the ERM objective on the training dataset D tr . Subsequently, we freeze the feature extractor g θ and focus solely on retraining the last linear layer h ϕ using the validation dataset D val as a held-out dataset.We randomly divide the validation set D val into two subsets, D LL and D M S which are used for last layer training and model selection, respectively. In Section 3.1 we explain how to sample a subset of D LL that statistically handles the group shifts inherent in the dataset. In Section 3.2 we describe how D M S is divided into different environments that are later used for model selection. The optimal number of selected samples from D LL and other hyperparameters is determined based on the worst environment accuracies among environments that are obtained from D M S . By combining our novel sampling and validation strategy, we aim to provide a robust linear classifier h ϕ * that significantly improves the accuracy of underrepresented groups without requiring group annotations of training or validation sets. Figure 1 illustrates the comprehensive workflow of the EVaLS methodology.Finally in Section 3.3, we provide theoretical support for the loss-based sampling procedure and its effectiveness.Following previous works [5, 10, 9] , we use the loss value as an indicator for identifying minority groups. We first evaluate classifier f on samples within D LL and choose k samples with the highest and lowest loss values for a given k. By combining these 2k samples from each class, we construct a balanced set D balanced , consisting of high-loss and low-loss samples (see Figure 1 (c)). D balanced is then used for the training of the last layer of the model.As depicted in Figure 2 , the proportion of minority samples among various percentiles of samples with the highest loss values increases as we select a smaller subset of samples with the highest loss.This suggests that high and low-loss samples could serve as effective representatives of minority and majority groups, respectively. In Section 3.3, we offer theoretical insights explaining why this approach could lead to the creation of group-balanced data.Contrary to common assumptions and practices in the field, precise group labels for the validation set are not essential for training models robust to spurious correlations. Our empirical findings, detailed in Section 4, reveal that partitioning the validation set into environments that exhibit significant subpopulation shifts can be used for model selection. Under these conditions, the worst environment accuracy (WEA) emerges as a viable metric for selecting the most effective model and hyperparameters.The concept of an environment, as frequently discussed in the invariant learning literature, denotes partitions of data that exhibit different distributions. A model that consistently excels across these varied environments, achieving impressive worst environment accuracy (WEA), is likely to perform equally well across different groups in the test set. Several methods for inferring environments with notable distribution shifts have been introduced [12, 22] . Environment Inference for Invariant Learning (EIIL) [12] , leverages the predictions from an earlier trained ERM model to divide the data into two distinct environments that significantly deviate from the invariant learning principle proposed by Arjovsky et al. [18] , thus creating environments with distribution shifts. Initially, EIIL is employed to split D M S into two environments. Subsequently, each environment is further divided based on sample labels, resulting in 2 × |Y| environments. To measure the difference between the distribution of environments, we define group shift of a class as the absolute difference in the proportion of a minority group between two environments of that class. A higher group shift suggests a more distinct separation between environments. As detailed in the Appendix, environments inferred by EIIL demonstrate an average group shift of 28.7% over datasets with spurious correlation. Further information about EIIL and the group shift quantities for each dataset can be found in the Appendix.We demonstrate that even more straightforward techniques, such as applying a random linear layer over the feature embedding space and distinguishing environments based on correctly and incorrectly classified samples of each class, can be effective to an extent in several cases (See Appendix E.2).It underscores that the feature space of a trained model is a valuable resource of information for identifying groups affected by spurious correlations. This supports the logic of previous research that employs clustering [23] or contrastive methods [24] in this space to differentiate between groups.In this subsection, we provide theoretical insights into why loss-based sampling in a class can be utilized to create a balanced dataset of each group under sufficient conditions. We will show the close relationship between the existence of a balanced dataset and the difference between the minority vs.majority group means, calculated based on the logits of an ERM-trained classifier. Such logits are known to depend on spurious features. Hence the mentioned group mean difference is expected to be high if spurious features are present in the dataset.Consider a binary classification problem with a cross-entropy loss function. Let logits be denoted asL.Because loss is a monotonic function of logits, the tails of the distribution of loss across samples are equivalent to that of the logits in each class.We assume that in feature space (output of g θ ) samples from the minority and majority of a class are derived from Gaussian distributions. So, we can consider N (µ min , σ 2 min ) and N (µ maj , σ 2 maj ) as the distribution of minority and majority samples in logits space. We provide an outline for proof of Proposition 3.1 here and leave the complete and formal proof and also exact bounds to Appendix C. We also analyze the conditions and effects of spurious correlation in satisfying these conditions. To proceed with the outline we first define a key concept to outline our proof.Definition 3.1 (Proportional Density Difference). For any interval I = (a, b] and a mixture distribution εP 1 (x) + (1 -ε)P 2 (x), the proportional density difference is defined as the difference of accumulation of two component distributions in the interval I and is denoted by ∆ ε P mixture (I).∆ ε P mixture (I)∆ = εP 1 x ∈ I -(1 -ε)P 2 x ∈ IProof outline Our proof proceeds with three steps. First, we reformulate the theorem as an equality of left-and right-tail proportional distribution differences. In other words, we show that the more mass the minority distribution has on one tail, the more mass the majority distribution must have on the other tail. Afterward, supposing µ min < µ maj WOLG, we propose a proper range for β values on the right tail. We show that when σ maj ≤ σ min , values for α trivially exist that can overcome the imbalance between the two distributions. In the last step, for the case in which the variance of the majority is higher than the minority, we discuss a necessary and sufficient condition for the existence of α and β based on the left-tail proportional density difference using the properties of its derivative with respect to α.Condition 1 suggests that for a given degree of spurious correlation ϵ and variations σ maj , σ min , an essential prerequisite for the efficacy of loss-based sampling is a sufficiently large disparity between the mean distributions of minority and majority samples, denoted by ∥µ maj -µ min ∥ 2 . This indicates that the groups should be distinctly separable in the logits space.Although the parameters α and β are theoretically established under certain conditions, their actual values are undetermined. Therefore, validation data is necessary to ascertain them. For practicality and simplicity in this study, we consider that α = β and explore its corresponding sample number (the count of high-and low-loss samples) from a predefined set of possibilities. By leveraging the worst environment accuracy, as elaborated in Section 3.2, we identify the optimal candidate that ensures uniform accuracy across all environments.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes]Justification: Codes and information of datasets that are constructed or reused in the paper are anonymized and included in the main paper and supplementary material.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiments: 
 In this section, we evaluate the effectiveness of our proposed method through comprehensive experiments on multiple datasets and compare it with various methods and baselines. We begin by briefly describing evaluation datasets and then introduce baselines and comparative methods. Finally, we report and fully explain the results.Datasets Our method, along with other baselines, is evaluated on Waterbirds [7] , CelebA [13] ,UrbanCars [14] , CivilComments [16] , and MultiNLI [15] . As per the study by Yang et al. [4] ,Waterbirds, CelebA, and UrbanCars among these datasets exhibit spurious correlation. Among the rest, CivilComments has class and attribute imbalance, whereas MultiNLI exhibits attribute imbalance.For additional details on the datasets, please refer to the Appendix.Baselines We compare our method with four baselines in addition to standard ERM. GroupDRO [7] trains a model on the data with the objective of minimizing its average loss on the minority samples.This method requires group labels of both the training and validation sets. DFR [1] Setup Similar to all the works mentioned in Section 4, we use ResNet-50 [25] pretrained on ImageNet [26] for image classification tasks. We used random crop and random horizontal flip as data augmentation, similar to [1] . For a fair comparison with the baselines, we did not employ any data augmentation techniques in the process of retraining the last layer of the model. For the CivilComments and MultiNLI, we use pretrained BERT [27] and crop sentences to 220 tokens length.In EvaLS, we use the implementation of EIIL by spuco package [28] for environments inference on the model selection set with 20000 steps, SGD optimizer, and learning rate 10 -2 for all datasets.Model selection and hyper-parameter fine-tuning are done according to the worst environment(or group if annotations are assumed to be available) accuracy on the validation set. For each dataset, we assess the performance of our model in two cases: fine-tuning the ERM classifier or retraining it.For all datasets except MultiNLI, retraining yielded better validation results. We report the results of our experiments in two settings: (i) EVaLS, which incorporates loss-based instance sampling for training the last layer, and environment inference for model selection. (ii) EVaLS-GL, similar to EVaLS except in using ground-truth group labels for model selection. For more details on the ERM training and last layer re-training hyperparameters refer to the Appendix.The results of our experiments along with the reported results on GroupDRO [7] , DFR [1] , JTT [5] ,and AFR [9] on five datasets are shown in Table 1 . The reported results for GroupDRO, DFR, JTT, and AFR except those for the UrbanCars are taken from Qiu et al. [9] . For EIIL+Group DRO, the results are reported from Zhang et al. [24] . We report only the worst group accuracy of methods in Table 1 . The average group accuracies are documented in the Appendix. The Group Info column shows whether group annotation is required for training or model selection entry for each method.When compared to other methods with the same level of supervision, EVaLS-GL outperforms on four of the five datasets, achieving near-optimal worst group accuracy on Waterbirds, demonstrating the effectiveness of loss-based sample selection compared to the weighting scheme in AFR [9] . Given that AFR employs exponential weights with a temperature parameter to assign a positive weight to all samples, proportional to the model's assigned probability of the correct class, an increase in the number of low-loss samples will lead to a corresponding rise in their cumulative weight.Consequently, in situations where spurious correlation is high and an uptick in majority samples leads to a greater proportion of low-loss over high-loss samples, determining an appropriate parameter becomes challenging.The comparison between EVaLS and Group DRO + EIIL indicates that when environments are available instead of groups, our method, which uses environments solely for model selection and utilizes loss-based sampling, is more effective than GroupDRO, a potent invariant learning method, which uses this annotation for training.Regarding the UrbanCars, which contains an un-annotated spurious attribute, Li et al. [14] has shown that shortcut mitigation methods often struggle to address multiple shortcuts simultaneously. Notably, techniques such as DFR [1] which are designed to reduce reliance on a specific shortcut feature, cannot make the model robust to an unknown shortcut. In contrast, our experiments suggest that loss-based methods can mitigate the impact of both labeled and unlabeled shortcut features more effectively. Also, in the case of CivilComments, which is viewed as a benchmark for class imbalance, EVaLS-GL exceeds all prior methods, even those with complete group annotation, thanks to the class balancing for the training of the last layer.Our evaluation of EVaLS is based on the spurious correlation benchmarks. This is because, in other instances of subpopulation shift, the attributes that differ across groups are not predictive of the label, thereby reducing the visibility of these attributes' effects in the model's final layers [29] .Consequently, EIIL, which depends on output logits for prediction, might not effectively separate the groups. This observation is further supported by our findings related to the degree of group shift between the environments inferred by EIIL for each class in the CivilComments and MultiNLI datasets. The average group shift (defined in the Section 3.2) in the environments of the minority class of CivilComments is only 5.6 ±0.8 %. Also, environments associated with Classes 1 and 2 in MultiNLI show only 1.1 ±0.3 % and 1.9 ±1.0 % group shift respectively. More results and ablation studies can be found in the Appendix.To evaluate the performance of our method in the case of unknown spurious correlations, we train a ResNet-18 [25] model on the Dominoes-CMF dataset.We apply DFR [1] , EVaLS-GL, and EVaLS on top of the trained ERMs to assess their ability to mitigate multiple shortcuts. For the last layer training set, we consider the MNIST/Fashion-MNIST feature as the known group label, and the color as the unknown attribute. The results are shown in Table 2 . To clarify, we calculate the worst-group accuracy on the validation set considering only the label of one shortcut, i.e., the lowest accuracy among the four groups based on the combination of the target label and the single known shortcut label. Note that EVaLS does not require group annotations.Our results confirm findings by Li et al. [14] , suggesting that methods using group labels mitigate reliance on the known shortcut but not necessarily on the unknown one. EVaLS-GL mitigates this phenomenon using its loss-based sampling approach, but surprisingly EVaLS even outperforms EVaLS-GL. Combining a loss-based sampling approach for last layer training and environment-based model selection, results in a completely group-annotation-free method in a multi-shortcut setting and successfully re-weights features to perform well with respect to both spurious attributes. Robustness to spurious correlation is a critical concern across various machine learning subfields.It is a form of out-of-distribution generalization [31] where the distribution shift arises from the disproportionate representation of minority groups-those instances that are devoid of the correlated spurious patterns associated with their labels [4] . The issue of spurious correlation also intersects with the discourse on fairness in machine learning. [32, 33] .Past studies have proposed a range of strategies to mitigate the models' reliance on spurious correlation. Broadly speaking, these methods can be categorized according to the degree of supervision they require regarding group labels.Invariant learning (IL) methods [18, 34, 35] operate under the assumption of having access to a collection of environments that comprise group shift. By imposing invariant conditions on these environments, IL methods strive to create classifiers robust against group-sensitive features. IRM [18] is designed to learn a feature extractor, which, when utilized, guarantees the existence of a classifier that would be optimal in all training environments. VREx [34] aims to decrease the risk variance among different training environments. PGI [36] works by minimizing the distance between the expected softmax distribution of labels, conditioned on inputs across both majority and minority environments.Lastly, Fishr [35] focuses on bringing the variance of risk gradients closer together across different training environments. For scenarios which environments are not available, environment inference methods [12, 22] are used to obtain a set of environments. Creager et al. [12] introduce environment inference for invariant learning (EIIL), which tries to partition samples into two groups such that the objective of IRM [18] is maximized. HRM [22] aims to optimize both an environment inference module and an invariant prediction module jointly, with the goal of achieving an invariant predictor.When group annotations are accessible, various methods leverage this information to equalize the impact of different groups on the model's loss. The Group Distributionally Robust Optimization (GDRO) approach [7] , for instance, focuses on optimizing the loss for the worst-performing group during training. Kirichenko et al. [1] has shown that models can still learn and extract core data features even in the presence high spurious correlation. Consequently, They suggest that retraining just the last layer of a model initially trained with Empirical Risk Minimization (ERM) can effectively reduce reliance on spurious correlation for predicting class labels. This method, termed Deep Feature Re-weighting (DFR), has been validated as not only highly effective but also significantly more efficient than earlier techniques that necessitated retraining the full model [8, 7] . However, availability of group annotations is considered a serious restrictive assumption.Several recent studies have endeavored to enhance model robustness against spurious correlation, even in the absence of group annotations [5, 24, 9, 2, 6] . Liu et al. [5] introduce a two-stage method that involves training a model using ERM for a number of epochs before retraining it to give more weight to misclassified samples. The study by Zhang et al. [24] employs the same two-stage training process, but with a twist for the second stage: they utilize contrastive methods. The goal is to bring samples from the same class but with divergent predictions closer in the feature space, while simultaneously increasing the separation between samples from different classes that have similar predictions. Another method, known as automatic feature reweighting (AFR) [9] , reweights the last layer of an ERM-pretrained model to favor samples that the original model was less accurate on.LaBonte et al. [2] refine the last layer of an ERM-trained model through class-balanced finetuning, identifying challenging data points by comparing the classifier's predictions with those of an earlystopped version. While these methods have significantly reduced the reliance on group annotations, some are still required for validation and model selection. This remains a constraint, particularly when the spurious correlation is completely unknown.For making a trained model robust to spurious correlation with zero group annotations, recently, LaBonte et al. [2] have empirically demonstrated that the class-balanced retraining of a model pretrained with ERM can effectively improve the WGA for certain datasets. However, this approach fails in datasets with a high degree of spurious correlation.Table 3 : The average and variation percentage (%)(across 3 seeds) of group shift between the inferred environments using EIIL [12] for each class, which is the absolute difference between the proportion of a minority group in the two environments of a class. Higher group shift indicates better separation of environments. In most cases, a significant group shift is observed between the inferred environments. Consider the training dataset D tr = {(x (i) , y (i) )|x (i) ∈ X , y (i) ∈ Y}, where X and Y represent the input and output spaces, respectively. This dataset can be partitioned into different environmentsE tr = {e 1 , ..., e n }, such that for any i ̸ = j, the data distribution in e i and e j differs. The objective of invariant learning is to train a predictor that performs consistently across all environments in E tr .Under certain conditions, this predictor is also expected to perform well on e tst , a test environment with a distribution distinct from the training data. Invariant Risk Minimization (IRM) [18] approaches this problem by learning a feature extractor Φ(.) such that a classifier ω(.) exists, where ω • Φ(.)performs consistently across all training environments. The practical implementation of the IRM objective is to minimizeEQUATIONwhere ω is a constant scalar with a value of 1.0, λ is a hyperparameter, and R e (f ) = E (x,y)∼pe [l(f (x), y)] is referred to as the risk on environment e.In real-world scenarios, training environments might not always be available. To address this, Environment Inference for Invariant Learning (EIIL) [12] partitions samples into two environments in a way that maximizes the objective in Eq 2.During the training phase, the EIIL algorithm replaces the hard assignment of environments to samples with a soft assignment q i (e) = p(e|(x (i) , y (i) ), where q i is learnable. Consequently, the relaxed version of the risk function is defined as Re (Φ) = 1N N i q i (e)[l(Φ(x (i) ), y (i) )]. Given a model Φ that has been trained with ERM on the dataset, EIIL optimizesEQUATIONAs discussed in Creager et al. [12] , using a biased base model Φ could lead to environments exhibiting varying degrees of spurious correlation. During the inference phase, the soft assignment is converted to a hard assignment. The average group shift between the inferred environments using EIIL is illustrated in Table 3 .In this section, we establish a more formal description of loss-based sampling for balanced dataset creation and then prove it. We thoroughly analyze the close relationship between the availability of the balanced dataset and the gap between spurious features of minority and majority groups.Consider a binary classification problem with a cross-entropy loss function. Let logits be denoted as L. Because loss is a monotonic function of logits, the tails of the distribution of loss across samples are equivalent to that of the logits in each class. We assume that in feature space (output of g θ ) samples from the minority and majority of a class are derived from Gaussian distributions N (h min , t 2 min I d ) and N (h maj , t 2 maj I d ), respectively. Before diving into the group balance problem we initially show that the distribution of minority and majority samples in the logit space (output of h ϕ ) are Gaussian too.Lemma C.1 (Gaussain Distribution of Logits). if Z ∼ N (h, t 2 I d ) in feature space and W ∈ R d then logits L = ⟨W, Z⟩ ∼ N W h, t 2 ∥W ∥ 2 Proof. Let Z ∼ N (h, t 2 I d ).Consider the linear combination L = ⟨W, Z⟩ = W T Z, where W ∈ R d which is a univariate gaussian.To find the distribution of L, we need to determine its mean and variance.1.Mean of L E[L] = E[⟨W, Z⟩] = E[W T Z] = W T E[Z] = W T h = ⟨W, h⟩.Therefore, the mean of L is W h. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: The training details, hyperparameters, model selection criteria, etc. have are written in the Appendix and the data and metadata have been provided in our code.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: All tables report standard deviation and how it was computed and the plot contains error bar (also by standard deviation).Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [No] Justification: The paper does provide details about the hardware used for the experiments.However, since experiments were done on different hardwares, the computational resources needed for each individual experiment are not documented.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Justification: All codes and rules have been thoroughly reviewed and checked, with no instances of non-compliance found.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: In the Social Impacts section we discuss that our work can significantly contribute to fairness in machine learning. We did not find any negative social impacts of our work.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.• According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. Justification: The paper does not involve crowdsourcing nor researh with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.Note that in the CelebA dataset, only the "blond hair" class includes a minority group. 
: 
 • EVaLS attains near-optimal worst group accuracies or even exceeds them in spurious correlation benchmarks, all with zero group annotations.• When group annotations are available for model selection, EVaLS delivers state-of-the-art performance across a variety of subpopulation shift benchmarks.• We introduce a new dataset consisting of two spurious features in which partial supervision may negatively impact the performance of the underrepresented groups. 
 
 Appendix:  


Decision: While the premise and approach has novelties, the overall contribution of this paper is insufficient for publication in NeurIPS. One major concern is that theory is not informative of the method: while the method proposed is a reasonably complex pipeline that combines several (existing) ideas, the theory is about balancing tails of two scalar gaussians. A theory with meaningful guarantees for the authors' proposed method would improve the paper substantially.

Reviews: review: paper_summary: paper_summary: The paper studies how to improve the model’s robustness to multiple spurious correlations when the group labels (indicator for spurious correlation) are unknown in general. The proposed approach, EVaLS, leverages the loss from a base ERM model to sample a balanced subset to prevent learning from spurious correlations. In addition, a new synthetic dataset (Dominoes-CMF) for multiple spurious attributes is crafted. Empirically, the proposed approach sometimes has advantages over the rest of the baselines when using the same amount of additional information (group label).
strength_weakness: strength_weakness: 1. Figure 2 attempts to illustrate more minority samples have high loss while the majority samples have low loss. However, in each of the plots, only the % of one of the minority or majority groups is shown. The illustration can be improved by showing the % of both majority and minority groups in the same plot, and showing the actual distribution of the loss for the groups. 
2. Though the idea is straightforward, it is unclear how the loss-based instance sampling is actually implemented. It is helpful to provide an algorithm or pseudocode to improve the presentation. 
3. The theoretical analysis is generally sound but limited to a case without discussing the use of the loss (which may not be Gaussian) and the spurious correlations (which involve at least two dimensions of core and spurious features [1]). 
4. The experimental results are less polished and sometimes the advantages are not so clear over other baselines. Some results are missing for datasets such as UrbanCars and MultiNLI. Only a few baselines are compared for the new dataset in Table 2. There is also no convincing and fine-grained analysis (e.g., ablation study) to understand how the proposed approach ensures data balancing and improves group robustness. 
5. The paper initially focuses on improving group robustness when multiple spurious correlations are present, but the experimental results are lacking for these more challenging datasets. 

[1] Sagawa, Shiori, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. "An investigation of why overparameterization exacerbates spurious correlations." In *International Conference on Machine Learning*, pp. 8346-8356. PMLR, 2020.
questions: questions: 1. Since it’s an essential prerequisite for loss-based sampling, how well are the majority/minority groups separated in the logit space?
2. How is the number of balancing samples $k$ chosen? How balanced are the samples when they are selected from the optimal $k$? 
3. I am curious about why the result for Civilcomment dataset is so much higher than the other baselines. Is the evaluation consistent with the baseline methods?
4. Minor typo: line 233 should be without loss of generality (w.l.o.g.). 
5. Other minor issues with the experiments: The best-performing results in each category of approaches should be bold. The column header “best” should be “average” in Tables 4 and 5.
limitations: limitations: The authors have discussed the limitations in Section 5.
paper_summary: paper_summary: To address the issue of spurious correlations when group labels are unavailable, this paper proposes a new method called EVaLS. It first creates a balanced training dataset using loss-based sampling. Then, it evaluates the accuracy of the balanced training set based on the inferred environments from the validation set, and selects models accordingly.
strength_weakness: strength_weakness: 1. Why is the approach of using high-loss points (considered as the minority group) more effective than directly using misclassified points (considered as the minority group) in methods like JTT? Intuitively, compared to misclassified points, high-loss points are more "implicit" and no obvious thresholds, which could potentially result in high-loss points actually belonging to the majority group, thus exacerbating the imbalance in resampling.

2. If the author can show the balance level of samples obtained through loss-based sampling compared to directly using labels (misclassified points), it could further illustrate the advantages of loss-based sampling.

3. In Section "Mitigating Multiple Shortcut Attributes", if color is treated as a known spurious attribute and shape as an unknown spurious attribute, how would the performance of EVaLS be affected? Based on my understanding, there is a possibility that simplicity bias could cause the model to prioritize learning the simpler feature, color, and struggle to learn the more complex shape attribute. Therefore, considering color as known and shape as unknown can better show the performance of EVaLS in handling complex spurious features.
questions: questions: See weaknesses.
limitations: limitations: See weaknesses.
paper_summary: paper_summary: This paper addresses the problem of subpopulation generalization, also known as spurious correlations. Building on the Last Layer Retraining (DFR) method, it removes the constraints on a small subset of annotations. The paper introduces the Environment-based Validation and Loss-based Sampling (EVaLS) method. Unlike DFR, EVaLS divides the validation set $D^{val}$ into two parts: (1) $D^{LL}$,  where losses from an ERM-trained model are used as a proxy for identifying minority groups for retraining, and (2) $D^{MS}$, where environment inference methods are used for partitioning environments. The paper presents theoretical insights and empirical results demonstrating the effectiveness of EVaLS.
strength_weakness: strength_weakness: * The novelty and contribution of the proposed method may be limited for the following reasons: 1) The paper combines multiple previously proposed methods (*i.e.* DFR [1], EIIL [2]) all at once, which inherently guarantees a nontrivial performance; (2) The primary technical contribution, at least from my perspective, is the loss-based sampling, which has been already explored extensively in the noisy label literature and has been used as tools for pseudo-labeling. 
* The paper fails to discuss recently proposed methods that also require no group annotations, such as SELF [3], BAM [4], and BPA [5]. In particular, SELF is also a direct follow-up of DFR. The authors are encouraged to discuss the limitations and strengths of loss-based schemes against the class-based schemes advocated by SELF and BAM.
* More analyses can be included to provide further understanding of the selected loss-based samples. For example, given a threshold, how much percent of the high-loss and low-loss samples are indeed the minority and majority samples and how does this percentage change with the threshold?

[1] Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations, ICLR 2023

[2] Environment inference for invariant learning. ICML 2021

[3] Towards Last-layer Retraining for Group Robustness with Fewer Annotations. NeurIPS 2023 

[4] Bias Amplification Enhances Minority Performance. TMLR 2024

[5] Unsupervised learning of debiased representations with pseudo-attributes. CVPR 2022
questions: questions: * Is EVaLS sensitive to hyperparameters? 
* [minor] There seem to be typos in your corollary C.1.
* Can the authors make clarifications on how the conditions in proposition 3.1 are met or relaxed in practice? How does the distribution of actual experimental benchmark datasets compare to your assumptions?
limitations: limitations: Aforementioned in Weaknesses and Questions.

Comments: title: Reviewer Response
Evaluation on CivilComments Dataset
Discussion Summary
Clarification Regarding the Algorithm
Reviewer Response
Response to Reviewer RE7s
Reviewer Response
Response to Reviewer 7E4z
Thanks for your response
comment: Dear Reviewer RE7s,

Thank you for your constructive review. We’re pleased that our rebuttal successfully addressed your concerns. We ensure that the final version addresses these points to avoid any potential confusion or unfairness.

Best regards,

Authors of Submission 19478
Thank you for promptly checking and running the results for CivilComments in the correct setting. Please make sure to present a clear and fair comparison with detailed experimental setup in the revised version to avoid potential confusion and unfairness. With no other major issues left, I have increased my score accordingly.
Dear Reviewer RE7s,

We deeply appreciate your notice. Our setting follows the practice in the GitHub repository `izmailovpavel/spurious_feature_learning` in `datasets.py`, which is referenced in the official implementation of DFR as an extension of their code functionality. Both (line 266 in their code and line 129 in `data/civilcomments.py` in our code) collapse grouping as “Has Identities”/”No Identities” in each class (4 groups in total). We realize that all the methods that report results around ~79-80% worst group accuracy on CivilComments (like the class-balanced schemes suggested by Reviewer 7E4z and previously reported results for EVaLS-GL) use the 4-group setting of CivilComments, while the methods (as you mentioned) that report on the 16-group setting, result in around ~69-70% worst group accuracies.

We carefully considered your notice and reran the experiment of EVaLS-GL on the CivilComments dataset with 16 groups. **EVaLS-GL on CivilComments with 16 groups has worst group accuracy and average accuracy (in parentheses) of $\boldsymbol{68.0_{\pm0.5}(89.2_{\pm0.3})}$%**. Evaluating ERM on worst accuracy for 16 groups also resulted in $56.3_{\pm4.8}$ (average accuracy does not differ with the 4 group setting).

We should restate that, as mentioned by previous works (reference [4]-Table 2) and as seen in the following table, CivilComments is not an instance of a dataset with spurious correlation (L313). In other words, no identity attribute in CivilComments is predictive for any of the labels (see L316-318). However, loss-based sampling with ground-truth group annotations (EVaLS-GL) still significantly improves worst group accuracy compared to initial ERM.

Proportion of attributes in each class for CivilComments dataset:
|Toxicity (Class)|Male|Female|LGBTQ|Christian|Muslim|Other Religions|Black|White|
|-|-|-|-|-|-|-|-|-|
|0|0.11|0.12|0.03|0.10|0.05|0.02|0.03|0.05|
|1|0.14|0.15|0.08|0.08|0.10|0.03|0.1|0.14|

### Conclusion
EVaLS-GL results in $68.0_{\pm0.5}$% worst group accuracy when CivilComments is examined with 16 groups (identity attributes in the table above in each class), and $80.5_{\pm0.4}$% when it is examined with 4 groups (“Has Identities”/”No Identities” attributes in each class).

We acknowledge the discrepancy in the reported results of EVaLS-GL on the CivilComments dataset and appreciate your feedback. We hope the above explanations provide clarity for a thorough comparison. We will incorporate this discussion regarding the EVaLS-GL results on CivilComments into the final version of the paper.

Finally, we again greatly appreciate your attention to this matter and your notice regarding this.

---

Below, we present the updated main table (Table 1) of the paper, including the results you requested to be added to it in this rebuttal thread for a more comprehensive comparison. If a method’s worst group accuracy is better than others with a similar or weaker level of assumption about the availability of group annotations during training/validation phases, it is underlined. Worst group accuracies are bolded if a method outperforms all other methods. SC stands for spurious correlation datasets. Results for CivilComments are calculated for the 16-group setting (∗ denotes results for the 4-group setting). Other notations are similar to the main table. We would like to remind you that environment-based validation only works for spurious correlation subpopulation shift (L316-324).

|Method|Group Info (Train/Val)|Waterbirds (SC)|CelebA (SC)|UrbanCars (SC)|CivilComments|MultiNLI|
|-|-|-|-|-|-|-|
|GDRO|$\checkmark/\checkmark$|$91.4$|$\underline{\boldsymbol{88.9}}$|$73.1_{\pm 2.0}$|$69.9$|$\underline{\boldsymbol{77.7}}$|
|DFR|$\text{x}/\checkmark\checkmark$|$92.9_{\pm0.2}$|$88.3_{\pm1.1}$|$79.6_{\pm2.2}$|$\underline{\boldsymbol{70.1_{\pm0.8}}}$|$74.7_{\pm0.7}$|
—-------------------------------
|GDRO+EIIL|$\text{x}/\checkmark$|$77.2_{\pm1.0}$|$81.7_{\pm0.8}$|$76.5_{\pm2.6}$|$67.0_{\pm2.4}$|$61.2_{\pm0.5}$|
|ES disagreement SELF|$\text{x}/\checkmark$|$\underline{\boldsymbol{93.0_{\pm0.3}}}$|$83.9_{\pm0.9}$|$82.1_{\pm1.8}$|$79.1_{\pm2.1}^*$|$70.7_{\pm2.5}$|
|JTT|$\text{x}/\checkmark$|$86.7$|$81.1$|$79.5_{\pm 5.1}$|$\underline{69.3}$|$72.6$|
|AFR|$\text{x}/\checkmark$|$90.4_{\pm1.1}$|$82.0_{\pm0.5}$|$80.2_{\pm2.0}$|$68.7_{\pm0.6}$|$73.4_{\pm0.6}$|
|EVaLS-GL (Ours)|$\text{x}/\checkmark$|$89.4_{\pm0.3}$|$84.6_{\pm1.6}$|$\underline{\boldsymbol{82.3_{\pm1.2}}}$|$68.0_{\pm0.5}$/$80.5_{\pm0.4}^*$|$\underline{75.1_{\pm1.2}}$|
—-------------------------------
|EVaLS (Ours)|$\text{x}/\text{x}$|$88.4_{\pm3.1}$|$\underline{85.3_{\pm0.4}}$|$\underline{82.1_{\pm1.0}}$|Not SC|Not SC|
|Class-balanced last-layer retraining|$\text{x}/\text{x}$|$\underline{92.6_{\pm0.8}}$|$73.7_{\pm2.8}$|$21.9_{\pm13.0}$|$80.4_{\pm0.8}^*$|$64.7_{\pm1.1}$|
|ERM|$\text{x}/\text{x}$|$66.4_{\pm2.3}$|$47.4_{\pm2.3}$|$18.7_{\pm2.0}$|$56.3_{\pm4.8}$|$64.8_{\pm1.9}$|
Previously in Q3, I asked about why the performance of the proposed approach is much higher for the CivilComments dataset. 

The author responded that "The CivilComments dataset has class imbalance, so fine-tuning/retraining on class-balanced data can significantly improve group robustness."

However, as I looked closely into the details, that might not be the case. In SELF[2], the worst-group accuracy is calculated over **4** groups by aggregating all identity categories into one spurious feature. In other benchmarks such as JTT, DFR, and AFR, the worst-group accuracy is calculated over **16** groups (8 attributes and 2 labels). Therefore, the result of EVaLS-GL in CivilComments column might not be directly comparable to each other except with SELF (added later) and the ERM/DFR implemented by SELF. Please look into this carefully and correct/rerun the misleading results if necessary. 

Relevant public GitHub repos: 

https://github.com/tmlabonte/last-layer-retraining 

https://github.com/AndPotap/afr
Dear Reviewer RE7s,

We’re glad our clarification was helpful. It’s good to know that the pseudo-code and highlights have been effective in preventing confusion. We appreciate your suggestion to include them in the final version and will incorporate them.

### Regarding the limitations that you have stated

As you mentioned, we state in the Discussion section (L356-357) that environment-based validation is limited to spurious correlation and not other types of subpopulation shifts.

### Regarding SELF [2]

Please refer to the response to Reviewer 7E4z. As we described there and also in Appendix A-Related Work (L547-554), reference [2] in the paper proposes two methods:

*Class-balanced (CB) last layer retraining*: See L552-554. As we discuss comprehensively in the response to Reviewer 7E4z, this method is not helpful in targeting robustness to spurious correlation, but *it improves robustness to class-imbalance, another type of subpopulation shift which is independent of spurious correlation*. For details, refer to the formulations in the response to Reviewer 7E4z.

*Early-stop (ES) disagreement SELF*: As noted in the response to Reviewer 7E4z, ES disagreement requires group annotations for model selection, but not for retraining (L547-551). It also needs an early-stopped version of the trained ERM and the final model.

In the following table, we restate the worst group accuracy for our methods and those of SELF on spurious correlation benchmarks. We also add the results of CB last-layer retraining and ES disagreement SELF for Dominoes-CMF.

|Dataset|Group Info (Train/Val)|Waterbirds|CelebA|UrbanCars|Dominoes-CMF|
|-|-|-|-|-|-|
|ES disagreement (SELF)|$\text{x}/\checkmark$|$93.0_{\pm0.3}$|$83.9_{\pm0.9}$|$82.1_{\pm1.8}$|$60.5_{\pm1.3}$|
|EVaLS-GL (ours)|$\text{x}/\checkmark$|$89.4_{\pm0.3}$|$84.6_{\pm1.6}$|$82.3_{\pm1.2}$|$63.6_{\pm1.3}$|
|CB last-layer retraining (SELF)|$\text{x}/\text{x}$|$92.6_{\pm0.8}$|$73.7_{\pm2.8}$|$21.9_{\pm13.0}$|$53.3_{\pm5.0}$|
|EVaLS (ours)|$\text{x}/\text{x}$|$88.4_{\pm3.1}$|$85.3_{\pm0.4}$|$82.1_{\pm0.9}$|$67.1_{\pm4.2}$|
|ERM|$\text{x}/\text{x}$|$66.4_{\pm2.3}$|$47.4_{\pm2.3}$|$18.67_{\pm2.0}$|$50.6_{\pm1.0}$|

*Waterbirds*: SELF’s methods outperform EVaLS. As SELF illustrates in their Sec. 3 (Preliminaries), since CB last-layer retraining is performed on validation data, which is group-balanced for Waterbirds, CB last-layer retraining is also group-balanced for Waterbirds. When the dataset for is not group-balanced, CB last-layer retraining achieves a worst group accuracy of $77.4_{\pm0.3}$ (Table 7 in reference [2] of the paper (SELF)).

*CelebA*: As you can see, ***EVaLS and EVaLS-GL outperform both CB last-layer retraining and ES disagreement***. We believe there may have been an oversight in your response, as it incorrectly suggests their method outperforms ours. CB last-layer retraining underperforms all other benchmarked methods in Table 1.

*UrbanCars*: EVaLS-GL outperforms all other methods. Both EVaLS and ES disagreement (with higher group supervision) show similar worst group accuracy. CB last-layer retraining shows no significant improvement over ERM. This is expected, as UrbanCars’ training data is class-balanced, unlike Waterbirds and CelebA. Therefore, further class-balancing for CB last-layer retraining makes no difference.

*Dominoes-CMF*: 
EVaLS outperforms other methods. EVaLS-GL also outperforms methods in SELF [2]. CB last-layer retraining does not show any significant improvement compared to ERM. This is expected, as Dominoes-CMF also has class-balanced training data like UrbanCars.

#### Conclusion: 

CB last-layer retraining targets class-imbalance shifts and is ineffective for spurious correlation. Except for the Waterbirds dataset, our methods ٍ(EVaLS and EVaLS-GL) outperform those of SELF [2] with similar group supervision on all other datasets. EVaLS shows comparable or better results (***e.g., in CelebA***) compared to methods with higher group supervision. See Table 1 in the paper for more comparisons.
Dear Reviewer SN9B,

Thank you for taking the time to read our rebuttal. We are pleased to hear that you find our research complete and detailed, with sufficient experiments demonstrating its effectiveness.

We must emphasize that the title of our work reflects its content. We propose a method to derive all necessary information from the model itself to enhance its robustness against spurious correlations. Section 3, which introduces EVaLS, presents solutions such as loss-based sampling for creating a balanced dataset and partitioning the validation set into environments for model selection. We have also conducted multiple ablations in Section E to evaluate different modules while maintaining the overall framework. Thus, our work has never proposed using and combining previous methods as a novelty (as Reviewer 7E4z has stated).

Our work demonstrates the following points, as stated at the beginning of the general author rebuttal:

- In contrast to what is considered a requirement for previous methods (see L528-549 in Appendix A-Related Work), we show that ideal group discovery is not required for model selection in robustness to spurious correlations. Instead, identifying environments with group shifts (Sec. 3.2) works effectively (see Table 1).
- Relying on what a trained model learns, rather than auxiliary (even ground-truth) information (as in methods like AFR, DFR, etc.), could be more effective in the case of unknown spurious correlations (see Table 2 and Table 1-UrbanCars).
- Loss-based sampling (Sec. 3.1) is not only an effective method for robustness to group shifts (compared to other methods such as loss-based weighting schemes like AFR or upweighting misclassified samples like JTT; see EVaLS and EVaLS-GL in Table 1), but it is also supported by a theory for data balancing with general assumptions (Sec. 3.3 - Theoretical Analysis).

Best regards,
Authors of Submission 19478
Thank you for the detailed clarification on having a class-wise loss-based sampling strategy. In this case, I agree that the theoretical analysis in proposition 3.1 aligns and supports the proposed approach very well. I also apologize for not realizing the sampling strategy applies to each class, and that's why I asked for pseudocode for clarification at first. I believe the paper can be improved after including the pseudocode and more highlights to avoid confusion, which the authors have already done well in the rebuttal. 

The major limitation still lies in the experimental results. The proposed algorithm EVaLS is only somewhat better than other baselines, and the results are quite arguable when compared to SELF[2], since SELF is more flexible and outperforms EVaLS-GL on Waterbirds and CelebA. When using EIIL for validation split instead of using the ground-truth group labels, it's also limited to the case where the spurious correlations are strong and not applicable to the two NLP datasets. The authors have correctly identified and discussed these limitations. 

After checking the comments from other reviewers, there are no other obvious major concerns. I have increased my score accordingly.
Dear reviewer RE7s,

In the general author rebuttal, we aimed to keep the pseudocode minimal and straightforward to highlight the main components of the algorithm and framework. We greatly appreciate your attention to detail in noticing that the pseudocode in the general author rebuttal did not demonstrate the class-dependency of the algorithm. Details regarding this matter have been added to the pseudo-code below.

We have clarified in the paper that loss-based sampling occurs within each class. In Section 3.1, where we introduce loss-based sampling, we clearly state: *“By combining these **$2k$ samples from each class**, we construct a balanced set $D^{balanced}$”, consisting of high-loss and low-loss samples (see Figure 1(c))* (L171-172). This matches what you state in the response. Moreover, in the caption of the main figure, Figure 1(c) in the paper, it is stated: *“We evaluate train split samples on the initial ERM classifier and sort high-loss and low-loss samples of **each class** for loss-based sampling.”* Additionally, in Figures 1(c) and 1(d) in the paper, you can see that in each class, samples with the highest and lowest loss are selected.

It is worth mentioning that our environment-based validation also applies to each class separately, as stated in Sec. 3.2 Partitioning Validation Set into Environments (L194-195): *“Subsequently, each environment is further divided based on sample labels, resulting in $2\times|y|$ environments.”* You can also observe that all our reported group shifts in Table 3 are class-dependent, and other environment inference methods used are class-dependent as well (L201-203).

We sincerely apologize for any confusion in this matter. Please let us know if anything further needs to be clarified.

-------------

Class-dependency details for loss-based sampling **become** ***bold*** in the pseudo-code below.

**Input:** Held-out dataset `D`, ERM-trained model `f_ERM`, maximum `k` value `maxK`  
**Output:** Optimal number of samples `k*`, Best model `f*`
1. Split the held-out dataset `D` into train and validation:
   - $D_{MS}$, $D_{LL}$ = `splitDataset(D)`
2. Infer environments from the validation split:
   - `envs = inferEnvs(D_MS)`
3. ***For each class $y\in\mathcal{Y}$***
    - ***Sort $D_{LL}$ samples from class $y$ by their loss:***
      - ***`sortedSamples[y] = sortByLoss(f_ERM, `$D_{LL}^y$`)`***
4. Initialize  `wea* = 0, k*= 0, f*= None`
5. For `k` from 1 to `maxK`:
   - ***For each class $y\in\mathcal{Y}$:***
     - Select top-`k` high-loss samples:
       - ***`highLossSamples[y] = sortedSamples[y][:k]`***
     - Select top-`k` low-loss samples:
       - ***`lowLossSamples[y] = sortedSamples[y][-k:]`***
   - Combine samples:
     - ***`selectedSamples = {highLossSamples[y], lowLossSamples[y]} for all `$y\in\mathcal{Y}$***
   - Retrain the last layer:
     - `f = retrainLastLayer(selectedSamples)`
   - Evaluate the retrained model:
     - `wea = evaluateWEA(f, envs)`
   - If `wea > wea*`:
     - `wea* = wea`
     - `f* = f`
     - `k* = k`
6. return `f*`
Thanks for the explanation! I think we should clarify whether your loss-base sampling approach is class-wise or not. It seems to be so from our discussion, but your pseudo-code at the top in the general rebuttal did not consider the class, and I could not verify you are selecting $|\mathcal{Y}|*2k$ samples in total from the main paper.
Dear author, 

Thank you for your reply and the additional experimental results. 

After reading the reply and the opinions of other reviewers, I maintain my score. Although I agree with Reviewer 7E4z's point that this work is of limited novelty, I think the author's research is complete and detailed with enough experiments to show the effectiveness of EVaLS. In addition, I appreciate the discussion on multiple (unknown) spurious features. Therefore, I keep my score.

Best,

Reviewer SN9B
Thank you for considering our rebuttal and paying special attention to the theoretical aspects of our work. We have tried to answer all your concerns in our response and hope you find them helpful. Please let us know if there is anything that needs to be clarified, as we find these discussions very helpful. 

**W3**: We would like to thank you for clarifying your arguments. We think we now understand better what you meant.


We should emphasize that **both** the loss-based sampling and theoretical analysis are **class**-dependent. We state that our theoretical arguments are class-dependent in L215-218, and also L584-587 in Appendix C. Consider the setting we have assumed for the classification task and the loss function (L214 in Sec. 3.3, L583 in Appendix C). As you have stated, in the case of using logistic loss (sigmoid function), loss values are calculated as $-log(\sigma(L))$ and $-log(1-\sigma(L))$ (each for one of the classes). Therefore, the order of samples in logit and loss spaces is monotonic (L215, L584) in **each class**. The only difference between the two classes is that the direction of the monotonicity is different for them. But as our analysis are **class**-dependent, there is no worry about applying two different transformations, as the transformations themselves are **class**-dependent.

Please note that our theoretical analysis aims to show that loss-based sampling can create a balanced dataset. Focusing on logit space is sufficient to demonstrate sampling thresholds. Consequently, as seen in Proposition 3.1 (and C.1), we assume that we have $1-\epsilon$ spurious correlation, and minority (proportion $\epsilon$) and majority (proportion $1-\epsilon$) samples in each class are drawn from different Gaussian distributions in the feature and hence logit spaces (as you have noted). Additional assumptions about the feature space structure (like in the paper [1] you mentioned) are more restrictive. We do not fully comprehend how assuming a more structured feature space is helpful to further develop the theory as the current assumptions are less restrictive.

**W4**: Thank you. As shown in our response to reviewer 7E4z, while the mentioned methods target different subpopulation shifts or have higher supervision, our method outperforms them in almost all experiments (the only exception is Waterbirds, where some have higher WGA but lower average accuracy). We will include the new baseline results in the final paper.

As stated in Sec. 2.1 (L112-L123), There exist multiple types of subpopulation shifts (see response to reviewer 7E4z for class-imbalance). Refer to reference [4] in the paper for a categorization of subpopulation shifts and their formulation (Table 1 in their paper). Our focus in designing environment-based validation (EV) is on spurious correlation. The types of subpopulation shifts in CivilComments and MultiNLI are class-imbalance and attribute-imbalance, respectively (see the definition of these datasets in Appendix D.3, L714-725).

Regarding your question, please refer to L316-324. As explained there, patterns distinguishing groups in CivilComments and MultiNLI are not predictive for the target class (see tables below). This reduces their visibility in the model’s final layers (reference [29] in the paper discusses the role of various layers of neural networks in different types of shifts). Since environment inference algorithms like EIIL (see Appendix E.2 for further investigation) depend on the last layers of a trained model, they cannot infer environments with notable group shifts (defined in L195-198) in CivilComments and MultiNLI. The group shifts in CivilComments and MultiNLI (L323-324) are significantly lower than those of the datasets that are reported in Table 3. Thus, the focus of environment-based validation is on datasets with spurious correlation.

Nevertheless, loss-based sampling (LS) is effective for CivilComments and MultiNLI. Our EVaLS-GL, using ground-truth group labels for model selection and loss-based sampling for retraining, outperforms all other methods on CivilComments and those with similar group supervision on MultiNLI. Only GDRO, with full group annotations during training, performs better on MultiNLI.

**CivilComments and MultiNLI training sets statistics**

CivilComments:
|Group |Class|Attribute |# Train Data|
|-|:-:|:-:|:-:|
|$G_1$|0|No Identities|148186 (55%)|
|$G_2$|0|Has Identities|90337 (33%)|
|$G_3$|1|No Identities|12731 (5%)|
|$G_4$|1|Has Identities|17784 (7%)|


MultiNLI:
|Group|Class|Attribute|# Train Data|
|-|:-:|:-:|:-:|
|$G_1$|0|No Negations|57498 (28%)|
|$G_2$|0|Has Negations|11158 (5%)|
|$G_3$|1|No Negations|67376 (32%)|
|$G_4$|1|Has Negations|1521 (1%)|
|$G_5$|2|No Negations|66630 (32%)|
|$G_6$|2|Has Negations|1992 (1%)|
Thanks for providing the detailed responses including experimental results, pseudo code, useful statistics, etc.

W3: To clarify, what I meant was that the proposed approach utilizes “**loss**-based sampling” (class-dependent), but the analysis in proposition 3.1 is based on the **logits** (class-independent). It’s perfectly fine to assume Gaussian distribution in the features space and hence logit space, but when the loss is involved, it becomes a bit more complicated. This is because now we have different classes even for logistic regression. The logistic loss is monotonic for each class but when the label flips it’s no longer the same function. As assumed there are two Gaussian distributions for the **logits**, for binary classification, there will be four resulting distributions on the **losses**, because there are two transformations $-log(\sigma(L))$ and $-log(1-\sigma(L))$. Does the analysis for logits still hold in this scenario with losses?

Furthermore, I referenced [1] because they considered two feature dimensions (core and spurious), the majority/minority groups are clearly defined as spuriously-correlated/non-spuriously-correlated w.r.t. the features and labels. When you reduce the analytical setup to majority and minority distributions, the relationship with spurious correlation becomes blurred. That’s why I mentioned it as a limitation.

W4: Thanks for filling in the missing results. Please also include the few baselines pointed out by reviewer 7E4z to make the empirical comparison more comprehensive, but just to clarify why CivilComments and MNLI are out of the scope of EVaLS?
We deeply appreciate your consideration of our rebuttal. Thank you for suggesting the inclusion of the discussions in the final version of the paper; we will certainly do so.

We have carefully addressed all your concerns based on your feedback. If there are any specific areas where you feel further clarification or justification is needed, we are eager to address them promptly. Your guidance helps us achieve the highest standards, and we hope this will be reflected in a higher score.
Thank you for your detailed response. I think it generally addresses my concerns so I will increase my score. I would encourage the authors to include these additional discussions in the final version of the paper.