The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations
Authors: Tyler LaBonte
John Collins Hill
Xinchen zhang
Vidya Muthukumar
Abhishek Kumar
Venue: NeurIPS 2024 poster (2024)
Keywords: spurious correlations
group robustness
distribution shift
class balancing
Primary area: fairness
TL;DR: We identify surprising and nuanced behavior of finetuned models on worst-group accuracy in settings including class-balancing, model scaling, and spectral analysis.

Abstract: Modern machine learning models are prone to over-reliance on spurious correlations, which can often lead to poor performance on minority groups. In this paper, we identify surprising and nuanced behavior of finetuned models on worst-group accuracy via comprehensive experiments on four well-established benchmarks across vision and language tasks. We first show that the commonly used class-balancing techniques of mini-batch upsampling and loss upweighting can induce a decrease in worst-group accuracy (WGA) with training epochs, leading to performance no better than without class-balancing. While in some scenarios, removing data to create a class-balanced subset is more effective, we show this depends on group structure and propose a mixture method which can outperform both techniques. Next, we show that scaling pretrained models is generally beneficial for worst-group accuracy, but only in conjunction with appropriate class-balancing. Finally, we identify spectral imbalance in finetuning features as a potential source of group disparities --- minority group covariance matrices incur a larger spectral norm than majority groups once conditioned on the classes. Our results show more nuanced interactions of modern finetuned models with group robustness than was previously known. Our code is available at https://github.com/tmlabonte/revisiting-finetuning.

The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations
Tyler LaBonte John C. Hill? Xinchen Zhang?
Vidya Muthukumar?. Abhishek Kumari
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology School of Electrical and Computer Engineering, Georgia Institute of Technology
Abstract
Modern machine learning models are prone over-reliance spurious corre- lations, which can often lead to poor performance on minority groups. In this paper, we identify surprising and nuanced behavior of finetuned models on worst- group accuracy via comprehensive experiments on four well-establiseed bench- marks across vision and language tasks. We first show that the commonly used class-balancing techniques of mini- batch upsampling and loss upweighting can induce decrease in worst-group accuracy (WGA) with training epochs, leading to performance no better than without class-balancing. While in some scenar- ios, removing data to create Blassbbllncee subset more effective, we show this depends on group structure and propose a mixture method which can outper- form both techniques. Next, we show that scaling pretrained models is generally beneficial for worst-group accuracy, but only in conjunction with appropriate class bass-balanncig. Finally, we identify spectral imbalance finctuning features potential source of group disparities minority group covariance matri- ces incur a larger spectral norm than majority groups once conditioned on the classes. Our results show more nuanced interactions of modern finetuned mod- els with group robustness than was previously known. Our code is available at https: / gi thub. cww/cmt//iioocc com/ttmaboonee.. wewoonttereviiiiingg.... wwwrrrsiiiinnninntititee finetuning.
Introduction
Classification performance in machine learning is sensitive to spurious correlations: patterns which are predictive of the target class in the training dataset but not at test time. For example, in computer vision tasks, neural networks known to utilize the backgrounds of images proxies for their content [1. 50, 68]. Beyond simple settings, spurious correlations have been identified high- consequence applications such criminal justice [8], medicine [70], and facial recognition [33]. In particular, a model's 's reliance on spurious correlations disproportionately affects its accuracy on minority groups which are under-represenned in the training dataset; we therefore desire maximizing the model's group robustness, quantified by its minimum accuracy on any group [50].
The standard workflow in modern machine learning involves initializing from a pretrained model and finetuning on the downstream dataset using empirical risk minimization (ERM) [62], which minimizes the average training loss. When group annotations are available in the training dataset, practitioners utilize a rich literature of techniques to improve worst-group accuracy (WGA) [50, 39, 26). However, group annotations are often unknown problematic to obtain (e.g., due to financial, privacy, or fairness concerns). While group robustness methods have been adapted to work without group annotations [31, 72, 47, 29], they remain complex variants the standard finetuning procedure.
"Corresponding author. Email: tlabonte@gatech.com edu. I Work done at Google DeepMind.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Hence, it is often unclear to what extent the WGA dynamics of these methods are attributable to details of model finetuning.
In this paper, we take a complementary approach to the methodological literature by pursuing comprehensive understanding of the fundamental properties of model finetuning on four well- established group robustness benchmarks across vision and language tasks. We focus especially on the conjunction of model scaling and blaass-bllancng improve robustness on some datasets [22] - on the worst-group accuracy of the ERM baseline. These considerations enable to isolate the impact of group disparities on worst-group accuracy. thereby revealing more nuanced behaviors of finetuned models than previously known. In particular, we challenge overarching narratives that "overparameterization helps or hurts distributional robustness" and show striking differences in finetuning performance depending on class-balancing methodology.
which was recently shown to greatly
In more detail, our main contributions include:
Identifying two failure modes of common blass-balancing techniques during fine- tuning: (1) mini-batch upsampling and loss upweighting experience catastrophic collapse with standard hyperparameters on benchmark datasets, and (2) removing data create a class-balanced subset can harm WGA for certain datasets.
Proposing mixture balancing method which combines the advantages of two class-balancing techniques and can improve baseline WGA beyond either method.
Showing that while overparameterization can harm WGA in certain cases, model scaling is generally beneficial for robustness when applied in conjunction with appropriate pretraining and class-balancing.
Identifying a spectral imbalance in the top eigenvalues of the group covariances even when the classes are balanced and showing that minority group covariance matrices consistently have larger spectral norm conditioned on the classes.
1.1
Related work
Here we provide a brief summary of related work along three axes. Throughout the paper, we also provide detailed contextualiaations of our results with the most closely related work.
Spurious correlations. The proclivity of ERM to rely on spurious correlations has been widely studied [12, 37]. Rectifying this weakness is an important challenge for real-world deployment of machine learning algorithms. as spurious correlations can exacerbate unintended bias against demographic minorities [20, 2, 57, 17, 5] or cause failure in high-consequence applications [33, 8, 70, 42]. Reliance on spurious correlations manifests in image datasets as the usage of visual shorteuts including background [1, 50, 68], texture [11], and secondary objects [48, 52, 54), and in text datasets as the usage of syntactic or statistical heuristics as a substitute for semantic understanding [14, 41, 36).
or training with an equal number of samples from each group, has been proposed as a simple yet effective method improve robustness to spurious correlations [17, 51, 6, 55]. However, goulnbbaanniing requires group annotations, which are often unknown or problematic to obtain [31, 72, 47, 29]. On the other hand, class- balaancannn or training with an equal number of samples from each class, is well-studied method in long-tailed surprisingly powerful method for improving vorst-group accuracy which does not require group annotations [22, 29, 7, 53). In particular, [22] study the WGA dynamics of two common class-balancing methods: removing data from the larger classes (which we call subsetting) and upsampling the smaller classes (which we call upsampling). Our results complement those [22] and show more nuanced effects of class-balancing than previously known; provide additional contextualization with [22] in Section 3.1. We show similar nuanced behavior of upweighting smaller classes in the loss function, a popular method the sroup-balancing setting [31, 47, 55] which [22] did not study.
Class-balancing and group robustness.
classification [24, 15, 4]. Recent work has shown that class-balancing
Overparameterization and distributional robustness. While the accepted empirical wisdom that overparamettrizatioo improves in-distribution test accuracy [40, 71]. the relationship between overparameterization and robustness is incompletely understood. [51] considered class of ResNet- 18 architectures and showed that increasing model width reduces worst-group accuracy the

---

Waterbirds and CelebA datasets when trained with class insssimmaleneed ERM this contrasts with the improvement in average accuracy widely observed in practice (see, e.g.. [38]). Conversely, [19] showed a benefit of overparameterizatton in robustness to "natural" covariate shifts, which are quite different from spurious correlations [27]. On the mathematical front, [59, 35] showed that overparameterization in random feature models trained to completion improves robustness to wide class of covariate shifts.However both the optimization trajectory and statistical properties of random features are very different from neural networks (see, e.g., [13]). Closely related to our work, [46] investigated pretrained ResNet, VGG, and BERT models, and showed that overparameterization does not harm WGA. Our results complement those of [46] with richer setup and show that balascinnniing-.... - which they do not study - can greatly impact model scaling behavior.
2 Preliminaries
Setting. We consider classification tasks with imput domain E and target classes y N. Suppose S is a set of spurious features such that each example a E R" is associated with exactly one feature s(x) S. The dataset is then partitioned into groups G, defined by the Cartesian product of classes and spurious features G = y x S. Given dataset of m training examples, we define the set of indices of examples which belong to some group g E 9 or class y E 3 by Qg S (1..... m) and 2g S (1,..., m., respectively. Then, the majority group(s) is defined by the group(s) that maximize [Ωyl- All other groups are designated as minority groups. Further, the worst group(s)) is defined by the group(s) which incur minimal test accuracy. We define majority and minority classes similarly. Because groups are defined by the Cartesian product of classes and spurious features, all training examples particular group are identically labeled, and therefore a group is a subset of a class.
We desire a model which, despite group imbalance in the training dataset, enjoys roughly uniform performance over G. Therefore, we evaluate worst-group accuracy (WGA), i.e., the minimum accuracy among all groups [50]. We will also be interested in the relative performance on groups within the same class, and we thereby define the majority group within class y E Y as the group which maximizes [9gl over all {g E G E g}. Other groups are designated as the minority groups within that class. For example, referring to the Waterbirds section of Table 2, groups and are the minority groups within classes 0 and 1, respectively.
Class-balancing. A dataset considered to be class-balanced it is composed of an equal number of training examples from each class in expectation over the sampling probabilities. We compare three class- balanein techniques: subsetting, upsampling, and upweighting. We describe each below:
In subsetting, every class is set to the same size as the smallest class by removing the appropriate amount of data from each larger class uniformly at random. This procedure is performed only once, and the subset is fixed prior to training.
In upsampling, the entire dataset is utilized for training with typical stochastic optimization al algorithm, but the sampling probabilities of each class are adjusted so that mini-batches are class-balanced in expectation. To draw single example, we first sample Uni,())), then sample æ ~ p(- y) where ý is the empirical distribution on training examples.
In upweighting. the minority class samples are directly upweighted in the loss function according to the ratio of majority class data to minority class data, called the class-imbalance ratio. Specifically, if the loss function is (()())) y) for model f, example æ. and class label y. the upweighted loss function is ((( y) where is defined as the class-imbalance ratio for minority class data and for majority class data. It worth noting that upweighting is equivalent to upsampling in expectation over the sampling probabilities.
Note that the terminology for these class-balances techniques is not consistent across the literature. For example, [22] call subsetting subsampling (denoted SUBY) and upsampling reweighting (denoted RWY). On the other hand, [55] call ggroup-wise) subsetting downsampling and use upweighting to describe increasing the weight of minority group samples in the loss function.
"Note that, as is standard in the empirical literature on distributional robustness, majority, minority and worst groups are defined with respect to the empirical training distribution, as this is all that we have access to. Moreover, test accuracy is typically maximized by the majority group and minimized by a minority group, though this is not always the case.
3

---

Datasets and models. We study four classification datasets, two in the vision domain and two in the language domain, which are well-established as benchmarks for group robustness. We summarize each dataset below and provide additional numerical details in Appendix A.
Waterbirds [64, 63, 50] is an image dataset wherein birds are classified land species ("landbirds") or water species ("waterbirds"). The spurious feature is the image background: more landbirds are present land backgrounds and vice versa.*
CelebA [33, 50] is an image dataset classifying celebrities as blond or non-blond. The spurious feature gender, with more blond women than blond men in the training dataset.
CivilCommentts [3, 27] is a language dataset wherein online comments classified as toxic or non-toxic. The spurious feature is the presence of one of the following categories: male, female, LGBT, black, white, Christian, Muslim, or other religion. 2 More toxic comments contain one of these categories than non-toxic comments, and vice versa.
MultiNLI [65, 50] is a language dataset wherein pairs of sentences are classified as a contradiction, entailment, or neither. The spurious feature is a negation in the second sentence more contradictions have this property than entailments or neutral pairs.
Waterbirds is class innalaiebaleeeed with a majority/minority class ratio of 3.31:1, CelebA a ratio of 5.71:1, and CivilComments ratio of 7.85:1. MultiNLI is bass-balanced priori. Since the Waterbirds dataset has a shift in group proportion from train to test, we weight the group accuracies by their proportions in the training set when reporting the test average accuracy [50].
We utilize ResNet [18), ConyNeXt-V2 [67], and Swin Transformer [32] models pretrained on ImageNet-IK IK [49] for Waterbirds and CelebA, and BERT [9] model pretrained on Book Corpus [73] and English Wikipedia for CivilComments and MultiNLI. We use the Adam optimizer [34] for finetuning on three independent seeds, randomizing both mini-batch order and any other stochastic procedure such as subsetting, and we report error bars corresponding to one standard deviation. We do not utilize carly-stopping: instead, to consider the impact of overparameterization in a holistic way, we train models to completion to properly measure the overfitting effect. This can result in longer training than commonly seen in the literature (e.g., finetune CelebA for about 3x more gradient steps than is standard). See Appendix A.2 for further training details.
3 Nuanced effects of class-balancing on group robustness
We now present our first set of results, which shows that the choice of class balancing method greatly impacts the group robustness of the ERM baseline.
3.1 Catastrophic collapse of class-balanced upsampling and upweighting
In a recent paper. [29] observed that contrary to the central hypothesis underlying the Just Train Twice method [31], the worst-group accuracy of ERM decreases dramatically with training epochs on CelebA and CivilComments; however, they provide no explanation for this phenomenon. In this section, we show that this degradation of WGA due to their choice of class- baus-balanning method (i.e., upsampling). Specifically, ERM finetuned with upsampling experiences a catastrophic collapse in test WGA over the course training, a phenomenon that was previously only noticed in synthetic datasets with linear classifier [22]. Moreover, while [22] state that class-balanced subsetting is not recommended in practice, we show that can in fact improve WGA conditional on the lack of of small minority group within the majority class. Finally, we show that balanced upweighting a popular technique which [22] do not study - experiences a similar WGA collapse as upsampling.
We finetune a ConvNeXI-V2 Base on Waterbirds and CelebA and a BERT Base on CivilComments, and we compare the subsetting. upsampling, and upweighting techniques to a imbalanedddd
*We note that the Waterbirds dataset is known to contain incorrect labels [56]. We report results the original, un un-corrected version as standard in the literature.
'This version of CivilComments has four groups, used in this work and by [50. 22, 23, 26, 29). There is another version where the identity categories are collapsed into spurious feature; that version is used by [31. 72.47]. Both versions use the WILDS split [27].
"To be more specific, we finetune ConvNeett-V V2 Base roughly to a training loss of 10 on Waterbirds and 10-3 on CelebA, and BERT Base roughly training loss of 10 CivilComments and 1 on MultiNLI.

---

clessibllnccing
Subsetting
Upeampling
Upwegtning
80
70
75
70
Rest NGG
60
70
65
50
60
40
55
40
50
10
20
30
40
50
80 Epech
70
80
90
100
10
12 Epoch
14
16
18
20
10
14
16
18
20
Epoch
(a) Waterbirds
(b) CelebA
(c) CivilComments
Figure 1: Class-balanced upsampling and upweighting experience catastrophic collapse. We compare subsetting, wherein data is removed to set every class to the same size as the smallest class, upsampling. wherein the sampling probabilities of each class are adjusted so that the mini-batches are class-balanced in expectation, and upweighting, wherein the loss for the smaller classes is scaled by the class-imbalance ratio. We observe a catastrophic collapse over the course of training of upsampling and upweighting on CelebA and CivilComments, the more class-imbalanced datasets. Subsetting reduces WGA on Waterbirds because removes data from the small minority group within the majority class. MultiNLI is class-balanced priori, we do not include it here.
baseline. Our results are displayed in Figure 1, with additional models in Appendix B. On CelebA and CivilComments, the more insalanebae datasets, upsampling and upweighting both experience catastrophic collapse over the course of training. We believe this collapse is caused by overfitting to the minority group within the minority class; any individual point from this group is sampled far more often during upsampling and weighted far more heavily during upweighting, causing overfitting during long training runs. In fact, upsampling does even worse on CelebA than observed in [29] because train 3x longer ensure convergence. With that said, optimally tuned early-stopping appears to mitigate the collapse (as previously noticed by [22] in a toy setting).
Our experiments also highlight previously unnoticed disadvantage of class-balanced subsetting: if there small minority group the majority class, subsetting will further reduce its proportion and harm WGA. For example, in the Waterbirds dataset, the species (landbirds/waterbirds) is the class label and the background (land/water) the spurious feature; landbirds/water is small minority group within the majority class (landbirds), When landbirds is cut by 3.31x, the landbirds/ group greatly suffers, harming WGA. On the other hand, in the CelebA dataset, the hair color monnblondd/blond) is the class label and the gender (female/male) is the spurious feature; the only small minority group is blond/male, while the groups are nearly balanced the majority class. In this case, subsetting preserves blond/male examples and increases their proportion, helping WGA.
Finally, while upsampling and upweighting have similar WGA dynamics perhaps as expected, as they are equivalent in expectation over the sampling mechanism both differ greatly from subsetting. Recently, [55] proved a theoretical equivalence between subsetting and upsampling the groups in the population setting, iee, assuming access to the training distribution. The equivalence of upsampling and upweighting would then imply that all three objectives are optimized by the same solution. However, our results suggest this may not hold in the real-world empirical setting, where subsetting has distinctly different behavior, and model parameters may outnumber training examples. As previously mentioned, this may be due to overfitting to minority class data repeated often during training; theoretically investigating this discrepancy is an important future direction.
Contextualization with previous work. Our observations explain the decrease in WGA of CelebA and CivilComments noticed by [29], phenomenon which they left unresolved. Our result implies that group robustness methods which assume that WGA increases during training, such as Just Train Twice [31], may only be justified with appropriate class Llass-balancing [22] show that upsampling can cause catastrophic collapse WGA, but only in synthetic dataset with a linear classifier. In realistic datasets, [22] perform extensive hyperparameter tuning (using group labels, which may be unrealistic) to achieve good results with upsampling, while we show that catastrophic collapse can occur the same datasets when standard hyperparameters used. Moreover, [22] state that class- bass-balanced
5

---

Subsetting
Uptemping
Upoeighting
80
75
70
70
7bx8 WC24
65
50-
60
40
10
20
30
40
50
80 Epech
70
80
90
8
10
12 Epoch
14
16
18
20
10
14
16
18
20
Epoch
(a) Waterbirds
(b) CelebA
(c) CivilComments
Figure Mixture balancing mitigates catastrophic collapse of upsampling and upweighting. We mixture method, which combines subsetting and upsampling by first drawing blass-imbalanceed subset uniformly at random from the dataset, then adjusting sampling probabilities so that mini batches are balanced in expectation. Our method increases exposure to majority class data without over-sampling the minority class. Remarkably, mixture balancing outperforms all three blass-balancing methods on Waterbirds and CivilComments, and while it does not outperform subsetting on CelebA, it significantly alleviates the WGA collapse experienced by upsampling and upweighting. MultiNLI is baas-balaanced a priori, so we do not include it here.
propose a
subsetting is not recommended in practice, but we show that subsetting can be effective except when there is small minority group within the majority class, previously unnoticed nuance. Finally, show that subsetting experiences different WGA dynamics from upsampling and upweighting in the empirical setting, suggesting additional complexity compared the population setting results of [55].
Without extensive tuning, class-balanced upsampling and upweighting can induce WGA no better than without class-balancing. While class-balanced subsetting can improve WGA, practitioners should use caution a small minority group is present within the majority class.
3.2 Mixture balancing: interpolating between subsetting and upsampling
To mitigate the catastrophic collapse of class-balanced upsampling and upweighting, we propose simple mixture method which interpolates between subsetting and upsampling. Our method increases exposure to majority class data without over-sampling the minority class, which improve WGA and mitigate overfitting to the minority group. We first create a data subset with a specified class- imbalance ratio by removing data from the larger classes uniformly random until the desired (smaller) ratio is achieved. Next, we perform ERM finetuning on this subset by adjusting sampling probabilities so that mini-batcher reduces to subsetting, and using the original insalanne ratio reduces to upsampling.
balanced in expectation. Using class inbalance ratio of 1:1
We finetune ConvNeXt-V2 V2 Base on Waterbirds and CelebA and BERT Base CivilComments, and we compare our class-balanced mixture method to the subsetting. upsampling. and upweighting techniques. The results of our experiments are displayed in Figure 2. We plot the performance of our mixture method with the best class innalaimae ratio during validation; an ablation study varying the ratio is included in Appendix B. Remarkably, mixture balancing outperforms all three class-balancing methods on Waterbirds and CivilComments, and while it does not outperform subsetting on CelebA, it significantly alleviates the WGA collapse experienced by upsampling.
Next, we perform an ablation of the necessity of subsetting in mixture balancing. We compare our method with an implementation which eschews subsetting, instead adjusting sampling probabilities so that the mini-batcher have particular class ratio expectation. upsampling on 2:1 class-imbalanced subset, we upsample the majority class by ratio of 2:1 on the entire dataset. The results of our ablation are included in Appendix B; our mixture method outperforms the alternative, which incompletely corrects for class imbalance.
example, instead of performing

---

Table Mixture balancing robust to model selection without group annotations. We compare the best ttass-balancing method during validation with and without group annotations. Both worst- class accuracy [69] and the biappussueer validation score of [60] are effective for model selection without group annotations, often choosing the same method or mixture ratio as worst-group accuracy (WGA) validation. We list the method maximizing each metric and its average WGA over 3 seeds.
Validation Metric
Group Anns
Waterbirds
CelebA
CivilComments
Bias- claspussueervised Score
x
Upsampling (79.9)
Subsetting (74.1)
Mixture 3:1 (77.6)
Worst-class Accuracy
x
Mixture 2:1 (81.1)
Subsetting (74.1)
Mixture 3:1 (77.6)
Worst-group Accuracy
Mixture 2:1 (81.1)
Subsetting (74.1)
Mixture 3:1 (77.6)
Note on validation. In Figure 2. plot the best class-imbalie imbalanceeaeeeee ratio achieved using validation on a group annotated held-out set. While this is a common assumption in the literature [50, 31, 23, 26], it is nevertheless unrealistic when the training set does have any group annotations. Therefore, we compare with both worst-class accuracy [69] and the sias-ussueereised validation score of [60], which do not use any group annotations for model selection. In Table we list the method which maximizes each validation metric as well as its average WGA. Overall, we show both methods are effective for model selection, often choosing the same method mixture ratio WGA validation.
Contextualization with previous work. Increasing exposure to majority class data without over- sampling the minority class was previously explored by [26]. who proposed averaging the weights of logistic regression models trained on ten independent baass-balanced subsets. However, this method only works for linear models- as nonlinear models cannot be naively averaged and requires multiple training runs, which is computationally infeasible for neural networks. In comparison, our mixture method is simple and efficient alternative which extends easily to nonlinear models.
The catastrophic collapse of balsncallnced upsampling and upweighting can be mitigated by mixture method. It increases exposure majority class data without over-sampling the minority class and can improve baseline WGA beyond either technique.
4 Model scaling improves WGA of class-balanced finetuning
The relationship between overparameterriztion and group robustness has been well-studied, with often conflicting conclusions [51, 59]. this section, we study the impact of model scaling on sorst-group accuracy in setting- finetuning pretrained models which more closely resembles practical use-cases. Importantly, we evaluate the impact of model scaling in conjunction with class- class-balancing to isolate the impact of group incquities on WGA appropriate balancinggg overparameterization can in fact significantly improve WGA over very wide range of parameter scales, including before and after the interpolation threshold. On the other hand, scaling on imbalanced datasets or with the wrong balancing technique can harm robustness.
function of model size. We find that with
We take advantage of advancements in efficient architectures [61, 67) to finetune pretrained models in a wide range of scales from 3.4M to 101M parameters. We study six different sizes of ImageNet K- pretrained Vo- and five different sizes of Book Corpus/English Wikipedia pretrained BERT; specifications for each model size are included in Appendix A.2. Our results are displayed in Figure 3, and we include results for Swin Transformer in Appendix C.
We find that model scaling is beneficial for group robustness in conjunction with appropriate class- balancing, with improvements of up to 12% WGA for interpolating models and 40% WGA for non-interpolating models. This comes in stark contrast to scaling on class-imbalanced datasets or with the wrong class-balancing technique, which shows either a neutral trend or decrease in WGA the most severe examples being on CivilComments. With respect to interpolating models, CivilComments WGA decreases slightly after the interpolation threshold, while Waterbirds and CelebA continue to improve well beyond interpolation; on the other hand, BERT never interpolates MultiNLL greatly increasing robustness at scale. It is unclear why Waterbirds and CelebA experience
7

---

beluncing
Upomperg
Used
Wataro
Hopplitonn Trratst
00
30
32
403
$600
13000
2000
350
Doda
147
17903
60301
B690
099
0901
oral
madin
(a) Waterbirds
(b) CelebA
(c) CivilComments
(d) MultiNLI
Figure 3: Scaling class-balanced pretrained models improve worst-group accuracy. We finetune each model size starting from pretrained checkpoints and plot the test worst-group accuracy (WGA) as well as the interpolation threshold, where model reaches 100% training accuracy. We find model scaling is generally beneficial for WGA only in conjunction with appropriate class-balancing and scaling on imbalanced datasets or with the wrong method can harm robustness. Note MultiNLI is class- balanced priori and is not interpolated. See Appendix C for training accuracy plots.
(a) Waterbirds (last layer only)
(b) Waterbirds (finetuning)
(c) CelebA (finctuning)
Figure 4: Class-balancing greatly affects ResNet scaling results of [46]. We contrast the ResNet scaling behavior of [46] who do not use dass-balanciig-
the scaling of class-balanced ResNets. We finetune each model size starting from pretrained checkpoints and plot the test worst- group accuracy (WGA), well as the interpolation threshold, where the model reaches 100% training accuracy. On Waterbirds, we find that class-balanche balancing enables a much more beneficial trend during model scaling. On CelebA, class-balancing greatly increases baseline WGA but does not affect scaling behavior (in contrast to the ConvNeXt-V3? V2 plots in Figure 3). We use SGD for last-layer training and AdamW for full finetuning. See Appendix C for training accuracy plots.
different behavior from CivilComments interpolation the toy linear model of [51] suggests a benign "spurious-core information ratio", but complete understanding is left to future investigation.
The most closely related work models on group robustness. However, because their experiments do not employ any form of glass-balancing, their conclusions may overly pessimistic. We replicate their experiments with our hyperparameters and contrast with our results using class-balancing in Figure 4. We find that class-balancing greatly affects their results: on Waterbirds, balannin enables a much more beneficial trend during model scaling regardless of whether linear probe or the entire model is trained. Moreover, while Mlas-balancinn increases baseline WGA on CelebA but does not affect scaling behavior, we observe a more positive WGA trend when scaling ConvNeXt- V2 in Figure 3.
is [46], who study the impact of scaling pretrained ResNet
Contextualization with previous work. While previous work has primarily studied either linear probing pretrained weights or training small models from scratch [51, 59], we study full finetuning of lcage-scale pretrained models and show that class-balancing can have major impact on scaling behavior. We compare directly with the most closely related work, [46], and show that class-balancing can either induce strikingly different scaling behavior greatly increase baseline WGA. Overall, training with balancinnn allows us to isolate the impact of group inequities on robustness and more precisely observe the
trend of model scaling for worst-group accuracy.

---

Gmap 0
Graa
Crow
50
Group
ones
1000
290
(a) Waterbirds
(b) CelebA
(c) CivilComments
(d) MultiLI
Figure 5: Group disparities are visible the top eigenvalues of the group covariance matrices.
We visualize the mean, across 3 experimental trials, of the top 10 eigenvalues of the group covariance matrices for a ConvNeXI-V V2 Nano finetuned on Waterbirds and CelebA and BERT Small finetuned on CivilComments and MultiNLI. The standard deviations are omitted for clarity. The models are finetuned using the best balas-bblnniing method from Section 3 for each dataset. The group numbers detailed in Table and the minority groups within each class are denoted with an asterisk. The largest A in each case belongs to a minority group, though may not be the worst group, and minority group eigenvalues are overall larger than majority group eigenvalues within the same class.
While overparameterizzttio can sometimes harm WGA, pretraining and appropriate class- balancing make scaling generally beneficial. Moreover, modern language datasets are com- plex enough that standard models do not interpolate, greatly improving robustness at scale.
Spectral imbalance may exacerbate group disparities
In a recent paper, [25] propose spectral imbalance of class covariance matrices, or differences in their eigenspectrum, as source of disparities in accuracy across classes even when balanced. Here, we examine whether similar insights hold in the group robustness setting. Our observations reveal surprising nuances in the behavior of group-wise spectral imbalance; nevertheless, we conclude that spectral imbalance may play a similar role in modulating WGA after class-balancing applied.
Let us denote by the feature vector corresponding to sample æ; (i.e., the vectorized output of the penultimate layer). Recall from Section that S2g the set of indices of samples which belong to group We further define 2, to be the empirical mean of features with group To obtain the estimated eigenspectrum, we first compute the empirical covariance matrix for group g E g by
Σg =
Σ (zi - Zz)( -2g) Tg) iE
We then compute the eigenvalue decomposition Eg = V AgAvV where Ag a diagonal matrix with non-negative entries {(9) and the columns of V, 9 are the eigenvectors of Σg. Without loss of generality, assume X(5) X2 (g) .... (g) where is the rank of Eg.
We compute the group covariance matrices using a ConvNeXt-V Nano model for Waterbirds and CelebA, and a BERT Small model for CivilComments and MultiNLI We plot the top 10 eigenvalues of each group covariance matrix Figure Even though we finetune with class-ba,ancing, disparities in eigenvalues across groups are clearly visualized in Figure 5, especially for the largest eigenvalues. We include extensions to the top eigenvalues and class covariance matrices in Appendix D.
Close observation of Figure 5 yields interesting findings. First, the group g" that maximizes x(") in each case belongs minority group; though, importantly, it may not belong the worst group. This is different from the findings of [25], who showed that the largest eigenvalues typically belong the worst-performing class. Second, we find that minority group eigenvalues are overall larger than majority group eigenvalues, but only when conditioned on the class. A majority group belonging to one class may have larger eigenvalues than minority group belonging to another class, but there exists a consistent spectral imbalance between majority and minority groups within the same class."
Forr example. in Figure 5c, the spectrum for group 3 (the majority group within class 1) is larger than the spectrum for group (the minority group within class 0). However, conditioning the class, we find that the

---

14
14
12
12
10
4,0
18
50
66
16
(a) Waterbirds
(b) CelebA
(c) CivilComments
(d) MultiNLI
Figure 6: Group-wise spectral imbalance is apparent once conditioned on the classes. We plot the mean and standard deviation, across 3 experimental trials, of the intra-class spectral norm ratio p(v), the ratio of the top eigenvalues of the minority and majority group covariance matrices, for each class E y. We compute this metric using a finetuned ConvNeXt-V Nano on Waterbirds and CclebA and finetuned BERT Small on CivilComments and MultiNLI, each using the best class-balancing method from Section for each dataset. The key observation is that g is at least one for all classes y E y (except a single seed for class on CelebA), illustrating a group disparity captured by the eigenspectrum once condition the classes.
To quantify this group- group-wise wise spectral imbalance, we introduce a new metric called the intra-class spectral norm ratio. Suppose 9min(y) and Imaj(1) the minority and majority groups within par-
ticular class y E y. Then, we define the intra-class spectral norm ratio by p(v) == x((mma()) /x] (9mg(()) While (عمر only considers the top eigenvalue and not the entire spectrum, the absolute magnitude of individual eigenvalues was found in [25] to correlate best with worst-class accuracy. We note that p()) considers only the top eigenvalue and not the entire spectrum, since the magnitude of the top eigenvalues was found in [25] to correlate best with worst-class accuracy. We plot the intra-class spectral norm ratios for each dataset Figure 6; notably, they are always least one (except for single seed on CelebA), showing the group disparity captured by the eigenspectrum.
Finally, in Table 5 (deferred to Appendix D). we compare the class with the largest plv) to the class with the largest disparity in group test accuracies, i.e., Acc(Amag most cases these classes correspond, suggesting an explanatory power of the intra-class spectral norm ratio. In particular, this correspondence consistent throughout all trials of CelebA and CivilComments, the most glass-imbalanced datasets we study.
Acc(gmii (y)). We that
Contextualization with previous work. Our spectral analysis of the group covariance matrices inspired by [25]. We both study class-balanced settings, with the key difference that they study class disparities instead of group disparities. However, show imbalance across both classes and groups, i.e., spectral imbalance is more prevalent between majority and minority groups within the same class, rather than across groups globally.
nuanced impact of spectral
Spectral imbalance in the group covariance matrices may exacerbate group disparities even when the classes are balanced. While the worst-group covariance may not have largest spectral norm, the minority group spectra are consistently larger conditioned on the class.
Discussion
In this paper, we identified nuanced impacts class-balancing and model scaling on worst-group accuracy, well as spectral imbalance in the group covariance matrices. Overall, our work calls for a more thorough investigation of generalization in the presence spurious correlations to unify the sometimes contradictory perspectives in the literature. We hope that, the community continues to develop group robustness methods with increasing performance and complexity. researchers and practitioners alike remain cognizant of the disproportionate impact of the details.
spectrum for group (the minority group within class 1) is larger than that of group 3, and the spectrum of group 1 larger than that of group (the majority group within class O).
10

---

Acknowledgments. We thank Google Cloud for the gift of compute credits, Jacob Abernethy for additional compute assistance, and Chiraag Kaushik for helpful discussions. T.L. acknowledges support from the DolD NDSEG Fellowship. V.M. acknowledges support from the NSF (awards CCF-223915 and IIS-2212182), Google Research, Adobe Research and Amazon Research.
References
[1] Sara Beery, Grant van Horn, and Pietro Perona. Recognition in terra incognita. In European Conference on Computer Vision (ECCV), 2018.
[2] Su Lin Blodgett, Lisa Green, and Brendan O'Connor. Demographic dialectal variation in social media: A case study of Afmricn-American English. In Empirical Methods in Natural Language Processing (EMNLP). 2016.
[3] Daniel Borkan, Lucas Dixon, Jeffrey Sorenson, Nithium Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In World Wide Web (WWW), 2019.
[4] Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106(1):249-259, 2018.
[5] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability, and Transparency in Machine Learning (FATML), 2018.
[6] Niladri S Chatterji, Saminul Haque, and Tatsunori Hashimoto. Undersampling a minimax optimal robustness intervention in nonparametric classification. Transactions Machine Learning Research (TMLR), 2023.
[7] Kamalika Chaudburi, Kartik Ahuja, Martin Arjovsky, and David Lopez-Paz. Why does throwing away data improve worst-group error? In International Conference on Machine Learning (ICML), 2023. 2023.
[8] Alexandra Chouldechova. Fair prediction with disparate impact: study of bias in recidivism prediction instruments. In Conference on Fairness, Accountability, and Transparency in Machine Learning (FATML). 2016.
[9] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In North American Association for Computational Linguistics (NAACL), 2019.
[10] William Falcon and the PyTorch Lightning maintainers and contributors. PyTorch Lightning. GitHub, 2019.
[11] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. srggeeett-triinee CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations (ICLR), 2019.
[12] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2:665-673, 2020.
[13] Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz, and Andrea Montanari. Limitations of lazy training of lwo-rs neural network. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
[14] Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In North American Association for Computational Linguistics (NAACL), 2018.
[15] Guo Haixiang. Li Yijing, Jennifer Shang, Gu Mingyun, Huang Yuanyue, and Gong Bing. Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, (111):220-293 2017.

---

[16] Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Couraapea,, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel Smith, Robert Kern. Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fer- nández del Río, Mark Wiehe, Pearu Peterson, Pierre Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585(1):357-32,, 2020.
Kevin Sheppard, Tyler
[17] Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning (ICML), 2018.
[18) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[19] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustmess: A critical analysis of out-of-distribution generalization. In International Conference on Computer Vision (ICCV), 2021.
[20] Dirk Hovy and Anders Søgaard. Tagging performance correlates with author age. In Association for Computational Linguistics (ACL), 2015.
[21] John D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9(3):90-99 95, 2007.
[22] Badr Youbi Idrissi, Martín Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst-group-accuracy. Conference on Causal Learning and Reasoning (CLeaR), 2022.
[23] Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew Gordon Wilson. On feature learn- ing in the presence of spurious correlations. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
[24] Nathalie Japkowicz and Shaju Stephen. The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5):429 449, 2002.
[25] Chiraag Kaushik, Ran Liu, Chi-Heng Lin, Amrit Khera, Matthew Y Jin, Wenrui Ma. Vidya Muthukumar, and Eva L. Dyer. Balanced data, imbalanced spectra: Unveiling class disparities with spectral imbalance. In International Conference on Machine Learning (ICML), 2024.
[26] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness spurious correlations. In International Conference Learning Representations (ICLR), 2023.
[27] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga Richard Lanas Phillips, Sara Beery, Jure Leskovee, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning (ICML), 2021.
[28] Tyler LaBonte. Milkshake: Quick and extendable experimentation with classification models. http://gitan com/ wwwabbnt/mmikkshaataaaae 2023.
[29] Tyler LaBonte, Vidya Muthukumar, and Abhishek Kumar. Towards last-layer retraining for group robustness with fewer annotations. In Conference on Neural Information Processing Systems (NeurIPS), 2023.
[30] Yoonho Lee, Huaxiu Yao, and Chelsea Finn. Diversify and disambiguate: Learning from underspecified data. In International Conference on Learning Representations (ICLR), 2023.
[31] Evan Zheran Liu, Behzad Haghgoo. Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning (ICML), 2021.
12

---

[32] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows, In International Conference on Computer Vision (ICCV), 2021.
[33] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In International Conference on Computer Vision (ICCV), 2015.
[34] Nya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR). 2019.
[35] Subha Maity, Saptarshi Roy, Songkai Xue, Mikhail Yurochkin, and Yuekai Sun. How does overparametrization affect performance on minority groups? arXiv preprint arXiv:2206.035/5, 2022.
[36] Tom McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Association for Computational Linguistics (ACL), 2019.
[37] Mazda Moayeri, Wenxiao Wang. Sahil Singla, and Soheil Feizi. Spuriosity rankings: Sorting data to measure and mitigate biases. In Conference on Neural Information Processing Systems (NeurlPS), 2023.
[38] Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021 (12)::24003, 2021.
[39] Junhyun Nam, Jachyung Kim, Jacho Lee, and Jinwoo Shin. Spread spurious attribute: Improving worst-group accuracy with spurious attribute estimation. In International Conference on Learning Representations (ICLR), 2022.
[40] Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. In search of the real inductive bias: On the role of implicit regularization in deep learning. In International Conference Learning Representations (ICLR), 2015.
[41] Timothy Niven and Hung- Yu Kao. Probing neural network comprchension of natural language arguments. In Association for Computational Linguistics (ACL), 2019.
[42] Luke Oakden-Rayne Jared Dunnmon, Gustavo Carneiro, and Christopher Ré. Hidden strat- ification causes clinically meaningful failures in machine learning for medical imaging. In Conference on Neural Information Processing Systems (NeurIPS) Workshop on Machine Learn- ing Health, 2019.
[43] Matteo Pagliardini, Martin Jaggi, François Fleuret, and Sai Praneeth Karimireddy. Agree to disagree: Diversity through disagreement for better transferability. In International Conference on Learning Representations (ICLR), 2023.
[44] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary De Vito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. In Conference on Neural Information Processing Systems (NeurIPS) Workshop on Automatic Disferentiation, 2017.
[45] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang. Zach Detoto, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Conference on Neural Information Processing Systems (NeurIPS), 2019.
[46] Alan Pham, Eunice Chan, Vikranth Srivatsa, Dhruba Ghosh, Yaoqing Yang, Yaodong Yu, Ruiqi Zhong, Joseph E Gonzalez, and Jacob Steinhardt. The effect of model size on worst-group generalization. In Conference on Neural Information Processing Systems (NeurIPS) Workshop on Distribution Shifts, 2021.
13

---

[47] Shikai Qiu, Andres Potapczynski, Pavel Izmailov, and Andrew Gordon Wilson. Simple and fast group robustness by automatic feature reweighting. In International Conference on Machine Learning (ICML), 2023.
[48] Amir Rosenfeld, Richard Zemel, and John K. Tsotsos. The elephant in the room. arXiv preprint 0008.03305, 2018.
[49] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet large scale visual recognition challenge. International Journal of Computer Vision
(IJCV), 115(1):211 152, 2015.
[50] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In International Conference on Learning Representations (ICLR), 2020.
[51] Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameteriizatoon exacerbates spurious correlations. In International Conference on Machine Learning (ICML), 2020.
[52] Rakshith Shetty, Bernt Schiele, and Mario Fritz. Not using the car to see the sidewalk: Quanti- fying and controlling the effects of context in classification and segmentation. In Conference Computer Vision and Pattern Recognition (CVPR), 2019.
[53] Ravid Shwartz-Zii Ziv, Micah Goldblum, Yucen Lily Li, C Bayan Bruss, and Andrew Gordon Wilson. Simplifying neural network training under class imbalance. In Conference on Neural Information Processing Systems (NeurIPS), 2023.
[54] Sahil Singla and Soheil Feizi. Salient ImageNet: how to discover spurious features in deep learning? In International Conference on Learning Representattinss (ICLR), 2022.
[55] Nathan Stromberg, Rohan Ayyagari, Monica Welfert, Sanmi Koyejo, and Lalitha Sankar. Robustness to subpopulation shift with domain label noise via regularized annotation domains. arXiv preprint arXiv:2402.110399 2024.
[56] Saeid Asgari Taghanaki, Aliasghar Khani, Fereshte Khani, Ali Gholami, Linh Trana, Ali Mahdavi-Amiri, and Ghassan Hamarneh. MaskTune: mitigating spurious correlations by forcing to explore. In Conference on Neural Information Processing Systems (NeurIPS), 2022.
[57] Rachael Tatman. Gender and dialect bias in YouTube's 's automatic captions. In Association for Computational Linguistics (ACL) Workshop on Ethics in Natural Language Processing, 2017.
[58] TorchVision maintainers and contributors. Vision: PyTorch's computer vision library. GitHub, 2016.
[59] Nilesh Tripuraneni, Ben Adlam, and Jeffrey Pennington. Overparameterization improves ro- bustness to covariate shift in high dimensions. In Conference on Neural Information Processing Systems (NeurIPS), 2021.
[60] Christos Tsirigotis, Joao Monteiro, Pau Rodriguez, David Vazquez, and Aaron Courville. Group robust classification without any group information. In Conference on Neural Information Processing Systems (NeurIPS), 2023.
[61] Iulia Turc, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint
2019.
[62] Vladimir Vapnik. Statistical Learning Theory. Wiley, 1998.
[63] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongic. The Caltech-UCSD firds-20--2011 dataset. Technical report, California Institute of Technology, 2011.
[64] Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-UCSD birds 200. Technical report, California Institute of Technology, 2010.
14

---

[65] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In North American Association for Computational Linguisties (NAACI.), 2018.
[66]
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the. art natural language processing. In Conference on Empirical Methods in Natural Language Processing (EMNLP) System Demonstrations, 2020.
[67] Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. ConvNeXt V2: co-designing and scaling convnets with masked autoencoders. In Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[68] Kai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise or signal: The role of image backgrounds in object recognition. International Conference on Learning Representations (ICI.R), 2021.
[69] Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzyeh Ghassemi. Change is hard: A closer look at subpopulation shift. In International Conference on Machine Learning (ICML), 2023.
[70] John R. Zech, Marcus A. Badgeley, Manway Liu, Anthony B. Costa, Joseph J. Titano, and Eric Karl Oermann. Variable generalization performance of deep learning model detect pneumonia in chest radiographs: A cross-sectional study. PLoS Medicine, 15:e1002683, 2018.
[71] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3): 107-115, 2021.
[72] Michael Zhang, Nimit S. Sohoni, Hongyang R. Zhang, Chelsea Finn, and Christopher Ré. Correct-n-contrast: contrastive approach for improving robustness to spurious correlations. In International Conference on Machine Learning (ICML), 2022.
[73] Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In International Conference Computer Vision (ICCV), 2015.
15

---

A Additional Details for Section 2
A.1 Dataset Composition
Table 2:
Dataset composition. We study four well-established benchmarks for group robustness across vision and language tasks. The class probabilities change dramatically when conditioned on the spurious feature. Note that Waterbirds is the only dataset that has a distribution shift and MultiNLI is the only dataset which is class-balanced a priori. The minority groups within each class are denoted by an asterisk in the "Num" column. Probabilities may not sum to due to rounding.
Group g
Training distribution
Data quantity
Datasct
Num
Class
Spurious 8
p(y)
p(g)
p(yss)
Train
Val
Test
landbird
land
.730
.984
3498
467
2225
.768
Waterbirds
landbird
water
.038
.148
466
2225
waterbird
.012
.016
56
133
642
.232
waterbird water
.220
.852
1057
133
642
0
non-blond female non-blond male
.400
.758
71629
8535
9767
.851
1
.411
.980
66874
8276
7535
CelebA
hlond female
.141
.242
22880
2874
2480
.149
blond male
.009
.020
1387
182
180
neutral
identity identity identity identity
.551
.921
148186
25159
74780
.887
1
neutral
.336
.836
90337
14966
43778
CivilComments
2
toxic
.047
.079
12731
2111
6455
.113
3
toxic
.066
.164
17784
2944
8769
contradiction contradiction
no negation negation
.79
.300
57498
22814
34597
.333
1
.054
.761
11158
4634
6655
2
entailment
negation negation negation
.327
.352
67376
26949
40496
MultiNLI
.344
3+
entailment
.007
.104
1521
613
886
4
neither
.323
.348
66630
26655
39930
5*
neither
negation
.010
.136
1992
797
1148
A.2 Training details
We utilize ResNet [18], ConvNeXt-VE V2 [67], and Swin Transformer [32] models pretrained on ImageNet-IK [49] for Waterbirds and CelebA, and a BERT [9] model pretrained on Book Cor- pus [73] and English Wikipedia for CivilComments and MultiNLI. These pretrained models are used as the initialization for ERM finetuning under the cross-entropy loss. We use standard ImageNet normalization with standard flip and crop data augmentation for the vision tasks and BERT tok- enization for the language tasks [23]. Our implementation uses the following packages: NumPy [16]. PyTorch [44, 45], Lightning [10], TorchVision [58], Matplotlib [21], Transformers [66]. and Milkshake [28].
To knowledge, the licenses of Waterbirds and CelebA unknown. CivilComments is released under the CCO license, and information about MultiNLI's license may be found in [65].
Our experiments were conducted on four Google Cloud Platform (GCP) 16GB Nvidia Tesla P100 GPUs and two local 24GB Nvidia RTX A5000 GPUs The spectral imbalance experiments in Section 5 were conducted on a GCP system with 16-core CPU and I 128GB of RAM. We believe our work could be reproduced for under $5000 in GCP compute credits, with majority of that compute going multiple random seeds.
towards running experiments
We list model scaling parameters in Table 3 and hyperparameters used for each dataset in Table ConvNeXt-V2 V2, ResNet and Swin Transformers are composed of four separate "stages", and we list the depths of these stages individually in Table 3. All of these configurations are standard in the literature. The smaller BERT models were introduced by [61]. We perform model selection only for our mixture balancing method (see Table 1) and not for the ERM finetuning hyperparameters, most of which are standard in the literature [50, 22, 23]. For the last-layer training experiments in Figure 4 and Figure 1, we use SGD with learning rate 10-3 and train for 20 epochs. Different from previous work, train CclebA for about 3x more gradient steps than usual to ensure convergence, and
16

---

double the batch size for CivilComments and MultiNLI to increase training stability (we also double
the epochs to hold the number of gradient steps constant).
Table 3: Model scaling parameters.
(a) ConvNeXt-V2 parameters.
(b) BERT parameters.
Size
Width
Depth (4 stages)
Params
Size
Width
Depth
Params
Atto
40
(2, 2, 6, 2)
3.4M
Tiny
2
128
4.4M
Femto
48
(2.2,6,7
4.8M
Mini
256
11.2M
Pico
64
(2,2,6,2
8.6M
Small
4
512
28.8M
Nano
80
(2.2,8.2)
15.0M
Medium
512
41.4M
Tiny
96
(3,3,9.3)
27.9M
Base
12
768
109M
Base
128
(3,3,27,) 27,3)
87.7M
(c) ResNet parameters.
Size
Width (4 stages)
Depth (4 stages)
Params
18
(64, 128, 256, 512)
22,
11.2M
34
(64, 128, 256, 512)
(3,4,6,3
21.3M
(256, 512, 1024, 2048)
(3,4,6,3)
23.5M
101
(256, 512, 1024, 2048)
23,423,3)
42.5M
152
(256, 512, 1024, 2048)
(3, 36,36,
58.1M
(d) Swin Transformer parameters.
Size
Width
Depth (4 stages)
Params
Tiny
96
(,,2, 6,
29M
Small
96
(2, 2, 18,
50M
Base
128
(2, 2, 18,2)
88M
Table 4: ERM finetuning hyperparameters.
Dataset
Optimizer
Initial
LR schedule
Batch size
Weight decay
Epochs
Waterbirds
Adam W
x 10-5
Cosine
32
x 10-4
100
CelebA
Adam W
x 10
Cosine
32
10-4
20
CivilComments
Adam W
1 X 10
Linear
32
x 10
20
MultiNLI
Adam W
X 10-5
Linear
32
x 10-1
20
17

---

B Additional Experiments for Section 3
Mixturc baloncing subscc)
Mixturc balancing Ino
75
77.5
75.0
83
73-
70.0
67.5
70
55
65
62.5
80.0
45
57.5
1.1
2.3
3.31:1
1.1
2.1
5.711
111
3.1
7.85.1
(a) Waterbirds
(b) CelehA
(c) CivilComments
mixture balancing method. First, we vary the llass-imbllance ratio across the x axis. On the left- left-hand side, using class-imbalann imbalance ratio of 1:1 reduces to the subsetting technique; the original class-imbalance ratio in the dataset reduces to upsampling. Second, we perform an ablation of whether subsetting is essential in mixture balancing. We plot proposed method (which takes a subset of data based on the class-imbalance ratio, then performs upsampling) against the same method without subsetting. instead adjusting the class probabilities on the entire dataset as specified by the iass-imbalance ratio. MultiNLI is llassbblaaced a priori, so we do not include it here.
Figure 7: Mixture balancing ablation studies. We perform two ablation studies
the right-hand side, using
No class balancing
Upsampling
Subsetting
Upweighting
Mixture 2:1
1est WGA
Tost WGA
82.5
77.5
75.0
72.5
Epoch
Epoch
(a) Waterbirds
(b) CelebA
Figure 8: Balancing behavior is consistent with Swin Transformer. We demonstrate the effec-
tiveness of our (compare the ConvNeXt-V2 V2 results in Figure 2). Overall, we find our results are consistent across pretrained model families, with the model affecting the raw accuracies but typically not the relative performance of class-balancing techniques. We also corroborate poor performance subsetting on Waterbirds and the catastrophic collapse of upsampling and upweighting on Celeba from Figure 1.
mixture method when used in conjunction with Swin Transformer
18

---

No class-balancing
Upsampling
Subsetting
Upweighting
Mixture 2:1
70
Test VIGA
WGA
60
80 100
2.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0
Epoch
Epach
(a) Waterbirds
(b) CelebA
Figure 9: Balancing behavior is consistent with ResNet model family. We demonstrate the effectiveness our class- balanced mixture method on another model family, ResNet (compare to the ConvNeXt-V2 results in Figure 2). Again, find that results consistent and that the model architecture affects the raw accuracies but typically not the relative performance of class- balancing techniques. We also corroborate the poor performance of subsetting Waterbirds and the catastrophic collapse of upsampling and upweighting on Celeba from Figure l.
19

---

C Additional Experiments for Section
cles-baarcingg
Trasheli
11.0
61.
(a) Waterbirds
(b) CelebA
(c) CivilComments
(d) MultiNLI
Figure 10: Average accuracy of scaled models. We finetune each model size starting from pretrained checkpoints and plot the train average accuracy (AA) as well as the interpolation threshold, where at least one seed of the non-class-balance CelebA does not interpolate with all three seeds). Average accuracy consistently increases with model size regardless of class-balancing, implying the scaling dynamics for AA and WGA are starkly different. Note that MultiNLI mass-balanced priori and does not interpolate any size.
model reaches 100% training accuracy. (For example,
100
150
(a) Waterbirds (last layer only)
Waterbirds (finetuning)
CelebA (finetuning)
Figure 11: Average accuracy of scaled ResNets. We contrast the ResNet scaling behavior of [46] who do not use class-balancing - to the scaling class-balanced ResNets. We finetune each model size starting from pretrained checkpoints and plot the train average accuracy (AA) as well as the interpolation threshold, where the model reaches 100% training accuracy. Similarly to Figure 10, average accuracy consistently increases with model size. We use SGD for last-layer training and AdamW for full finetuning.
20

---

No class balancing
Upsampling
Subsetting
Upweighting
Mixture 2:1
Interpolation Threshold
90
70
65
Test WGA
Test ISA
45
thn
small
base
tiny
small
base
(a) Waterbirds
(b) CelebA
Figure 12: Scaling behavior is consistent with Swin Transformer. We exhibit the model scaling behaviour of a Swin Transformer, and compare it to that of ConvNeXt-V2 (shown in Figure 3). We see that the scaling behaviour consistent across pretrained model families, with the model affecting the raw accuracies but not the relative performance of class-balane balancing techniques.
21

---

D Additional Experiments for Section
Grage
Chaap una Gra.p
Graps Gowe
Group
0oop4
400
1000
Indec
irdec
(a) Waterbirds
(b) CelebA
CivilComments
(d) MultiNLI
Figure 13: Additional eigenvalues of the group covariance matrices. In contrast to Figure 5, we visualize the top eigenvalues of the group covariance matrices for ConvNeXt-V Nano finetuned on Waterbirds and CelebA and BERT Small finetuned on CivilComments and MultiNLI. The models are finetuned using the best class-balancing method from Section for each dataset. The group numbers are detailed in Table and minority groups are marked with an asterisk. It becomes difficult to distinguish patterns between the groups the lower cigenvalues, which why we focus only on local properties of the top eigenvalues (e.g., the spectral norm and the relative ordering of the groups). With that said, would be interesting to explore power-law decay metrics [25], which characterize relatively global properties of the eigenspectrum, in future work.
030
Ciua
Davi
-30
2000
400
1000
200
D00
DCCCC
19d6
(a) Waterhirds
(b) CelebA
CivilCommenns
(d) MultiNLI
Figure 14: Class disparities are visible in the top eigenvalues of the class covariance matrices. We visualize the mean, across 3 experimental trials, of the top 10 eigenvalues of the class covariance matrices for a ConvNeXt-V V2 Nano finetuned on Waterbirds and CelebA and BERT Small finetuned on CivilComments and MultiNLI. The standard deviations are omitted for clarity. The models are finetuned using the best are detailed Table 2. The minority class eigenvalues for CelebA and CivilComments are overall larger, while the reverse is true for Waterbirds, slightly different conclusion than [25].
balancing method from Section 3 for each dataset. The class numbers
200
2
Clrss
Das
E30
0lx57
2000
1000
3
12
22
40
10 40
33
20 20 40
(a) Waterbirds
(b) CelebA
(c) CivilComments
(d) MultiLI
Figure 15: Additional eigenvalues of the class covariance matrices. In contrast Figure 14, we visualize the top 50 eigenvalues of the class covariance matrices for a ConvNeXt-V2 Nano finetuned on Waterbirds and CelebA and BERT Small finetuned on CivilComments and MultiNLI. The models are finetuned using the best class-balancing method from Section 3 for each dataset. The class numbers are detailed in Table Similar the groups, it becomes difficult distinguish patterns between the classes in the lower eigenvalues, which is why we again focus only on local properties of the top eigenvalues (e.g., the spectral norm and the relative ordering of the classes).
22

---

106
Gre45
Gres
Gmap Grup
Grupo
Grep
SPENE
Urep
2
22
10
1830
20
190
(a) No Class Balancing
(b) Subsetting
Upsampling
(d) Mixture
Figure 16: Group eigenvalue decay is consistent across balancing methods. We visualize the mean, across 3 experimental trials, of the top 10 eigenvalues of the group covariance matrices for ConvNeXt-VV X2 Nano finetuned on Waterbirds across all class-balancing methods. The standard deviations are omitted for clarity. Overall, we found that the magnitude of the eigenvalues is significantly affected by the chosen balss-balancing method. However, the relative ordering of minority/majority group eigenvalues is consistent across class-balancing techniques. We note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal baasncal may bring about additional stability in the representation.
00
(a) No Class Balancing
(b) Subsetting
(c) Upsampling
(d) Mixture
Figure 17: Class eigenvalue decay is consistent across balancing methods. We visualize the mean,
across 3 experimental trials, of the top 10 eigenvalues of the class covariance matrices for ConvNeXt- V2 Nano finetuned on Waterbirds across all class- balancing methods. The standard deviations are omitted for clarity. Overall, found that the magnitude of the eigenvalues is significantly affected by the chosen class- balancing method. However, the relative ordering of minority/maaority group eigenvalues consistent across dlss-balaaccing techniques. We note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class blass-balancing may bring about additional stability in the representation.
Table 5: Correspondence between p(t) and intra-class group accuracy disparity. We compare p(v), the intra-class spectral norm ratio, to the difference in intra-class group accuracy. Each row represents different experimental seed. Each cell contains tuple with the class label for the class with largest value of p(u) paired with the class label for the class with the largest intra-class group test accuracy disparity, i.e., Acc(smai(y)) - Acc(gmin(y)) We see that in most cases these classes correspond, suggesting an explanatory power of the spectral norm ratio. In particular, this correspondence is consistent throughout all trials of CelebA and CivilComments, the most class- imbalanced datasets we study.
Seed
Waterbirds
CelebA
CivilComments
MultiNLI
1
(1,
(1, 1)
(0, 0)
(0, 0)
2
(1, l)
(1, I)
(0, 0)
(0, I)
3
(0, 1)
(1.1)
(0, 0)
(2, 0)
23

---

14
14
14
1,2
1,32
1,19
16
88
18
66
66
(a) No Class Balancing
(b) Subsetting
(c) Upsampling
(d) Mixture
Figure 18: Spectral imbalance consistent across balaneing methods. We plot the mean and standard deviation, across experimental trials, of the intra-class spectral norm ratio أاأ) or the ratio of the top eigenvalues of the minority and majority group covariance matrices, for each class y E V. We compute this metric using finetuned ConvNeXt-V2 V2 Nano on Waterbirds. Overall, we found that the relative magnitudes of p(3) consistent across class-balancing methods. We note that the most drastic change in the relative magnitudes of p(v) is induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class- balancing may bring about additional stability in the representation.
24

---

E Broader impacts, limitations, and compute
Broader impacts. We hope our work contributes to the safe and equitable application of machine learning and motivates further research in ML faireess. With that said, potential negative outcome may arise if practitioners simply apply our techniques in place of conducting rigorous bias studies. Indeed, while our methods show improved fairness with respect to the worst-group accuracy metric, necessary to perform comprehensive evaluations with respect to multiple additional fairness criteria prior to model deployment.
Limitations. Our methods take advantage of the structure of spurious correlations; our insights would likely not transfer over to datasets which exhibit more extreme complete correlation (i.e., contain zero minority group data) [43, 30] or to more generic settings. A limitation of our mixture balancing method is that to achieve optimal performance, it requires a validation set with group annotations for selection of the best class-imbalance ratio [50, 31, 23, 26]. With that said, show in Table that worst-class accuracy [69] and the bias-unsupervisee validation score of [60] are sufficient for model selection the benchmarks we study.
-iloo-ddistribution generalization
Compute. Our experiments were conducted two Google Cloud Platform (GCP) 16GB Nvidia Tesla PI00 GPUs and two local 24GB Nvidia RTX A5000 GPUs. The spectral imbalance experiments in Section 5 were conducted on a GCP system with 16-core CPU and 128GB of RAM. We believe our work could be reproduced for under $5000 GCP compute credits, with a majority of that compute going towards running experiments over multiple random seeds.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claims stated clearly and supported by empirical evidence. Several rigorous benchmarks are considered across vision and language tasks using state-of-the-ar models.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We provide discussion of limitations in Appendix E.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
26

---

Justification:
No theoretical results are included.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our experiments are performed with fixed seeds for reproducibility and we have released the code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
27

---

Answer: [Yes]
Justification: Our experiments are performed with fixed seeds for reproducibility and we have released the code. Our datasets open benchmarks provided by the community.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https:
ctpps.cc/public//gg/ guwdee/iee///ee CCodeSubeesiaceecceee Sws.Suubiiiiin..... wsssonoliccyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe the main experimental setting in Section and additional model configuration information is located in Appendix A.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide error bars representing one standard deviation over three indepen- dent seeds. We state factors of variability captured by the error bars in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
28

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide compute statement in Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with NeurlPS Code of Ethics https /nnuriiur....// htcp/pllic//thcstuieeeeeeeeeeeee
Answer: [Yes]
Justification: We have reviewed the code of ethics and confirm that our work follows them in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We provide a discussion of broader impacts in Appendix E.
The experiments in the paper are aimed at understanding modern machine learn- ing algorithms and promoting their fair and equitable use. We have included a discussion of social impacts in Appendix E.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
29

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
The experiments in the paper are aimed at understanding modern machine learning algorithms and promoting their fair and equitable use. We believe the methodologies described in the paper do not have high risk for misuse, but nevertheless have included discussion of social impacts in Appendix E.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We credit creators of datasets and models used in the paper via citation and additionally in Appendix A.2.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
30

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have released the code and license information with additional documenta- tion located in the code.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not perform experiments with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not perform experiments with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
31

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
32


Reviews:
[{"review_id": "PfNoSvH7Jj", "reviewer": "Reviewer_Rwfv", "summary": "This paper experimentally analyzes the impact of class balancing on group robustness.", "questions": "See the weaknesses", "limitations": "Yes, they have addressed.", "rating": 2, "confidence": 5, "soundness": 1, "presentation": 1, "contribution": 1, "strengths": "This research contributes to the community by investigating spurious correlations in machine learning models' reliance.", "weaknesses": "- Due to significant variations in experimental results across datasets, generalization based on these results appears challenging. The case-by-case description of experimental outcomes seems more akin to a technical report rather than a conference paper.\n\n- The format of the citations is incorrect. The incorrect basic formatting makes the paper appear unprofessional.\n\n- The Related work subsection should be a section.\n\n- Lines 112-117 are not essential for the main content of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We warmly thank Reviewer Rwfv for their comments and suggestions. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.\n\n## Weakness 1\n\nWe thank the reviewer for the relevant comments. While robustness performance indeed differs across datasets, our explicit goal is to elucidate these nuances, analyze their causes and effects, and provide the community with a rigorous foundation for further research. Furthermore, we do find a potentially generalizable pattern for our class-balancing results. Specifically, we investigate two new failure modes of class-balancing which depend on the *particular group/class structure of each dataset*: (1) mini-batch upsampling and loss upweighting experience catastrophic collapse with standard hyperparameters when imbalance is large, and (2) removing data to create a class-balanced subset can harm WGA when a small minority group is present in the majority class.\n\nWe further investigate the generalizability of our results in the global response. In particular, our observations remain consistent even when tested on new class-balancing methods, without group labels for model selection, and in new settings including two different model families.\n\nFinally, we remark that case-by-case descriptions of experimental outcomes have strong precedent in the community. For example, Izmailov et al. [8] investigate the case-by-case effects of architecture, pretraining dataset, and regularization on multiple benchmark datasets. Similarly to us, they analyze how their results vary across different datasets and leverage their insights to improve model performance and make practical recommendations.\n\n## Weakness 2\n\nWe believe the reviewer is referencing the fact that we utilized alphabetical/lastname citations instead of numerical citations. We respectfully refer the reviewer to line 150 in the NeurIPS 2024 Formatting Instructions, which states “Any choice of citation style is acceptable as long as you are consistent.” If the reviewer was referencing some other aspect of our citations, please feel free to clarify the specific concerns, and we would be happy to address them during the discussion phase.\n\n## Weakness 3, Weakness 4\n\nWe thank the reviewer for the suggestions and we will include them in our updated version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DEUZ5ENGd7", "reviewer": "Reviewer_E5Mf", "summary": "The paper tries to study the fundamental properties of fine-tuning DNNs and worst group accuracy in the presence of spurious correlations. The effort focuses on revealing nuances that were not clear. They conducted experiments with both 2 vision and 2 NLP datasets with spurious correlations and subgroup labels present. Within the authors' setup, the experiments discuss the (1) impacts of class-balancing on group robustness (2) proposed approach to mitigate (3) model scaling's impact and (4) spectral imbalance's effect.", "questions": "1. Authors have put summaries at the end of each experiments. However, do the authors have a clearer guidelines for what practitioners should do under each \"nuanced\" scenarios, or just suggest there are more \"nuanced\" scenarios?\n\n2. Datasets authors used are all binary classifications, except MultiNLI (trinary), what would the nuances look like in a realistic multi-class setting? Would it create more nuances? [1] has an not ideal but acceptable multi-class dataset with group labels.\n\n3. What would the nuances look like in a ViT or its variants setting? As ViTs are know to require significant pretraining and downstream finetuning.\n\n---------------------------------------\n[1] Spawrious [Lynch, et. al, 2023]", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper conduct experiments to challenge the belief of (1) \"overparameterization helps or hurts robustness\" and (2) show the impact of different class-balancing methods,\n\n2. They conducted comprehensive experiments to explore and reveal the \"nuances\" in different directions across vision & language tasks, and propose suggestions when facing those scenarios.\n\n3. They propose a mixture method to outperform 2 prior common practices, subsetting and mini-batch class-balancing upsampling. The central idea of the mixture method is interpolate subsetting and upsampling.", "weaknesses": "1. This paper aim to study the \"fundamental properties of fine-tuning\" but focused on ConvNext only and didn't experiment with another popular pre-trained model family such as ViTs.\n\n2. The paper keeps reiterating the \"nuanced\", however, I believe the those nuances are general prior beliefs, I would hope the authors can provide some ablations to clarify those nuances and potentially provide clear guidelines, since all methods have nuances.\n\n3. The findings of this paper have some dependencies, the success cases seem to depend on a specific or \"appropriate\" setup to work.\n\n4. The datasets the authors use has group labels in training set, the authors should at least study the how maybe other training paradigms such as GroupDRO, etc., fare in the authors setting and proposed method, otherwise I believe the scope would be relatively narrow.\n\n5. The datasets used are binary or trinary. In a more realistic setting multi-class setting, the nuances should be discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We graciously thank Reviewer E5Mf for their insightful comments and questions. We appreciate that the reviewer recognizes our comprehensive experiments which challenge existing notions in the literature and leverage previously unknown nuances to improve model performance. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.\n\n## Weakness 1, Question 3\n\nWe appreciate the suggestion to replicate our experiments with other popular pretrained model families. We have implemented these experiments in two settings: Swin Transformer [10], a more efficient variant of Vision Transformer [11] with three pretrained sizes available, and ResNet [12], the most popular convolutional backbone with five pretrained sizes available. Overall, we find our results are consistent across pretrained model families, with the model affecting the raw accuracies but typically not the relative performance of class-balancing techniques or scaling behavior. Please see the global response PDF for figures.\n\n## Weakness 2, Question 1\n\nWe appreciate the reviewer’s comments and we agree that we could be more explicit about our recommendations for practitioners. Below, we provide more practical recommendations based on the “nuances” we propose in our paper.\n\n1. Since group proportions are unknown in practice, a good proxy for which class-balancing technique to use is the size of one’s dataset. If the dataset is on the smaller side (a couple thousand points, like Waterbirds), one should use upsampling or mixture balancing. If the dataset size is larger than 100K (like CivilComments), subsetting or mixture balancing is recommended.\n\n2. With appropriate pretraining and class-balancing, model scaling is generally beneficial for group robustness. Therefore, practitioners should utilize the largest model appropriate for their use case (especially if it does not interpolate, like MultiNLI).\n\n3. Practitioners should typically apply a post-hoc group robustness technique like AFR [2] or SELF [3] after finetuning, since as seen in the global response, class-balancing can only push WGA so far. With that said, the performance of the initial finetuned model affects downstream WGA (especially due to representation quality for last-layer methods [8]), so it is still worthwhile to carefully select model pretraining parameters.\n\nWe would appreciate it if you could point out the general prior beliefs you refer to. The literature appears to reflect a lack of consensus even with respect to phenomena as seemingly simple as model scaling. For example, in our related work section, we discuss conclusions from the literature which state that model scaling could either be beneficial or detrimental to robustness performance, depending on the particular setting and assumptions used for analysis.\n\nIn the global response, we address the reviewer’s request for additional experiments/ablations by investigating an additional balancing technique (upweighting), proposing model selection without group labels, and performing additional experiments/ablations across two additional model families. If there is a specific ablation the reviewer wishes to see that has not been addressed in the global response, please let us know and we would be happy to address it during the discussion phase.\n\n## Weakness 4\n\nWe agree with the reviewer that a more comprehensive discussion of where our mixture balancing method lies within the broader context of group robustness methods is appropriate. We would like to clarify that, in contrast to Group DRO [9], our methods **do not** use group labels in the training dataset (they are discarded before training). Please see the global response for a more explicit comparison with Group DRO and other methods, where we clearly delineate if held-out data or group labels are utilized.\n\nFurthermore, while performing model selection with respect to worst-group accuracy is a common assumption in the literature [1, 4, 8, 9], it is nevertheless unrealistic when group labels are not available. We investigate several methods for model selection and conclude that our methods are robust to validation that does not use group labels (indeed, validation with respect to worst-class accuracy is sufficient). Please see the global response for results and analysis.\n\n## Weakness 5, Question 2\n\nWe appreciate the reviewer’s suggestion and we agree that it would be interesting to observe the behavior of our methods on multi-class datasets. However, the dataset Spawrious [15] suggested by the reviewer is class-balanced *a priori* and hence not suitable for evaluating class-balancing methods (similarly to MultiNLI). With that said, we were able to run model scaling experiments. We used the most rigorous O2O Hard split and include the results below:\n\n| ConvNeXt-V2 Version | WGA (3 seeds) |\n|---------------------|---------------|\n| Atto (3.4M params)  | 24.8 +/- 2.0  |\n| Femto (4.8M params) | 17.3 +/- 1.4  |\n| Pico (8.6M params)  | 44.3 +/- 2.6  |\n| Nano (15.0M params) | 38.8 +/- 2.7  |\n| Tiny (27.9M params) | 37.3 +/- 3.4  |\n| Base (87.7M params) | 57.1 +/- 4.0  |\n\nDespite the trend being less monotone than other datasets, it is clear that scaling pretrained and class-balanced models is broadly beneficial for robustness on Spawrious, especially as one nears the 100M parameters mark.\n\nWe anticipate that our class-balancing methods would generalize to multi-class datasets without additional nuance, since the failure modes we identify for each method remain the same no matter the number of classes. Specifically: (1) for upsampling, any small class will be over-represented during long training runs no matter the number of classes, so we expect overfitting to the minority group within this class and poor WGA as a result; and (2) for subsetting, any minority group within a large class will be severely downsampled no matter the number of classes, so subsetting will disproportionately harm that group.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jH10g5FEVD", "reviewer": "Reviewer_kRBT", "summary": "The authors propose a class balancing scheme that both discards samples from the majority classes and upsamples majority classes as a way to improve worst group accuracy/robustness.", "questions": "Did you tune the mixture ratio? Why don't you compare to other class balancing techniques apart from subsetting and oversampling?\nHave you tried other datasets or other group-imbalance settings?", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The authors propose a simple yet effective method that provides good performance for all of the datasets tested. The over-parametrization and spectral imbalance empirical analysis are of independent interest.", "weaknesses": "Limited empirical Evidence: Although the benchmark datasets are taken from prior work (Idrissi et. al)  and indeed popular,  most of the claims are substantiated by experiments on only three settings, in which the two baseline methods show a different behaviour. The claim that their method works better on a wider range of settings (without hyperparameter tuning) could be supported by more empirical evidence.\n\nRelevance of the Feature Spectral Analysis: Although it is interesting and informative/insightful, there is no discussion on how the spectral features change between the proposed method and baseline  methods (subsetting/upsampling) - i.e. , how the balanced is achieved. Thus it seems unrelated to the discussion about the proposed method.\n\nRatio ablation and choice: There is not much discussion about the choice of mixture ratio  (again this is important because there is emphasis in this method performing well *without* hyperparameter tuning). The ratio ablation included in the supplementary (figure 7) only includes two values per dataset (apart from the baselines). The two values shown do have similar performance so if this holds with more points, this could better illustrate the robustness of the method w.r.t this hyperparameter.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We warmly thank Reviewer kRBT for their detailed comments and questions. We appreciate that the reviewer recognizes the improved performance of our mixture balancing method and the independent interest of our model scaling and spectral analysis experiments. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.\n\n## Weakness 1, Question 3\n\nWe appreciate the reviewer’s request for further empirical evidence and evaluation settings. In the global response, we investigate an additional balancing technique (upweighting), propose model selection without group labels, and perform additional experiments/ablations across two additional model families. Please see the global response for results and analysis.\n\n## Weakness 2\n\nFor the reviewer’s request of connecting spectral analysis to class-balancing, we re-ran our spectral analysis for Waterbirds with all balancing methods from the paper. Overall, we found that the magnitude of the eigenvalues is significantly affected by the chosen class-balancing method. However, the relative ordering of minority/majority group eigenvalues is consistent across class-balancing techniques.\n\nWe note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class-balancing may bring about additional stability in the representation. Please see the global response PDF for figures.\n\n## Weakness 3, Question 1\n\nWe agree with the reviewer that while performing model selection with respect to worst-group accuracy is a common assumption in the literature [1, 4, 8, 9], it is nevertheless unrealistic when group labels are not available. In the global response, we address these concerns by investigating several methods for model selection without group labels -- using 3-4 mixture ratios per dataset -- and conclude that our methods are robust to validation without group labels (indeed, validation with respect to worst-class accuracy is sufficient). Please see the global response for results and analysis.\n\n## Question 2\n\nWe thank the reviewer for the question and we address it in the global response. In addition to subsetting, upsampling, and mixture balancing, we have investigated another common class-balancing technique called *upweighting* not studied by [7]. In this method, minority class samples are directly upweighted in the loss function by the class-imbalance ratio, and it is used by state-of-the-art group robustness algorithms including AFR [2]. We find that upweighting experiences a similar catastrophic collapse as upsampling, even though they are only equivalent *on average* over the upsampling probabilities (i.e., not in practice). Please see the global response PDF for figures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TKz9e0FQWN", "reviewer": "Reviewer_ax3f", "summary": "The subject of the work is over-reliance on spurious correlations, which can often lead to poor performance on minority groups. The authors identify two failure modes of common class-balancing techniques: (1) class-balanced mini-batch finetuning experiences catastrophic collapse with standard hyperparameters on benchmark datasets, and (2) finetuning on a balanced subset of data can harm WGA when a small minority group is present in the majority class.", "questions": "* Why was [1] referenced in Sec 3.2?\n* Why was [2] not referenced in Sec. 3.2?\n* What motivates the authors' choice to focus on subsetting and oversampling in Secs 3 and 4, and on spectral imbalance in 5, rather than a more general investigation of their proposed method's efficacy or a simple analysis of subsetting and oversampling in group robustness?\n\nSUMMARY\n\nAs the paper stands, I think it is a weak accept; all of the contributions seem to be well supported by the evidence, but none of them seem likely to have an overwhelming impact. I will consider raising my score, however, if the weaknesses I mention are addressed.", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors provide important incremental advances to our understanding of subsetting and upsampling in this subfield; I.E., subsetting can be effective except when there is a small minority group present in the majority class, and ERM with class-balanced upsampling experiences a catastrophic collapse in test accuracy over the course of training.\n2. The decision to study the impact of model scaling on worst-group accuracy in a new setting — finetuning pretrained models — which more closely resembles practical use-cases -- is a welcome incremental contribution.\n3. The finding documented in Fig. 5 that the largest eigenvalue in each dataset belongs to a minority group and minority group eigenvalues are overall larger than majority group eigenvalues within the same class is a useful, if not terribly surprising, extension of [4].", "weaknesses": "1. In 3.2, the authors overclaim the strengths of their proposed method and their investigation of prior work in this section is spotty. The authors mention [1] as prior work; however, Deep Feature Reweighting, the main method explored in [1], requires group labels, so it is unclear what aspect of this work they are referencing. The authors do not address Automatic Feature Reweighting [2] in this section, which resolves spurious correlations using a weighted loss on a held-out dataset drawn simply from the training distribution. [2] retrains the last layer only, with weights prioritizing datapoints on which the base model performs poorly. The authors cannot and do not claim superiority without comparisons to existing work; however, it would still be extremely informative to see how their mixture balancing method compares to that of [2].\n2. It would be helpful to have a section describing the motivation for why the authors chose to focus on subsetting and oversampling so heavily in the first section of the paper, included a section on spectral imbalance, but did not deeply investigate [3], which is also cited.\n\n[1] Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations\n\n[2] Simple and fast group robustness by automatic feature reweighting\n\n[3] Spuriosity Rankings: Sorting Data to Measure and Mitigate Biases\n\n[4] Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with Spectral Imbalance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We graciously thank Reviewer ax3f for their detailed comments, questions, and references. We appreciate that the reviewer recognizes our contributions to the understanding of class-balancing methods, robustness impact of scaling pretrained models, and spectral analysis of model representations. Below, we provide responses to each of the reviewer’s points, combining weaknesses and questions as appropriate.\n\n## Weakness 1, Question 1, Question 2\n\nWe thank the reviewer for the insightful comments. Our intention was to avoid a comprehensive discussion of last-layer retraining methods, as we exclusively focus on model finetuning without held-out data or group labels. Our results are not directly comparable to those of DFR [1] and AFR [2] as they train on held-out data (up to 40K additional points) not included in standard model finetuning, as well as group labels (for training in [1] and model selection in [2] -- see the global response for mixture balancing model selection without group labels). On the other hand, since models finetuned with class-balancing are used as initializations for last-layer retraining [3], we expect our deeper understanding of finetuning phenomena to influence more sophisticated robustness algorithms.\n\nOur key reason for discussion of DFR [1] in Section 3.2 is not to compare to last-layer retraining, but instead to compare our mixture balancing method to their technique of averaging the last layer weights over multiple independently-sampled class-balanced data subsets, which like our proposed method intends to increase exposure to majority group samples without over-sampling the minority group. To our knowledge, this is the only technique in the literature which attempts to balance this trade-off. We did not cite AFR [2] in Section 3.2 because they do not mention this averaging technique, instead using a method akin to upweighting, and therefore make no attempt to increase exposure to majority class samples without over-sampling the minority class. Their approach of upweighting points on which the model does poorly is closely related to Just Train Twice (JTT) [4] and orthogonal to our methods. We will clarify these points in the paper.\n\nWith that said, we agree with the reviewer that a more comprehensive discussion of where our mixture balancing method lies within the broader context of group robustness methods is appropriate. Please see the global response for further discussion.\n\n## Weakness 2, Question 3\n\nWe thank the reviewer for the detailed questions. While we considered including a simple theoretical analysis of subsetting and upsampling in class-balancing, a preliminary investigation revealed some interesting connections to the literature which we believe merit their own, purely theoretical submission in the future. In particular, we observed that while upsampling and upweighting have similar WGA dynamics, both differ greatly from subsetting, in contrast to [5] which proved a broader theoretical equivalence between the three techniques in the population setting. We believe theoretically investigating this discrepancy is an interesting direction, but it requires the development of some new techniques to analyze WGA dynamics when under-represented data is seen repeatedly by the model during training, which would be out of scope for this paper. We will add further technical discussion in our updated version.\n\nFor the reviewer’s request for further investigation of class-balancing methods, we investigated an additional balancing technique (upweighting), proposed model selection without group labels, and performed additional experiments/ablations across two additional model families. Please see the global response for results and analysis.\n\nFor the reviewer’s request of connecting spectral analysis to class-balancing, we re-ran our spectral analysis for Waterbirds with all balancing methods from the paper. Overall, we found that the magnitude of the eigenvalues is significantly affected by the chosen class-balancing method. However, the relative ordering of minority/majority group eigenvalues is consistent across class-balancing techniques. We note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class-balancing may bring about additional stability in the representation. Please see the global response PDF for figures.\n\nFinally, we did not discuss Spuriosity Rankings [6] because it is focused on explainability/interpretability methods for discovering spurious features, which is orthogonal to our contributions. If the reviewer has a specific comparison they would like us to make with [6], we would be happy to address their concerns during the discussion phase.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We graciously thank all reviewers for their time and insights. Here, we provide new comparisons and experiments of interest to multiple reviewers.
## Additional class-balancing technique (all reviewers)
In addition to subsetting, upsampling, and mixture balancing, we investigated another common class-balancing technique called *upweighting* not studied by [7]. In this method, minority class samples are directly upweighted in the loss function by the class-imbalance ratio, and it is used by state-of-the-art group robustness algorithms including AFR [2]. We found that upweighting experiences a similar catastrophic collapse as upsampling, even though they are only equivalent *on average* over the upsampling probabilities (i.e., not in practice). Please see the attached PDF for figures.
## Model selection without group labels (all reviewers)
While performing model selection with respect to worst-group accuracy is a common assumption in the literature [1, 4, 8, 9], it is nevertheless unrealistic when group labels are not available. To address this, we re-ran mixture ratio tuning with respect to both worst-class accuracy [13] and the recently proposed *bias-unsupervised validation score* [14], which do not use any group labels for model selection. We performed model selection over at least 3-4 mixture ratios per dataset, and below we list the ratio which maximized each metric as well as its average WGA over 3 seeds.
| Validation Metric            | Group Labels | Waterbirds    | CelebA     | CivilComments |
|----------------------------|--------------|---------------|------------|---------------|
| Bias-unsupervised Score [14] | No           | 3.31:1 (79.9) | 1:1 (74.1) | 3:1 (77.6)    |
| Worst-class accuracy [13]    | No           | 2:1 (81.1)    | 1:1 (74.1) | 3:1 (77.6)    |
| Worst-group accuracy         | Yes          | 2:1 (81.1)    | 1:1 (74.1) | 3:1 (77.6)    |

Overall, both worst-class accuracy and the bias-unsupervised validation score performed well for our use case; in fact, similarly to the commonly used worst-group-accuracy score. All of the scores enable tuning of the mixture ratio without any group labels.
## Additional model families (all reviewers)
We appreciate the suggestions to replicate our experiments with other popular pretrained model families. We implemented these experiments in two settings: Swin Transformer [10], a more efficient variant of Vision Transformer [11] with three pretrained sizes available, and ResNet [12], which has five pretrained sizes available.  Overall, we find our results are consistent across pretrained model families, with the model affecting the raw accuracies but typically not the relative performance of class-balancing techniques or the impact of model scaling. Please see the attached PDF for figures.

## Connection of spectral analysis to class-balancing (kRBT, ax3f)
We thank the reviewers for their suggestion to compare spectral properties among different class-balancing techniques. We re-ran our spectral analysis for Waterbirds with all balancing methods from the paper. Overall, we found that the magnitude of the eigenvalues is significantly affected by the chosen class-balancing method. However, the relative ordering of minority/majority group eigenvalues is consistent across class-balancing techniques. Please see the attached PDF for figures.

We note that the most drastic changes in the spectrum are induced by the subsetting method, which has the worst WGA by far for the Waterbirds dataset. These results suggest that optimal class-balancing may bring about additional stability in the representation.

## Contextualization with broader group robustness methods (E5Mf, kRBT, ax3f)

We agree with the reviewers that a more comprehensive discussion of where our mixture balancing method lies within the broader context of group robustness methods is appropriate. We view mixture balancing as a soft upper bound on the performance of the simple baseline of ERM with class-balancing alone, as we select the optimal interpolation between subsetting and upsampling. We naturally expect this simple baseline to still perform worse than more sophisticated (and often more computationally intensive) group robustness methods. We compare the WGA of referenced methods below:

| Method                   | Held-out Data | Group Labels | Waterbirds | CelebA | CivilComments |
|--------------------------|---------------|--------------|------------|--------|---------------|
| DFR [1]                  | Yes           | Yes (train)  | 91.1       | 89.4   | 78.8          |
| AFR [2]                  | Yes           | Yes (val)    | 90.4       | 82.0   | ---           |
| Group DRO-ES [8, 9]      | No            | Yes (train)  | 90.7       | 90.6   | 80.4          |
| Just Train Twice [4, 7]  | No            | No           | 85.6       | 75.6   | ---           |
| Mixture balancing (ours) | No            | No           | 81.1       | 74.1   | 77.6          |
| ERM (no balancing)       | No            | No           | 79.4       | 48.3   | 58.7          |

Keeping in mind that the goal of mixture balancing is **not** to achieve state-of-the-art group robustness performance, we can see that the addition of held-out data and group labels improves WGA significantly over JTT and mixture balancing on Waterbirds and CelebA. However, on CivilComments, mixture balancing is competitive with state-of-the-art robustness methods, corroborating the conclusion of [3] that *class-balancing is sufficient for group robustness* in some cases.

In terms of guidelines for practitioners: our results show they should typically apply a post-hoc group robustness technique like AFR [2] or SELF [3] after finetuning, since class-balancing can only push WGA so far. With that said, the performance of the initial finetuned model affects downstream WGA (especially due to representation quality for last-layer methods [8]), so it is still worthwhile to carefully understand model pretraining parameters.

Meta-review: This paper is concerned with spurious correlations, which are closely related to the issue of poor performance on minority data slices or groups. Typical solutions involve re-balancing through a variety of means, but this can accidentally cause issues in certain settings (but not in others). The authors dig into these issues via carefully-crafted experiments, leading to a solid set of insights. As an example, the authors describe behavior from previous published work that was mysterious: using a popular previous method decreases the worst group accuracy during training; this is explained by overfitting on repeatedly-sampled minority points. The authors also provided new approaches inspired by these observations; these are simple but powerful (for example, the mixture approach). 

Reviews for this paper were somewhat mixed. One reviewer was quite negative; however, I felt that the authors were able to respond to all of the concerns raised. Others were very positive. I tended to agree with the latter—the paper has a nice contribution overall. Papers that dig into critical details and explain difficult-to-understand behaviors are very valuable, and this paper does that in spades.
Decision: Accept (poster)