Boosting worst-group accuracy without any group annotations
{vbardenha,tifreaa,fan.yang}@ethz.ch
Abstract
Despite having good average test accuracy, classiﬁcation models can have poor performance on subpopulations that are not well represented in the training data.
In this work, we introduce a criterion to estimate the accuracy on these populations. This allows us to design a procedure that achieves good worst-group performance and unlike previous procedures requires no group labels.
We provide a sound empirical investigation of our procedure and show that it recovers the worst-group performance of methods that use oracle group annotations.
Introduction
Machine learning models have been remarkably successful on a broad range of prediction tasks, when performance is measured with the average error on a holdout i.i.d.
test set.
However, good performance on average does not guarantee that the error is uniformly low on different subpopulations.
In applications where the accuracy on the subpopulations matters (e.g. instances where fair prediction is critical), metrics other than the average validation error are more informative.
One common way for such subpopulations to emerge is when a subset of the features is spuriously correlated with the target variable in data sampled from the training distribution [Geirhos et al., 2020, Minderer et al., 2020].
If prediction models use these spurious features for the decision rule, then they produce incorrect outputs on samples for which the spurious correlation does not hold, leading to subpopulations (groups) which are consistently predicted incorrectly.
Ideally, we want that estimators have both good average accuracy, and also good worst-group performance.
When the training distribution has imbalanced subpopulations (see Figure 1 Right), the Bayes-optimal classiﬁer has poor performance on groups that cannot be predicted correctly using the spurious feature. Unsurprisingly, estimating the Bayes-optimal classiﬁer via empirical risk minimization (ERM) also suffers from poor worst-group performance, with the detrimental impact of overparameterization being studied at length in Sagawa et al.
[2020b].
A number of works have made progress on mitigating the consequences of the group imbalance problem, but all these approaches require some degree of access to clean group membership labels for the training or validation data.
Some of the most effective methods are inspired by distributionally- robust optimization (DRO) [Sagawa et al., 2020a] or importance weighting/sampling [Sagawa et al., 2020a,b, Menon et al., 2021], They rely crucially on strict regularization and introduce a number of additional hyperparameters that need to be carefully tuned with respect to worst-group performance.
As a consequence, all of these methods require that oracle group labels are available for some data points.
However, collecting group labels is often a difﬁcult or sometimes straight-out impossible task in practice.
For instance, in real-world applications it is often not clear what exactly the spurious features are (e.g. they could be as inconspicuous as the laterality artifacts of an X-ray device [Zech et al., 2018]), which makes it impossible to determine how to partition data into groups.
Furthermore, even if we know what the spurious attributes are, it might not be possible to collect them during the data acquisition process.
Public or private institutions are often not allowed to hold records on sensitive information such as race, religion, sexual orientation etc.
Despite a surge in approaches that can achieve good worst-group accuracy without group annotations for the training data, the problem of model selection without requiring validation group labels remains a largely unaddressed challenge.
All prior works either use an annotated validation set for hyperparameter tuning [Nam et al., 2020, Liu et al., 2021] or they avoid model selection altogether and use hyperparameters identiﬁed by other works, known to lead to good worst-group performance [Sohoni et al., 2020, Creager et al., 2021].
Contributions.
In this work we address this issue and propose a model selection criterion that can be applied without any group labels.
To this aim we adapt a two-stage procedure introduced in prior work (e.g. [Nam et al., 2020, Liu et al., 2021, Bahng et al., 2020]) and devise proxy criteria for hyperparameter selection for the models trained during both stages.
Our experiments reveal that our method signiﬁcantly outperforms baselines that use no group annotated data and achieves similar performance to approaches that require group annotations on a variety of prediction tasks for image and tabular data sets.
Problem setting
In this section we describe the setting that we assume throughout our work.
We consider classiﬁcation problems in which the covariates are drawn from a mixture of groups, similar to Sagawa et al. [2020b].
All the samples from one group are assigned the same ground truth label, but a class can comprise several groups.
Elements from different groups differ by features that are not predictive of the class label for all samples. However, these features can be spuriously correlated with the class labels for samples coming from certain groups.
In Figure 1 we illustrate the setting for the simple case of 2D binary data with 2 groups per class, balanced classes and isotropic Gaussian group conditional distributions. The marginal distribution of covariates is given by x ∼ P X = (cid:80) y ∈Y (cid:80) g ∈{ 0 , 1 } P [ X | G = g, Y =
y ] P g | y P y . If the groups are balanced (i.e. P 0 | y =
P 1 | y , ∀ y ∈ Y ), then the Bayes classiﬁer f Bayes has simultaneously both optimal average and worst-group error.
However, if some groups are signiﬁcantly more represented in the training distribution than others (i.e. P 0 | y >> P 1 | y for some y ∈ Y ), then f Bayes is optimal with respect to the average error, but does not have the best error among all possible classiﬁers on the minority groups (Figure 1 right). This problem is particularly challenging when the spurious feature is highly correlated with the class label for the majority groups, meaning that a classiﬁer could easily rely on spurious attributes for its predictions.
This relationship between the worst-group and the average accuracy for scenarios with group imbalance is reminiscent of other well-studied trade-offs such as the one involving the standard and the adversarial robust risk [Tsipras et al., 2019]. Related work.
As Figure 1 shows, even when given inﬁnite training data and a well-speciﬁed hypothesis class that includes the Bayes classiﬁer, ERM still arrives at a solution with subpar worstgroup performance.
Instead, with group labels for the training samples, we change the original training distribution (e.g. via importance weighting/sampling) such that the Bayes classiﬁer of this new distribution has optimal worst-group performance.
Prior works propose implicit (e.g. GDRO [Sagawa et al., 2020a]) and explicit (e.g. generating minority points [Goel et al., 2020], removing majority points [Menon et al., 2021, Sagawa et al., 2020b]) ways to “alter” the training distribution.
Furthermore, a number of recent approaches do not use group annotations for the training data (e.g.
Liu et al.
[2021],Nam et al.
[2020],Creager et al.
However, since these approaches are sensitive to the choice of some crucial hyperparameters (e.g. regularization strength, weight values for importance weighting/sampling etc), effective model selection is of utmost importance. All prior approaches rely on access to group annotations for the validation set and select hyperparameters that lead to optimal worst-group accuracy.
Proposed method
We now sketch a general framework that covers a number of two-stage approaches that have been proposed for learning unbiased classiﬁers.
We then describe a procedure for selecting hyperparameters without using group annotations. 3.1 Overview of two-stage approaches for learning unbiased predictors
First stage – Detecting error-prone samples. In the absence of group annotations for the training set, the ﬁrst step consists in identifying a stand-in for the minority groups. Recall that the minority samples are also the ones for which the spurious correlation does not hold.
Assuming that the spurious feature is easier to pick up from the data than the core features (e.g. spurious features have a signiﬁcantly higher signal-to-noise ratio [Nam et al., 2020]), then one can recover the minority points by using a biased classiﬁer that relies more heavily on the spurious attribute for its predictions. One can achieve the desired biased predictors by using carefully tailored loss functions [Nam et al., 2020], or through strong regularization, such as early stopping [Bahng et al., 2020, Liu et al., 2021]. Training the biased ﬁrst-stage predictors amounts to solving the following optimization problem:
[Equation] 1) where T 1 denotes the set of possible early stopping times and Θ 1 is the set of all hyperparameter conﬁgurations that we consider. Ideally, the biased classiﬁer ˆ f t 1 ,θ 1 produces incorrect predictions at inference time on most of the minority samples. Thus, one can use the error set of ˆ f t 1 ,θ 1 as a surrogate for the minority groups.
We assign group label g = 1 to points in the error set of ˆ f t 1 ,θ 1 to get the annotated training set: [Equation] Similarly, if group labels are not provided for the validation set, we can annotate it to obtain ¯ V ( ˆ f t 1 ,θ 1 ) . Second stage – Correcting the mistakes of the biased predictor.
Once we have identiﬁed the error-prone training samples, we can use a procedure like importance sampling/weighting or GDRO to mitigate the group imbalance and ﬁnd classiﬁers that perform well uniformly across groups. For instance, one could use an importance weighted loss L IW to up-weigh minority samples, in a procedure that resembles boosting algorithms [Breiman, 1996]. Once again, these techniques introduce additional hyperparameters (e.g. regularization strength, importance weight values etc) and choosing good values for them is essential to achieving satisfactory worst-group performance. The optimization problem for training the de-biased predictor takes the following form:
[Equation] where T 2 and Θ 2 are the set of early stopping times and the hyperparameter grid for the second stage. 3.2 Prior work: Model selection with oracle group labels
Prior work [Nam et al., 2020, Liu et al., 2021] propose to tune hyperparameters by maximizing the worst-group validation accuracy directly, since group annotations are available on the validation set.
In this setting, there are two possible strategies for hyperparameter tuning, as illustrated in Table 1. One option is to consider separate model selection criteria for the two stages [Nam et al., 2020,
Bahng et al., 2020]. Hyperparameters for the ﬁrst stage are chosen so as to bias the classiﬁer towards making mistakes on minority-group samples, while predicting majority points well. Then secondstage hyperparameters are selected such that worst-group accuracy is maximized.
Alternatively, one can choose both ﬁrst and second-stage parameters using a single objective: improving the ﬁnal performance on minority groups [Liu et al., 2021]. 3.3 Proposed model selection without group annotations Model selection for the ﬁrst stage. We propose to ﬁx the regularization strength, and select the remaining hyperparameters so that the average validation accuracy is maximized, which implicitly promotes more biased classiﬁers.
Concretely, for a given model class, we only search for empirical risk minimizers in the set of predictors that can be obtained after only one epoch of gradient descent training. This severe early stopping restricts the set of possible classiﬁers to simple functions that can
[Table]
achieve better validation accuracy by relying on the easier to learn, spurious features, and ignoring the core features altogether. To construct the error set of the training data, we perform two-fold cross-validation and repeatedly extract samples with incorrect predictions from the holdout fold. Model selection for the second stage.
To approximate minority group accuracy we could measure the accuracy on the error set of the biased model trained during the ﬁrst stage. A naive approach that uses the accuracy on this error set as a criterion for hyperparameter tuning, leads to suboptimal results.
The reason is that one error set contains minority points, but also many majority samples that are predicted incorrectly (see Appendix G.2 for more details). Instead, we propose to use an ensemble of biased ﬁrst-stage predictors and the intersection of the error sets that they produce as a proxy for the validation minority group. This procedure runs the risk of removing some of the few minority points present in the validation set. Therefore, instead of the actual intersection of the error sets, we use a “soft” version of the set intersection operator: the average of the error set accuracies over the ensemble.
For an ensemble of K models, each producing different group annotations for the validation set ¯ V ( ˆ f i ) we can write the model selection criterion C ( f θ, { ¯ V ( ˆ f i ) } Ki =1 ) as follows, for an arbitrary function f θ with hyperparameters θ :
[Equation]
Recall that the group label g is 1 for the samples that belong to the error set of the biased predictor trained in the ﬁrst stage. By using this criterion we amplify the contribution of samples that are consistently predicted incorrectly by biased classiﬁers, while not discarding any of the samples in the error sets.
In our experiments, we ﬁx the ensemble size to 4 , and show in Appendix G how the performance of the proposed approach varies when we use smaller ensembles.
Experiments
We now show empirical results obtained with our procedure and provide insight into the role played by our modiﬁcations to each of the stages.
Data sets.
We highlight that one inconspicuous pitfall when developing new methodology is inadvertent overﬁtting, that is tailoring a method to particular data sets.
In order to prevent this, we only use two data sets for designing and analyzing our approach (i.e. Corrupted MNIST and
Waterbirds), and then run it on a number of other new data sets from various domains, ranging from tabular data (e.g. the Adult data set) to natural images (e.g. CelebA).
We stress that we do not change our approach in any way in order to improve performance on these unseen data sets.
Baselines.
There is no previous work that tackles the challenge of worst subgroup performance without using any subgroup information.
Thus, we compare our method to baselines with different degree of required group label information: ERM w/o group labels, ERM tuned with a validation set with group annotations (ERM WG), JTT and GDRO.
Detailed information on the tuning of the baselines can be found in Appendix D.
Main results.
The results of our empirical analysis are summarized in Table 2.
We point out that our method consistently outperforms the ERM baseline, the only other method that does not use any group annotations for training or model selection.
Moreover, our approach often achieves a worst-group accuracy comparable to JTT, which has the advantage of oracle group label knowledge for the validation data.
We note that with a group-annotated validation set even simple ERM training (ERM WG) can sometimes match the performance of the more sophisticated two-stage approaches like JTT.
[Table]
4.1 Ablation studies
[Figure]
In this section we break down our approach and give insights into why accurate model selection is possible even without any access to group labels.
Extreme regularization leads to biased classiﬁers.
We begin by showing that neural networks trained according to our proposed ﬁrst stage procedure are indeed biased towards making more mistakes on minority points.
We train a LeNet5 model to perform even/odd digit binary classiﬁcation on the Color MNIST dataset.
We project the input space into a 2D representation where one of the coordinates correlates with the core features, and the other with the spurious features (see Appendix C for more details on how we obtained the projection).
As shown in Figure 2, the classiﬁer trained for one epoch and with hyperparameters tuned for average validation accuracy is more aligned with a predictor that relies exclusively on spurious features. Therefore, our ﬁrst-stage classiﬁers will produce incorrect predictions primarily on the minority samples.
Conversely, a classiﬁer trained to convergence will resemble the Bayes optimal predictor that does not make mistakes uniformly over the minority groups, leading to error sets that contain fewer representative minority samples.
Precision and recall of detecting minority samples.
We show in Appendix F the precision and recall of our ﬁrst-stage predictors and how they compare to those of JTT. In particular, we note that our procedure for hyperparameter tuning achieves similar precision compared to JTT.
On the other hand, JTT has the advantage of using oracle group labels on the validation set, and hence, obtains slightly better recall. We stress, however, that ultimately our goal is not to accurately detect minority points, but rather to improve prediction on minority subpopulations.
As both JTT and our method show, an imperfect ﬁrst-stage error set (i.e. with low precision and/or recall) can still signiﬁcantly improve worst-group performance in the second stage on the data sets that we have considered.
Correlation between the C criterion and worst-group accuracy.
We show in Appendix G that the criterion in Equation 4 correlates well with the actual metric that we are interested in, i.e. worstgroup accuracy. Furthermore, we analyze the impact of using an ensemble of ﬁrst-stage models when computing the proxy metric C and ﬁnd that this procedure is necessary for selecting the optimal early-stopping time.
Discussion and future work
In summary, we introduce a model selection criterion that requires no group label information and show that it successfully achieves good worst-group performance on a variety of real-world data sets.
Our current approach relies on a number of implicit assumptions.
For instance, when selecting the error set in the ﬁrst stage, data with large amounts of label noise could potentially render our method infeasible.
We leave a more thorough investigation on ways to prevent the catastrophic effects of noise for future work.
Moreover, we implicitly assume that we know which classes contain minority groups, which helps especially with model selection in the second stage of our method.
Relaxing this assumption is another interesting direction for future research.
A Optimality in the presence of subgroups
In this section we conceptually and theoretically analyse a data model for the subgroup performance challenge.
In this discussion we have two goals: First, analyse the fundamental performance limit of the Bayes classiﬁer for certain data generating processes with rare subgroups.
Second, analyse to what extend this fundamental limit is relevant for the behaviour of over-parameterized models and in that light contribute a different perspective to the question of memorization and feature learning raised in Sagawa et al.
[2020b] and Menon et al.
To achieve these goals, we introduce and parameterize an explicit example of the data generating process in form of a Gaussian mixture model (GMM).
For this GMM we demonstrate that the Bayes classiﬁer has low subgroup performance and show that an over-parameterized random feature model learns a decision rule that is very similar to the Bayes classiﬁer in the Gaussian mixture setting with rare subgroups.
B Explicit data model
Following the general description of the data generating process in 2 we deﬁne a concrete parametric example.
We take the data generating model used in Sagawa et al.
[2020b] to get sample data for experiments on implicit memorization.
In this case each conditional distribution is a Gaussian with diagonal co-variance matrix.
This "Gaussian mixture with minority subgroups" is deﬁned as the following.
Deﬁnition B.1 (Gaussian Mixture with Minority Subgroups) .
The data of our Gaussian mixture with minority subgroups x ∈ R d with d ∈ 2 Z + is distributed as a mixture of four Gaussian with each of them distributed according to p x ( x | s, c ) ∼ N ( µ s,c , Σ) .
Of the d dimensions d/ 2 are spurious features x s ∈ R d/ 2 and d/ 2 are core features x c ∈ R d/ 2 .
The mean values µ s,c are dependent on the core c ∈ { 0 , 1 } and the spurious category ( s ∈ { 0 , 1 } ) .
For simplicity of calculation we assume symmetry around #» 0 with the centers µ 0 ,[-
 =-]
[-[ #» k , #» k ] , µ 1 , 
 = [ − #» k , − #» k ] , µ 1 , 0 = [ − #» k , #» k ] , µ 0 , 1 = [ #» k , − #» k ] .-][Equation]
The variance Σ = diag #» σ s , #» σ c is a diagonal matrix with the spurious variance σ s < σ c .
The class conditional distribution are mixtures of two of the groups each:
[Equation]
We deﬁne that each class label contains a minority group 0 < p ( a = 1 | c = 0) (cid:28) p ( a = {+
+} | c = 0) and 0 < p ( a = 0 |
c = 1) (cid:28) p ( a = 1 | c = 1) .
For this model it is simple to calculate the Bayes classiﬁer and analyse its properties.
Using Bayes rule we have
[Equation]
In this simple case of binary classiﬁcation, the classiﬁer can be rewritten as an inequality.
[Equation] [Table]
The worst subgroup performance for this case transforms to:
[Equation]
The integration cannot be solved analytically for Gaussian distributions, but numerical approximations are simple especially for low dimensions.
In high dimensions one can use Monte Carlo simulations to get a good approximation of the integral.
We compute numerical values for d = 2 through numerical double integration computation.
In the ﬁrst step we analyse the worst-group and average accuracy values that Sagawa et al.
[2020b] observed for overparameterized models.
While they attributed low worst-group performance to the overparameterized models, we hypothesize that the accuracy values are not a model phenomenon but correspond to properties of the underlying data distribution and the optimization objective.
To have exact results we carry out the analysis for d = 2 .
The analysis is to change the relative probability of minority points in the range from [0 . 5 , 0 . 99] and change the variance σ c .
The resulting graph is displayed in (4).
The general tendency aligns with Sagawa et al.
[2020b] who described that increasing label spurious feature correlation leads to a sharp drop in worst subgroup performance of overparameterized models.
Thus, the behaviour previously attributed to overparameterization properties arises for the Bayes classiﬁer (compare 4).
The same holds true for the variance analysis where the beavhiour of the overparameterized model and the Bayes classiﬁer align.
Furthermore, we analyse the average accuracy if only the core or spurious feature can be used for classiﬁcation.
Additionally, we compute numerical approximations through sampling for the exact settings from Sagawa et al.
[2020b] with d = 200 and evaluate the overparameterized random feature model from their work (a logistic regression on a random feature model of size 3500) in terms of worst subgroup and average accuracy.
We then compare it to the Bayes classiﬁer.
We ﬁnd that the over-parameterized model almost attains the performance of the Bayes classiﬁer in terms of worst subgroup as well as average accuracy (compare 3).
Consequently, we conclude that worst subgroup performance can be a consequence of Bayes Error minimization on the data distribution.
The experiments and analysis demonstrate that in the synthetic setting of Sagawa et al.
[2020b] the
Bayes Classiﬁer has low worst subgroup performance. In addition, they demonstrate that the low worst subgroup performance of overparameterized models is not due to problematic model behaviour but due to the underlying data distribution.
If improving worst subgroup performance is the target, it is important to realize that there usually is a trade-off with average accuracy.
Thus, the optimization target needs to be shifted away from the cross-entropy loss as a proxy for the Bayes Error to achieve good worst subgroup performance.
C Mapping images to spurious-core representation
In a general setting it is computationally infeasible to analyse distributional properties over image spaces.
Especially, if the data is not generated with full control over the manifold.
However, we utilize our assumptions on the underlying data distribution and limited control over the data generating process to get a 2D- representation of core and spurious features for some image datasets.
This analysis is only feasible if core and spurious feature can be manipulated separately, this is the case for the worst subgroup datasets of Color-MNIST and waterbirds.
To facilitate the analysis, it is necessary to generate a mapping from the image domain x to the core feature x c and the spurious feature x s which are assumed to be one dimensional.
Given this mapping it is possible to estimate the distributions p ( x c ) and p ( x s ) , visualize the Bayes Classiﬁer, compute its accuracy and compare it accuracy increases with the visualization and accuracies of CNN classiﬁcation models.
Furthermore, it is possible to get an intuition of the model behaviour in the ﬁrst stage of our procedure.
C.1 Identifying Core and Spurious Feature Distribution
First, we train a mapping f from the core features to a one-dimensional representation.
The closer f represents the ranking of the ground truth conditional distribution p ( y | x c ) meaning p ( y = 1 | x 1 c ) > p ( y = 1 | x 2 c ) =
⇒ f
( x 1 c ) >
f ( x 2 c ) of the labels given the input, the better.
In practice, we take the training images from MNIST, add a neutral background color to all images and train a LeNet5 LeCun et al.
to minimize the cross-entropy loss on the odd / even classiﬁcation task.
For waterbirds, we take the cropped bird images and place them on a black background.
We take the non-normalized output of the last layer to get the mapping
[Equation]
Second, for Color-MNIST we deﬁne a bijection between the spurious feature color and a onedimensional space by replacing the one color channel of the background with a value between 60 and 120.
[Equation]
For waterbirds we repeat the ﬁrst step, but instead of having a classiﬁer learning the core feature, this time the classiﬁer learns the spurious feature.
We take only the background images of water and land and train a classiﬁer to classify land and water correctly with a mapping:
[Equation]
Third, we combine these two mappings and create a complete mapping of the input images to a 2d representation with the spurious
[Equation]
Fourth, we create an artiﬁcial grid of images by combining the input factors.
For Color-MNIST it is simple to add a different background color to the images from the validation set.
For waterbirds we repeat the original data creation pipeline from Sagawa et al.
[2020a] and place the birds in the centre of the landscapes of the background.
We assume that x c and x s remain identical when we combine the inputs.
For Color-MNIST this assumption is satisﬁed.
For waterbirds some issues arise when the bird in the foreground covers important parts of the background, or the background itself contains birds or birdlike objects.
However, this issue arises rarely.
C.2 Analysis of the Representation Quality
The analysis of the mapping can be conducted somewhat like the analysis of disentanglement features in VAE.
However, here images are not created but sorted in a learned representation space.
The ﬁrst method is to create a traversal starting with images that have a very low expression of the core feature over ambiguous examples to very high expression of the core feature.
We can visually check if the hierarchy matches with the human perception.
The second method is to check for model consistency.
Namely, how strong do models learned with different initialization differ in their recovered distribution.
The traversal in 5 shows that the middle image is clearly ambiguous and the intended number could be even (4) or odd (9).
And the numbers to the right and left of the centre image look less stereotypical and clearly written than the ones at the corners.
We check three model random seeds and compute the pairwise Pearson correlation for the data points conditioned on the ground truth class label.
This is important as for us it is not only relevant if the models learn a similar classiﬁcation rule, but also if the feature expression within each class carries some ground truth.
We ﬁnd signiﬁcant correlation of above 80%.
The results for these consistency check appear promising and indicate that the learned mapping can indeed be used for qualitative and with some uncertainty even quantitative analysis.
D Hyperparameter Tuning D.1 Baselines
For the ERM baselines we tune over learning rate, batch size, weight decay and if the dataset has class imbalance over reweighted and standard cross entropy loss.
For the baselines tuned for average accuracy we select the hyperparameters and early stopping time with respect to average accuracy on the validation set.
We then run three seeds of the best hyperparameter (7, 9, 15) and report accuracy values on the test set.
For the ERM baselines with worst-group early stopping we use the same tuning procedure.
The only adaptation is that we select the model and early stopping time based on the best worst subgroup accuracy on a validation set.
It is important to recognize that the high worst-group accuracy might not be consistent over runs with different seeds.
In this setting high worst subgroup accuracy arises as a random deviation from the optimization target.
Thus, hyperparameter search to a certain extent is the search over the best seed and parameter conﬁguration.
However, the good worst-group performance in the validation set generalizes to the test data, given sufﬁcient minority samples in the validation set.
For the JTT baselines with worst-group early stopping we have a larger grid. In addition to the standard optimizer parameters, learning rate, batch size and weight decay there are the method speciﬁc selection epoch and upweighting factor.
Selection epoch is the epoch from which training mistakes of the ﬁrst stage are selected for upweighting in the second stage.
The upweighting factor is the weighting that these samples receive during training of the second stage.
We select the best model at best early stopping time based on worst subgroup performance on the validation set.
For the GDRO baselines with worst-group early stopping we perform a grid search that in addition to the standard optimizer parameters, learning rate, batch size and weight decay selects the best value for generalization adjustment.
The generalization adjustment is an additional group speciﬁc upweighting factor for the loss that increases the importance of groups with few samples.
D.2 Ours
For our method hyperparameter tuning is carried out in two stages.
In the ﬁrst stage we ﬁx the model class to models that can be trained from scratch with gradient descent in one epoch. The metric is overall accuracy on the validation set.
The grid consists of different values for learning rate, batch size, optimizer, and weight decay.
We carry out hyperparameter tuning in a twofold cross validation because the models in our ﬁnal ensemble are also trained on half of the data and evaluated on the other half.
In the second stage we tune model selection and early stopping for our proxy metric.
In addition to the standard parameters of learning rate, weight decay and batch size, we tune for the best upweighting factor for the train error set samples.
To improve precision, we experiment with forming a hard intersection of two train error sets to select the samples to be upweighted.
The proxy metric only selects this option for the poverty dataset that is in general very hard to optimize and might contain a lot of label noise in the minority groups.
For the results of our approach, we rerun the second stage with four different seeds (7,9,15,22) with the same hyperparameters and select the best early stopping epoch based on the proxy metric.
The resulting standard deviations are in Table 4.
[Table] E Label Noise/ No subgroups
By design our procedure assumes a minority group in each label category.
If this assumption is violated, the error set for the class without minority subgroups will consist of samples that are not classiﬁed correctly by the "biased" classiﬁer for other reasons than the misleading spurious feature. One possibility is the presence of samples with uniform label noise.
In the smooth intersection proxy, the samples classiﬁed wrongly by all the ensemble models will dominate the metric.
Optimizing for this metric will inevitably lead to poor average accuracy and hardly improvement on the worst subgroups and the proxy metric.
The violation of the "minority in each class assumption" might be identiﬁed by a sharp decrease in average accuracy without much of an increase in the proxy metric.
If this violation is detected in one class, the proxy metric should be limited to the other classes.
If there are minority subgroups in the class uniform label noise is less of a problem.
The error set will be a mixture of minority and label noise samples.
The label noise samples are in general impossible to optimize for. Thus, the proxy metric will be affected by noise from these points, but it will still be correlated with the minority accuracy.
The boosting and upweighting of the label noise samples during training seem to be even less of a problem.
In CelebA, where we almost exclusively upweight label noise for the class of non-blond people the achieved overall and average accuracy are still high.
In future research it would be interesting to transform these qualitative insights from CelebA to a quantitative insight on other datasets. Especially the hypothesis that the poverty dataset contains a large amount of label noise and how this inﬂuences the results, would be interesting to analyse. For this direction one can utilize synthetic datasets where the amount of label noise can be controlled.
F Precision Recall
Our procedure is targeted at achieving high majority accuracy in the ﬁrst stage.
Thus, avoiding many majority points to be present in the error set and achieving high precision.
However, for some datasets one epoch is enough for a well optimized classiﬁers to pick up on both the core and the spurious signal. In this case, minority points are also classiﬁed correctly which results in a lower recall.
In the development procedure we analysed using an ensemble of models to create error sets on the training data
[Figure]
but there was no clear improvement across datasets.
Thus [Figure] we decided to keep the procedure as simple as possible. [Figure]
Precision
Recall
Precision
Recall
[Figure]
Ours 0.
JTT 0.
CelebA
Ours
[Figure]
JTT
[Figure]
Waterbirds
Ours 0.
JTT 0.
[Figure]
[Figure]
[Figure]
[Figure]
G Proxy Metric
The goal of the proxy metric used for model selection in the second stage is to provide a reasonable surrogate for the worst-group accuracy. Note that a good model selection metric does not necessarily need to be a good estimate of the exact worst-group accuracy. Instead for hyperparameter selection it sufﬁces if the two quantities have a similar ranking especially for large values. We split the analysis of the metric into two sections.
First, we analyse the correlation across the whole range of parameters and epochs during the hyperparameter search. In this section small and large values of the proxy metric and the ground truth receive the same importance.
Second, we analyse the selection of the early stopping time, given the selected optimizer parameters and thus zoom in on the important range of values. G.1 Identiﬁcation of hyperparameters
To analyse the hyperparameter selection performance we compute the rank correlation between the different metrics and the ground truth on the validation set. We use the worst-group validation accuracy as the ground truth. We consider models obtained for all the conﬁgurations in the hyperparameter search grid. Our proposed surrogate model selection criterion greatly outperforms the average validation accuracy criterion, as indicated in the table below.
The number of ﬁrst-stage biased models that are used for computing the criterion C introduced in Section 3.3 does not inﬂuence the rank correlation with the ground truth signiﬁcantly. However, as we will see in the next section, the ensemble plays a role for selecting the regularization strength.
[Table]
The right early stopping time is crucial for the performance of the procedure. Given good optimization hyperparameters the model learns majority as well as minority samples well. Worst-group accuracy ﬂuctuates relatively strongly and thus selecting an epoch with good worst-group accuracy is crucial.
To compare the metrics on this task, we compute the validation accuracy if the best hyperparameter run is early stopped on either of the metrics. The displayed results are averaged over the four seed runs of the second stage. The result is that the epoch selection with our selected metric is more accurate across most datasets and the ﬂuctuation between the seeds is smaller.
For waterbirds this does not hold a possible reason is the extremely small amount of just 18 points in the smallest minority group in the validation set. Accordingly, the indirect up-weighting of the points that appear across validation error sets generally makes the selection procedure more stable. [Table] G.3 Effect of ensembling in the proxy metric
Next, we try to get an intuition why the ensembling is beneﬁcial when selecting early stopping time. To do so we compare precision and recall for the points that appear only in one error set and those that appear in all four error sets (are in the intersection). The relative frequency of minority points in the intersection is larger than in the individual error sets.
This shows in a generally increased precision.
However, the absolute frequency is also strongly reduced resulting in a lower recall. The agreement between error sets from different models is larger for minority points than for majority points. Thus, their importance is increased through the ensembling in our metric, which might explain the better results.
[Table]
H Datasets
Waterbirds: Designed by Sagawa et al.
[2020a] to analyse ERM behaviour with respect to spurious and core features the waterbirds dataset is a binary classiﬁcation task of birds in to land birds and water birds.
The dataset is synthetically created by extracting these birds from their original photos and placing them in front of water or land backgrounds.
The spurious feature is the background with waterbirds in front of water in 95% of their occurrence and land birds in front of land in 95% of their occurrence.
While the classiﬁer must learn features related to coloring and form of the bird to correctly classify the images, the easy shortcut is to follow the color and structure of the background.
The ofﬁcial publication comes with train, validation and test split and the experiments follow these ofﬁcial splits with one adaptation.
The ofﬁcial validation and test sets are balanced with respect to subgroups for accurate estimation of worst subgroup performance.
To analyse our approach under realistic conditions an i.i.d. validation set is necessary that was created by splitting the test set in half randomly. One half is used as a new test set. The other half is under sampled according to the training distribution to create this i.i.d.
validation set.
To ensure comparability no data augmentation is used, and the images are rescaled and cropped to 224 x 224 pixels.
Afterwards, the images are normalized with the mean and standard deviation values of the ImageNet pretraining.
CelebA
The CelebA dataset Dua and Graff [2017] is a collection of face images of celebrities annotated with a variety of attributes, face landmark points and many more.
Following Sagawa et al.
[2020a] we take the classiﬁcation of hair color into blonde and non-blonde as an example for the worst subgroup question.
The spurious feature in this situation is the annotated gender.
In contrast to waterbirds there is only one rarely occurring subgroup: blond males.
The easy feature to pick up is the gender (hair length) and the core feature hair colour is harder to learn for the classiﬁer.
To ensure comparability no data augmentation is used.
The images are cropped to have a square format and then rescaled to the 224 x 224 pixels resolution.
The last preprocessing step is a normalization with the image net mean and standard deviation values.
Poverty The poverty dataset Koh et al.
[2021] is a collection of 8 channel satellite imagery from various regions in Africa with annotated information regarding country and if the picture is taken in an urban/ rural area.
While the original task is to predict a continuous poverty index and generalize over country related domains, we create a new target and a new train, val, test split.
We bin the continuous index into the classes "wealthy" and "poor" by taking 0 as a decision boundary and discarding images with a value between [-0.1,0.1] and try to classify images into these categories.
Furthermore, we identify the urbanization label as spurious feature and analyse classiﬁcation performance on the subgroups.
The train, val, test splits are generated i.i.d. from all available data with 50% of the data used for training.
No data augmentation and no transformation of the inputs is applied.
C-MNIST
The corruption biased MNIST dataset was developed on the basis of the work in Goel et al.
[2020] that combine the standard MNIST dataset LeCun and Cortes [2010] and the zigzag corruption of Corrupted MNIST.
The classiﬁcation task is to tell if a digit is even or odd and the spurious feature is having or not having a corruption.
In comparison to the previous dataset we made a few adoptions: we reduced the fraction of subgroup samples from 5% to 1% to make the task more difﬁcult and increased the overall amount of samples to demonstrate that the root for the problem is not to be found in too few samples.
The ofﬁcial publication of MNIST and corruption MNIST has a train test split and we only use data from the training split for training and split the testing data into validation, i.i.d. validation and testing.
No data augmentation and no transformation of the inputs is applied.
Color-MNIST
The colour biased MNIST dataset is created based on the idea of Kim et al.
They color the MNIST letters with different colors while leaving the background unaffected.
In our adaptation we add color to the background and pose the binary classiﬁcation task of even and odd numbers.
The colored background is created as a spurious feature with two base colors with some random color noise, with frequent color and even/odd combinations and rare once.
In the experiments we vary the label, color correlation and sometimes use monocolor MNIST for demonstrative experiments.
The ofﬁcial publication of MNIST has a train test split and we only use data from the training split for training and split the testing data into validation, i.i.d. validation and testing.
No data augmentation and no transformation of the inputs are applied.
Adult The Adult dataset is taken from Jiang and Li [2011] and the pre-processing and subgroup deﬁnition was replicated from Lahoti et al.
The dataset is a tabular dataset extracted from a census in the USA.
The binary prediction tasks is to estimate if the individuals yearly income is above or below 50.000 USD based on the given characteristics such as education and marital status.
We deﬁne the gender and race to be protected attributes and deﬁne the following subgroups per class:
G = {{ male,non-black } , { female,non-black } , { male,black } , { female,black }}