Simple data balancing achieves competitive worst-group-accuracy
Editors: Bernhard Sch¨olkopf, Caroline Uhler Kun Zhang
Abstract
We study the problem of learning classifiers that perform well across (known or unknown) groups of data.
After observing that common worst-group-accuracy datasets suffer from substantial imbalances, we set out to compare state-of-the-art methods to simple balancing of classes and groups by either subsampling or reweighting data.
Our results show that these data balancing baselines achieve state-of-the-art-accuracy, while being faster to train and requiring no additional hyper-parameters.
In addition, we highlight that access to group information is most critical for model selection purposes, and not so much during training.
All in all, our findings beg closer examination of benchmarks and methods for research in worst-group-accuracy optimization.
1. Introduction
Machine learning classifiers achieve excellent test average classification accuracy when both training and testing data originate from the same distribution (Vapnik, 1995; LeCun et al., 2015).
In contrast, small discrepancies between training and testing distributions cause these classifiers to fail in spectacular ways (Alcorn et al., 2019).
While training and testing distributions can differ in multiple ways, we focus on the problem of worst-group-accuracy (Sagawa et al., 2019).
In this setup, we discriminate between multiple classes , where each example also exhibits some (labeled or unlabeled) attributes .
We call each class-attribute combination a group , and assume that the training and testing distributions differ in their group proportions.
Then, our goal is to learn classifiers maximizing worst test performance across groups.
Optimizing worst-group-accuracy is relevant because it reduces the reliance of machine learning classifiers on spurious correlations (Arjovsky et al., 2019), that is, patterns that discriminate classes only between specific groups (Shah et al., 2020; Geirhos et al., 2018, 2020).
The problem of worstgroup-accuracy is also related to building fair machine learning classifiers (Barocas et al., 2019), where groups may have societal importance (Datta et al., 2014; Chouldechova, 2017; Rahmattalabi et al., 2020; Metz and Satariano, 2020).
Maximizing worst-group-accuracy is an active area of research, producing two main strands of methods (reviewed in Section 3).
On the one hand, there are methods that consider access to attribute information during training, such as the popular group Distributionally Robust Optimization (Sagawa et al., 2019, gDRO).
On the other hand, there are methods that consider access only to class information during training, such as the recently proposed Just Train Twice (Liu et al., 2021,
JTT).
Unsurprisingly, methods using attribute information achieve the best worst-group-accuracy.
But, since labeling attributes for all examples is a costly human endeavour, alternatives such as JTT are of special interest when building machine learning systems featuring strong generalization and requiring weak supervision.
This work takes a step back and studies the characteristics of four common datasets to benchmark worst-group-accuracy models (CelebA, Waterbirds, MultiNLI, CivilComments).
In particular, we observe that these datasets exhibit a large class imbalance which, in turn, correlates with a large group imbalance (Section 2).
In light of this observation, we study the efficacy of training systems under data subsampling or reweighting to balance classes and groups (Section 4).
Our experiments (Section 5) provide the following takeaways:
• Due to class or attribute imbalance, simple data balancing baselines achieve competitive performance in four common worst-group-accuracy benchmarks, are faster to train, and require no additional hyper-parameters.
• While we obtained the best results by balancing groups, simple class balancing is also a powerful baseline when attribute information is unavailable.
• Access to attribute information is most critical for model selection (in the validation set), and not so much during training.
• We recommend practitioners to try data subsampling first, since (i) it is faster to train, (ii) is less sensitive to regularization hyper-parameters, and (iii) has a stable performance during long training sessions.
• Given the efficacy of subsampling methods, question the mantra “just collect more data”, particularly when optimizing for test worst-group-accuracy (as opposed to the classic goal of optimizing test average accuracy).
In essence, we beg for closer examination of both benchmarks and methods for future research in worst-group-accuracy optimization.
Popular worst-group- accuracy benchmarks
We consider datasets { ( x i , y i , a i ) } ni =1 , where each example is a triplet containing an input x i , a class label y i , and an attribute label a i .
The sequel studies four popular worst-group accuracy benchmarks that follow this structure.
• CelebA (Liu et al., 2015; Sagawa et al., 2019) consists of images of aligned celebrity faces.
Each face image is annotated with multiple traits.
Here, our task is to classify if the person has blond hair.
The attribute indicates whether the person in the image is male or female. [Table]
exhibit large class ( y ) and group imbalance.
In particular, class probabilities shift significantly when conditioning on the attribute ( a ) value.
For instance, the CelebA dataset has only 15% of examples of class “blond”.
Moreover, the probability of “blond” is different when the attribute value is “female” ( 24% ) or “male” ( 2% ), creating a spurious correlation.
• Waterbirds (Wah et al., 2011; Sagawa et al., 2019) contains images of birds cut and pasted on different backgrounds.
The task is to classify specimens into water birds or land birds.
The attribute indicates whether the bird appears on its natural habitat or not.
• MultiNLI
(Williams et al., 2017; Sagawa et al., 2019) is a dataset containing pairs of sentences.
The task is to classify the relationship between the two sentences as being a contradiction, an entailment, or none of the two.
The attribute indicates the presence of the negation words: ’no’, ’never’, ’nobody’ or ’nothing’ in the second sentence.
The presence of these words makes the ’contradiction’ label more likely in this dataset.
We’ve kept this limited list of negation words to have comparable results with previous literature.
• CivilComments (Borkan et al., 2019;
Koh et al., 2021) is a dataset containing comments from online forums.
The task is to classify whether a comment is toxic or not.
There are multiple attributes annotating the content of each comment, relating to: male, female, LGBT, black, white, Christian, Muslim, other religion.
Following Sagawa et al.
(2019), we consider a coarse version of the CivilComments dataset to train gDRO.
This coarse version provides a binary attribute indicating if any of the eight attributes listed above appears in the comment.
Table 1 lists the number of examples per class and group for these four datasets.
The data reveals that three out of four datasets exhibit a large class imbalance, and that all of them exhibit a large group imbalance.
Furthermore, these imbalances are highly correlated: class probabilities vary significantly when conditioning on the attribute value.
In the Waterbirds dataset, class probabilities invert when swapping attribute values.
In the MultiNLI dataset, the class “contradiction” is much more likely when there is a negation in the second sentence.
In CelebA, it is unlikely to find examples from the “male” class when the attribute is “blond”.
Therefore, these datasets contain spurious correlations helpful to discriminate only between some groups.
When such groups represent most of the dataset, learning algorithms latch onto the spurious correlations, and resort to memorization to achieve zero training error (Sagawa et al., 2020).
These observations immediately motivate training classifiers under data subsampling or reweighting to balance out classes and groups.
After group balancing, we expect the spurious correlations between classes and attributes to vanish, improving test worst-group-accuracy.
Before exploring the efficacy of these simple balancing baselines, we first review some popular state-of-the-art methods proposed to optimize worst-group-accuracy.
3. Popular worst-group-accuracy methods
We review three popular methods appearing in the literature of worst-group-accuracy optimization.
• Empirical Risk Minimization (Vapnik, 1995, ERM ) chooses the predictor minimizing the empirical risk 1 n (cid:80) ni
=1 ℓ ( f ( x i ) , y i ) .
ERM does not use attribute labels.
• Just Train Twice
(Liu et al., 2021, JTT ) proceeds in two steps.
First, JTT trains an ERM model for a small amount of epochs T .
Assuming that this “simplistic” ERM model classifies examples based on spurious correlations, its errors should correlate to the subset of examples where the spurious pattern does not appear.
Following this assumption, JTT trains a final ERM model on a dataset where the mistakes from the “simplistic” ERM model appear λ up times.
JTT does not use attribute labels.
• Group Distributionally Robust Optimization (Sagawa et al., 2019, gDRO ) minimizes the maximum loss across groups: sup q ∈ ∆ | G | (cid:80) | G | g
=1 q g n g (cid:80) n g i =1 ℓ ( f ( x i ) , y i ) , where G =
Y × A is the set of all groups, ∆ | G | is the | G |− dimensional simplex and n g is the number of examples from group g ∈ G contained in the dataset.
Therefore, gDRO uses attribute labels.
In particular, gDRO allocates a dynamic weight q g to the minimization of the empirical loss of each group, proportional to its current error.
Other methods
The literature in robust optimization is flourishing, so the comparison of all possible methods renders itself impossible.
Some further examples of robust learners not using attribute information are Learning from Failure (Nam et al., 2020), the Too-Good-to-be-True prior (Dagaev et al., 2021), Spectral Decoupling (Pezeshki et al., 2020), Environment Inference for
Invariant Learning (Creager et al., 2021), and the G EORGE clustering algorithm
(Sohoni et al., 2020).
Other examples of methods that use attribute information include Conditional Value at Risk (Duchi et al., 2019), Predict then Interpolate (Bao et al., 2021), Invariant Risk Minimization (Arjovsky et al., 2019), and a plethora of domain-generalization algorithms (Gulrajani and Lopez-Paz, 2020).
4. Simple data balancing baselines
Given the class and group imbalance shown in Table 1, we explore the effectiveness of four data balancing baselines on worst-group-accuracy:
• Subsampling large classes ( SUBY ), so every class is the same size as the smallest class.
Such subsampling is performed once and fixed before training starts.
This baseline does not use attribute labels.
• Similarly, subsampling large groups (Sagawa et al., 2020, SUBG ), so every group is the same size as the smallest group.
This baseline does use attribute labels.
• Reweighting the sampling probability of each example, so mini-batches are class-balanced in expectation ( RWY ).
This baseline does not use attribute labels.
• Similarly, reweighting the sampling probability of each example, so mini-batches are groupbalanced in expectation ( RWG ).
This baseline does use attribute labels.
Toy example
To motivate our baselines, we consider a synthetic logistic regression example (Sagawa et al., 2020, Section 5.1.).
The classes y ∈ {− 1 , +1 } are dependent on two attributes a core , a spu ∈ {− 1 , +1 } with correlations ρ core , ρ spu ∈ [ − 1 , 1] .
While ρ core remains invariant between the training and the test data, ρ spu varies from the training to the test set.
Each attribute dictates a Gaussian distribution over input features.
In particular, each input example x is a concatenation of the following three components,
[Equation]
where
[Equation]
where ( σ 2 , γ i ) are the variance and scale of each of the features.
The scaling factors γ i control the rate at which the model learns each feature: the larger γ
i , the faster the model learns about x i .
Moreover, the noise features x noise are independent from the class labels y , and therefore uncorrelated in expectation .
However, in over-parameterized settings where d is greater than the number of training examples, there exists an empirical correlation between the noise features and the class labels.
Therefore, and depending on the values of γ i , over-parametrized models can exploit noise features to memorize training examples on their path to achieving zero training error.
Figure 1 implements one instance of this example where ρ core = 1 and ρ spu = 0 . 8 .
Furthermore, γ spu = 4 , γ core = 1 , γ noise = 20 , and σ = 0 . 15 .
Given these correlation and scaling coefficients, the spurious feature is learnable much faster than the core and noise features.
As shown on the first two panels of Figure 1, a vanilla ERM model mainly relies on spurious features, and therefore achieves poor test worst-group-accuracy.
On the other hand, subsampling the majority group (SUBG) decorrelates the spurious feature from the labels, leading to a model that relies on the core feature, discards the spurious feature, and achieves good test worst-group-accuracy.
While reweighting groups (RWG) also solves this toy example, one has to pay special attention to model selection, since test worst-group-accuracy degrades as the number of training iterations increases.
We note that the probability of misclasifying when using just x core is P ( yx core < 0) =
P ( y (1 + yσZ ) < 0) =
P (cid:0) Z < − yσ (cid:1) =
1 − Φ (cid:0) − yσ (cid:1) =
1 − Φ (cid:0) 1 σ (cid:1) (since Φ is symmetrical) which is 10 − 11 in this particular case.
This means that the problem is separable using just x core with high probability.
- -    x spu * gamma spu - -    x c o r e * g a mm a c o r e
I II III IV
ERM - -    x spu * gamma spu - -    x c o r e * g a mm a c
o r e
I
II
III
IV
SUBG - -    x spu * gamma spu - -    x c o r e * g a mm a c
o r e
I
II
III
IV
RWG
Iterations 0.
w o r s tg r oupacc u r ac y
ERM
Iterations 0.
w o r s tg r oupacc u r ac y
SUBG
Iterations 0.
w o r s tg r oupacc u r ac y RWG valid test train adding 5. Experiments We implement ERM, JTT, gDRO, SUBY, SUBG, RWY and RWG, as well as the necessary infrastructure to experiment on the Waterbirds, CelebA, MultiNLI, and CivilComments benchmarks.
Our implementation follows closely the ones of (Sagawa et al., 2019, gDRO) and (Liu et al., 2021, JTT).
For the image datasets Waterbirds and CelebA, we train ResNet50 models pre-trained on ImageNET (He et al., 2016) using the SGD optimizer.
For the NLP datasets MultiNLI and CivilComments, we train BERT models pre-trained on Book Corpus and English Wikipedia (Devlin et al., 2018) using the
AdamW optimizer (Loshchilov and Hutter, 2017).
We tune the learning rate in { 10 − 5 , 10 − 4 , 10 − 3 } , weight decay in { 10 − 4 , 10 − 3 , 10 − 2 , 10 − 1 , 1 } , and JTT’s λ up in { 4 , 5 , 6 , 20 , 50 , 100 } .
We tune the batch size in { 2 , 4 , 8 , 16 , 32 , 64 , 128 } for CelebA and Waterbirds and { 2 , 4 , 8 , 16 , 32 } for MultiNLI and CivilComments.
We tune JTT’s T in { 40 , 50 , 60 } for Waterbirds, { 1 , 5 , 10 } for CelebA, and { 1 , 2 } for MultiNLI and CivilComments.
We fix gDRO’s η to 0 .
We allow 50 random combinations of hyper-parameters for each method and dataset.
In contrast to previous literature, we run each hyper-parameter random combination 5 times to compute the average and standard deviation of the reported test worst-group-accuracies.
These error-bars relate to data shuffling, data subsampling, and random initialization of last linear layers.
We train Waterbirds for 360 epochs, CelebA for 60 epochs, and both MultiNLI and CivilComments for 7 epochs.
We select best models (hyper-parameter combination and epoch) by computing the worst-group-accuracy on a validation set.
The table of the best hyper-parameters is in table 5 in the appendix.
Our code is available at https://github.com/facebookresearch/BalancingGroups .
5.1. Results
Table 2 reports test worst-group-accuracies for all methods and benchmarks.
While some methods do not require the use of attribute labels for training, we emphasize that all methods require a validation set with attribute labels to perform model selection.
As shown in the second row of Table 3, the performance of all methods degrades when one performs model selection based on the average validation accuracy (e.g., not assuming access to attribute labels in the validation set).
Table 2 also lists the number of hyper-parameters tuned by each method, four being the minimal achieved by
ERM, SUBG, SUBY, RWG, RWY (learning rate, weight decay, batch size, early stopping epoch).
In summary, reweighting baselines perform competitively: SUBG scores only 1.7 points less than gDRO on average, while RWY scores 2.1 points more than JTT on average.
Subsampling SUBY performs below its reweighting counterpart RWY, while SUBG outperforms RWG by a small margin.
Finally, the fourth row of Table 3 reports the running times employed to find the best models discussed above.
This shows that ERM and RWY is 1.9 times faster than its competitor JTT, and that RWG is 1.
times faster than its competitor gDRO.
The subsampling baselines are 3.8 times faster than JTT and 7 times faster than gDRO while only having slightly worse worst-group-accuracy.
5.2. Analysis of exceptions
Table 2 shows two exceptions to our claim that balancing baselines have competitive results with more complicated methods.
The performance of gDRO largely surpasses the rest on MultiNLI with a 8.4 points difference with the second best method.
We conjecture that gDRO is performing best in Multi-NLI because of the nature of the dataset.
Indeed, the spurious attribute “presence of: ’no’, ’never’, ’nobody’ or ’nothing’ in the second sentence” only helps the classifier in a small proportion
[Table]
#HP is the number of tuned hyper-parameters.
Simple data balancing baselines match the performance of state-of-the-art methods within error bars, with two exceptions.
Green backgrounds indicate datasets where algorithms exhibit a statistically different performance at a significance level of α = 0 .
This is determined using an Alexander-Govern test for the equality of means of multiple sets of samples with heterogeneous variance (Alexander and Govern, 1994).
All algorithms not using attribute information perform similarly with the exception of SUBY, under-performing in CivilComments.
All algorithms using attribute information perform similarly, with the exception of gDRO being better on MultiNLI.
[Table]
of the data (see table 1).
It is therefore not a dominating spurious correlation and the classifier still needs to extract other features to achieve good accuracy.
Group dro minimizes a soft maximum of the group losses, thus enabling a more flexible way to penalize this mild spurious correlation.
Therefore, the simpler baselines are less effective in this case because balancing either through subsampling or reweighting is a strong measure that is meant to completely decorrelate the spurious feature with the label.
The gains of getting rid of this mild spurious correlation are canceled by the harsh capacity control imposed by balancing.
Indeed, subsampling throws away a big proportion of the data, and reweighting has a similar effect when stopped early, which is the case for MultiNLI (training for only a few epochs).
We provide an illustrative example of the above in figure 3 in the appendix.
SUBY, on the other hand, seems to consistently underperform on the datasets in this paper, even performing worse than ERM.
We therefore don’t recommend its use in practice.
One possible explanation for its poor performance is the fact that worst group samples could easily not be picked at all during subsampling, because of their small number, and lead to poor performance on those groups.
From these exceptions, we conclude that simple balancing baselines might only work on simpler cases where the spurious correlation is present in a big majority of the data.
In more nuanced cases, such as MultiNLI, where the spurious correlation doesn’t dominate the data, more complex methods such as gDRO might be useful.
Table 4 summarizes the top 5 best hyper-parameters for each dataset and method, together with their associated test worst-group-accuracies.
We make three observations.
First, the range of test worst-group-accuracies (top 1st worst-group-accuracy - top 5th worst-group-accuracy) is smaller for methods accessing group information.
This means that if we used less than 50 hyperparameter tuning runs, we would still get a good worst-group-accuracy, which implies that these methods are less sensitive to hyper-parameter choice.
Second, methods are most sensitive to the choice of learning rate, with multiple sets of top-5 runs preferring the same value.
Third, RWG prefers small-capacity models by choosing small learning rates, high weight decays, and early epochs.
5.4. Evolution of worst-group-accuracy during training
Figure 2 shows the evolution of the train and test worst-group-accuracy for all methods and datasets.
First, we observe that RWY, RWG, and gDRO peak in worst-group-accuracy early, and then degrade in performance.
On the contrary, SUBG has a more stable performance during long sessions of training, for all datasets and especially in Waterbirds.
Second, there is a consistent generalization gap for all methods and datasets regardless of regularization strength.
Since some models reach 100% train worst-group-accuracy, they must have memorized some of the worst-group examples.
5.5. Differences between reweighting and subsampling groups
While similar at a first glance, training models with data subsampling or reweighting may lead to different decision boundaries due to different interactions with regularization (Sagawa et al., 2020).
For instance, in the absence of regularization, logistic regression converges to the maximum margin classifier (Soudry et al., 2018) in linear realizable problems.
Therefore, since any strictly positive reweighting of example probabilities has no effect on support vectors , we conclude that regularization
[Table]
is necessary for reweighting to have any effect on the resulting classifier.
In contrast, subsampling changes the support of the dataset, likely removing support vectors and affecting the final classifier even in the absence of regularization .
While the previous are proven facts only for linear problems, Table 3 shows similar findings for the over-parametrized deep models used in this work.
In particular, the reweighting methods [Figure] (RWG and RWY) and gDRO, both degrade when removing regularization.
That is consistent with findings of Słowik and Bottou (2021) where they establish close theoretical connections between gDRO and reweighting mechanisms.
On the other hand, the subsampling method (SUBG) maintains its performance in the long run without the need of regularization.
Byrd and Lipton (2019); Sagawa et al.
(2019) reach a similar conclusion: strong regularization is necessary to benefit from data reweighting.
Table 4 ratifies this, since SUBG prefers smaller weight decays than RWG.
The superior performance of SUBG suggests two conclusions.
On the one hand, we favor subsampling under tight computational budgets, since the resulting models are faster to train and depend less on regularization hyper-parameters.
On the other hand, the considered benchmarks seem solvable with small data, showing that either the tasks at hand are too easy or that the reweighting methods fail to make good use of all the data .
To conclude, we comment on one similarity between early-stopped reweighting and subsampling.
Reweighting uses a weighted random sampler to produce minibatches containing an equal amount of minority and majority examples (in expectation).
This weighted random sampler is with replacement due to the scarcity of minority examples.
Therefore, for a small amount of epochs, the model has likely seen all the minority examples while only observed a subsample of the majority examples.
More specifically, the number of observed unique majority examples after sampling k times is on average N maj (1 − (1 − 1 N maj ) k ) , where N maj is the number of majority examples contained in the dataset.
Given that the best RWG model stops after 3 epochs for CelebA, it observes only 44% of majority examples on average, which amounts to subsampling the majority group.
6. Conclusion
We have shown that simple data balancing baselines achieve state-of-the-art performance in four popular worst-group-accuracy benchmarks.
While balancing groups leads to best worst-groupaccuracy, balancing class labels obtains competitive performance even in the absence of attribute information.
We have also revisited the critical importance of having access to attribute information in the validation set, necessary to perform model selection based on worst-group-accuracy.
Therefore, hyper-parameter tuning for domain generalization under weak supervision remains an open problem (Gulrajani and Lopez-Paz, 2020).
We have illustrated some differences between data reweighting and data subsampling, advocating to try data subsampling first, since (i) it is faster to train and thus allows more hyper-parameter exploration, (ii) has less reliance on regularization, and (iii) has a more stable performance during long training sessions.
All in all, our results raise two questions.
First, are our current worst-group-accuracy benchmarks expressing a real problem?
If so, is there room to outperform simple data balancing baselines in these datasets? Appendix A. Supplementary material [Table]
The highlight in green at position i shows the norm of the gradient the contradiction’s class probability with respect to the input embedding at position i .
i.e. ∥∇ e i ˆ p ( label = contradiction | x ) ∥ , where ˆ p is the model’s output probability given a sentence x .
It is a proxy to the importance of each word in the final decision of the model as to whether it is a contradiction or not.
Group DRO picked up on the crucial word “remembering” in the first sentence and therefore could correctly classify the relation between the two sentences as a contradiction.
The other methods lack this “feature” and therefore fail.
This sentence serves only an illustrative purpose.