Learning Debiased Classiﬁer with Biased Committee
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Neural networks are prone to be biased towards spurious correlations betweenclasses and latent attributes exhibited in a major portion of training data, whichruins their generalization capability.
This paper proposes a new method for trainingdebiased classiﬁers with no spurious attribute label.
The key idea of the methodis to employ a committee of classiﬁers as an auxiliary module that identiﬁes biasconﬂicting data, i .
e ., data without spurious correlations, and assigns large weights tothem when training the main classiﬁer.
The committee is learned as a bootstrappedensemble so that a majority of its classiﬁers are biased as well as being diverse,and intentionally fail to predict classes of bias-conﬂicting data accordingly.
Theconsensus within the committee on prediction difﬁculty thus provides a reliablecue for identifying and weighting bias-conﬂicting data.
Moreover, the committee isalso trained with knowledge transferred from the main classiﬁer so that it graduallybecomes debiased along with the main classiﬁer and emphasizes more difﬁcultdata as training progresses.
On ﬁve real-world datasets, our method outperformsexisting methods using no spurious attribute label like ours and even surpassesthose relying on bias labels occasionally.
Introduction
Most supervised learning algorithms for classiﬁcation rely on the empirical risk minimization (ERM)principle [37].
However, ERM has been known to cause a learned classiﬁer to be biased towardspurious correlations between predeﬁned classes and latent attributes that appear in a majority oftraining data [11].
In the case of hair color classiﬁcation, for example, when most people withblond-hair ( i . e ., target class) are female ( i . e ., latent attribute) in a dataset, a classiﬁer learnedby ERM exploits female as a shortcut for the classiﬁcation due to its spurious correlation withblond-hair , and often mis-classiﬁes non-blonde-haired women as blond-hair in consequence.
We call data with such spurious correlations and holding a majority of training data bias-guidingsamples , and the other bias-conﬂicting samples , respectively.
The issue of model bias has often beenaddressed by exploiting explicit spurious attribute labels [21, 27, 34, 1, 36, 35, 42] or knowledgeabout bias types given a priori [2].
However, these methods are impractical because such supervisionand prior knowledge are costly, and the methods demand extensive post hoc analysis.
Hence, a body of research has been conducted for learning debiased classiﬁers with no additional labelfor spurious attributes [39, 26, 28, 31, 22, 25].
A common approach in this line of work is to employan intentionally biased classiﬁer as an auxiliary module [28, 31, 22, 25].
In this approach, samplesthat the biased classiﬁer has trouble handling are regarded as bias-conﬂicting ones and assigned largeweights when used for training the main classiﬁer to reduce the effect of bias-guiding counterparts.
Although it has driven remarkable success, this approach has drawbacks due to the use of a singlebiased classiﬁer.
First, the quality of the biased classiﬁer could vary by hyper-parameters [28] and itsinitial parameter values [10].
Further, data that the biased classiﬁer fails to handle could include notonly bias-conﬂicting samples but also bias-guiding ones, which differs by the quality of the classiﬁer.
[Figure]
These drawbacks limit the reliability and performance of debiasing methods depending on a singlebiased classiﬁer, as demonstrated in Figure 1.
To overcome these limitations, we propose a new method using a committee of biased classiﬁers asthe auxiliary module, coined learning with biased committee (LBC).
LBC identiﬁes bias-conﬂictingsamples and determines their weights through consensus on their prediction difﬁculty within thecommittee.
To this end, the committee is built as a bootstrapped ensemble, i .
e ., each of its classiﬁersis trained from a randomly sampled subset of the entire training dataset.
This strategy not onlyguarantees the diversity among the classiﬁers, but also lets a majority of the classiﬁers be biasedsince random subsets of training data are highly likely to be dominated by bias-guiding samples.
Accordingly, a majority of the committee tends to classify bias-guiding samples correctly and failto deal with bias-conﬂicting ones.
The consensus on prediction difﬁculty within the committee thusgives a strong cue for identifying and weighting bias-conﬂicting samples.
Also, using the consensusof multiple classiﬁers enables LBC to be robust to the varying quality of individual classiﬁers andconsequently to focus more precisely on bias-conﬂicting samples, as shown in Figure 1.
Moreover, unlike the biased classiﬁer trained independently of the main classiﬁer in the previouswork, the committee in LBC is trained with knowledge of the main classiﬁer as well as the randomsubsets of training data to serve the main classiﬁer better.
Speciﬁcally, the knowledge is distilledin the form of classiﬁcation logits of the main classiﬁer [17], and each classiﬁer of the committeeutilizes the knowledge as pseudo labels of training data other than its own training set.
We expectthat this strategy allows the committee to become debiased gradually so that it does not give largeweights to easy bias-conﬂicting samples, i .
e ., those already well handled by the main classiﬁer, andfocuses more on difﬁcult ones.
Note that, even with this strategy, the classiﬁers of the committee arestill biased differently due to their different training sets with ground-truth labels.
Finally, we further improve the proposed method by adopting a self-supervised representation asthe frozen backbone of the committee and the main classiﬁer.
Since self-supervised learning is notdependent on class labels, it is less affected by the spurious correlations between classes and latentattributes, leading to a robust and less-biased representation.
Also, by installing the committee andthe main classiﬁer on top of the representation, the classiﬁers can be implemented efﬁciently in bothspace and time while enjoying the rich and bias-free features given by the backbone.
LBC is validated extensively on ﬁve real-world datasets.
It substantially outperforms existing methodsusing no bias label and even occasionally surpasses previous arts demanding bias labels.
We alsodemonstrate that all of the main components, i .
e ., the use of the committee, its training with knowledge transfer, and the self-supervised learning, contribute to the outstanding performance.
The maincontribution of this paper is four-fold:
• We present LBC, a new approach to learning a debiased classiﬁer with no spurious attribute label.
The use of consensus within the committee allows LBC to address limitations of previous workrelying on a single biased classiﬁer.• We propose to learn the committee using knowledge of the main classiﬁer, unlike the previouswork whose auxiliary modules do not consider the main classiﬁer.• We investigate the potential of self-supervised learning for debiasing, and ﬁnd that it is a solidyet unexplored baseline for the task.
• LBC demonstrates superior performance on ﬁve real-world datasets.
It outperforms existingmethods using no additional supervision like ours and even surpasses those relying on spuriousattribute labels occasionally.
Related work
2.1 Debiasing with prior knowledge on bias
A majority of previous work on learning debiased classiﬁers mitigates the bias by leveraging explicitlabels on bias types [21, 27, 34, 1, 36, 35, 42].
For instance, Kim et al.
[21] build a debiasedmodel to classify digits by leveraging RGB color labels explicitly in the Colored MNIST dataset.
Other methods are designed to handle predeﬁned domain-speciﬁc bias types ( e .
g ., texture bias in
ImageNet) [38, 4, 2].
Recent methods try to reduce annotation costs for bias supervision by only utilizing a small set ofbias-labeled data.
For instance, Nam et al.
[32] and Jung et al.
[20] train auxiliary bias predictorwith a small set of bias-labeled data and assign pseudo bias labels to the entire training set usingits prediction.
These methods have two inherent limitations.
First, they cannot handle biases whosetypes are not predeﬁned in training.
Second, manually annotating all bias types is often expensiveand laborious.
Even obtaining a small set of bias labels could be expensive since bias labels existingin a real-world dataset is often long-tailed [29] or multi-labeled [
Debiasing without spurious attribute labels
Recent debiasing methods without any bias supervision [18, 7, 31, 25, 22, 28, 6] are based on theassumption that attributes of malignant bias are easier to learn than those of target classes.
Throughthis assumption, previous methods identify the amount of biases within the samples and train arobust models by emphasizing bias-conﬂicting samples.
To identify biases of samples, previous workleverage a high gradient of the latent vector [18, 7] or employ an intentionally biased classiﬁer asan auxiliary module [31, 22, 25] by using generalized cross-entropy (GCE) loss [41].
Liu et al.
[28]consider misclassiﬁed samples as bias-conﬂicting samples, and Creager et al.
[6] derive bias-guidingand bias-conﬂicting partition that maximally violates the invariance principle.
On the other hand,
Bahng et al.
[2] propose a method tailored to the texture bias, building a debiased model to learnindependent features from a biased model in which receptive ﬁeld size is limited to capture the texturebias in image classiﬁcation.
While LBC is also based on the aforementioned assumption, we ﬁrstly propose to identify biasconﬂicting samples and up-weight them through the consensus of a committee of the biased classiﬁers.
By exploiting the consensus of multiple classiﬁers, LBC is robust against the varying quality ofindividual classiﬁers and allows to learn debiased classiﬁers more reliably and effectively.
Proposed method
We propose a new method that learns a debiased classiﬁer with a committee of biased classiﬁers,dubbed LBC.
It ﬁrst learns a feature representation with self supervision, which is used as the frozenbackbone providing rich and bias-free features to downstream modules (Section 3.1).
Next, it trainsa committee of m auxiliary classiﬁers f 1 , f 2 , . . .
, f m and the main classiﬁer g on top of the selfsupervised representation (Section 3. (cid:2278) (cid:2907)(cid:2887)(cid:2889) (cid:1859) :
Bias-guiding sample : Bias-conflicting sample Sample weight (cid:1875) (cid:1858) (cid:2870) (cid:1858) (cid:2869) (cid:1858) (cid:3040) (cid:2278) (cid:2887)(cid:2889) (cid:1085) (cid:2278) (cid:2895)(cid:2888) Committee (cid:1875) (cid:851) correct (cid:1858) (cid:4666)(cid:1668)(cid:4667) Sample weighting
Knowledge distillation
[Figure]
Weightingfunction (cid:1710) (cid:1845) (cid:2869) (cid:1845) (cid:3040)
A committee of auxiliary classiﬁers f
i is trained on top of the backbone; a random subset of training data is assigned to each classiﬁer of the committee as labeled data for supervised learning (Eq.
The committee determines the weight of each training sample based on consensus of its members, i .
e ., the number of members that correctly predict the class of the sample (Eq.
The main classiﬁer g is trained using the weights by minimizing the weighted cross entropy loss (Eq.
In turn, knowledge of the main classiﬁer is transferred to the committee through knowledge distillation (Eq.
In a nutshell, the committee identiﬁes bias-conﬂicting samples and assigns them large weights toreduce the effect of bias-guiding samples during the training of the main classiﬁer.
To this end, thecommittee is trained as a bootstrapped ensemble of classiﬁers so that a majority of its classiﬁersare biased as well as diverse, and intentionally fail to predict classes of bias-conﬂicting samplesaccordingly.
Hence, the consensus within the committee on prediction difﬁculty of a sample ( e .
g .,the number of classiﬁers that fail to predict its class label) indicates how much likely the sampleis bias-conﬂicting, and is used to compute weights for training samples.
Moreover, the committeeis trained also with knowledge of the main classiﬁer so that it gradually becomes debiased alongwith the main classiﬁer and emphasizes more difﬁcult samples as training progresses.
Note thatthe committee is an auxiliary module used only in training and thus does not impose additionalcomputation or memory footprint in testing.
The overall process of LBC is illustrated conceptually in Figure 2 and given formally in Algorithm 1.
The following sections elaborate on each step of LBC.
3.1 Self-supervised representation learning
As the feature extractor, we train a backbone network by self-supervised learning with BYOL [13]on the target dataset.
During the self-supervised learning, a random patch of input image is cropped,resized to 224 ⇥ 224 pixels, ﬂipped horizontally at random, and distorted by a random sequence ofbrightness, contrast, saturation, hue adjustments, and grayscale conversion.
A self-supervised model can capture diverse patterns shared by data without being biased towardsa particular class even the training set is biased.
We empirically demonstrate that adopting a selfsupervised representation leads to a model less biased compared with a supervised representation.
Although the representation offers rich and less-biased features, the main classiﬁer can be still biasedwhen it is trained by ERM.
In other words, the self-supervised representation alone is not enough tolearn a debiased model, and the need to explore a debiasing method for a classiﬁer arises.
Hence, wepropose LBC, a new debiasing method illustrated in the next section.
3.2 Learning a debiased classiﬁer with a biased committee
First, we randomly sample m subsets of the same size, denoted by S 1 , ..., S m , from the entire trainingdataset D with replacement.
Then m auxiliary classiﬁers f 1 , ..., f m of the committee are initializedrandomly, and each of the subsets S l is assigned to each auxiliary classiﬁer f l as its training data.
The ﬁrst step of LBC is warm-up training of the committee; this is required to ensure that thecommittee is capable of identifying and weighting bias-conﬂicting samples at the beginning of themain training process.
Given a mini-batch B at each warm-up iteration, the committee is trained by Eq minimizing the cross-entropy loss below:
[Equation]
Since each subset is sampled from the training set dominated by bias-guiding samples, a majority ofauxiliary classiﬁers are also biased.
At the same time, the classiﬁers are diverse due to their differencein initialization and training data.
After the warm-up stage, the main classiﬁer and the committee are trained while interacting witheach other.
First, the main classiﬁer is trained by the weighted cross entropy loss with the entiretraining set, where the sample weights are computed by considering consensus within the committeeon prediction difﬁculty of the samples.
Since a majority of auxiliary classiﬁers have trouble to handlebias-conﬂicting samples, we identify and weight bias-conﬂicting samples based on the number ofauxiliary classiﬁers whose predictions are correct for the samples.
The weight function w is given by
[Equation]
where m is the size of the committee, f l means the l -th classiﬁer of the committee, and ↵ is a scalehyper-parameter.
The weight reﬂects how much the sample x is likely to be bias-conﬂicting anddecreases rapidly
when the number of correctly predicting classiﬁers increases
Then we train themain classiﬁer g with emphasis on the bias-conﬂicting samples through the weight function w .
Theweighted cross entropy loss L WCE is given by
[Equation]
where B is the mini-batch.
During training, as the main classiﬁer is gradually debiased, samples useful for debiasing the mainclassiﬁer change accordingly.
To focus more on bias-conﬂicting samples difﬁcult for the mainclassiﬁer, we inform the quality of the main classiﬁer to the committee by distilling the knowledgeof the main classiﬁer in the form of its classiﬁcation logits [17] and transferring the knowledge byminimizing the following KD loss:
[Equation]
where ⌧ is a temperature parameter.
Note that we apply L KD to the complement set of S l to avoid [Table]
[Table]
After the warm-up training, the main classiﬁer and the auxiliary classiﬁers are alternately updated witha given mini-batch at each iteration.
First, we forward every sample in a mini-batch to each auxiliaryclassiﬁer and then compute weights of the samples using predictions of the auxiliary classiﬁers(Eq.
With the weights, the main classiﬁer is updated by Eq.
(3) and the knowledge of the updatedmain classiﬁer is transferred to the auxiliary classiﬁers by Eq.
Thus, the auxiliary classiﬁers areupdated by Eq.
(1) and Eq.
(4) as follows:
[Equation]
Experiments
4.1 Setup
Implementation details.
For the training self-supervised model, we train ResNet-18 [14] followingself-supervised learning with BYOL [13] on the target dataset.
Since the color information is keyfeature to classify HairColor class on the CelebA dataset, we adopt only brightness and contrastas color distortion when it trains celebA. Since NICO and BAR are very small dataset, we trainself-supervised model from ImageNet [8] pretrained parameters as an initial point.
Except for theexperiments using BAR and NICO, we train self-supervised models from scratch.
We use the selfsupervised ResNet-18 as a backbone network except for the last fully connected layer.
We set thebatch size to {64, 64, 128, 256}, learning rate {1e-3, 1e-3, 1e-4, 6e-3}, the size of the committee mto {30, 30, 30, 40}, the size of subset S l to {10, 10, 80, 300},  to {0.9, 0.6, 0.6, 0.6}, and ⌧ to {1, 1,1, 2.5}, respectively for {BAR, NICO, Imagenet-9, CelebA} and ↵ to 0.02.
Note that we average theaccuracy of each training over three independent trials with random seeds.
Evaluation metrics.
For evaluation, we adopt six metrics.
V ALIDATION / T EST : average accuracy onvalidation / test splits.
G UIDING : average accuracy on bias guiding samples per class.
C ONFLICTING :
[Table] [Table]
4.2 Datasets
CelebA. CelebA [30] is a dataset for face recognition where each sample is labeled with 40 attributes.
Following the experiment conﬁguration suggested by Nam et al.
[31], we focus on HairColorand HeavyMakeup attributes that are spuriously correlated with Gender attributes, i .
e ., most of the
CelebA images with blond hair are women.
As a result, the biased model suffers from performancedegradation when predicting HairColor attribute on males.
Therefore, we use HairColor as thetarget attribute and Gender as a spurious attribute, the same as HeavyMakeup .
ImageNet-9.
ImageNet-9 [19] is a subset of ImageNet [33] containing nine super-classes.
Followingthe setting adopted by [2], we conduct experiments with 54,600 training images and 2,100 validationimages.
ImageNet-9 tempts to have a correlated object class and image texture.
We follow theevaluation scheme adopted by [2], and we report the unbiased accuracy of the validation set, which iscomputed as average accuracy on every object-texture combination.
ImageNet-A. ImageNet-A [16] contains real-world images misclassiﬁed by ImageNet-trained ResNet50 [14].
Since such failure is caused when a model too heavily relies on the color, textures, andbackgrounds.
The ImageNet-A dataset could be a bias-conﬂicting set w.r.t.
various ImageNet biases.
This dataset is used only for evaluating a model trained on ImageNet-9.
BAR.
The Biased Action Recognition (BAR) dataset [31] is a real-world image dataset designed toevaluate the spurious correlation between human action and place on real-world images.
Originallythe given training set of BAR consists of only 100% bias-guiding samples, and its test set consistsof only bias-conﬂicting samples.
In our setting, we use 10% of the original BAR training set asvalidation and set the bias-conﬂicting ratio of the training set to 1%.
NICO.
NICO [15] is a real-world dataset for simulating the out-of-distribution image classiﬁcationtask.
Following the setting used by Wang et al.
[39], we use an animal subset of NICO, which islabeled with 10 object and 10 context classes for evaluating the debiasing methods.
The training setconsists 7 context classes per object class and they are long-tailed distributed (e.g., more dog imagesappear on the ‘on grass’ context than any of the other 6 contexts).
The validation and test set consistsof 10 context classes with 3 unseen context classes per object class.
We verify the ability of debiasinga model from object-context correlations by evaluating on NICO.
4.3 Quantitative results
LBC shows superior classiﬁcation accuracy among the methods that do not use the spurious attributelabel on the ﬁve real-world datasets.
In Tables 1 & 2, we observe LBC outperforms existing debiasingmethods using no spurious attribute label and shows comparable C ONFLICTING performance with
[Table] [Table]
[Figure]
methods that exploit spurious attribute labels on CelebA, which reﬂects gender bias in the real world. Especially, the gap between the average accuracy of groups and worst-group accuracy of LBC ismuch smaller than the other methods, this means that our model fairly predicts a sample belongsto each group.
Table 3 shows the results on the ImageNet-9, which is dominated by texture bias,and the ImageNet-A, which is regarded as a bias-conﬂicting set of ImageNet.
LBC is 9.7% betterthan CaaM
[40] on the ImageNet-A dataset, i .
e ., LBC is robust to texture bias and various ImageNetbiases.
Table 4a shows results on the BAR dataset.
LBC is 18.6% better than LDD, which is theprevious state of the art.
Compared to LfF [31] and LDD [25], which are debiasing methods with asingle biased classiﬁer, we demonstrate that learning a debiased classiﬁer with the biased committeeis more effective than a single classiﬁer scheme.
Table 4b shows the results on the NICO dataset.
LBC is 13.3% better than CaaM, i .
e ., LBC is well generalized to the unseen spurious attributes.
Notethat the validation and test set of NICO have unseen context classes and are unbiased.
[Figure]
[Figure]
[Figure]
Self-supervised representation as a solid baseline.
We empirically investigate the potential ofself-supervised representation as a solid baseline for the debiasing task.
We train a classiﬁer on the topof the self-supervised representation by ERM.
The results are denoted by ‘SSL+ERM’ and comparedwith ‘ERM’, which is a fully supervised classiﬁcation model in Table 1, 2, 3, 4a, 4b.
‘SSL+ERM’outperforms not only ERM but also the previous state-of-the-art on all the datasets except for CelebA.‘SSL+ERM’ is less biased than the model trained by fully supervised learning.
Importance of each module in LBC.
Table 5 demonstrates through ablation studies [28, 31, 22, 25]:(1) learning from a single biased classiﬁer, (2) learning with committee, and (3) transferring theknowledge of the main classiﬁer.
First, we train a classiﬁer by ERM (row 1) then assign a weight value50 to the wrongly predicted samples and 1 to other samples.
Then re-training the classiﬁer with theweights (row 2).
Comparing these two results demonstrates that up-weighting scheme with a biasedclassiﬁer is effective to debiasing a classiﬁer.
Then we increase the number of biased classiﬁers andcompute the sample weights using our weight function Eq.
(2) (row 3).
Learning with the committeeshows a remarkable improvement in the worst-group accuracy.
Moreover, knowledge distillation thatenables the committee to interact with the main classiﬁer further improves performance (row 4).
Effectiveness of transferring knowledge of the main classiﬁer.
Figure 3(a) shows the range ofunbiased validation accuracy of classiﬁers in the committee and unbiased validation accuracy of themain classiﬁer during training.
The mean accuracy of classiﬁers in the committee gradually increasesfollowing the accuracy of the main classiﬁer.
Also, the accuracy of the main classiﬁer graduallyincreases as training progresses.
On the other hand, the accuracy of classiﬁers in the committeetrained without KD loss does not increase or even decrease.
Number of classiﬁers.
We compare the results of the main classiﬁer trained with the different numberof auxiliary classiﬁers.
Figure 3(b) shows the worst-group accuracy versus the number of auxiliaryclassiﬁers m on CelebA. With a single auxiliary classiﬁer, the main classiﬁer shows the lowestworst-group accuracy, but the accuracy increases as m increases.
When larger than 40, the number ofclassiﬁers has little effect on learning the main classiﬁer.
Conclusion
We have proposed a new method for learning a debiased classiﬁer with a committee of auxiliaryclassiﬁers.
The committee is learned in a way that consensus on predictions of its classiﬁers offersa strong and reliable cue to identify and weight bias-conﬂicting data.
The main debiased classiﬁeris then trained with an emphasis on the bias-conﬂicting data to reduce the effect of bias-guidingcounterparts.
The committee is also trained to be debiased gradually along with the main classiﬁer sothat it highlights more challenging data as training progresses.
Moreover, we demonstrated that selfsupervised learning is a solid yet unexplored baseline for debiasing.
Coupled with a self-supervisedfeature extractor, our method achieved state-of-the-art by large margins on most existing real-worlddatasets.
Limitations.
Self-supervised learning increases overall training complexity as it usually takes alonger time for convergence than supervised learning.
Also, since the committee is designed as abootstrapped ensemble, it involves randomness in training and thus causes noticeable performanceﬂuctuation on CelebA, although it demonstrates stable performance on the other datasets.