Learning Debiased Classifier with Biased Committee
Anonymous Author(s)
AffiliationAddress email
Abstract
Neural networks are prone to be biased towards spurious correlations betweenclasses and latent attributes exhibited in a major portion of training data, whichruins their generalization capability.
This paper proposes a new method for trainingdebiased classifiers with no spurious attribute label.
The key idea of the methodis to employ a committee of classifiers as an auxiliary module that identifies biasconflicting data, i .
e ., data without spurious correlations, and assigns large weights tothem when training the main classifier.
The committee is learned as a bootstrappedensemble so that a majority of its classifiers are biased as well as being diverse,and intentionally fail to predict classes of bias-conflicting data accordingly.
Theconsensus within the committee on prediction difficulty thus provides a reliablecue for identifying and weighting bias-conflicting data.
Moreover, the committee isalso trained with knowledge transferred from the main classifier so that it graduallybecomes debiased along with the target and emphasizes more difficult data astraining progresses.
On five real-world datasets, our method outperforms existingmethods using no spurious attribute label like ours and even surpasses those relyingon bias label occasionally.
Introduction
Most supervised learning algorithms for classification rely on the empirical risk minimization (ERM)principle [37].
However, ERM has been known to cause a learned classifier to be biased towardspurious correlations between predefined classes and latent attributes that appear in a majority oftraining data [11].
In the case of hair color classification, for example, when most people withblond-hair ( i e ., target class) are female ( i e ., latent attribute) in a dataset, a classifier learnedby ERM exploits female as a shortcut for the classification due to its spurious correlation withblond-hair , and often mis-classifies non-blonde-haired women as blond-hair in consequence.
We call data with such spurious correlations and holding a majority of training data bias-guidingsamples and the other bias-conflicting samples , respectively.
The issue of model bias has often beenaddressed by exploiting explicit spurious attribute labels [21, 26, 34, 1, 36, 35, 42] or knowledgeabout bias types given a priori [2].
However, these methods are impractical because such supervisionand prior knowledge are costly, and the methods demand extensive post hoc analysis.
Hence, a body of research has been conducted for learning debiased classifiers with no additional labelfor spurious attributes [39, 25, 27, 31, 22, 24].
A common approach in this line of work is to employan intentionally biased classifier as an auxiliary module [27, 31, 22, 24].
In this approach, samplesthat the biased classifier has trouble handling are considered bias-conflicting ones and assigned largeweights when used for training the main classifier to reduce the effect of bias-guiding counterparts.
Although it has driven remarkable success, this approach has drawbacks due to the use of a singlebiased classifier.
First, the quality of the biased classifier could vary by hyper-parameters [28] and itsinitial parameter values [10].
Further, data that the biased classifier fails to handle could include notonly bias-conflicting samples but also bias-guiding ones, which differs by the quality of the classifier.
[Figure]
These drawbacks limit the reliability and performance of debiasing methods depending on a singlebiased classifier, as demonstrated in Figure 1.
To overcome these limitations, we propose a new method using a committee of biased classifiers asthe auxiliary module, coined learning with biased committee (LBC).
LBC identifies bias-conflictingsamples and determines their weights through consensus on their prediction difficulty within thecommittee.
To this end, the committee is built as a bootstrapped ensemble, i .
e ., each of its classifiersis trained from a randomly sampled subset of the entire training dataset.
This strategy not onlyguarantees the diversity among the classifiers, but also lets a majority of the classifiers be biasedsince random subsets of training data are highly likely to be dominated by bias-guiding samples.
Accordingly, a majority of the committee tends to classify bias-guiding samples correctly and failto deal with bias-conflicting ones.
The consensus on prediction difficulty within the committee thusgives a strong cue for identifying and weighting bias-conflicting samples.
Also, using the consensusof multiple classifiers enables LCB to be robust to the varying quality of individual classifiers andconsequently to focus more precisely on bias-conflicting samples, as shown in Figure 1.
Moreover, unlike the biased classifier trained independently of the main classifier in the previouswork, the committee in LCB is trained with knowledge of the main classifier as well as the randomsubsets of training data to serve the main classifier better.
Specifically, the knowledge is distilledin the form of classification logits of the main classifier [17], and each classifier of the committeeutilizes the knowledge as pseudo labels of training data other than its own training set.
We expectthat this strategy allows the committee to become debiased gradually so that it does not give largeweights to easy bias-conflicting samples, i .
e ., those already well handled by the main classifier, andfocuses more on difficult ones.
Note that, even with this strategy, the classifiers of the committee arestill biased differently due to their different training sets with ground-truth labels.
Finally, we further improve the proposed method by adopting a self-supervised representation asthe frozen backbone of the committee and the main classifier.
Since self-supervised learning is notdependent on class labels, it is less affected by the spurious correlations between classes and latentattributes, leading to a robust and less-biased representation.
Also, by installing the committee andthe main classifier on top of the representation, the classifiers can be implemented efficiently in bothspace and time while enjoying the rich and bias-free features given by the backbone.
LCB is validated extensively on five real-world datasets.
It substantially outperforms existing methodsusing no bias label and even occasionally surpasses previous arts demanding bias labels.
We alsodemonstrate that all of the main components, i .
e ., the use of the committee, its training with knowledgetransfer, and the self-supervised learning, contribute to the outstanding performance.
The maincontribution of this paper is four-fold:
• We present LBC, a new approach to learning a debiased classifier with no spurious attribute label.
The use of consensus within the committee allows LBC to address limitations of previous workrelying on a single biased classifier.• We propose to learn the committee using knowledge of the main classifier, unlike the previouswork whose auxiliary modules do not consider the main classifier.• We investigate the potential of self-supervised learning for debiasing, and find that it is a solidyet unexplored baseline for the task.
• LCB demonstrates superior performance on five real-world datasets.
It outperforms existingmethods using no additional supervision like ours and even surpasses those relying on spuriousattribute labels occasionally.
Related work2.1 Debiasing with prior knowledge on bias
A majority of previous work on learning debiased classifiers mitigates the bias by leveraging explicitlabels on bias types [21, 26, 34, 1, 36, 35, 42].
For instance, Kim et al.
[21] build a debiasedmodel to classify digits by leveraging RGB color labels explicitly in the Colored MNIST dataset.
Other methods are designed to handle predefined domain-specific bias types ( e .
g ., texture bias in
ImageNet) [38, 4, 2].
Recent methods try to reduce annotation costs for bias supervision by only utilizing a small set ofbias-labeled data.
For instance, Nam et al.
[32] and Jung et al.
[20] train auxiliary bias predictorwith a small set of bias-labeled data and assign pseudo bias labels to the entire training set usingits prediction.
These methods have two inherent limitations.
First, they cannot handle biases whosetypes are not predefined in training.
Second, manually annotating all bias types is often expensiveand laborious.
Even obtaining a small set of bias labels could be expensive since bias labels existingin a real-world dataset is often long-tailed [29] or multi-labeled [15].
2.2 Debiasing without spurious attribute labels
Recent debiasing methods without any bias supervision [18, 7, 31, 24, 22, 27, 6] are based on theassumption that attributes of malignant bias are easier to learn than those of target classes.
Throughthis assumption, previous methods identify the amount of biases within the samples and train arobust models by emphasizing bias-conflicting samples.
To identify biases of samples, previous workleverage a high gradient of the latent vector [18, 7] or employ an intentionally biased classifier asan auxiliary module [31, 22, 24] by using generalized cross-entropy (GCE) loss [41].
Liu et al.
[27]consider misclassified samples as bias-conflicting samples, and Creager et al.
[6] derive bias-guidingand bias-conflicting partition that maximally violates the invariance principle.
On the other hand,
Bahng et al.
[2] propose a method tailored to the texture bias, building a debiased model to learnindependent features from a biased model in which receptive field size is limited to capture the texturebias in image classification.
While LBC is also based on the aforementioned assumption, we firstly propose to identify biasconflicting samples and up-weight them through the consensus of a committee of the biased classifiers.
By exploiting the consensus of multiple classifiers, LBC is robust against the varying quality ofindividual classifiers and allows to learn debiased classifiers more reliably and effectively.
Proposed method
We propose a new method that learns a debiased classifier with a committee of biased classifiers,dubbed LBC.
It first learns a feature representation with self supervision, which is used as the frozenbackbone providing rich and bias-free features to downstream modules (Section 3.1).
Next, it trainsa committee of m auxiliary classifiers f 1 , f 2 , . . .
, f m and the main classifier g on top of the selfsupervised representation (Section 3.2); thanks to the self-supervised representation, the classifiersare designed concisely, using only two fully-connected layers for each.
In a nutshell, the committee identifies bias-conflicting samples and assigns them large weights toreduce the effect of bias-guiding samples during the training of the main classifier.
To this end, the
[Figure]
A committee of auxiliary classifiers f
i is trained on top of the backbone; a random subset of training data is assigned to each classifier of the committee as labeled data for supervised learning (Eq.
The committee determines the weight of each training sample based on consensus of its members, i .
e ., the number of members that correctly predict the class of the sample (Eq.
The main classifier g is trained using the weights by minimizing the weighted cross entropy loss (Eq.
In turn, knowledge of the main classifier is transferred to the committee through knowledge distillation (Eq.
committee is trained as a bootstrapped ensemble of classifiers so that a majority of its classifiersare biased as well as diverse, and intentionally fail to predict classes of bias-conflicting samplesaccordingly.
Hence, the consensus within the committee on prediction difficulty of a sample ( e .
g .,the number of classifiers that fail to predict its class label) indicates how much likely the sampleis bias-conflicting, and is used to compute weights for training samples.
Moreover, the committeeis trained also with knowledge of the main classifier so that it gradually becomes debiased alongwith the main classifier and emphasizes more difficult samples as training progresses.
Note thatthe committee is an auxiliary module used only in training and thus does not impose additionalcomputation or memory footprint in testing.
The overall process of LBC is illustrated conceptually in Figure 2 and given formally in Algorithm 1.
The following sections elaborate on each step of LBC.
3.1 Self-supervised representation learning
As the feature extractor, we train a backbone network by self-supervised learning with BYOL [13]on the target dataset.
During the self-supervised learning, a random patch of input image is cropped,resized to 224 × 224 pixels, flipped horizontally at random, and distorted by a random sequence ofbrightness, contrast, saturation, hue adjustments, and grayscale conversion.
A self-supervised model can capture diverse patterns shared by data without being biased towardsa particular class even the training set is biased.
We empirically demonstrate that adopting a selfsupervised representation leads to a model less biased compared with a supervised representation.
Although the representation offers rich and less-biased features, the main classifier can be still biasedwhen it is trained by ERM.
In other words, the self-supervised representation alone is not enough tolearn a debiased model, and the need to explore a debiasing method for a classifier arises.
Hence, wepropose LBC, a new debiasing method illustrated in the next section.
3.2 Learning a debiased classifier with a biased committee
First, we randomly sample m subsets of the same size, denoted by S 1 , ..., S m , from the entire trainingdataset D with replacement.
Then m auxiliary classifiers f 1 , ..., f m of the committee are initializedrandomly, and each of the subsets S l is assigned to each auxiliary classifier f l as its training data.
The first step of LBC is warm-up training of the committee; this is required to ensure that thecommittee is capable of identifying and weighting bias-conflicting samples at the beginning of themain training process.
Given a mini-batch B at each warm-up iteration, the committee is trained byminimizing the cross-entropy loss below:
[Equation]
Eq.
Eq. Eq. (1), Eq. Since each subset is sampled from the training set dominated by bias-guiding samples, a majority ofauxiliary classifiers are also biased.
At the same time, the classifiers are diverse due to their differencein initialization and training data.
After the warm-up stage, the main classifier and the committee are trained while interacting witheach other.
First, the main classifier is trained by the weighted cross entropy loss with the entiretraining set, where the sample weights are computed by considering consensus within the committeeon prediction difficulty of the samples.
Since a majority of auxiliary classifiers have trouble to handlebias-conflicting samples, we identity and weight bias-conflicting samples based on the number ofauxiliary classifiers whose predictions are correct for the samples.
The weight function w is given by
[Equation]
where m is the size of the committee, f l means the l -th classifier of the committee, and α is a scalehyper-parameter.
The weight reflects how much the sample x is likely to be bias-conflicting anddecreases rapidly
when the number of correctly predicting classifiers increases
Then we train themain classifier g with emphasis on the bias-conflicting samples through the weight function w .
Theweighted cross entropy loss L WCE is given by
[Equation]
where B is the mini-batch.
During training, as the main classifier is gradually debiased, samples useful for debiasing the mainclassifier change accordingly.
To focus more on bias-conflicting samples difficult for the mainclassifier, we inform the quality of the main classifier to the committee by distilling the knowledgeof the main classifier in the form of its classification logits [17] and transferring the knowledge byminimizing the following KD loss:
[Equation]
where τ is a temperature parameter.
Note that we apply L KD to the complement set of S l to avoidauxiliary classifiers in the committee being identical to each other.
By interacting with the mainclassifier, the committee gradually becomes debiased along with the main classifier. Since samplescorrectly predicted by the main classifier are less weighted and those with incorrect predictions aremore weighted by the committee.
After the warm-up training, the main classifier and the auxiliary classifiers are alternately updated witha given mini-batch at each iteration.
First, we forward every sample in a mini-batch to each auxiliaryclassifier and then compute weights of the samples using predictions of the auxiliary classifiers [Table] [Table] (Eq.
With the weights, the main classifier is updated by Eq.
(3) and the knowledge of the updatedmain classifier is transferred to the auxiliary classifiers by Eq.
Thus, the auxiliary classifiers areupdated by Eq [Table]
(1) and Eq.
(4) as follows:
[Equation]
Experiments
4.1 Setup
Implementation details.
For the training self-supervised model, we train ResNet-18 [14] followingself-supervised learning with BYOL [13] on the target dataset.
Since the color information is keyfeature to classify HairColor class on the celebA dataset, we adopt only brightness and contrastas color distortion when it trains celebA. Since NICO and BAR are very small dataset, we trainself-supervised model from ImageNet [8] pretrained parameters as an initial point.
Except for theexperiments using BAR and NICO, we train self-supervised models from scratch.
We use the selfsupervised ResNet-18 as a backbone network except for the last fully connected layer.
We set thebatch size to {64, 64, 128, 256}, learning rate {1e-3, 1e-3, 1e-4, 6e-3}, the size of the committee mto {30, 30, 30, 40}, the size of subset S l to {10, 10, 80, 300}, λ to {0.9, 0.6, 0.6, 0.6}, and τ to {1, 1,1, 2.5}, respectively for {BAR, NICO, Imagenet-9, CelebA} and α to 0.02.
Note that we average theaccuracy of each training over three independent trials with random seeds.
Evaluation metrics.
For evaluation, we adopt six metrics.
V ALIDATION / T EST : average accuracy onvalidation / test splits.
G UIDING : average accuracy on bias guiding samples per class.
C ONFLICTING :average accuracy on bias conflicting samples per class. U NBIASED : average of ‘Guiding’ and‘Conflicting’ per class. W ORST - GROUP : minimum average accuracy of group.
[Table] [Table]
CelebA. CelebA [30] is a dataset for face recognition where each sample is labeled with 40 attributes.
Following the experiment configuration suggested by Nam et al.
[31], we focus on HairColorand HeavyMakeup attributes that are spuriously correlated with Gender attributes, i .
e ., most of the
CelebA images with blond hair are women.
As a result, the biased model suffers from performancedegradation when predicting HairColor attribute on males.
Therefore, we use HairColor as thetarget attribute and Gender as a spurious attribute, the same as HeavyMakeup .
ImageNet-9.
ImageNet-9 [19] is a subset of ImageNet [33] containing nine super-classes.
Followingthe setting adopted by [2], we conduct experiments with 54,600 training images and 2,100 validationimages.
ImageNet-9 tempts to have a correlated object class and image texture.
We follow theevaluation scheme adopted by [2], and we report the unbiased accuracy of the validation set, which iscomputed as average accuracy on every object-texture combination.
ImageNet-A. ImageNet-A [16] contains real-world images misclassified by ImageNet-trained ResNet50 [14].
Since such failure is caused when a model too heavily relies on the color, textures, andbackgrounds.
The ImageNet-A dataset could be a bias-conflicting set w.r.t.
various ImageNet biases.
This dataset is used only for evaluating a model trained on ImageNet-9.
BAR.
The Biased Action Recognition (BAR) dataset [31] is a real-world image dataset designed toevaluate the spurious correlation between human action and place on real-world images.
Originallythe given training set of BAR consists of only 100% bias-guiding samples, and its test set consistsof only bias-conflicting samples.
In our setting, we use 10% of the original BAR training set asvalidation and set the bias-conflicting ratio of the training set to 1%.
NICO.
NICO [15] is a real-world dataset for simulating the out-of-distribution image classificationtask.
Following the setting used by Wang et al.
[39], we use an animal subset of NICO, which islabeled with 10 object and 10 context classes for evaluating the debiasing methods.
The training setconsists 7 context classes per object class and they are long-tailed distributed (e.g., most dog imagesare appeared ‘on grass’ the others are appeared on 6 contexts).
The validation and test set consists of10 context classes with 3 unseen context classes per object class.
We verify the ability of debiasing amodel from object-context correlations by evaluating on NICO.
4.3 Quantitative results
LBC shows superior classification accuracy among the methods that do not use the spurious attributelabel on the five real-world datasets.
In Tables 1 & 2, we observe LBC outperforms existing debiasingmethods using no spurious attribute label and shows comparable C ONFLICTING performance withmethods that exploit spurious attribute labels on CelebA, which reflects gender bias in the real world.
Especially, the gap between the average accuracy of groups and worst-group accuracy of LBC ismuch smaller than the other methods, this means that our model fairly predicts a sample belongs
[Table] [Table]
[Figure]
to each group.
Table 3 shows the results on the ImageNet-9, which is dominated by texture bias,and the ImageNet-A, which is regarded as a bias-conflicting set of ImageNet.
LBC is 9.7% betterthan CaaM
[40] on the ImageNet-A dataset, i .
e ., LBC is robust to texture bias and various ImageNetbiases.
Table 4a shows results on the BAR dataset.
LBC is 18.6% better than LDD, which is theprevious state of the art.
Compared to LfF [31] and LDD [24], which are debiasing methods with asingle biased classifier, we demonstrate that learning a debiased classifier with the biased committeeis more effective than a single classifier scheme.
Table 4b shows the results on the NICO dataset.
LBC is 13.3% better than CaaM, i .
e ., LBC is well generalized to the unseen spurious attributes.
Notethat the validation and test set of NICO have unseen context classes and are unbiased.
[Figure]
4.4 Ablation
[Figure]
Self-supervised representation as a solid baseline.
We empirically investigate the potential ofself-supervised representation as a solid baseline for the debiasing task.
We train a classifier on the topof the self-supervised representation by ERM.
The results are denoted by ‘SSL+ERM’ and comparedwith ‘ERM’, which is a fully supervised classification model in Table 1, 2, 3, 4a, 4b.
‘SSL+ERM’outperforms not only ERM but also the previous state of the art on all the datasets except for CelebA.‘SSL+ERM’ is less biased than the model trained by fully supervised learning.
Importance of each module in LBC.
Table 5 demonstrates through ablation studies [27, 31, 22, 24]:(1) learning from a single biased classifier, (2) learning with committee, and (3) transferring theknowledge of the main classifier.
First, we train a classifier by ERM (row 1) then assign a weight value50 to the wrongly predicted samples and 1 to other samples.
Then re-training the classifier with theweights (row 2).
Comparing these two results demonstrates that up-weighting scheme with a biasedclassifier is effective to debiasing a classifier.
Then we increase the number of biased classifier andcompute the sample weights using our weight function Eq.
(2) (row 3).
Learning with the committeeshows a remarkable improvement on the worst-group accuracy.
Moreover, knowledge distillation thatenables the committee to interact with the main classifier further improves performance (row 4).
Effectiveness of transferring knowledge of the main classifier.
Figure 3(a) shows the range ofunbiased validation accuracy of classifiers in the committee and unbiased validation accuracy ofmain classifier during training.
The mean accuracy of classifiers in the committee gradually increasefollowing the accuracy of the main classifier.
Also, the accuracy of the main classifier graduallyincreases as training progresses.
On the other hand, the accuracy of classifiers in the committeetrained without KD loss does not increase or even decrease.
Number of classifiers.
We compare the results of the main classifier trained with the different numberof auxiliary classifiers.
Figure 3(b) shows the worst-group accuracy versus the number of auxiliaryclassifiers m on CelebA. With a single auxiliary classifier, the main classifier shows the lowestworst-group accuracy, but the accuracy increases as m increases.
When larger than 40, the number ofclassifiers has little effect on learning the main classifier.
Conclusion
We have proposed a new method for learning a debiased classifier with a committee of auxiliaryclassifiers.
The committee is learned in a way that consensus on predictions of its classifiers offersa strong and reliable cue to identify and weight bias-conflicting data.
The main debiased classifieris then trained with emphasis on the bias-conflicting data to reduce the effect of bias-guidingcounterparts.
The committee is also trained to be debiased gradually along with the main classifier sothat it highlights more challenging data as training progresses.
Moreover, we demonstrated that selfsupervised learning is a solid yet unexplored baseline for debiasing.
Coupled with a self-supervisedfeature extractor, our method achieved the state of the art by large margins on most of existingreal-world datasets.
Limitations.
Self-supervised learning increases overall training complexity as it usually takes alonger time for convergence than supervised learning.
Also, since the committee is designed as abootstrapped ensemble, it involves randomness in training and thus causes noticeable performancefluctuation on CelebA, although it demonstrates stable performance on the other datasets. Checklist For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[No] In fact, ourmethod relieves societal discrimination problem of neural network.(d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See supplemen-
tary materials.(b)
Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section 4.1 and supplementary materials.(c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We average the accuracy of each training over threeindependent trials with random seeds.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 4.1 and supplementarymaterials.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes](b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]