Learning Debiased Classifier with Biased Committee
Authors: Nayeong Kim
Sehyun Hwang
Sungsoo Ahn
Jaesik Park
Suha Kwak
Venue: NeurIPS 2022 Accept (2022)
Keywords: Debiasing
bootstrap ensemble
self-supervised learning
image classification
spurious correlation

Abstract: Neural networks are prone to be biased towards spurious correlations between classes and latent attributes exhibited in a major portion of training data, which ruins their generalization capability. We propose a new method for training debiased classifiers with no spurious attribute label. The key idea is to employ a committee of classifiers as an auxiliary module that identifies bias-conflicting data, i.e., data without spurious correlation, and assigns large weights to them when training the main classifier. The committee is learned as a bootstrapped ensemble so that a majority of its classifiers are biased as well as being diverse, and intentionally fail to predict classes of bias-conflicting data accordingly. The consensus within the committee on prediction difficulty thus provides a reliable cue for identifying and weighting bias-conflicting data. Moreover, the committee is also trained with knowledge transferred from the main classifier so that it gradually becomes debiased along with the main classifier and emphasizes more difficult data as training progresses. On five real-world datasets, our method outperforms prior arts using no spurious attribute label like ours and even surpasses those relying on bias labels occasionally. Our code is available at https://github.com/nayeong-v-kim/LWBC.

Learning Debiased Classifier with Biased Committee
Nayeong Kim
Sehyun Hwang
Sungsoo Ahn
Jaesik Park
Suha Kwak
Pohang University of Science and Technology (POSTECH), South Korea (nayeong.kim, schyum03, sungsoo.ahn jaesik. park, suha.kwak @ @postech.ac.kr
Abstract
Neural networks are prone to be biased towards spurious correlations between classes and latent attributes exhibited a major portion of training data, which ru- ins their generalization capability. We propose new method for training debiased classifiers with no spurious attribute label. The key idea is to employ a committee of classifiers an auxiliary module that identifies bias-conflicting data, i.e., data with- out spurious correlation, and assigns large weights to them when training the main bootstrapped ensemble so that majority
classifier. The committee is learned of its classifiers are biased as well as being diverse, and intentionally fail to predict classes of bias-conflicting data accordingly. The consensus within the committee prediction difficulty thus provides reliable cue for identifying and weighting cinffcoon data. Moreover, the committee is also trained with knowledge transferred from the main classifier so that it gradually becomes debiased along with the main classifier and emphasizes more difficult data as training progresses. On five real-world datasets, our method outperforms prior arts using no spurious attribute label like ours and even surpasses those relying on bias labels occasionally. Our code is available at hhttttt////// //gtthub..///t//g/// tiigthu..... com woom/nayeong--oe Biiioogg---iiiiBB--------
Introduction
Most supervised learning algorithms for classification rely on the empirical risk minimization (ERM) principle [40]. However, ERM has been known to cause learned classifier be biased toward spurious correlations between predefined classes and latent attributes that appear in a majority of training data [12]. In the case of hair color classification, for example, when most people with blond-hair (i.e., target class) are female (i.e., latent attribute) in a dataset, a classifier learned shortcut for the classification due its spurious correlation with blond-hair, and often mis-classifies non-blonde-haires women as bl blond-hain in consequence.
by ERM exploits female
We call data with such spurious correlations and holding a majority of training data bias-guiding samples, and the other bias-conflicting sumples, respectively. The issue of model bias has often been
addressed by exploiting explicit spurious attribute labels [22 28 36
39
38 45)
or knowledge
about bias types given a priori (3]. However, these methods are impractical because such supervision and prior knowledge are costly, and the methods demand extensive post hoc analysis.
Hence, a body of research has been conducted for learning debiased classifiers with no additional label 23 26 A common approach in this line of work is to employ an intentionally biased classifier as an auxiliary module (2 32 23 26]. In this approach, samples that the biased classifier has trouble handling are regarded as bias-conficctig ones and assigned large weights when used for training the main classifier to reduce the effect of bias-guiding counterparts. Although has driven remarkable success, this approach has drawbacks due the of a single biased classifier. First, the quality of the biased classifier could vary by hyper-parameter (2) and its initial parameter values [IT]. Further, data that the biased classifier fails handle could include not only bias-conflicting samples but also bias-guiding ones, which differs by the quality of the classifier.
for spurious attributes 42 29 32
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

(blond, male)
sample ra50
0:12
Ennichment
Model ID
0.10
Committee
Single (maan) Single trange)
0.08
range
10
16.6
5e-4
1o-4
16-3
100
Learning rate
Model ID
Scale factor
(h)
(c)
Figure 1: Analysis on the instability of single biased classifier in mining and weighting bias- conflicting samples. The experiments are conducted on the CelebA dataset, in which samples with blond and male attributes incorrectly predicted by a single biased classifier. The ratio highly fluctuates by the learning rate of the classifier and varies up to 4%p due to different initialization even with a fixed learning rate, meaning that a single biased classifier is sensitive to predictions of biased classifiers. For pairs of biased classifiers initialized differently, we measure the number of caassconflicting samples for which the classifiers predict differently. The results suggest (c) Comparisons between a single biased classifier and a committee of biased classifiers in terms of enrichment [29]. Higher enrichment implies more precise mining and weighting of bias-conflicting samples; the formal definition of
bias-conflicting. (a) The ratio of bias-confficting samples all
(b) Disagreement on
that individual biased classifiers are sensitive initialization.
enrichment is given in Appendix ??. The committee clearly outperforms the single biased classifier in terms of the enrichment.
These drawbacks limit the reliability and performance of debiasing methods depending on a single biased classifier, as demonstrated in Figure
To overcome these limitations, we propose a new method using a committee biased classifiers as the auxiliary module, coined learning with biased commillee (LWBC). LWBC identifies bias-conflicting samples and determines their weights through consensus on their prediction difficulty within the committee. To this end, the committe is built as a bootstrapped ensemble, i.e., each of its classifiers is trained from a randomly sampled subset the entire training dataset. This strategy not only guarantees the diversity among the classifiers, but also lets majority of the classifiers be biased since random subsets of training data are highly likely to be dominated by bias- guiidig samples. Accordingly, majority of the committee tends classify bias- gaas-igidnng samples correctly and fail to deal with bias- caass-offficting ones. The consensus on prediction difficulty within the committee thus gives strong cue for identifying and weighting bias-conflicting samples. Also, using the consensus of multiple classifiers enables LWBC to be robust to the varying quality of individual classifiers and consequently focus more precisely on bias-conflicting samples, shown in Figure T
Moreover, unlike the biased classifier trained independently of the main classifier in the previous work, the committee in LWBC is trained with knowledge of the main classifier as well as the random subsets of training data to serve the main classifier better. Specifically, the knowledge is distilled in the form of classification logits of the main classifier [18), and each classifier of the committee utilizes the knowledge as pseudo labels of training data other than its own training set. We expect that this strategy allows the committee to become debiased gradually so that it does not give large weights to easy bias-conflicting samples, i.e., those already well handled by the main classifier, and focuses more on difficult ones. Note that, even with this strategy. the classifiers of the committee still biased differently due to their different training sets with ground-truth labels.
Finally, we further improve the proposed method by adopting self-supervised representation as the frozen backbone of the committee and the main classifier. Since self-supervised learning is not dependent on class labels, it is less affected by the spurious correlations between classes and latent attributes, leading to a robust and less-biased biased representation. Also, by installing the committee and the main classifier on top of the representation, the classifiers be implemented efficiently in both space and time while enjoying the rich and bias-free features given by the backbone.
LWBC is validated extensively methods using no bias label and even occasionally surpasses previous arts demanding bias labels. We also demonstrate that all of the main components, i.e., the use of the committee, its training with
five real-world datasets. It substantially outperforms existing

---

knowledge transfer, and the self-supervised learning, contribute to the outstanding performance. The main contribution of this paper is four-fold:
We present LWBC, new approach learning debiased classifier with spurious attribute label. The use consensus within the committee allows LWBC to address limitations of previous work relying on a single biased classifier.
We propose to learn the committee using knowledge of the main classifier, unlike the previous work whose auxiliary modules do not consider the main classifier.
We investigate the potential of self-supervised learning for debiasing, and find that it is solid yet unexplored baseline for the task.
LWBC demonstrates superior performance five real-world datasets. It outperforms existing methods using no additional supervision like ours and even surpasses those relying on spurious attribute labels occasionally.
2 Related work
2.1 Debiasing with prior knowledge on bias
majority of previous work on learning debiased classifiers mitigates the bias by leveraging explicit
38 45). For instance, Kim et al. [22] build a debiased model to classify digits by leveraging RGB color labels explicitly in the Colored MNIST dataset. Other methods are designed to handle predefined domain-specific bias types (e.g., texture bias in
labels on bias types [22] 28 36
39
ImageNet) 41 3..
Recent methods try to reduce annotation costs for bias supervision by only utilizing a small set of bias-labeled data. For instance, Nam et al. 3 and Jung et al. (2 train auxiliary bias predictor with a small set bias-labeled data and assign pseudo bias labels to the entire training set using its prediction. These methods have inherent limitations. First, they cannot handle biases whose types are not predefined in training. Second, manually annotating all bias types is often expensive and laborious, Even obtaining small set of bias labels could be expensive since bias labels existing in real-world dataset is often long-tailed (30 or multi-labeled [16].
2.2 Debiasing without spurious attribute labels
Recent debiasing methods without any bias supervision [19 based on the assumption that attributes of malignant bias are easier learn than those of target classes. Through this assumption, previous methods identify the amount of biases within the samples and train robust models by emphasizing bias-conflicting samples. To identify biases of samples, previous work leverage a high gradient of the latent vector [19] g or employ an intentionally biased
32 26 23
37
25 34
are
classifier as an auxiliary module [32 23 26 by using generalized cross-entropy (GCE) loss [44]. Liu et al. [29] consider misclassified samples as bias-conflicting samples, and Creager al. 7 derive bias-guiding and bias-conflicting partition that maximally violates the invariance principle. On the other hand, Bahng et al. [3] propose a method tailored to the texture bias, building debiased model to learn independent features from a biased model in which receptive field size is limited to capture the texture bias in image classification.
While LWBC also based on the aforementioned assumption, we firstly propose to identify bias- conflicting samples and up- weight them through the consensus of a committee of the biased classifiers. By exploiting the consensus of multiple classifiers, LWBC is robust against the varying quality of individual classifiers and allows learn debiased classifiers more reliably and effectively.
Proposed method
We propose new method that learns debiased classifier with a committee biased classifiers, dubbed LWBC. It first learns feature representation with self supervision, which is used as the frozen backbone providing rich and bias-free features to downstream modules (Section (.T Next, it trains committee of m auxiliary classifiers f1, f2....., fm and the main classifier g top of
3

---

Committee
Sample weighting
Sm
Weighting function
9
LWCE
fi
# correct fo
Input images
Sample weight W
Bias Mlas-guiding sample
Bias-conflicting sample
CE Lx
Knowledge distiliation
Figure 2:
The overall pipeline of LWBC. It adopts a frozen backbone trained by self-supervised
learning, A committee of auxiliary classifiers is trained top of the backbone; random subset of training data is assigned to each classifier of the committee as labeled data for supervised learning The committec determines the weight of each training sample based consensus of its members, i.e. the number of members that correctly predict the class of the sample (Eq. (2). The main classifier g is trained using the weights by minimizing the weighted cross entropy loss (Eq. (3). In turn, knowledge of the main classifter is transferred to the committee through knowledge distillation (Eq.
(Eq. (T.
thanks to the self-supervised representation, the classifiers are designed conciscly, using only two fully-connected layers for each.
the self-supervised representation (Section 3.2
In a nutshell, the committee identifies bias-conflicting samples and assigns them large weights to reduce the effect of bias-guie hlas-ggiiing samples during the training of the main classifier. To this end, the committee trained as a bootstrapped ensemble of classifiers so that a majority of its classifiers are biased as well as diverse, and intentionally fail to predict classes of bias-conf classconffictin samples accordingly. Hence, the consensus within the committee on prediction difficulty of sample (e-g., the number of classifiers that fail to predict its class label) indicates how much likely the sample bias-confleting, and is used to compute weights for training samples. Moreover, the committee is trained also with knowledge of the main classifier so that it gradually becomes debiased along with the main classifier and emphasizes more difficult samples as training progresses. Note that the committee is an auxiliary module used only in training and thus does not impose additional computation or memory footprint in testing.
The overall process of LWBC is illustrated conceptually in Figure given formally in Algorithm
The following sections elaborate on each step of LWBC.
3.1 Self-supervised representation learning
As the feature extractor, train backbone network by self-supervised learning with BYOL [14 on the target dataset. During the self-supervised learning, random patch of input image is cropped. resized to 224 224 pixels, flipped horizontally at random, and distorted by random sequence of brightness, contrast, saturation, hue adjustments, and grayscale conversion.
self-supervised model capture diverse patterns shared by data without being biased towards a particular class even the training set is biased. We empirically demonstrate that adopting self- supervised representation leads model less biased compared with supervised representation.
Although the representation offers rich and less-biased features, the main classifier can still biased when it is trained by ERM. In other words, the self-supervised representation alone is not enough to learn a debiased model, and the need to explore debiasing method for classifier arises. Hence, we propose LWBC, new debiasing method illustrated in the next section.
3.2 Learning a debiased classifier with a biased committee
First, we randomly sample m subsets of the same size, denoted by Si, Sm. from the entire training dataset with replacement. Then m auxiliary classifiers f1, fm of the committee are initialized randomly, and each of the subsets S is assigned each auxiliary classifier fi its training data.

---

Algorithm Learning a debiased classifier with a biased committee
input training set D, batch size b, size of the committee m, learning rate y,, scale hyper-parameter o, balancing nyper-parameter X, total number of iterations t, number of warm-up iterations tw 1: Draw m random subsets Si, Sm of D.
2: Initialize auxiliary classifiers of the committee {fi(2; 01))
3: Initialize the main classifier g(;;
4: for 1,... to do
5: Draw mini-batch B = { U: D:) from D.
6: 81 < 8, v) 8. LCB 4/ 1, ,,.,
Eq.
7: end for
8:
for j j- Lu + 1,.., L
9:
Draw mini-batch B = [1 Ui) v:)) from D.
10:
w(x;) = 1-11/11 1/(_== [(f((z)) = 11)/m + aa) vz,
Eq.
11:
② - 6 VV &WCE
Eq.
12:
O1 + 0: - (1 Aim Cce - LKD = 1, ,,,, m
Eq.
Eq.
13: end for
The first step of LWBC is warm-up training of the committee; this is required to ensure that the committee is capable of identifying and weighting bias-conflicting samples the beginning of the main training process. Given a mini-batee batch B at each warm-up iteration, the committee is trained by minimizing the cross-entropy loss below:
27
P-= -
LCB
Σ
CE(f((z).))
(2.9)ESinB
Since each subset is sampled from the training set dominated by bias- bias-guiding samples, majority of auxiliary classifiers are also biased. At the same time, the classifiers are diverse due to their difference in initialization and training data.
After the warm-up stage, the main classifier and the committee are trained while interacting with each other. First, the main classifier is trained by the weighted cross entropy loss with the entire training set, where the sample weights are computed by considering consensus within the committee on prediction difficulty of the samples. Since majority of auxiliary classifiers have trouble to handle bias-come coassconffiting samples, we identify and weight bias-con bias-conflicting samples based on the number of auxiliary classifiers whose predictions are correct for the samples. The weight function w is given by
w(x) E ((f(z) v)mm+
where m is the size of the committee, ft means the I-th classifier of the committee, and a is scale hyper-parameter. The weight reflects how much the sample x is likely to be bias-conflicting and decreases rapidly when the number of correctly predicting classifiers increases. Then we train the main classifier g with emphasis on the clas-confictiig samples through the weight function w. The weighted cross entropy loss LWCE is given by
LWCF Σ (Z:))
w(x) CE(g(x))
(3)
where B is the mini-batch.
During training, the main classifier is gradually debiased, samples useful for debiasing the main classifier change accordingly. To focus more on bias-conflicting samples difficult for the main classifier, we inform the quality of the main classifier the committee by distilling the knowledge of the main classifier in the form of its classification logits [18 and transferring the knowledge by minimizing the following KD loss:
(g(z)
f1(x)) T
CKD
Σ (R.))EB
KL softmax
softmax
where 7 is temperature parameter. Note that we apply LKD to the complement set of S, to avoid auxiliary classifiers in the committee being identical to each other. By interacting with the main
5

---

Table GUIDING, UNBIASED, and CONFLICTING metrics (%) for the CelebA dataset. For methods without spurious attribute labels, we mark the best and the second-bes performance in bold and underline, respectively.
Spurious attribute label
CelebA HairColor
CelebA HeuvyMukeup
Methed
GUIDING
UNBIASED
CONFLICTING
GUIDING
UNBIASED
CONFLICTING
Gmup DRO
87.46
85,43
83,40
64.881042
50.24
EnD 38
/ 94.97
91.21
87.4521m
98.16
75.931131
53.70
CSAD 45
/
91.19
89,36
87.53
82.32
67.88
53.44
ERM
X
87.98
70,25
52.5220.1
90.25
62.00gam
33.75
LIF
X
87.24
84.24+0.37
81,2441138
86,92
16.20
45 48.48
SSI SSI+
X
94.15
80.48
666.992220
93.00
66.30
39.50
LWBC
X
90,57
88,90
87.22
86.42
70,29
51.28
Table 2: UNBIASED and WORST-GROUP metries (%) for the CelebA dataset. We also report the difference between UNBIASED and WORST-GROUS as GAP. For methods without spurious attribute labels, mark the best and the second-bes) performance in bold and underline, respectively.
Backbone
Spurious
CelebA HairColor
Method
network
attribute label
UNBLASED
WORST-GROUP
GAP
Group DRO
36
Resnet50
93.1
88.5
4.6
SSA 33
Resnet50
92.8
89.8
3.0
FRM
Resnet50
95.6
47.2
48.4
CVaR DRO
27
Resnet50
82.4
64.4
18.0
LIF 32
Resnet50
86.0
70.6
15.4
EIIL
7
Resnet50
X
91.9
83.3
8.6
JTT 29
Resnet50
88.0
81.1
6.9
SSL+ERM
Resnet 8
80.5
38.5
42.0
LWBC
Resnet1 8
88.9
85.5
3.4
classifier, the committee gradually becomes debiased along with the main classifier. Hence, samples correctly predicted by the main classifier are less weighted and those with incorrect predictions are more weighted by the committee.
After the warm-up training, the main classifier and the auxiliary classifiers are alternately updated with given mini-batch at each iteration. First, we forward every sample in mini-batch each auxiliary classifier and then compute weights of the samples using predictions of the auxiliary classifiers (Eq. (2). With the weights, the main classifier is updated by Eq. (3) and the knowledge of the updated main classifier is transferred to the auxiliary classifiers by Eq. (4). Thus, the auxiliary classifiers are
updated by minimizing both losses of Eq. and Eq. 44
Loommitee = (1 CCE + ACKD.
(5)
where À is a balancing dyper-paramen
4
Experiments
4.1 Setup
Implementation details. We adopt ResNet-18 [15] as the self-supervised model and train it following the strategy of BYOL (14] on the target dataset. Since the color information is key feature for HairColor classification on the CelebA dataset, we vary only brightness and contrast for data augmentation of color distortion when training the model on CelebA. Since NICO and BAR are very small datasets, we initialize the self self-supervised model with ImageNet 9 pretrained parameters; except for the experiments using BAR and NICO, self-supervised model trained from scratch. We use the self-supervised ResNet-18 18 as backbone network except for the last fully connected layer. We set the batch size (64, 64, 128, 256), learning rate to (1e-3, le-3, le-4, 6e-3), the size of the committee to (30, 30, 30, 40), the size of subset Si to (10, 10, 80, 300), X to (0.9, 0.6, 0.6, 0.6),

---

Table 3: VALIDATION, UNBIASED, and TEST metrics (%) evaluated on the ImageNet-9 and ImageNet- A datasets. For methods without spurious attribute labels, we mark the best and the second-best performance in bold and underline, respectively.
Spurious
ImageNet-9
ImageNet-A
Method
attribute label
VALIDATION
UNBIASED
TEST
StylisedIN 13
88.4 88.4
86.0 86.6
24.6
LeannedMixin
/
64. 1:40
62.7
15.0
RUBi 5
/
90.5
88.6204
27.7+2.1
ERM
X
90.8
88, 88.8
24. 1.11
Biased (BagNet18) 4
X
67.7
65.9
18.8 18.8
ReBias
X
91.9
90.5
29.6+16
LIF 32
X
86.0
85.0
24.6
CaaM
X
95.7
95.2
32.8
SSL+ERM
X
94_18
93.18
34.21
LWBC
X
94.03
93.04=0.33
35.97
and r {1, 1, 1, 2.5), respectively for (BAR, NICO, Imagenet-9, CelebA and 0.02. Note that we run LWBC on 3 random seeds and report the average and the standard deviation.
Evaluation metrics. Six metrics are adopted for evaluation. VALIDATION TEST: average accuracy on validation test splits. GUIDING: average accuracy on bias guiding samples per class. CONFLICT- ING: average accuracy on bias conflicting samples per class. UNBIASED: average of GUIDING and CONFLICTING per class. WORST-GROUP: minimum average accuracy of group; we can group the validation and test samples by the target and the bias.
4.2 Datasets
CelebA. CelebA 31 is a dataset for face recognition where each sample is labeled with 40 attributes. Following the experiment configuration suggested by Nam et al. [32]. we focus on HairColor or and HeavyMakeup attributes that are spuriously correlated with Gender attributes. i.e., most of the CelebA images with blond-hair are women. As result, the biased model suffers from performance degradation when predicting HairColor attribute on males. Therefore, we use HairColor as the target attribute and Gender as a spurious attribute, the same HeavyMakeup.
ImageNet-9. ImageNet-99 20 is a subset of ImageNet 35] containing nine super-classes. Following the setting adopted by Bahng et al. (3), we conduct experiments with 54,600 training images and 2, 2,100 validation images. Netge has been known to have a correlations between object class and image texture. We follow the evaluation scheme adopted by Bahng al. (3. and we report the unbiased accuracy of the validation set, which is computed as average accuracy on every object-texture combination.
ImageNet-A. ImageNet-A 17
contains real-world images misclassified by an ImageNet- ResNet 50 [13]. Since such failures caused when the model too heavily relies colors, textures, and backgrounds, ImageNet-A could be regarded as a bias-conflicting set w.r.t. various ImageNet biases. This dataset is used only for evaluating model trained ImageNet-9.
BAR. The Biased Action Recognition (BAR) dataset [32 is real-world image dataset intentionally designed to exhibit spurious correlations between human action and place on its images. Originally the training set of BAR consists of only bias- bias-guiding samples, and its test set consists of only bias- conflicting samples. In our setting, use 10% of the original BAR training set as validation and set the bias-conflicting ratio of the training set to 1%.
NICO. NICO [6 is real-world dataset for simulating distriibuttonno image classification
scenarios. Following the setting used by Wang et [42), we use an animal subset of NICO, which is labeled with 10 object and 10 context classes for evaluating the debiasing methods. The training set consists of context classes per object class and they are long-tailed distributed (e.g., dog images are more frequently coupled with the 'on grass' context than any of the other 6 contexts). The validation and test sets consist of seen context classes and 3 unseen context classes per object class. We verify the ability of debiasing model from object-context correlations through evaluation NICO.
7

---

Table 4: CONFLICTING, VALIDATION, and TEST metrics (%) evaluated on the BAR dataset (a) and the NICO dataset (b). For methods without spurious attribute labels, we mark the best and the second-best performance in bold and underline, respectively.
Spurious attribute label
NICO
Method
VALIDATION
TEST
Spurious attribute label
BAR
Cutout RUBi
43.69
43.77
Method
43.86
44.37
CONFLICTING
IRM
40.62
41.46
ERM
X
35.32
Unshuffie
59
43.15
43.00
ReBias
3
X
37.02.+0.26
REx 24
V
41.00
41.15
LIF
32
X
48. 48.1510.99
ERM
X
43.77
42.61
LDD
26
X
52.31
CBAM
45
X
42.15
42.46
SSL+ERM
X
60.88
ReBias
X
44,92
45.23
LWBC
X
62.03
LIF 32
X 41.83
40.18
CaaM 42
X 46.38
46.62
(a) BAR dataset
SSL+ERM
X
55.63.10.54
52.24
LWBC
X
56.05.10.45
52.84103
(b) NICO dataset
Table Ablation studies using WORST-GROUP metric (%) on the CelebA HairColor dataset. We
study the impact of learning from single biased classifier (row 2), learning by committee (row 3),
and transferring the knowledge of the main classifier (row 4). We mark the best performance bold.
Method
CelebA HairColor
SSL
ERM
Single
Committee
KD
WORST-GROUP
38.5
X
/
X
X
64. +24
X
X
/
X
81.3+2.
X
X
/
85.5
100
98
Unb ased val accuracy
Irain set accuracy
02-
02
classifier
90
classifeer wto
89
clessil w/o
Committee trange]
Commitee trange!
mean
88
Committee wio KD (range)
Committee w/o KD (range)
std
20
87
Epoch
(a) ImageNet-9
(b) CelebA HairColor
Figure 3: Effect of knowledge distillation (Eq. 4b and the size of the committee m. (a) Accuracy of the main classifier and the committee with or without KD loss in ImageNet-9 train (left) and unbiased validation set (right). (b) WORST- accuracy of LWBC versus the number of classifiers within the committee, where the green dot indicates the value used in the main paper.
4.3 Quantitative results
LWBC shows superior classification accuracy among the methods that do not use the spurious attribute label on the five real-world datasets. In Tables & [2, we observe LWBC outperforms existing debiasing methods using no spurious attribute label and shows comparable CONFLICTING performance with methods that exploit spurious attribute labels on CelebA, which reflects gender bias in the real world. Especially, the gap between the average accuracy of groups and worst-group accuracy of LWBC much smaller than the other methods, suggesting that our model fairly predicts sample that belongs to each group. Table 3 shows the results on ImageNet-9, which is dominated by

---

50
Sample weight
30
20
10
0.2
0.4
1.0
The ratio of correctly predicted classifiers
(a) CelehA
Test sample (hias-conflicting)
50
Sample weight
40
30
20
10
0
0.2
0.4
0.6
1.0
The ratio of correctly predicted classifiers
(b) BAR
Figure 4: Qualitative examples of sample weighting by LWBC on CelebA and BAR. The graph represents the weight function in Eq. (2) along with example images corresponding to specific weights. (a) The examples the CelebA dataset. HairColor is the target and Gender is the bias. A majority of blond-haiz images on the training set is female. (b) The examples diving class on the BAR dataset. Action is the target and Place is the bias. A majority of diving images on the training set include body water or the surface a body of water. The red bordered images are test samples which are bias-confficting samples.
texture bias, and ImageNet-A, which is regarded bias- bias-confficting set of ImageNet. LWBC is 9.7% better than CaaM [43] on the ImageNet-A dataset, i.e., LWBC is robust texture bias and various ImageNet biases. Table shows results on the BAR dataset. LWBC is 18.6% better than LDD, the previous state of the art. Compared with LfF 32 and L.D (26,, which are debiasing methods with single biased classifier, learning a debiased classifier with the biased committee is more effective. Table shows the results on the NICO dataset. LWBC is 13.3% better than CaaM, which suggests that LWBC is better generalized unseen spurious attributes. Note that the validation and test set of NICO have unseen context classes and are unbiased.
4.4 Ablation study
Self-supervised representation solid baseline. We empirically investigate the potential of self-supervised representation as a solid bascline for the debiasing task. We train classifier on the top of the self-supervised representation by ERM. The results are denoted by 'SSL+ERM" and compared
with 'ERM', which is fully supervised classification model in Table I 2 4a 46
'SSL+ERM"
outperforms not only ERM but also the previous state-of-the-art all the datasets except for CelebA. 'SSL+ERM' is less biased than the model trained by fully supervised learning.
Importance of each module in LWBC. Table 5 demonstrates through ablation studies (29 32 23
26]: (1) learning from a single biased classifier, (2) learning with committee, and (3) transferring knowledge of the main classifier: First, we train a classifier by ERM (row 1) then assign a weight value
50 to the wrongly predicted samples and other samples. Then re-training the classifier with the weights (row 2). Comparing these two results demonstrates that ue-weighting scheme with biased

---

classifier is effective to debiasing a classifier. Then we increase the number of biased classifiers and compute the sample weights using our weight function Eq. (2) (row 3). Learning with the committee shows a remarkable improvement in the worst-group accuracy. Moreover, knowledge distillation that enables the committee to interact with the main classifier further improves performance (row 4).
Effectiveness of transterring knowledge of the main classifier. Figure 3h shows the range of unbiased validation accuracy of classifiers the committee and unbiased validation accuracy of the main classifier during training. The mean accuracy of classifiers in the committee gradually increases following the accuracy of the main classifier. Also, the accuracy of the main classifier gradually increases as training progresses. On the other hand, the accuracy of classifiers in the committee trained without KD loss does not increase or even decrease.
Number of classifiers. We compare the results of the main classifier trained with the different number of auxiliary classifiers. Figure 3 shows the worst-group accuracy versus the number of auxiliary classifiers m on CelebA. With a single auxiliary classifier, the main classifier shows the lowest worst-group accuracy, but the accuracy increases as m increases. When larger than 40, the number of classifiers has little effect on learning the main classifier.
4.5 Qualitative results
F shows the graph of the weight function in Eq. (2) along with example images corresponding to specific weights. LWBC not only up-weights the bias-confiicting samples and down- wwinhw the bias-guiding samples but also imposes fine-grained weights according to the difficulty of samples. For example, majority of training samples of diving class on BAR are images of scuba diving or those of falling into the water with a blue background. In contrast, test samples of diving class sport images of falling into water from platform springboard. As illustrated in Figure 4b, LWBC imposes fine-gre one-grained weights based on the proportion of the background in the image as well as down- weighting scuba diving images and up- weiehtttn falling from platform images. The results demonstrate that LWBC BBC more precisely distinguishes samples according to bias attributes.
5 Limitations
Self-supervised learning increases overall training complexity as it usually takes longer time for convergence than supervised learning. Also, since the committee is designed as a bootstrapped ensemble, it involves randomness in training and thus causes noticeable performance fluctuation on CelebA, although it demonstrates stable performance on the other datasets.
6 Conclusion
We have proposed new method for learning debiased classifier with committee of auxiliary classifiers. The committee is learned in a way that consensus on predictions of its classifiers offers strong and reliable cue identify and weight bias-conflicting data. The main debiased classifier is then trained with an emphasis on the bjas-conffictin data to reduce the effect of bias-guiding counterparts. The committee also trained to be debiased gradually along with the main classifier so that it highlights more challenging data as training progresses. Moreover, we demonstrated that self- supervised learning a solid yet unexplored baseline for debiasing. Coupled with self-supervised feature extractor, our method achieved state-of- state-of-the-ar by large margins on most existing real-world datasets.
Acknowledgments and Disclosure of Funding
This work was supported by the NRF grants and the IITP grants funded by the Ministry of Education and the Ministry of Science and ICT, Korea 2022R AAALA00052254. 20%; LTTP-2022--000290 30%; IITP- LTT-2001-0-01196, 10%).
40%; NRF.
10

---

References
[1] Faruk Ahmed, Yoshua Bengio, Harm van Seijen, and Aaron Courville. Systematic generalisation with group invariant predictions. In Proc. International Conference on Learning Representations (ICLR), 2021.
[2] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXi::/907.02800 1907.0283, 2019.
[3] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In Proc. International Conference on Machine Learning (ICML), pages 528 328-539. 2020.
[4] Wieland Brendel and Matthias Bethge. Approximating enns with bag-of- bag-of-local-Fert lectl-features models works surprisingly well on imagenet. arXiv preprint arXiv:/90k 00760, 2019.
[5] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, al. Rubi: Reducing unimodal biases for visual question answering. In Proc. Neural Information Processing Systems (NeurIPS), volume 32, pages 841-852, 2019.
[6] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't 't take the easy way out: Ensemble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4069-4082, 2019.
[7] Elliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. In International Conference on Machine Learning, pages 2189-22000 PMLR, 2021,
[8] Luke Darlow, Stanistaw Jastrzębski. and Amos Storkey. Latent adversarial debiasing: Mitigating collider bias in deep neural networks. arXiv preprint arXiv:2201..1448 2020.
[9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: a large- scale hierarchical image database. In Proc. IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2009.
[10] Terrance VeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04555, 2017.
[11] Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv: 1rX1::19/2200257,2011
[12] Robert Geirhos, Jörn Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence 2(11)):66 1)66666733 2020.
[13] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet- trained CNNs are biased towards texture; increasing shape bias im- proves accuracy and robustness. In Proc. International Conference on Learning Representations (ICLR), 2019.
[14] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach self self-supervised learning. In Proc. Neural Information Processing Systems (NeurIPS), 2020.
[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. IEEE Conference Computer Vision and Pattern Recognition ( (CVPR), June 2016.
[16] Yue He, Zheyan Shen, and Peng Cui. Towards non-iid image classification: A dataset and baselines. In Pattern Recognition, volume 110, page 107383. Elsevier, 2021.
[17] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. Proc. IEEE Conference on Computer Vision and Patlern Recognition (CVPR), pages 15262-15271, 2021.

---

[18] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: 150..0233, 2015.
[19] Zeyi Huang, Haohan Wang. Eric P Xing, and Dong Huang. Self-challenging improves cross- domain generalization. In Proc. European Conference on Computer Vision (ECCV), pages 124-140. Springer, 2020.
[20] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Alek- sander Madry. Adversarial examples are not bugs, they are features. In Proc. Neural Information Processing Systems (NeurIPS), volume 32, pages 125-136, 2019.
[21] Sangwon Jung, Sanghyuk Chun, and Taesup Moon. Learning fair classifiers with partially annotated group labels. arXiv preprint arXiv:21 14581,, 2021.
[22] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9012-9022 2019.
[23] Eungyeup Kim, Jihyeon Lee, and Jaegul Choo. Biaswap: Removing dataset bias with bias- tailored swapping augmentation. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 14992-15001, 2021.
[24] David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapo- lation (rex). In Proc. International Conference on Machine Learning (ICML), pages 5815-5826.
PMLR, 2021.
[25] Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning. In Proc. Neural Information Processing Systems (NeurIPS), volume 33, pages 728-740 2020.
[26] Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning debiased representation via disentangled feature augmentation. In Proc. Neural Information Processing Systems (NeurIPS), volume 34, 2021.
[27] Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distri- butionally robust optimization. In Proc. Neural Information Processing Systems (NeurIPS), volume 33, pages 8847-8860, 2020.
[28] Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 9572-9581, 2019.
[29] Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Proc. International Conference on Muchine Learning (ICML), pages 6781-6792. PMLR, 2021.
[30] Z. Liu, X. Li, P. Luo, C. C. Loy, and X. Tang. Semantic image segmentation via deep parsing network. In Proc. IEEE International Conference on Computer Vision (ICCV) 2015.
[31] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. Proc. IEEE International Conference on Computer Vision (ICCV), pages 3730-3738, 2015.
[32] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Proc. Neural Information Processing Systems (NeurIPS), 2020.
[33] Junhyun Nam, Jachyung Kim, Jaeho Lee, and Jinwoo Shin, Spread spurious attribute: Improving worst-group accuracy with spurious attribute estimation. In Proc. International Conference on Learning Representations (ICLR), 2022.
12

---

[34] Mohammad Pezeshki, Oumar Kaba, Yoshua Bengio, Aaron C Courville, Doina Precup, and Guillaume Lajoie. Gradient starvation: A learning proclivity in neural networks. In Proc. Neural Information Processing Systems (NeurIPS), volume 34, pages 1256-1272, 2021.
[35] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), pages 1-42, April 2015.
[36] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In Proc. International Conference on Learning Representations (ICLR), 2019.
[37] Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine- tine-grained robustness in coarse- coarse-grained classification problems. In Proc. Neural Information Processing Systems (NeurIPS), volume 33, pages 19339 19352,, 2020.
[38] Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentan- gling deep representations for bias correction. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 13508-13517, 2021.
[39] Damien Teney, Ehsan Abbasnejad, and Anton van den Hengel. Unshuffling data for improved generalization in visual question answering. In Proc. IEEE International Conference Computer Vision (ICCV), pages 1417-1427, 2021.
[40] Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
[41] Haohan Wang. Zexue He, Zachary C Lipton, and Eric Xing. Learning robust representa- tions by projecting superficial statistics out. In Proc. International Conference on Learning Representations (ICI.R), 2018.
[42] Tan Wang, Chang Zhou, Qianru Sun, and Hanwang Zhang. Causal attention for unbiased visual recognition. In Proc. IEEE International Conference Computer Vision (ICCV), pages 3091-3100, 2021.
[43] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proc. European Conference on Computer Vision (ECCV), pages 3-19,0016
[44] Zhilu Zhang and Mert R Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Proc. Neural Information Processing Systems (NeurIPS), 2018.
[45] Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, and Jiebo Luo. Learning bias-invariant representation by cross-sample mutual information minimization. In Proc. IEEE International Conference on Computer Vision (ICCV), pages 15002-15012, October 2021.
13


Reviews:
[{"review_id": "yHkWrirg-Lr", "reviewer": "Reviewer_Ff7t", "summary": "The paper proposes a new approach to learning a debiased classifier with no spurious attribute label. Samples are weighted according to the consensus. The main classifier and committee classifier interacts. Committee classifier uses information about samples difficult to classify by the main classifier as distilled knowledge and integrates this into the loss function used to train the committee classifiers. Self-supervised representation obtained by BYOL is used as backbone.", "questions": "- BYOL is trained on the target dataset. It would be interesting to see how the representations learned this way on a limited data would compare to those learned from DINO (base or small VIT) pretrained on ImageNet. \n\n- It looks like different hyperparameters are used for each dataset. How are these (learning rate, batch size, committee size, \\lambda, \\tau  etc.} optimized?\n\n- LBC outperforms SSL+ERM on the CelebA dataset but is on par with it on the ImageNet (as well as BAR and NICO) datasets. Is this because bias is not as strong in texture-heavy images of ImageNet as in CelebA dataset?\n\n- In the light of the limitations stated in the paper how can you ensure the improvement in CelebA dataset is real and is not due to fluctuations of randomness in the experimental process that affect smaller datasets more?\n\n- Lines 206-207: “Therefore, we use HairColor as the target attribute and Gender as a spurious attribute, the same as HeavyMakeup.” This sentence is confusing. The proposed approach is not supposed to use the explicit knowledge of attributes causing bias. It is not clear  what it means to select these attributes as “target” and “spurious” attributes and how this information was used during training. \n\n\nCorrections\nLine 160: We identity", "limitations": "Algorithmic limitations are explained but societal impact is not.  What happens if the proposed algorithm fails to debias the data? This should have some ethical implications.\n\n-------------------------------\nAuthors responses address my main concerns. As a result I update my score to Weak Accept.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Distilling the knowledge of difficult samples from the main classifier to train committee classifiers is an intriguing idea. Doing this with a self-supervised backbone is also carefully considered.\n\n- Random subsets of training may not necessarily be dominated by bias-guiding samples especially in long-tailed datasets\n\n- Figure 1 does not demonstrate that a committee of classifiers focus more on bias-conflicting samples. Higher enrichment can possibly be achieved due to other factors as well?\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "yDhet_7WhyV", "reviewer": "Reviewer_bx4Y", "summary": "The paper proposes LBC to mitigate biases without bias labels. Motivated by the limitation of LfF’s single biased classifier, LBC overcomes this limitation by learning a committee of biased classifiers. The LBC is also trained with the knowledge of the main classifier through knowledge distillation. Furthermore, the paper explored the role of self-supervised representation in bias mitigation. The experiments are conducted on multiple datasets, showing better performance of the proposed method.", "questions": "Questions\n* Clarify the inconsistent results of ERM and LfF between Table 1 and Table 2.\n* Clarify the split used on CelebA.\n* Clarify if the new split of the BAR dataset will be released for reproducibility.\n\n\nSuggestions\n* Cite and add discussion on other debiasing methods without spurious attribute labels.\n", "limitations": "Yes. The limitations are discussed in Sec. 5.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n* The paper is well-written (except for a few typos) and easy to follow.\n* The paper is well-motivated by the limitation of LfF’s single biased classifier (Fig. 1 and Sec. 1). The proposed method uses a committee with multiple biased classifiers to address this problem well, which has a novel contribution to this field.\n* Using self-supervised representation for bias mitigation is also a novel exploration.\n\n\nWeaknesses\n* A few methods should also be cited and discussed in Section 2.2: PGI (Ahmed et al. 2021), ARL (Lahoti et al. 2020), George (Sohoni et al. 2020), and SD (Pezeshki et al. 2021), which are all debiasing methods without using spurious attribute labels.\nCould the authors clarify why the results of ERM and LfF differ between Table 1 and Table 2? For example, ERM’s unbiased accuracy in Table 1 is 70.25 but becomes 95.6 in Table 2.\n\n* Could the authors clarify which split is used to report the results on CelebA? The paper mentioned that the configuration follows [31] (L203), which uses the validation set to report the results. However, the experiment also uses CelebA’s validation set to tune the hyperparameter (L452-453).\n\n* Could the authors promise to release the new split on BAR (L220-221)? Otherwise, it will become hard for future methods to benchmark.\n\nTypos\n\n* L51, 54, 68, and caption of Fig. 3: LCB -> LBC\n* L160: identity -> identify\n* L188: celebA -> CelebA\n\n\nAlthough I have a few questions about the paper, the proposed method is generally novel and performs well on various datasets. My current rating is “Borderline accept.” I am willing to raise my rating if the authors can address my concerns in the response.\n\nReference\n\nAhmed, Faruk, Yoshua Bengio, Harm van Seijen, and Aaron Courville. 2021. “Systematic Generalisation with Group Invariant Predictions.” In International Conference on Learning Representations. https://openreview.net/forum?id=b9PoimzZFJ.\n\nSohoni, Nimit S., Jared A. Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. 2020. “No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems.” In Advances in Neural Information Processing Systems. http://arxiv.org/abs/2011.12945.\n\nLahoti, Preethi, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. 2020. “Fairness without Demographics through Adversarially Reweighted Learning.” In Advances in Neural Information Processing Systems. Vol. 33. https://papers.nips.cc/paper/2020/hash/07fc15c9d169ee48573edd749d25945d-Abstract.html.\n\nPezeshki, Mohammad, Sékou-Oumar Kaba, Yoshua Bengio, Aaron Courville, Doina Precup, and Guillaume Lajoie. 2021. “Gradient Starvation: A Learning Proclivity in Neural Networks.” In Advances in Neural Information Processing Systems. https://openreview.net/forum?id=aExAsh1UHZo.\n\n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "NQAsyOVLv1", "reviewer": "Reviewer_Diuc", "summary": "This paper considers the problem of learning to classify from spuriously-correlated data when annotations for the spurious attributes are absent. The authors propose to use the disagreement between an ensemble of classifiers trained on random subsets of the data (bagging) \nto generate sample-weights for the classification loss. In addition to being trained on their respective subsets with supervision\nfrom the ground-truth labels, they are provided with an additional error signal from the remaining data in the form of a distillation loss\nwith respect to the 'main' classifier, trained using the full dataset. The authors make use of self-supervision (BYOL) to train the backbone, on top of which the auxiliary classifiers are trained, finding this alone to enhance robustness and complementary to the aforementioned reweighting. The proposed method, LBC, is evaluated on five datasets exhibiting with known spurious correlations and compared against a variety of baselines that are both supervised and unsupervised w.r.t. the spurious attribute. Qualitative results are additionally illustrated for the CelebA dataset to provide insight into the sample weights and whether they align with the biases in the dataset.\n", "questions": "- Is the lack of a stop-gradient on the main classifier, $g$, intentional? Generally one doesn't allow the gradient to propagate through to the teacher network when performing knowledge distillation. If this is intentional, was it the result of testing or is there some theoretical justification for it?\n- How essential is bagging? Could different random initialisations alone provide sufficient diversity, as suggested by the literature on deep ensembles?\n- Why does the performance degrade past 40 ensemble members, rather than just plateauing?\n- How important is the warm-up and how many iterations of it are typically required for stable performance?\n", "limitations": "I am generally satisfied with the extent to which the limitations of the proposed method have been discussed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- The idea of using a disagreement between ensemble members to generate sample-weights with the view to\nimprove fairness seems to be relatively novel\n-  The authors do a good job of setting up the problem, establishing sufficient context for it in light of previous work, and motivating the use of a committee for identifying bias-conflicting samples through the analysis conducted in Figure 1.\n- The proposed method is simple, intuitive, and demonstrably effective at alleviating spurious correlations.\n- Evaluation was conducted using a good range of relevant datasets and baseline methods.\n- The paper is well-presented, in that the results are easy to interpret, the proposed method can be understood based on its visual representation in Figure 2 alone, and the pseudocode is sufficiently detailed to make clear the more technical elements.\n - The paper is generally generally clear and well-written/structured, though there are some moderate grammatical errors, such as on line 225 (elaborated on below).\n - Good set of ablation studies demonstrating the contribution of each component of LBC, in turn, as well as the relationship\n between the size of the committee and performance. That said, it would be of interest to ablate the self-supervised component\n on its own to garner insight into how complementary it is to the proposed method, particularly as the dataset-task combination on which\n the ablations were performed show worse performance for SSL+ERM than ERM on its own (by a significant margin). Also of interest would be  how LBC compares with deep ensembling, where all ensemble members are trained on the full dataset with no distillation component.\n - Results are computed as the average over several replicates, alongside the with associated error intervals.\n\n### Weaknesses\n- The claim that this is the first paper to show the relationship between self-supervision and debiasing isn't entirely justified. \nThere are, for instance, [1] showing that large-scaling pre-training with SWAV improves fairness on downstream computer vision tasks,\nand [2] showing that models trained with self-supervision are more robust to adversarial perturbations and label corruptions.\n- Figure 4 isn't referenced in the main text. There should be some discussion as to what is shown here and whether it aligns with expectation (which it does seem to given that disagreement is highest for those samples from the minority group - Blond-Male).\n- While the related work section is reasonably thorough, there is a gap when it comes to discussion \nensemble-based-debiasing methods (e.g. [3, 4]) and ensembling in the context of robustness generally, having obvious relevance to the proposed method.\n- Despite the motivation encapsulated in Figure 1, there is a lack of theoretical guarantees regarding why and when should this bootstrapping approach should work \nfor spurious correlations in general.\n- The term 'enrichment', as used in Figure 1, is not explicitly defined.\n\n### Misc.\n- Liu et al. (Just Train Twice) is cited using two different references.\n- 'Self sup' should be SSL in the table of ablations for consistency with 'SSL+ERM'.\n- The parenthetical remark \"e.g., most dog images are appeared ‘on grass’ the others are appeared on 6 context\" doesn't make complete sense -- I assume this is supposed to read something to the effect of \"More dog images appear on the 'on grass' context than any of the other 6 contexts\"\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0R1uxeUfDM", "reviewer": "Reviewer_zA6x", "summary": "The paper proposes a method to de-bias classifiers and make them robust to spurious correlations, which is mainly based on learning an ensemble of intentionally biased models which will help to identify minority (bias-conflicting) examples in train data and increase their importance / weight while training a final de-biased model. Additional details of the method include (1) pre-training representations with self-supervised learning and learning shallow models on top of these representations, and (2) alternate updating of the biased ensemble and de-biased classifier where the knowledge is transferred from the classifier to the ensemble. The approach shows strong performance on a number of benchmarks: CelebA, ImageNet-9, ImageNet-A, BAR and NICO.", "questions": "Please see section \"Quality\" in \"Strengths And Weaknesses\".", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Originality\nThe proposed method is novel, to the best of my knowledge. Although prior works proposed to use an internationally biased model to detect minority examples in the past (e.g., [1], [2]) this papers is the first to proposed a biased bootstrapped ensemble and shows that it increases robustness of the final classifier model. The paper is based on a combination of known techniques (bootstrapped ensemble, using auxiliary module to identify biases, self-supervised learning, knowledge distillation), however, the proposed approach shows strong performance in practice, and, thus, I believe is a valuable contribution.\n\n\n### Quality\n\n1. My main concern for the paper evaluation is regarding the hyperparameter tuning: the authors described which hyperparameters they picked in lines 192-195, but it is not mentioned how exactly they were tuned and which range of values was considered for each hyper-parameter. It was shown in prior works on spurious correlation robustness that the worst-group results may vary a lot depending on whether \"in-distribution\" or \"out-of-distribution\" data was used for hyper-parameter tuning (i.e. which portion of the validation set was bias-guiding vs bias-conflicting) and whether the hyper-parameters were chosen based on average accuracy or worst-group accuracy on validation (the latter would imply that we assume having access to bias labels for validation data). Also e.g. on Figure 3(b) it is shown that worst-group performance varies quite a bit depending on the number of classifiers in the committee and it is mentioned that $m$ was chosen to be the one corresponding to the best performance -- but it is not clear whether this number was chosen based on best performance on validation or test.\n \nIt would be helpful to know which assumptions on validation data the authors made and the exact strategy for hyper-parameter tuning. \n \n2. While the observation that self-supervised (SSL) training improves the robustness to biases on most benchmarks is interesting, it sounds a bit disconnected from the rest of the paper. I think it is also not fair to compare SSL+LBC to other prior methods which were not using SSL pretraining (especially considering that SSL+ERM is already beating most methods on all benchmarks except CelebA). I think to perform a careful and fair ablation it would be needed to (1) compare LBC w/o SSL pretraining to prior works (to isolate the effect of improved representations), (2) with fixed SSL features, train and compare different prior methods and LBC on top of these features. \n\n3. I found it a bit surprising that such small subsets of data $S_l$ were used to train individual networks in the committee, I guess it is related to the fact that the authors train quite shallow MLP models on top of SSL features. I would be really interested to see how important boostrapping is in general for the method, e.g. what if one swapped bootstrapped ensemble with regular deep ensemble? Would the performance change much in cases when SSL pretraining is used and when there is no SSL pretraining (so we are training not a shallow but deep model)? It would also be interesting to see how the two parameters the size of $S_l$ and the number of members in the committee interact: is it the case that we might get the same performance with the smaller number of networks in the committee if they are trained on bigger splits of train data?\n\n4. It would be helpful for understanding of the method to see the plot similar to Figure 1(a) but for the proposed approach showing what the ratio of bias-conflicting examples among the examples identified by a biased committee is.\n\n5. Since the method consists of a few independent parts (biased committee, SSL, knowledge distillation) it would be helpful to see more ablation studies varying just one of those components, e.g. how much would the KD be helpful when using only one biased model as opposed to the committee? Similarly, what would the performance be with a single biased model on SSL features? (That would be quite related to reproducing a method like JTT [2] on SSL featured as mentioned in point 3 above) \n\n6. It would be interesting how this method works on other data modalities, e.g. NLP spurious correlation tasks such as CivilComments or MultiNLI.\n\n\n### Clarity\nThe paper is clearly written and easy to follow. A minor note is that it might be more natural to swap 4.1 and 4.2 and first describe the datasets in the Experiments section since they are mentioned in 4.1. \n\n\n### Significance\nI think although the method is based on a combination of known techniques, it significantly advances the performance on robustness benchmarks and especially if the authors address the evaluation concern described above the paper will be interesting for the robustness community.\n\n\nReferences\n\n[1] Nam et al, Learning from Failure: Training Debiased Classifier from Biased Classifier\n\n[2] Liu et al, Just Train Twice: Improving Group Robustness without Training Group Information\n\n[3] Lee et al, Diversify and Disambiguate: Learning From Underspecified Data", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept