Unbiased Classification through Bias-Contrastive and Bias-Balanced Learning
Authors: Youngkyu Hong
Eunho Yang
Venue: NeurIPS 2021 Poster (2021)
Keywords: Debiasing
Fairness
Contrastive Learning

Abstract: Datasets for training machine learning models tend to be biased unless the data is collected with complete care. In such a biased dataset, models are susceptible to making predictions based on the biased features of the data. The biased model fails to generalize to the case where correlations between biases and targets are shifted. To mitigate this, we propose Bias-Contrastive (BiasCon) loss based on the contrastive learning framework, which effectively leverages the knowledge of bias labels. We further suggest Bias-Balanced (BiasBal) regression which trains the classification model toward the data distribution with balanced target-bias correlation. Furthermore, we propose Soft Bias-Contrastive (SoftCon) loss which handles the dataset without bias labels by softening the pair assignment of the BiasCon loss based on the distance in the feature space of the bias-capturing model. Our experiments show that our proposed methods significantly improve previous debiasing methods in various realistic datasets.

Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning
Youngkyu Hong Naver Al Lab
Eunho Yang KAIST, AITRICS eunhoy@kaist ac.
Abstract
Datasets for training machine learning models tend to be biased unless the data is collected with complete care. In such biased dataset, models are susceptible making predictions based on the biased features of the data. The biased model fails to generalize to the case where correlations between biases and targets are shifted. To mitigate this, we propose Bias-Contrastive (BiasCon) loss based on the contrastive learning framework, which effectively leverages the knowledge of bias labels. We further suggest Balanced (BiasBal) regression which trains the classification model toward the data distribution with balanced target- -bas correlation. Furthermore, we propose Soft Ctas-Contr (SoftCon) loss which handles the dataset without bias labels by softening the pair assignment of the BiasCon loss based on the distance in the feature space of the bias-capturing model. Our experiments show that our proposed methods significantly improve previous debiasing methods in various realistic datasets.
Introduction
Machine learning models have achieved extremely high performance in variety of tasks and domains such as computer vision and natural language processing [15, 39]. Recently, however, many concerns have arisen that such evaluation does not reflect the real-world performance of the model when deployed [35, 41J. Among others, failing especially due to the biases existing dataset can lead to serious societal side effects such prejudice or racism beyond simple algorithm failure [45, 34, 5]. If bias features are highly correlated with the object class the dataset, models tend to the bias cue for the prediction they are casier to learn but enough to achieve high accuracy even though they are not actually related to the target class [3, 33].
The failure of learning due to the bias existing in the dataset appears in various fields and tasks. To name few, in image classification task, [18, 30] discovered that state-of-thee -he-trt CNNs have texture biases. Visual question answering (VQA) model is also known to be susceptible to biases as it only uses the word occurrence in the question to generate answer [1, 11]. For example, if most of the images of banana in the train sets are yellow, the model directly answers the question "What is the color of the banana?" as "Yellow", without looking at the image of green banana. Surveillance models, which can cause serious societal problems if misclassified, tend to give biased predictions toward sensitive attributes such as race [34]. Considering the importance of the problem, many approaches have been proposed to mitigate such biases the training dataset. Depending on the presence or absence of bias information, the approaches can be divided into two categories. When the bias label is available, [28, 43] add the bias prediction branch for the unbiased prediction. [40, 10] directly regularize the feature embedding to be indistinguishable across the bias classes. When the bias label is unavailable, [11, 3, 8, 20] design
t This work done student at KAIST
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

an auxiliary biattcap models that learn only bias features and train the main model to learn orthogonal features from those models.
In this paper, we propose two powerful debiasing approaches that can be complementarly applied when the bias lahel is available. We first propose Cras-Contrastive (BiasCon) loss that extends the approaches that directly regularize the feature space by adapting the recent advances of contrastive learning in representation learning. The BiasCon loss utilizes the power of contrastive learning to promote pulling the same target class but different bias class samples closer in the feature space. We further propose the Bias- Balanced (BiasBal) loss that suppresses the utilization of bias features by optimizing toward the data distribution where the target-bias correlation is balanced. Each loss achieves ofat--of-the-rr debiasing performance and shows even higher performance when they are jointly used as they give orthogonal debiasing effects.
To extend to more realistic cases where the bias label is unavailable, feature space of the bias-capturing model can be used in estimating whether a pair of samples have the same bias features. We thus propose Soft loss weighted with the cosine distance between samples in the feature space of the bias-capturing model.
the observation that the
(SofiCon) loss, which is the BiasCon
We conduct experiments to evaluate the debiasing performance of the proposed methods. For the case where the bias label is available, we evaluate the methods on CelebA [31] and UTKFace [46], which have biases toward sensitive attributes such as gender or race. For the case where the bias label is unavailable, we use ImageNet [36] and ImageNet-A [23] assess whether the bias of our model has been removed. Our method improves the unbiased accuracy of previous methods by a large margin across all datasets in both with and without bias labels.
Our contributions can be stated as follows:
We propose a powerful debiasing method, the BiasCon loss, that effectively adapts recent advancements of contrastive learning and the BiasBal loss that further enhances debias- ing performance by optimizing the model toward distribution with uniform target- correlation.
We introduce the SoftCon loss, the extension the BiasCon loss to the case where the bias label is unavailable, that utilizes the feature space of the bias-capturing model.
We show that our losses successfully improve the debiasing performance with a large gap in various real-world datasets for both bias label available/unavailable cases.
2
Related work
Bias existing in the data and the vulnerability of machine learning algorithms to such bias has been recently studied as an important problem in various domains and tasks. [35, 41] revealed that various training sets have regularity conditions (e.g., objects should not be occluded) that are unlikely to hold in practical settings, and machine learning models trained on such data fail to generalize in the absence of such conditions. More specifically, [23, 44, 30] showed that state-of-the-art object recognition models are biased toward backgrounds or textures associated with the object class. [38] explored that overparametrized models prone the spurious correlations and can have worse test errors on minority groups. Natural language processing models are also susceptible to dataset biases. Question Answering (QA) [19, 25, 13, 12] and Visual QA [1, 11] systems make predictions using shortcuts such as the occurrence of specific words in the question rather than the context. Even large language models tend to give biased prediction toward certain gender or race [10]. Meanwhile, the severity of such biased prediction and fairness issues of deployed models are thoroughly investigated in various domains [45, 34, 5]. This brings the importance and necessity of the debiasing method, and recent research has begun to proceed in two main directions depending on the presence or absence of bias labels.
Debiasing when bias label is available An important line of work assumed the situation where the bias factors are categorical attributes (e.g., gender, race) and the meta-information about the bias label of each training data is available. [43] suggested to use per-bias classifier head to equalize the effect of bias features. [45, 32, 14] proposed fairness metrics such as demographic parity (DP) or equalized odds (EO) that measure the biasedness of the model and devised algorithms that can

---

SoftCon
Main Model
7
7
8
7 8
g
Distance-weightedd Contrastive Loss
Bias capturing Model
Debiased Feature Space
Visualized explanation of our SoftCon loss. As samples with similar bias features are close in the feature space of the bias-capton blas-capturing model, we design SoftCon loss that does the distance- weighted contrastive learning to pull pair with the same target class but with different bias features
Figure 1:
induce the fair prediction. [28] extended the idea of the domain generalization [2. 17] by adding a bias prediction branch to the model and train the model with an additional regularization term that induces the learned features to be invariant to the bias features. In the risk minimization perspective, [4, 16, 37] proposed distributionally robust optimization that aims optimize toward the worst-case group distribution shift. [40] devised regularization term with a triplet loss formulation minimize the entanglement of bias features. For the pretrained language model, [10] appended additional filtering layer and corrects the sentence embedding by maximizing mutual information. We propose novel regularization term that work upon the success of the contrastive learning framework self-supervised learning [9, 21, 27] and leverages the bias label assigning positive pair samples.
Debiasing when the bias label is unavailable It is more realistic to assume that we know the form of biases that have to be removed, but no bias label available, nor the type of bias is not limited to categorical data (e.g., texture bias of CNNs). Still, there are approaches that mitigate biases in such situations. The first line of research is to utilize the bias-capturing model [11, 3, 8, 20]. The bias-capturing model is an auxiliary model designed to learn bias features. For example, texture biases can captured by CNNs with small receptive fields [3]. these approaches, the model is promoted to learn independent features from the bias-capturing models, thus Icarning debiased features. [33] further relaxed the assumption about the prior knowledge about bias based on the observation that the bias features are learned before the target features. Specific to CNNs for the image classification task, [18, 30] removed texture bias by randomizing the texture of the training data. Our method also utilizes bias- capturpg model, but we propose more effective way to the model by considering the distance between training samples in the feature space of the model.
Method
This section begins by explaining our problem setup. Then, we propose two debiasing objectives, Slas-Contrastive (BiasCon) loss and Bias Balancing (BiasBal) loss, that effective in situations where the bias is categorical and the bias label is available. We further propose Soft Ciat-Coote (SoftCon) loss which extends the BiasCon loss more general and challenging situations where the bias is not limited to categorical data and the bias label is not available.
3.1 Problem definition
To formulate our problem, we define a target Y and bias B that both affects the generation of data X. In such data, we assume that X can be divided into signal feature X, explaining the target Y and bias feature X,, from B. For example, in the data for hair color classification task that is biased toward race (that is, Y=hair and B=race), X, includes brown hair feature, and X, includes white skin feature. Now, the bias in the dataset can be understood as a high correlation between the target class and the bias feature, i.e., the prior p(Y BB) is imbalanced. Then the model may ignore the true signal X, and instead use X2 in prediction and output p(Y[X>) as can still give high accuracy. In the hair color example above, if most white people have blonde hair, the model may output blonde every time sees the white skin color. Therefore, the biased model fails on the test data distribution with the shifted correlation between Y and B, i.c., Ptroin (Y, B) * Prest (Y, B).
3

---

Our objective is to train the model to make prediction using the true signal X, only; that is, the model takes the whole feature X and outputs p(Y [X,, X2) pY]] conditionally independent on Xs. so that the model makes indiscriminative predictions across the biases.
3.2 Bias-contrastive loss
We first consider the situation where the bias label is categorical and known for all training samples (e.g.. gender or race information of each training sample is available). Here we note the fact that a biased model tends to learn features from B and make predictions using those features, hence the samples within the same bias class are close to each other in the feature space. Thus, we devise a loss explicitly incentivise the model to pull the same target class but different bias class sample pairs closer than the other pairs.
Toward this, we borrow and revise the main idea of [27] where supervised contrastive (SupCon) loss pulls the same class samples closer using the contrastive loss and achieves better performance than training solely with the cross-entropy loss. Specifically, we turn the SupCon loss into effective loss for debiasing. named Bias- Contrastive (BiasCon) loss, that pulls samples with the same class but different bias closer while pushing the other pairs.
Following the formulation of the SupCon loss, we first construct 2/V size "multiviewed batch" by applying pair of randomized augmentation (e.g., random crop, random flip) on N data samples in single batch: ((x,, Mis b,))2² We then apply the same SupCon loss but to different positive pairs (pairs of same target class but different bias class) and formulate the BiasCon loss as follows:
exp (z) 2//T)
LiinsCon
Σ TT
Σ log jEJ()
2N
[J(I)
EaElmo 100 exp (2+ Za/T)
where Imme == [1..... 2N is the index set of multiviewed batch, J(i) = {j E Lmme - * b.} is the index set of positive samples paired with i-th sample, 24 f(x,) ) /ll (x:))ll is normalized
feature of i-th sample extracted from f. the network up to the penultimate layer, and 7 a temperature hyperparameter.
The final objective is combination of our BiasCon loss and the standard cross-entropy (CE) loss as follows:
LCE
1 N
Σ SEI
log h(xx))uu)
(2)
a T.CE + TiiisConn
where [v) = p(u[z) is the model prediction for class y. T= (1, N) is the index set of the original batch and a is a weight hyperparameter. Originally, [27] used the SupCon loss solely for the representation learning up to the penultimate layer and additionally trained the fully-connected (FC) layer upon the frozen representation with the CE loss. However in our case, samples with the same negative pairs. Therefore, it is more effective to jointly train
target and bias class are considered with CE loss as it can give signals to incorporate all the same class samples. Thus, unlike [27], we propose use the BiasCon loss as regularizer, in (3). In the Appendix D.2, we empirically show that the joint training with CE loss achieves better performance.
Another specific issue here that contrasts with the SupCon loss is that when p(bly) is highly imbal- anced, a single batch may not contain enough positive pairs because most of the same class samples in the batch will have the same bias features. This may result in unstable optimization. To handle this issue, for training set D = {x;). we separately sample the mini-batch for the BiasCon loss with a modified sampling frequency Q(1) of i-th sample. We design Q(i) to oversample I; with low p(b:ly:) so that more positive pairs are available in the batch:
if p(b.l < Y min y.b p(hlv)
p(bllv))
Q(1) α
(4)
7 min n.b
O.W.
The mini-batch of size N is then sampled as (a(t)) : z00 = x, where i i~ Q(i) for j 1,..., N).
Note that this formulation clips the largest possible unnormalized Q(i) to be 7 min Q(i) so that every training data sampled with some chance. We pre-calculate p(bly) using the Y and B labels in the training set before the training.

---

It is also instructive to note that EnD [40] has the similar motivation of entangling the features of different bias class samples. However, EnD uses the triplet loss which has limitation that penalties the same class samples even after the model successfully learned the target features. Experimental results in Section 4 show the huge performance gap between ours and EnD across all tasks. More detailed comparison between two methods is available in Appendix A.2.
3.3 Bias-balanced regression
Highly imbalanced p(ylb) inhibits the learning of features (i.e., X22) that can correctly classify the minor data under the same bias label. Here, we propose novel method called bias-balanced regression, which debiases the model by optimizing the model toward the data distribution with the
uniform correlation between Y and B. Specifically, let Pirain (X, Y. B) be the data distribution of the training set and let Pa be the unbiased test data distribution with the uniform correlation between Y and B. Here, the test data can be understood as the shifted distribution from Piraén:
Pa Y, B) = Pirain (X BY,,
(5)
1 C P
B
= P. (B)
(6)
where C' is the number of target classes. Biased model fails to generalize to Pu as the correlation between Y and B in Pirain not applicable in P.- To handle this, we aim to use (X Y. B) in training to optimize the model performance on the data distribution without correlation between Y and B. However, due the bias in the training data, we cannot directly access the samples of Pu(X, Y, B) and need to achieve this goal from biased Ptrain.
Toward this, consider the model prediction p(y]or, b) and the corresponding Bayesian interpretation:
p(ulo, b) p(aly, b)p(ulb)
p(x(b)
As can be seen from (7), the bias of training data causes a distributional shift of ply(b) in the unbiased test phase, resulting in discrepancy of predictive distributions p(y |a: b) during training and test. To correct this discrepancy, inspired by [26], we derive the theorem below:
Theorem 1. (Bias-balanced Regression) Assume the multinomial logistic regression with the model (()))()) = E where Tl is the logit for class y. and the training data distribution
Perain (X,Y Y, B). Let Pa be the data distribution shifted from Ptrain to have the uniform correlation between and 1fe model h estimates the conditional probability over Pa, Le., h(x)(y) = (u(l))) then the estimate of the conditional probability over Ptrain is:
exp (Ny + log Ptrain (y (y)b))
Ptrain (y/z)) - Ey exp (Tyy + log Perain (3/ b))
(8)
and the cross-entropy loss risk associated with h over Perain is E E [L(()z),, b)], where L(h(z),y,b) is defined as follows:
exp (my + log Ptrasn (y(b))
L(h(z), b) = log Eg exp (nyy log Pevain (M' (b))
(9)
We name the loss function defined in (9) as Bias Hhus-Balaccing (BiasBal) loss:
L Bul
1
>E L(((2z)) v., iET
(10)
N
where I inherits the definition in (2). The BiasBal loss can be used as a replacement for the CE loss, hence it can be seamlessly combined with other existing debiasing methods based on CE loss. In the Appendix D.3, we empirically show that the BiasBal loss further improves the performance of existing methods. Especially, it well suited with the BiasCon loss, resulting superior performance in all tasks (Section 4...
5

---

14
Different blea
12
2,5
0.0
-7.5
Cosine distance
(a) Cosine distance distribution
(b) t-SNE of biased features
Figure 2: (a) Distribution of cosine distances between sample pairs of same different bias classes in the feature space of the bias-capturing model trained on BiasedMNIST [3]. (b) t-SNE [42] analysis of biased features shows the clear separation between bias classes.
3.4 Extension to the unknown bias label case
We extend our algorithm to more general settings that do not assume the type of biases to be categorical or the availability of bias labels. However, following [3], we still assume that have prior knowledge about the form of the bias and we can design a bias-capturing model that inherently make prediction only with Ts. i..., model predicts p(y(2)). For example, CNN model with small receptive field can only learn local features, thus it only allows to use texture biases to make predictions. (e.g.. predict zebra only with stripe features.) We pre-trained bias-capturing model as an auxiliary model for debiasing the main model.
To gain information about the bias of the dataset through the lens of the bias-capturing model, we focus on the feature space of the bias-capturing model. Intuitively, when a pair of samples are close to each other in the feature space of the bias-capturing model, it is likely that two samples have similar bias features in order for the bias crasscatuuing model to work. To check this intuition, we consider BiasedMNIST [3] dataset, MNIST [29] dataset with background color highly correlated with each target class. The data of class y has the background color of 6(y) with the probability of p and has random background color among the rest colors with the probability of (1 p) [3]. Here, use م = 0.995. We then observe the cosine distance between same/different bias class samples in the feature space of the bias-capturing model. As shown in Figure 2a, the same bias class sample pairs have near distances while different bias class sample pairs have far distances. Based on this observation, weights the BiasCon loss of each same class sample pair with cosine distance in the feature space of the baps-cap model as follows:
propose Soft Cas-Contrasttie (SoftCon) loss for unknown bias label case that
exp (zi 2//T)
LSofCCon
Σ JEJ(()
(w(2))) w(x;)) log Eae mm 1/2) exp (zi za/T)'
(11)
2N
C(i)
VEAmo
where Imv. 2; and inherit the definitions in BiasCon loss, J(1) {j Imv yj vi), w(x)) is feature embedding of sample x; extracted from the penultimate layer of the bias- capturin model, deos (u, v) : ा cosine distance and C(i) Ejen() does (w(x;), w(x))). With the help of the bipsucgp model, the SoftCon loss in (11) can be utilized without the bias label information, but perhaps someone may be concerned that its performance is highly affected by the behaviors of the bias-cap model. To address this concern, we empirically show in the Appendix D.1 that the SoftCon loss persists reasonable performance even when the bias-captu model severely fails to capture the bias features only.
The SoftCon loss also suffers from the problem that a single batch may not contain enough positive pairs, i.e., pairs with far cosine distances. However, when the bias label is unavailable, we do not know p(blg). Instead, we define the bias-sparseness of the sample x of class y(r):
((x) =
Σ deas (wox) w((a));
(12)
z' 2'ED
where D, a set of class y samples. the sample bias-sparse, only few same class samples have the same bias features, thus many positive pairs available. Therefore, oversampling the training

---

Table 1: Unbiased accuracy and standard error evaluated the BiasedMNIST [3] dataset with various target-bias correlations.
Corr
Vanilla
LNL [28]
DI [43]
EnD [40]
BiasCon
BiasBal
BC+BB
0.999
11.8
18.2
15.7
59.5
94.5:10.4
76.8#11
94,0=26
0.997
62.5#29
57.2=22
60.5
82.7.0.3
97.0=00
91.2a02
97.3
0,995
a9.
72.5.09
89,8=20
94.0=00.6
97.4410.
93,9
97.7
0.99
90.8200
86.0=22
96.9201
94.82:0.
97.7=a0
96.32n2
98.1
0.95
97.3
96.4
98.6=001
98.3+.0.1
98.6=0.1
98.1
98,9
0.9
98.2
94.0=103
98.8=0.
98.7=10.
98,9=to.
98.5=10
99.3:n00
data based on the bias-sparseness gives the similar effect with oversampling based on 1/p(bly) used in (4), In specific, we define the sampling frequency Q(i) of the i-th sample as follows:
(txx))
if t(x:) < min ((x)
Q(i) x l7 min min((x)) ((x)
(13)
o.w.
and y is the clipping hyperparameter. In practice, for each class y, we randomly select 1,024 samples and calculate ((x) over those samples.
4 Experiments
We conduct experiments to evaluate how well our proposed method performs debiasing. To do this, we train the model with a biased dataset and measure the debiasing performance with various metrics. The main metric that we focus on is unbiased accuracy [3, 33]. We first assess BiasCon and BiasBal loss on the dataset where the bias label is categorical and explicitly available. We conduct controlled experiment
BiasedMNIST [3], where each digit is highly correlated with certain background color. We also evaluate on the real-world datasets CelebA [31] and UTKFaae [46] where the dataset is biased toward sensitive attributes (e.g., gender, race). Next, we evaluate the SoftCon loss on BNNST but without bias label and 9-Class ImageNet [36, 24] that the CNN models tend to show the texture bias. We conduct all experiments with three different seeds and report the mean and standard error of each metric. Source code for our experiments publicly available'.
4.1 Performance when the bias label is available
We first evaluate our method using datasets with bias labels. In these datasets, each target class is highly correlated with certain bias class and the vanilla model tends to use bias features in the prediction. We evaluate on BiasedMNIST [3], CelebA [31], and UTKFaae [46]. Note that the bias label unavailable at the test phase.
Baselines We compare our method with baselines that utilize the bias labels for training. [28] proposed framework called "Learning Not to Learn" (LNL), which uses an additional bias prediction branch to minimize the mutual information between the model feature and the bias label.
[43] proposed a ionain-indeeeedent (DI) training scheme that trains separated fully-connected head layers for each bias class and ensembles the outputs of each layer in the final prediction. EnD [40] uses a regularizer that disentangles the features of the same bias class samples.
Controlled experiments The first experiment to validate our method is a controlled experiment using BiasedMNIST [3] dataset. As explained in Section 3.4, BiasedMNIST is an MNIST [29] dataset that has colored background highly correlated with each target class. Vanilla CNN model conditions on the background color to predict the digit as it is easier to learn and achieve high accuracy. We use the correlation of each method in various imbalance ratio. We explicitly construct a validation set and report the test unbiased accuracy at the epoch with the highest validation unbiased accuracy.
{0.9, 0.95, 0.99, 0.995, 0.997, 0.999) to evaluate the effectiveness and robustness
Table shows the performance of each method in the BiasedMNIST dataset. Our BiasCon loss shows dramatic improvement in every Especially when the correlation gets very high to 0.999,
'nttps://github htttchuuuboommeeeeeeemmeeeeeeeee cwwc/oo/////aeeeee htt//ggraaanaaaaaooee
7

---

Table 2: The unbiased / bias-conflict accuracy and standard error of the model trained on the CelebA [31] dataset.
Target
Acc. Type
Vanilla
LNL [28]
DI [43]
EnD [40]
BiasCon
BiasBal
BC+BB
Unbiased
79.0
80.1
90.9=0a
86.9+10
90.0=0.2
91.1100
91.4:n0
Blonde
Bias-conflict
59.0
61.2=1.5
86.3
76.4119
80.1
87.4
87.2
Unbiased
76.0
76.4=23
74.3=11
74.8+18
77.5207
76.0=a.
78.6218
Makeup
Bias-conflict
55.2+19
57.2-4.
53.8=1.6
53.3+36
61.3=116
57.5+2.1
63.5
Table 3: The unbiased / bias-conflict accuracy of the model trained on the UTKFace [46] dataset.
Bias
Acc. Type
Vanilla
LNL [28]
DI [43]
EnD [40]
BiasCon
BiasBal
BC+BB
Unbiased Bias-conflict
87.4
87.3=03
88.9=12
88,4203
90.3=0.2
90.4
91.0:a2
Race
:9.3
78.8=0.
89.1
81,6403
88.8=0.5
89.9 1..
89.2+..1
Unbiased
72.3+
72.9
75.6=08
73.2++3
75.7=002
78. 78.84
79.1
Age
Bias conflict
46.5
47.0=0.1
60.0=0.2
47.9+06
61.7=0.5
76.7
71.7208
our BiasCon loss still performs debiasing very well while the baselines fail to learn the target features. showing low unbiased accuracy. This shows the effectiveness and robustness of our BiasCon loss. Moreover, BiasBal loss itself shows higher performance than the previous state-of-the-art method. This indicates that simply optimizing toward uniform p(ylb) indeed makes the bias features uninformative in solving the target task to some extent, thus prevents the model from learning those features. When two methods are jointly used (BC+BB in Table 1), the performance gets even better in most cases. This additional improvement implies that two methods are offering different aspects of debiasing.
Real-world dataset To evaluate our method in the real-world datasets, we consider CelebA [31] and UTKFace [46], both datasets contain the meta-information about each data. For CelebA, follow [33] and train the binary classification model where HeavyMakeup and BlondHair are target attributes and Male is bias attribute. However, for Bl ond attribute, we find that both target classes are biased toward a non-Male e bias class. Therefore, to ensure that the dataset is biased, for Bl ondHair attribute task, we randomly select subset of the dataset so that two classes biased toward different bias classes. For UTKFace, we do the binary classification with Gender as a target attribute and Race or Age as bias attributes. We also truncate a portion of data to force the correlation between Y and B to be ply|b) = 0.9. For both datasets, we use prrettrinnd ResNet18 [22]. As previous experiments, we explicitly construct a validation set and report the test unbiased accuracy at the epoch with the highest validation unbiased accuracy. Details are available in the Appendix C.3. Here, in addition to the unbiased accuracy, we report the bias-conflict accuracy, which an accuracy of minor subgroup samples.
Results in Table 2 show that our BiasCon and BiasBal loss both outperform the previous methods with large margin. We emphasize that both methods do not hurt the performance of bias-aligning samples they also increase the total unbiased accuracy. Unlike the results in the controlled experiment, the BiasBal loss shows better performance than the BiasCon loss in the BlondHair attribute task. We do not claim the superiority of one method over the other between two methods. They give orthogonal effects in debiasing and the joint use of two methods (BC+BB) leads to further improvement. Similarly, our methods also show the superior performance in UTKeaae, as shown in Table 3. As result, the consistently superior results in two real-world datasets imply that both BiasCon and BiasBal losses successfully scale real-world datasets.
4.2 Performance when the bias label is unavailable
We now extend to the cases where the bias labels are unavailable, and the type of the bias is beyond categorical data and evaluate our SoftCon loss. Even though the bias label is unavailable, we still assume that we have prior knowledge about the bias and we can design a bias-capturing model [11.
8

---

Table 4: Unbiased accuracy and standard error evaluated on the BiasedMNNIST [3] bias labels.
dataset without
Corr.
Vanilla
LM [11]
RUBi [8]
ReBias [3]
LiF [33]
SoftCon
0.999
11.8
10.5
10,6:05
26.5+14
15.3:22
65.0
0.997
57.2.0.9
56.0a4.3
49.6
65.8
63.7
88.6:10
0.995
74.5
80.9
71.8205
75.4
90.3
93,1:102
0.99
88.9
91.5=n
85,9=0.1
88.4::0
95.1=0.1
95.2:10.
0.95
97.1
93.6
96.6=0.1
97.0+..0
97.7=002
98.0=0.1
0,9
98.2+0.1
89,5=07
97.8=0.1
98.1+
96.1=111
98.4:1.1
Table 5: Accuracy and standard error on 9-Class ImageNet [24] where bias labels are unavailable. Row with denotes results directly borrowed from [3].
Acc. Types
Vanilla
SINt [18]
LM [11]
RUBi [8]
ReBias
LfF [33]
SoftCon
Biased
94.0a0.1
88,4
79.2
93,9a02
94.0=0.2
91.2a0.
95,3102
Unbiased
92.7amz
86.6+1a
76.6+12
92.52no
92.7202
89.6403
94.1:a0
IN-A
30.5405
24.6#24
19.0+12
31.0202
30.5
29.4
34.1:00
Baselines We compare debiasing methods that work without explicit bias labels. Learned-Mixin (LM) [11] and RUBi [8] utilize ensemble learning to promote the model to learn orthogonal features from the bias-capturing model, while ReBias [3] devises a regularizer that induces such learning. LfF [33] leverages the finding that bias-conflccing samples have high loss in the early stage of learning by weighting such samples. Additionally for the ImageNet, we compare with StylizedImageNet (SIN) [18] that randomizes the texture of ImageNet data to inhibit the model from conditioning on texture biases.
Controlled experiments We use the same configuration with the controlled experiment of the known bias label case, explained in Section 4.1. We evaluate on BiasedMNIST [3], now the bias label is unavailable. For all methods, we design the bias-capturing CNN model with 1x1 filter. For the SoftCon loss, we pre-train the bias-capturing model for 80 epochs. Since we not have access to the bias labels, we cannot construct validation set. Thus, report the final test unbiased accuracy.
In Table 4, report the performance of each method BiaSedMNNIST. Our SoftCon loss shows superior performance for all correlation cases. Even for the case with the highest correlation of 0.999, SoftCon loss fairly maintains its performance while previous methods severely fail. This indicates that our distance-best based weighting successfully captures the positive pairs, i.c., sample pairs that need to be pulled closer. This also implies that even though the positive pair selection is not perfect, the concept of the bias-contrastive learning framework is robust enough to give powerful debiasing effect.
Real-world dataset We test our method on more realistic settings, 9-Class ImageNet [24], a subset of ImageNet [36] with super-classes. Following [3], we aim to train the ResNet18 [22] model from scratch without texture bias which CNN models are susceptible to have. To achieve this, we use BagNet18 [6] as a bias- caas-cap model since it mainly uses a Ixl kernel and has small receptive fields, and thus more susceptible to the local texture bias. To measure the unbiased accuracy, the texture bias label assigned by [3] and average the accuracy measured on each texture group. We also use ImageNet-A (IN-A) [23] as an additional test set as it contains bias-conflicting samples that vanilla CNN models fail to classify correctly. Due to the lack of bias labels in training set, we report results of the model after the full training of 120 epochs. Details are available in the Appendix C.5.
We report the performance each method on 9-Class ImageNet in Table 5. Results show that our SoftCon loss shows the best unbiased and ImageNet-A accuracy. This indicates that the feature space of the bias-cap model can give meaningful information about the bias even in the real-world scenario, and our SoftCon loss is effective and scalable.

---

Discussion and conclusion
In this work, we proposed the powerful debiasing method named Bias- Bias-Contrastive (BiasCon) loss, which effectively revises the contrastive learning framework for debiasing. We further improved the debiasing performance with Blas-Balanced (BiasBal) regression which minimizes the risk of the cross-entropy loss toward unbiased data distribution. To handle the case where the bias label is unavailable, we propose Soft Dias-Contrastive (SoftCon) loss by utilizing the feature space of bias-capturing model. Experiments on various real-world datasets including CelebA, UTKFaae and ImageNet show that our method significantly improves the previous debiasing approaches.
Limitations of our work The first limitation of our work is that, as in [3], our work still assumes that we know the form of the bias (e.g., texture) and we can design the bias-capturing model that mostly uses the bias features in prediction. We believe that can relax this assumption by considering features learned earlier as bias features, as in [33]. Furthermore, it is challenging and interesting future work adapt model beyond the classification task such as VQA [1, 11].
As unbiasedness is the important property that machine learning models need to achieve, expect our work gives a positive societal impact.
Societal impact
Acknowledgement
This work was supported by the National Research Foundation of Korea (NRF) grants 2201R1A5AA05992., 2019R1CIC1009199) and Institute of Information & Communications Tech- nology Planning & Evaluation (IITP) grants N0722017-0-17779, A machine learning and statistical inference framework for explainable artificial intelligence. No. N0.2011---11371, Development of brain- inspired AI with human-like intelligence, No.201 10000000,, Artificial Intelligence Graduate School Program (KAIST)) funded by the Korea government (MSIT).
10

---

References
[1] Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don't just assume; look and answer: Overcoming priors for visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[2] Martín Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. CoRR, abs/1907.02899 2019.
[3] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In International Conference on Machine Learning, pages
PMLR, 2020.
[4] Aharon Ben- Ben-Tal, Dick den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Manag. Sci., (11))3341335,,2133
[5] Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai. Man is to computer programmer woman is to homemaker? debiasing word embeddings. In NIPS, pages 4349 4357, 2016.
[6] Wieland Brendel and Matthias Bethge. Approximating CNNs with bag-of-local lecal-feaess models works surprisingly well on imagenet. In International Conference on Learning Representations, 2019.
[7] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurlPS, 2020.
[8] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. Advances in Neural Information Processing Systems, 32:841-88 852, 2019.
[9] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PMLR, 2020.
[10] Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin, Fairfil: Contrastive neural debiasing method for pretrained text encoders. In International Conference on Learning Representations, 2021.
[11] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't It take the easy way out: Ensemble based methods for avoiding known dataset biases. In EMNLP/JJCNLPP (/), pages 4067-4080. Association for Computational Linguistics, 2019.
[12] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try are, the A12 reasoning challenge. CoRR, abs/1803.00557, 2018.
[13] Peter Clark, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter D. Turney, and Daniel Khashabi. Combining retrieval, statisties, and inference to answer elementary science questions. In AAAI, pages 2580-2586. AAAI Press, 2016.
[14] Elliot Creager, David Madras, Jörn-Henrik Jacobsen, Marissa Weis, Kevin Swersky, Toniann Pitassi, and Richard S. Zemel. Flexibly fair representation learning by disentanglement. In International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1436-144 1445. PMLR, 2019.

---

[15] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. NAACL-HLT (1). pages 4171-4186. Association for Computational Linguisticss 2019.
[16] John C Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionally robust losses against mixture covariate shifts. Under review, 2019.
[17] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-aaveersarial training of neural networks. The journal of machine learning research, (7(1):2096-2033 2016.
[18] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
[19] Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In NAACL-HLT (2), pages 107-112. Association for Computational Linguistics, 2018.
[20] He He, Sheng Zha, and Haohan Wang. Unlearn dataset bias in natural language inference by fit- ting the residual. In DeepLo@ EMNLP-IJCNLP, pages 132-142. Association for Computational Linguistics, 2019.
[21] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie. and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[22] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[23] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 15262-15271, June 2021.
[24] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In NeurlPS, pages 125-136, 2019.
[25] Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In EMNLP, pages 2021 2031.. Association for Computational Linguistics, 2017.
[26] Ren Jiawei, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, et al. Balanced meta-softmax for loig-tailed visual recognition. Advances in Neural Information Processing Systems, 33, 2020.
[27] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33, 2020.
[28] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9012-9020, 2019.
[29] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 01:2278-2324 1998.
[30] Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, and cihang xie. Shape-texture debiased neural network training. In International Conference on Learning Representations, 2021.
[31] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
12

---

[32] David Madras, Elliot Creager, Toniann Pitassi, and Richard S. Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 3381-3390 2018.
[33] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
[34] Safyy Umoja Noble. Algorithms of oppression: How search engines reinforce racism. nyu Press, 2018.
[35] Jean Ponce, Martial Hebert, Cordelia Schmid, and Andrew Zisserman, editors. Toward Category- Level Object Recognition, volume 4170 of Lecture Notes in Computer Science. Springer, 2006.
[36] Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International jovenaaaaonalouurall journal of computer vision, 1 115(3):211222) 2015.
[37] Shiori Sagawa", Pang Wei Koh*, Tatsumori B. Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference Learning Representations, 2020.
[38] Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterization exacerbates spurious correlations. In International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 8346-8356. PMLR, 2020.
[39] Mingxing Tan and Quoe V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105-6114 14, 2019.
[40] Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentan- gling deep representations for bias correction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13508 13508-13517, June 2021.
[41] Antonio Torralba and Alexei A. Efros. Unbiased look at dataset bias. In CVPR, pages 1521-1528. IEEE Computer Society, 2011.
[42] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
[43] Zeyu Wang. Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[44] Kai Yuanqing Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. Noise or signal: The role of image backgrounds in object recogmition. In International Conference on Learning Representations, 2021.
[45] Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork. Learning fair representations. In International Conference on Machine Learning, volume 28 of JMLR Workshop and Conference Proceedings, pages 325-333, 2013.
[46] Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regressinn by conditional adver- sarial autoencoder. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
13


Reviews:
[{"review_id": "Zs0avOIUr0", "reviewer": "Reviewer_bmYk", "summary": "The authors propose a Bias-Contrastive (BiasCon) loss, Soften Bias-Contrastive (SoftCon) loss and Bias-Balanced (BiasBal) regression to mitigate model biases.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The motivation is clear and the problem being solved is important.  The contribution seems iterative but results are strong.  I have some concerns/questions on some of the details and choices made.  Hopefully the authors can clear them up.\n\nBiasCon pulls samples with same class but different biases closer and other pairs apart.  Why would it be preferable to pull samples with the same class apart regardless of bias?  It seems more intuitive to pull apart samples with same bias and pull together samples with same class which would also utilize same class samples without shared bias as well as different class samples with shared bias.  It would also remove the issue of limited positive pairs in a batch.  Can the authors comment on this?\n\nCross entropy (CE) should be pulling together samples with the same class which when combined with BiasCon in Equation 3 should balance the issue of the model learning to separate the representations of same class samples, but it seems that biasCon here would reduce the effect of the CE loss.\n\nIn the case when the dataset has limited examples of same class pairs without shared biases, which may be the case in many biased datasets, couldn't oversampling these datapoints lead to other learned biases specific to the limited data.  Essentially the model may become overtrained on these limited datapoints.\n\nIt is confusing to refer to P_u as the unbiased test data distribution as it is defined using only the training data.\n\nEquation 8 is summing logits with probabilities.  Logits can be large, small and possibly negative depending on the model.  How is this taken into account here so that the combination with p(y|b) has an influential contribution?\n\nCould the bias-capturing model also be used to weakly label positive bias pairs in biasCon?  More info on the bias-capturing model would be helpful.\n\nWithout these labels how can we trust that the representations of the bias-capturing model are clustered according to biases?  Many close representations could be simply datapoints of a similar class.  How is it ensured that the bias-capturing model representation is highly sensitive to bias?  In many applications it is not as simple as reducing the size of the receptive field such as the case with gender or racial biases in data.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2 hours"}, "extra_text": {}}, {"review_id": "PeV04evbipO", "reviewer": "Reviewer_YM7p", "summary": "The authors propose three debiasing methods. First, Bias-Contrastive Loss intends to bring the feature representations for samples with the same target class but different bias labels together. Second, the authors derive a Bias-Balanced Loss, which optimizes for unbiased distribution i.e., a distribution where target class and bias labels have uniform correlations. Third, they propose Soften Bias-Contrastive loss, which does not require bias labels, but instead uses a bias-capturing model to modulate the contrastive loss. The experiments are conducted both with and without bias labels and show improvements on all of the benchmarks.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the proposed techniques are novel and the experiments are well set up to demonstrate their efficacies. Building bias-resistant techniques based on the recently rising self-supervised methods is a valuable contribution to the field. Both BiasCon and BiasBal are well motivated and show improvements over other techniques that rely on bias labels. The SoftCon loss also improves upon other techniques that do not require bias labels. For this, SoftCon uses feature similarity/distance from bias-capturing models, which is different from how the previous methods e.g., LfF used them. Overall, the results are impressive, the techniques will be useful and the paper is very well written.\n\nI will now move on to clarification questions, addressing which may help improve the paper:\n\n[Q1] The very first question I had with the contrastive loss was: did the multi view batches already encode invariances? E.g., using grayscale for BiasedMNIST would be a trivial solution. While the authors mention the exact augmentation strategies in the supplementary section, it might be worthwhile to have at least a sentence stating whether or not the creation of the multi-view batch itself would help learn the signal in the main text itself.\n\n[Q2] For the Biased Contrastive Loss, it wasn’t clear why the samples with the same class, same bias were used as negatives. Intuitively, all instances from the same class should have similar representations, so should be used as positive pairs. What happens if the same class, same bias samples are used as positives instead? Which one is better and why?\n\n[Q3] While the oversampling ablation is mentioned in the supplementary, it might be worthwhile to have resampling/loss reweighting techniques as comparison methods for all the datasets. \nWhile simple, loss reweighting can mitigate biases under sufficient regularization [1], and thus might have practical utility. Also, the intuition behind the simple, classical techniques of resampling/reweighting based on target and bias labels is the same as the goal of Theorem 1 i.e., to optimize for uniform correlation between target and bias labels. As such, both theoretical and empirical comparisons with them would be a huge plus. It would be particularly interesting to have theoretical analyses supported by empirical results as to why/if BiasBal loss is better than the classical techniques. What is lacking in these classical techniques, that are otherwise very intuitive?\nAt the very least, [1] should be cited here.\n\n[Q4] The experiments section would be more insightful if the authors ran SoftBiasCon on datasets where it is not obvious how one can develop a bias-capturing model. The two datasets tested for SoftBiasCon had similar bias-encoding models (geared towards capturing background color/texture), but it is not obvious how an architecture could be developed for capturing “gender” bias in CelebA. Of course, perhaps the bias amplified model from LfF could be used for this case. In any case, it would be nice to have SoftBiasCon loss be run on all datasets, since bias mitigation without bias labels would be highly desirable in practice.\n\n[Q5] While the bias labels were not used during training for the SoftCon loss, were they used for model selection/hyperparameter tuning? It would be good to explicitly mention this.\n\n[1] Sagawa, Shiori, et al. \"An investigation of why overparameterization exacerbates spurious correlations.\" International Conference on Machine Learning. PMLR, 2020.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "While the authors mention not testing on more realistic benchmarks e.g., VQA, it might be worthwhile to also add that in such cases, biases may stem from multiple sources and it may not be trivial to label all of them or design bias-capturing models for all of them (which are required by the proposed methods).", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "MOgvpja5s9v", "reviewer": "Reviewer_bHbH", "summary": "This paper proposed several debiasing losses for unbiased image classification. First, this paper proposed two losses for known bias labels. motivated by contrastive learning, the Bias-Contrastive (BiasCon) loss for better representation learning. The Bias-Balanced (BiasBal) regression is proposed for highly imbalanced target-bias correlation. Second, the Soften Bias-Contrastive (SoftCon) is proposed for unknown bias labels. Experiments on BiasedMNIST, CelebA and UTKFace show that the proposed debiasing method outperforms state-of-the-art methods by a large margin whenever the bias label is available.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n+ This paper reveals the potential and effectiveness of contrastive learning in debiased image classification. The contrastive-learning-based loss is simple but very effective.\n\n+ Experimental results on several benchmark datasets show that the proposed methods outperform state-of-the-art debiasing methods by a large margin with or without bias labels. \n\n+ Ablation studies are sufficient to show the effectiveness of each component.\n\nWeaknesses & Questions:\n\n- The connection between BiasCon and BiasBal losses are not tight enough. It seems that these two losses are like a bag of tricks rather than a whole. Also, BiasBal is not used without bias labels. Even without BiasBal, I still think this paper is good. However, BiasBal seems a little incompatible in this paper. Perhaps a better introduction of BiasBal would help with this.\n\n- Another question on BiasBal. While BiasBal achieves competitive results in Tables 2 and 3, the performance in Table 1 is much lower than BiasCon when the bias and label are highly correlated. Recall that BiasBal is proposed to tackle the issue of a highly imbalanced correlation between label and bias. Is there any explanation for this?\n\n- The performances of Vanilla models with bias labels were implemented (Table 1-3) while those without bias labels were borrowed from ReBias (Table 4, 5). I wonder (1) why the vanilla models were not implemented in Tables 4 and 5, (2) whether the increase comes from the implementation. Results of vanilla models obtained by the same reimplementation would answer these questions.\n\n- It is not clear how Eq. (5) and (6) are theoretically correct. In Eq. (5) and (6), why can we replace $P_{u}$ with $P_{train}$?\n\n- After reading the paper, I am supervised by the success of contrastive learning in debiased image classification, but still not clear why contrastive learning can outperform others by a large margin (e.g., 93.3 vs. 52.3 on BiasedMNIST with corr=0.999 in Table 1). Is there a more intuitive or convincing explanation for this increase?\n\n- As shown in Tables A4 and A6 in the appendix, the hyperparameters $\\alpha$ are various for different correlations between label and bias. Is it a fair experiment setting? Or, is the hyper-parameter determined based on the test set? It is okay if this is the commonly used setting in other related works. In my opinion, the hyperparameters should be the same for different correlations as the correlations are supposed to be unknown in real-world applications.\n\n\n==================== After rebuttal ======================\n\nAccording to the authors' feedback, most of my major concerns have been addressed, and I am happy to vote for acceptance.\n\nThe strengths and contributions of this paper are:\n\n(1) Two losses for known bias labels and one variant for unknown bias labels for unbiased image classification. BiasCon is proposed for better representation learning motivated by contrastive learning, and BiasBal is proposed for highly imbalanced target-bias correlation. A variant of BiasCon, SoftCon, is proposed for unknown bias labels. These losses are simple but effective for unbiased image classification.\n\n(2) The paper is well written and easy to follow. The contributions are clearly stated.\n\n(3) Experiments and ablation studies are well designed and demonstrate the effectiveness of the proposed losses.\n\nMy major concerns were:\n\n(1) The connection between BiasCon and BiasBal losses is not tight enough. According to the authors' feedback, I acknowledge the contribution of each loss to unbiased image classification. However, it would be greater if these two losses could be unified into one framework rather than two independent techniques. Due to this major concern, I would not give a higher score.\n\n(2) The reason why contrastive learning works. Reviewer bmYk and YM7p raise a similar concern via a more detailed question: how and why the choice of BiasCon is determined, especially positive and negative pairs. The authors' feedback provides more experimental results to empirically compare BiasCon and other variants, which I think answers the \"how\" question. For the \"why\" question, I have left a post-rebuttal note to request the authors a more comprehensive and conclusive explanation.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "-SGpsMy4ohh", "reviewer": "Reviewer_Eyr1", "summary": "The paper propose two method, BiasCon and BiasBal, to mitigate the dataset bias. In particular, BiasCon is a contrastive-like objective that pulls a pair that shares the same target class but different bias feature. BiasBal objective is derived via assuming the data generating process. The authors also propose SoftCon, an extension of BiasCon, to tackle the unknown bias label. The proposed methods are evaluated on BiasedMNIST, CelebA and UTKFace dataset.\n\n# POST-REBUTTAL\nThe additional results look good. After reading these and my co-reviewers comments, I still advocate for weak accept (score 6).", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the paper address an important problem in machine learning. The paper is well-written and the contributions are stated clearly. I have a few questions and concerns.\n\nPositive Points:\n\n1. The proposed BiasBal is interesting. It seems that it is a simple yet effective objective for tacking the dataset bias.\n\n2. The proposed approach can also be partially extended to the case when the bias label is missing. It could be very influential since in real world application such as medical system, the bias label (e.g., gender) is not accessible.\n\n\nNegative Points:\n\n1. The paper does not discuss or compare to existing works about fairness. The bias variable is essentially the sensitive attributes in fairness literatures, e.g., the gender attribute in CelebA and UTKFace experiments. There are existing works [1,2,3] that leverage the sensitive attribute to learn fair representations or build fair classifier. \n\n2. Besides machine learning fairness literatures, there is another line of works that solves similar problem under the domain generalization or casual inference framework. I encourage authors to include some standard baselines such as Invariant Risk Minimization [4] or Domain-invariant Networks [5]? In particular, these works treat the domain-specific features as biased features. Therefore, the variable B could be domain index.\n\n3. The main metric used in the paper is the unbiased accuracy. It would be great to measure and discuss the performance w.r.t. demographic parity or equalized odds on CelebA and UTKFace dataset. To ensure machine learning fairness, unbiased accuracy only provides little informations.\n\nOther comments:\n\n1. I suggest moving the graphical model in appendix to section 3.3. This would help readers understand equation (5) and (6) more.\n\n2. It would be great to perform some analyses to the BiasCon objective. For instance, maybe one can link this to the mutual information between the feature and feature bias B?\n\n3. The notation is a bit confusing. It seems that $p$ has been used to denote several distributions.\n\nOverall, I think this paper address an important problem that will be of interest to a wide audience in NeurIPS. The method is stated clear and the experiments are sound. Nevertheless, there is lack of discussions and comparison to previous works. The paper would be much more polished if the authors can provide more empirical comparison to other baselines and metrics. \n\n\n[1] Zemel et al., Learning Fair Representations, 2013\n\n[2] Madras et al., Learning Adversarially Fair and Transferable Representations, 2018\n\n[3] Creager et al., Flexibly fair representation learning by disentanglement, 2019\n\n[4] Arjovsky et al., Invariant Risk Minimization, 2020\n\n[5] Ganin et al, Domain-Adversarial Training of Neural Networks, 2015\n\n[6] Li et al., Information-Preserving Contrastive Learning for Self-Supervised Representations, 2021\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately addressed the limitations and potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes new loss functions for training on datasets with feature bias. The reviewers find the proposals to be novel and the experiment results to be strong, and recommend acceptance. There were some concerns raised about the rationale behind some of the design choices, and missing comparisons to some baselines. The authors have satisfactorily addressed these concerns over multiple back-and-forth discussions with the reviewers, and have provided additional supporting experimental results. We trust that the authors will include the additional results in the final version of the paper.
Decision: Accept (Poster)