Unbiased Classification through Bias-Contrastive and Bias-Balanced Learning

1 Introduction . Machine learning models have achieved extremely high performance in a variety of tasks and domains such as computer vision and natural language processing [ 15 , 7 , 39 ] . Recently , however , many concerns have arisen that such evaluation does not reflect the real-world performance of the model when it is deployed [ 35 , 41 ] . Among others , failing especially due to the biases existing in the dataset can lead to serious societal side effects such as prejudice or racism beyond simple algorithm failure [ 45 , 34 , 5 ] . If bias features are highly correlated with the object class in the dataset , models tend to use the bias as a cue for the prediction as they are easier to learn but enough to achieve high accuracy even though they are not actually related to the target class [ 3 , 33 ] . The failure of learning due to the bias existing in the dataset appears in various fields and tasks . To name a few , in image classification task , [ 18 , 30 ] discovered that state-of-the-art CNNs have texture biases . Visual question answering ( VQA ) model is also known to be susceptible to biases as it only uses the word occurrence in the question to generate answer [ 1 , 11 ] . For example , if most of the images of banana in the train sets are yellow , the model directly answers the question `` What is the color of the banana ? '' as `` Yellow '' , without looking at the image of green banana . Surveillance models , which can cause serious societal problems if misclassified , tend to give biased predictions toward sensitive attributes such as race [ 34 ] . Considering the importance of the problem , many approaches have been proposed to mitigate such biases in the training dataset . Depending on the presence or absence of bias information , the approaches can be divided into two categories . When the bias label is available , [ 28 , 43 ] add the bias prediction branch for the unbiased prediction . [ 40 , 10 ] directly regularize the feature embedding to be indistinguishable across the bias classes . When the bias label is unavailable , [ 11 , 3 , 8 , 20 ] design †This work was done as a student at KAIST . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . an auxiliary bias-capturing models that learn only bias features and train the main model to learn orthogonal features from those models . In this paper , we propose two powerful debiasing approaches that can be complementarily applied when the bias label is available . We first propose Bias-Contrastive ( BiasCon ) loss that extends the approaches that directly regularize the feature space by adapting the recent advances of contrastive learning in representation learning . The BiasCon loss utilizes the power of contrastive learning to promote pulling the same target class but different bias class samples closer in the feature space . We further propose the Bias-Balanced ( BiasBal ) loss that suppresses the utilization of bias features by optimizing toward the data distribution where the target-bias correlation is balanced . Each loss achieves state-of-the-art debiasing performance and shows even higher performance when they are jointly used as they give orthogonal debiasing effects . To extend to more realistic cases where the bias label is unavailable , we use the observation that the feature space of the bias-capturing model can be used in estimating whether a pair of samples have the same bias features . We thus propose Soft Bias-Contrastive ( SoftCon ) loss , which is the BiasCon loss weighted with the cosine distance between samples in the feature space of the bias-capturing model . We conduct experiments to evaluate the debiasing performance of the proposed methods . For the case where the bias label is available , we evaluate the methods on CelebA [ 31 ] and UTKFace [ 46 ] , which have biases toward sensitive attributes such as gender or race . For the case where the bias label is unavailable , we use ImageNet [ 36 ] and ImageNet-A [ 23 ] to assess whether the bias of our model has been removed . Our method improves the unbiased accuracy of previous methods by a large margin across all datasets in both with and without bias labels . Our contributions can be stated as follows : • We propose a powerful debiasing method , the BiasCon loss , that effectively adapts recent advancements of contrastive learning and the BiasBal loss that further enhances debiasing performance by optimizing the model toward distribution with uniform target-bias correlation . • We introduce the SoftCon loss , the extension of the BiasCon loss to the case where the bias label is unavailable , that utilizes the feature space of the bias-capturing model . • We show that our losses successfully improve the debiasing performance with a large gap in various real-world datasets for both bias label available/unavailable cases . 2 Related work . Bias existing in the data and the vulnerability of machine learning algorithms to such bias has been recently studied as an important problem in various domains and tasks . [ 35 , 41 ] revealed that various training sets have regularity conditions ( e.g. , objects should not be occluded ) that are unlikely to hold in practical settings , and machine learning models trained on such data fail to generalize in the absence of such conditions . More specifically , [ 23 , 44 , 30 ] showed that state-of-the-art object recognition models are biased toward backgrounds or textures associated with the object class . [ 38 ] explored that overparametrized models are prone to the spurious correlations and can have worse test errors on minority groups . Natural language processing models are also susceptible to dataset biases . Question Answering ( QA ) [ 19 , 25 , 13 , 12 ] and Visual QA [ 1 , 11 ] systems make predictions using shortcuts such as the occurrence of specific words in the question rather than the context . Even large language models tend to give biased prediction toward certain gender or race [ 10 ] . Meanwhile , the severity of such biased prediction and fairness issues of deployed models are thoroughly investigated in various domains [ 45 , 34 , 5 ] . This brings the importance and necessity of the debiasing method , and recent research has begun to proceed in two main directions depending on the presence or absence of bias labels . Debiasing when bias label is available An important line of work assumed the situation where the bias factors are categorical attributes ( e.g. , gender , race ) and the meta-information about the bias label of each training data is available . [ 43 ] suggested to use per-bias classifier head to equalize the effect of bias features . [ 45 , 32 , 14 ] proposed fairness metrics such as demographic parity ( DP ) or equalized odds ( EO ) that measure the biasedness of the model and devised algorithms that can induce the fair prediction . [ 28 ] extended the idea of the domain generalization [ 2 , 17 ] by adding a bias prediction branch to the model and train the model with an additional regularization term that induces the learned features to be invariant to the bias features . In the risk minimization perspective , [ 4 , 16 , 37 ] proposed distributionally robust optimization that aims to optimize toward the worst-case group distribution shift . [ 40 ] devised a regularization term with a triplet loss formulation to minimize the entanglement of bias features . For the pretrained language model , [ 10 ] appended additional filtering layer and corrects the sentence embedding by maximizing mutual information . We propose a novel regularization term that work upon the success of the contrastive learning framework on self-supervised learning [ 9 , 21 , 27 ] and leverages the bias label in assigning positive pair samples . Debiasing when the bias label is unavailable It is more realistic to assume that we know the form of biases that have to be removed , but no bias label is available , nor the type of bias is not limited to categorical data ( e.g. , texture bias of CNNs ) . Still , there are approaches that mitigate biases in such situations . The first line of research is to utilize the bias-capturing model [ 11 , 3 , 8 , 20 ] . The bias-capturing model is an auxiliary model designed to learn bias features . For example , texture biases can be captured by CNNs with small receptive fields [ 3 ] . In these approaches , the model is promoted to learn independent features from the bias-capturing models , thus learning debiased features . [ 33 ] further relaxed the assumption about the prior knowledge about bias based on the observation that the bias features are learned before the target features . Specific to CNNs for the image classification task , [ 18 , 30 ] removed texture bias by randomizing the texture of the training data . Our method also utilizes a bias-capturing model , but we propose a more effective way to use the model by considering the distance between training samples in the feature space of the model . 3 Method . This section begins by explaining our problem setup . Then , we propose two debiasing objectives , Bias-Contrastive ( BiasCon ) loss and Bias-Balancing ( BiasBal ) loss , that are effective in situations where the bias is categorical and the bias label is available . We further propose Soft Bias-Contrastive ( SoftCon ) loss which extends the BiasCon loss to more general and challenging situations where the bias is not limited to categorical data and the bias label is not available . 3.1 Problem definition . To formulate our problem , we define a target Y and a bias B that both affects the generation of data X . In such data , we assume that X can be divided into a signal feature Xs explaining the target Y and a bias feature Xb from B . For example , in the data for hair color classification task that is biased toward race ( that is , Y =hair and B=race ) , Xs includes brown hair feature , and Xb includes white skin feature . Now , the bias in the dataset can be understood as a high correlation between the target class and the bias feature , i.e. , the prior p ( Y |B ) is imbalanced . Then the model may ignore the true signal Xs and instead use Xb in prediction and output p ( Y |Xb ) as it can still give high accuracy . In the hair color example above , if most white people have blonde hair , the model may output blonde every time it sees the white skin color . Therefore , the biased model fails on the test data distribution with the shifted correlation between Y and B , i.e. , Ptrain ( Y , B ) ̸= Ptest ( Y , B ) . Our objective is to train the model to make a prediction using the true signal Xs only ; that is , the model takes the whole feature X and outputs p ( Y |Xs , Xb ) = p ( Y |Xs ) conditionally independent on Xb , so that the model makes indiscriminative predictions across the biases . 3.2 Bias-contrastive loss . We first consider the situation where the bias label is categorical and known for all training samples ( e.g. , gender or race information of each training sample is available ) . Here we note the fact that a biased model tends to learn features from B and make predictions using those features , hence the samples within the same bias class are close to each other in the feature space . Thus , we devise a loss to explicitly incentivise the model to pull the same target class but different bias class sample pairs closer than the other pairs . Toward this , we borrow and revise the main idea of [ 27 ] where supervised contrastive ( SupCon ) loss pulls the same class samples closer using the contrastive loss and achieves better performance than training solely with the cross-entropy loss . Specifically , we turn the SupCon loss into an effective loss for debiasing , named Bias-Contrastive ( BiasCon ) loss , that pulls samples with the same class but different bias closer while pushing the other pairs . Following the formulation of the SupCon loss , we first construct 2N size `` multiviewed batch '' by applying a pair of randomized augmentation ( e.g. , random crop , random flip ) on N data samples in a single batch : { ( xi , yi , bi ) } 2Ni=1 . We then apply the same SupCon loss but to different positive pairs ( pairs of same target class but different bias class ) and formulate the BiasCon loss as follows : LBiasCon = − 1 2N ∑ i∈Imv 1 |J ( i ) | ∑ j∈J ( i ) log exp ( zi · zj/τ ) ∑ a∈Imv\ { i } exp ( zi · za/τ ) ( 1 ) where Imv : = { 1 , ... , 2N } is the index set of multiviewed batch , J ( i ) : = { j ∈ Imv : yj = yi , bj ̸= bi } is the index set of positive samples paired with i-th sample , zi = f ( xi ) /∥f ( xi ) ∥ is a normalized feature of i-th sample extracted from f , the network up to the penultimate layer , and τ is a temperature hyperparameter . The final objective is a combination of our BiasCon loss and the standard cross-entropy ( CE ) loss as follows : LCE = − 1 N ∑ i∈I log h ( xi ) [ yi ] , ( 2 ) L = α · LCE + LBiasCon ( 3 ) where h ( x ) [ y ] = p ( y|x ) is the model prediction for class y , I : = { 1 , ... , N } is the index set of the original batch and α is a weight hyperparameter . Originally , [ 27 ] used the SupCon loss solely for the representation learning up to the penultimate layer and additionally trained the fully-connected ( FC ) layer upon the frozen representation with the CE loss . However in our case , samples with the same target and bias class are considered as a negative pairs . Therefore , it is more effective to jointly train with CE loss as it can give signals to incorporate all the same class samples . Thus , unlike [ 27 ] , we propose to use the BiasCon loss as a regularizer , as in ( 3 ) . In the Appendix D.2 , we empirically show that the joint training with CE loss achieves better performance . Another specific issue here that contrasts with the SupCon loss is that when p ( b|y ) is highly imbalanced , a single batch may not contain enough positive pairs because most of the same class samples in the batch will have the same bias features . This may result in unstable optimization . To handle this issue , for training set D = { xi } , we separately sample the mini-batch for the BiasCon loss with a modified sampling frequency Q ( i ) of i-th sample . We design Q ( i ) to oversample xi with low p ( bi|yi ) so that more positive pairs are available in the batch : Q ( i ) ∝ 1 p ( bi|yi ) if 1 p ( bi|yi ) < γ ·miny , b 1 p ( b|y ) γ ·min y , b 1 p ( b|y ) o.w. . ( 4 ) The mini-batch of size N is then sampled as { x ( j ) : x ( j ) = xi where i ∼ Q ( i ) for j = 1 , . . . , N } . Note that this formulation clips the largest possible unnormalized Q ( i ) to be γ ·minQ ( i ) so that every training data is sampled with some chance . We pre-calculate p ( b|y ) using the Y and B labels in the training set before the training . It is also instructive to note that EnD [ 40 ] has the similar motivation of entangling the features of different bias class samples . However , EnD uses the triplet loss which has a limitation that it penalties the same class samples even after the model successfully learned the target features . Experimental results in Section 4 show the huge performance gap between ours and EnD across all tasks . More detailed comparison between two methods is available in Appendix A.2 .

Summary: The paper propose two method, BiasCon and BiasBal, to mitigate the dataset bias. In particular, BiasCon is a contrastive-like objective that pulls a pair that shares the same target class but different bias feature. BiasBal objective is derived via assuming the data generating process. The authors also propose SoftCon, an extension of BiasCon, to tackle the unknown bias label. The proposed methods are evaluated on BiasedMNIST, CelebA and UTKFace dataset.  # POST-REBUTTAL The additional results look good. After reading these and my co-reviewers comments, I still advocate for weak accept (score 6).