Mitigating Dataset Bias Using Per-Sample Gradients From A Biased Classifier

1 INTRODUCTION . Over the past decade , deep neural networks ( DNNs ) have been a focus of research owing to humanlike performances in various tasks , e.g. , image classification ( Krizhevsky et al. , 2012 ) , object detection ( Girshick , 2015 ) , and image generation ( Goodfellow et al. , 2014 ) . Despite these impressive results , deploying DNNs directly in real-world problems remains a significant challenge due to the difficulties in obtaining well-curated training sets . Specifically , unintendedly biased information in training sets causes prejudice , resulting in wrong decisions at inference time ( Torralba & Efros , 2011 ; Shrestha et al. , 2021 ) . For instance , most “ ski ” images contain “ skier. ” This unintended correlation can recommend a wrong shortcut for “ ski ” by examining the person . Prior studies have employed human supervision ( i.e. , providing additional labels related to the bias ) ( Kim et al. , 2019 ; McDuff et al. , 2019 ; Singh et al. , 2020 ) or giving a specific information about a biased domain ( Lee et al. , 2019 ; Geirhos et al. , 2018 ) , which is very expensive , to reduce the influence of a “ skier ” on “ ski ” class images . Recently , studies on replacing human labor with DNNs have been actively discussed ( Li & Vasconcelos , 2019 ; Nam et al. , 2020 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Clark et al. , 2019 ; Le Bras et al. , 2020 ) . Typically , two separate networks have been used : the biased network and the de-biased network . A biased model learns to replace human intervention and teaches a de-biased model using the prejudiced knowledge of the biased model . Previous studies based on the two mentioned networks approaches have had two directions : adjusting objectives and resampling . Adjusting the objective refers to a method that obtains weighted loss or additional regularizers for each sample differently , where the weights or regularizers are obtained from the biased model ( Nam et al. , 2020 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Clark et al. , 2019 ) . Methods that adjust the objective have been proposed frequently due to their several advantages , such as the implementational simplicity . However , according to ( An et al. , 2020 ) , adjusting-objective approaches suffers instability with stochastic gradient descent ( SGD ) type optimizers owing to the high variance of learning weights per sample . Alternatively , resampling methods could reconstruct balanced sets . Previous research ( Le Bras et al. , 2020 ; Li & Vasconcelos , 2019 ; Li et al. , 2018b ; Kim et al. , 2021a ) has focused on which samples should be oversampled to build a balanced set . However , almost all prior works has focused on expensive human labor to tune hyperparameters related to the resampling densities of data points . For instance , it is difficult to set the amount of resampling of each sample without knowing how much biased the training set is . The de-biasing methods , belonging to both adjusting-objective and resampling approaches are designed to emphasize samples that are hard for a model to learn . However , a few noisy labels ( i.e. , mislabeled samples ) in the training set could significantly interfere with such de-biasing because it might be difficult to discriminate between two types of hard samples : noisy samples that the model should not learn and rare samples that have to learn . Although training sets have noisy labels in practice ( Natarajan et al. , 2013 ) , to the best of our knowledge , no study considers the de-biasing problem under noisy label cases . Contribution . In this study , we propose a score-based resampling scheme for de-biasing , which does not require hyperparameters related to training set reconstruction . Instead of hyperparameters , we construct two types of scores that leverage gradients of the biased model trained on the given biased set . The proposed scores determine the required proportions for each sample in training set reconstruction . Before designing scores , we explain our hypothesis that gradients have remarkable differences between samples generating prejudice and the others , and check it empirically . The hypothesis is based on the following observations . First , the gradient magnitudes ( especially the Euclidian norm ) of the samples which contributes to the prejudice are relatively smaller than those of the others . Second , samples that make prejudice and the others do not have significantly different gradient directions , measured by the estimated likelihood of the von Mises-Fisher ( vMF ) distribution ( Banerjee et al. , 2005 ; Lee et al. , 2018 ) . Based on these observations , we propose two types of scores and a resampling-based de-biasing method . The proposed method consists of three steps similar to previous resampling studies ( Li & Vasconcelos , 2019 ; Le Bras et al. , 2020 ; Kim et al. , 2021a ) . The first step is to train a biased model based on a biased raw training set . Afterward , the scores are obtained by the biased model . The mini-batch sampler generates balanced mini-batches using the scores , and it feeds them into an ultimate model for de-biased training . In addition , we investigate the side effects of de-biasing methods when incorrectly labeled samples are in the training set . We observe significant performance degradations of de-biasing methods with a small portion of noisy data . To alleviate the negative side effects of noisy data , we first de-noise the training set using loss values from the our observation that noisy label samples have higher losses than clean samples and then run de-noising algorithms . The de-noising step successfully protects the de-biasing performance . Finally , we demonstrate the effectiveness of our method using various biased benchmarks ( Colored MNIST ( Nam et al. , 2020 ; Kim et al. , 2019 ; Li & Vasconcelos , 2019 ; Bahng et al. , 2020 ) , Watermarked MNIST , Cartoon ( Royer et al. , 2020 ) , CelebA ( Liu et al. , 2015 ) , Biased action recognition ( Nam et al. , 2020 ) , and ImageNet / ImageNet-A ( Bahng et al. , 2020 ; Deng et al. , 2009 ; Hendrycks et al. , 2019 ) ) . In most experiments , the proposed method outperforms the current state-of-the-art methods . In particular , the proposed method improves the accuracy of the unbiased test by 68.00 % from the baseline with a small performance degradation of the biased test of 0.91 % . Moreover , we compare the proposed method with the previous resampling method , REPAIR ( Li & Vasconcelos , 2019 ) . The proposed method improves the average accuracy by 71.04 % → 98.14 % . 2 RELATED WORK . Mitigating bias with human supervision . In ( Goyal et al. , 2017 ; 2020 ) , generated a de-biased dataset using human labor . A collection of studies ( Alvi et al. , 2018 ; Kim et al. , 2019 ; McDuff et al. , 2019 ; Singh et al. , 2020 ; Teney et al. , 2021 ; Ramaswamy et al. , 2021 ; Tartaglione et al. , 2021 ; Geirhos et al. , 2018 ; Wang et al. , 2018 ; Lee et al. , 2019 ) aimed to mitigate bias based on one of two types of human supervision : explicit bias labels and implicit bias information . Other studies ( Alvi et al. , 2018 ; Kim et al. , 2019 ; McDuff et al. , 2019 ; Singh et al. , 2020 ) , have used bias labels for each sample to reduce the influence of the bias labels when classifying target labels . ( Tartaglione et al. , 2021 ) proposed the EnD regularizer , which entangles target correlated features and disentangles biased features . Several authors ( Alvi et al. , 2018 ; Kim et al. , 2019 ; Teney et al. , 2021 ) have designed DNNs as a shared feature extractor and multiple classifiers . In contrast to the shared feature extractor methods , ( McDuff et al. , 2019 ; Ramaswamy et al. , 2021 ) constructed a classifier and conditional generative adversarial networks , generating test samples to check whether the classifier is biased . ( Singh et al. , 2020 ) proposed a new overlap loss defined by a class activation map ( CAM ) . The overlap loss reduces the overlapping parts of the CAM outputs of two bias labels and target labels . Different approaches ( Geirhos et al. , 2018 ; Wang et al. , 2018 ; Lee et al. , 2019 ) have assumed that the characteristics of bias features are known and prevent learning the implicit bias features . To avoid the known texture bias of ImageNet ( Deng et al. , 2009 ) , ( Geirhos et al. , 2018 ) generated stylized ImageNet , and ( Lee et al. , 2019 ; Wang et al. , 2018 ) inserted filter in front of the models so that the influence of the backgrounds and colors of the images can be removed . Mitigating bias without human supervision . To reduce human intervention , recent research ( Clark et al. , 2019 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Nam et al. , 2020 ; Li & Vasconcelos , 2019 ; Li et al. , 2018b ; Le Bras et al. , 2020 ; Darlow et al. , 2020 ) has used ensemble-based methods , using two separate networks with different purposes . The biased model learns biased information and even empowers it . Then , the biased model is used to train the de-biased model as a pathfinder , which provides information about the bias features . The ensemble idea has been used as a key component in several studies , ( Clark et al. , 2019 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Nam et al. , 2020 ) . ( Cadene et al. , 2019 ) proposed a method known as RUBi , which multiplies the sigmoid output of the biased model by the softmax output of the de-biased model . ( Clark et al. , 2019 ) summed the output of the two models to train the de-biased model with an entropy regularizer . ( Bahng et al. , 2020 ) proposed ReBias algorithm using the Hilbert-Schmidt independence criterion ( HSIC ) as an objective of the de-biased model , by independent of the biased model . Learning from failure ( LfF ) ( Nam et al. , 2020 ) used generalized cross-entropy-based training that emphasizing the usual samples but ignoring unusual samples . In contrast , biased and de-biased models are trained sequentially ( Li & Vasconcelos , 2019 ; Li et al. , 2018b ; Le Bras et al. , 2020 ; Darlow et al. , 2020 ) . Researchers computed the weights to reconstruct the training set from the biased model and trained the de-biased network on the reconstructed training set . The REPAIR ( Li & Vasconcelos , 2019 ) and RESOUND ( Li et al. , 2018b ) methods proposed mutual-information-based weights to create a downsampled dataset . In addition , AFLite ( Le Bras et al. , 2020 ) proposed accuracy-based weights from multiple biased models , unlike LAD ( Darlow et al. , 2020 ) and BiaSwap ( Kim et al. , 2021a ) , which generated new images using an autoencoder based on the latent representation of the biased model .

Summary: The paper propose to use the per-sample gradients from the biased classifier to tackle the dataset bias. Following prior work, this work adopts a three-step approach: 1) train a classifier on biased dataset 2) identify which samples are from the minority and which are not 3) train a classifier with re-sampling. The proposed per-sample gradient-based scores are used in step 2. The motivation is purely from the empirical observation that samples from the majority group tend to have similar gradient magnitude and direction. From the observation, they derive a scoring function for re-sampling. The proposed method work pretty well empirically compared to various baselines.  Aside from the novel scoring function, since the noisy labeled data might be accidentally considered as minority group, the paper introduces a de-noising step to tackle the potential noise in the dataset. The de-noising step further improves the performances.