Mitigating Dataset Bias Using Per-Sample Gradients From A Biased Classifier

1 INTRODUCTION . Over the past decade , deep neural networks ( DNNs ) have been a focus of research owing to humanlike performances in various tasks , e.g. , image classification ( Krizhevsky et al. , 2012 ) , object detection ( Girshick , 2015 ) , and image generation ( Goodfellow et al. , 2014 ) . Despite these impressive results , deploying DNNs directly in real-world problems remains a significant challenge due to the difficulties in obtaining well-curated training sets . Specifically , unintendedly biased information in training sets causes prejudice , resulting in wrong decisions at inference time ( Torralba & Efros , 2011 ; Shrestha et al. , 2021 ) . For instance , most “ ski ” images contain “ skier. ” This unintended correlation can recommend a wrong shortcut for “ ski ” by examining the person . Prior studies have employed human supervision ( i.e. , providing additional labels related to the bias ) ( Kim et al. , 2019 ; McDuff et al. , 2019 ; Singh et al. , 2020 ) or giving a specific information about a biased domain ( Lee et al. , 2019 ; Geirhos et al. , 2018 ) , which is very expensive , to reduce the influence of a “ skier ” on “ ski ” class images . Recently , studies on replacing human labor with DNNs have been actively discussed ( Li & Vasconcelos , 2019 ; Nam et al. , 2020 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Clark et al. , 2019 ; Le Bras et al. , 2020 ) . Typically , two separate networks have been used : the biased network and the de-biased network . A biased model learns to replace human intervention and teaches a de-biased model using the prejudiced knowledge of the biased model . Previous studies based on the two mentioned networks approaches have had two directions : adjusting objectives and resampling . Adjusting the objective refers to a method that obtains weighted loss or additional regularizers for each sample differently , where the weights or regularizers are obtained from the biased model ( Nam et al. , 2020 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Clark et al. , 2019 ) . Methods that adjust the objective have been proposed frequently due to their several advantages , such as the implementational simplicity . However , according to ( An et al. , 2020 ) , adjusting-objective approaches suffers instability with stochastic gradient descent ( SGD ) type optimizers owing to the high variance of learning weights per sample . Alternatively , resampling methods could reconstruct balanced sets . Previous research ( Le Bras et al. , 2020 ; Li & Vasconcelos , 2019 ; Li et al. , 2018b ; Kim et al. , 2021a ) has focused on which samples should be oversampled to build a balanced set . However , almost all prior works has focused on expensive human labor to tune hyperparameters related to the resampling densities of data points . For instance , it is difficult to set the amount of resampling of each sample without knowing how much biased the training set is . The de-biasing methods , belonging to both adjusting-objective and resampling approaches are designed to emphasize samples that are hard for a model to learn . However , a few noisy labels ( i.e. , mislabeled samples ) in the training set could significantly interfere with such de-biasing because it might be difficult to discriminate between two types of hard samples : noisy samples that the model should not learn and rare samples that have to learn . Although training sets have noisy labels in practice ( Natarajan et al. , 2013 ) , to the best of our knowledge , no study considers the de-biasing problem under noisy label cases . Contribution . In this study , we propose a score-based resampling scheme for de-biasing , which does not require hyperparameters related to training set reconstruction . Instead of hyperparameters , we construct two types of scores that leverage gradients of the biased model trained on the given biased set . The proposed scores determine the required proportions for each sample in training set reconstruction . Before designing scores , we explain our hypothesis that gradients have remarkable differences between samples generating prejudice and the others , and check it empirically . The hypothesis is based on the following observations . First , the gradient magnitudes ( especially the Euclidian norm ) of the samples which contributes to the prejudice are relatively smaller than those of the others . Second , samples that make prejudice and the others do not have significantly different gradient directions , measured by the estimated likelihood of the von Mises-Fisher ( vMF ) distribution ( Banerjee et al. , 2005 ; Lee et al. , 2018 ) . Based on these observations , we propose two types of scores and a resampling-based de-biasing method . The proposed method consists of three steps similar to previous resampling studies ( Li & Vasconcelos , 2019 ; Le Bras et al. , 2020 ; Kim et al. , 2021a ) . The first step is to train a biased model based on a biased raw training set . Afterward , the scores are obtained by the biased model . The mini-batch sampler generates balanced mini-batches using the scores , and it feeds them into an ultimate model for de-biased training . In addition , we investigate the side effects of de-biasing methods when incorrectly labeled samples are in the training set . We observe significant performance degradations of de-biasing methods with a small portion of noisy data . To alleviate the negative side effects of noisy data , we first de-noise the training set using loss values from the our observation that noisy label samples have higher losses than clean samples and then run de-noising algorithms . The de-noising step successfully protects the de-biasing performance . Finally , we demonstrate the effectiveness of our method using various biased benchmarks ( Colored MNIST ( Nam et al. , 2020 ; Kim et al. , 2019 ; Li & Vasconcelos , 2019 ; Bahng et al. , 2020 ) , Watermarked MNIST , Cartoon ( Royer et al. , 2020 ) , CelebA ( Liu et al. , 2015 ) , Biased action recognition ( Nam et al. , 2020 ) , and ImageNet / ImageNet-A ( Bahng et al. , 2020 ; Deng et al. , 2009 ; Hendrycks et al. , 2019 ) ) . In most experiments , the proposed method outperforms the current state-of-the-art methods . In particular , the proposed method improves the accuracy of the unbiased test by 68.00 % from the baseline with a small performance degradation of the biased test of 0.91 % . Moreover , we compare the proposed method with the previous resampling method , REPAIR ( Li & Vasconcelos , 2019 ) . The proposed method improves the average accuracy by 71.04 % → 98.14 % . 2 RELATED WORK . Mitigating bias with human supervision . In ( Goyal et al. , 2017 ; 2020 ) , generated a de-biased dataset using human labor . A collection of studies ( Alvi et al. , 2018 ; Kim et al. , 2019 ; McDuff et al. , 2019 ; Singh et al. , 2020 ; Teney et al. , 2021 ; Ramaswamy et al. , 2021 ; Tartaglione et al. , 2021 ; Geirhos et al. , 2018 ; Wang et al. , 2018 ; Lee et al. , 2019 ) aimed to mitigate bias based on one of two types of human supervision : explicit bias labels and implicit bias information . Other studies ( Alvi et al. , 2018 ; Kim et al. , 2019 ; McDuff et al. , 2019 ; Singh et al. , 2020 ) , have used bias labels for each sample to reduce the influence of the bias labels when classifying target labels . ( Tartaglione et al. , 2021 ) proposed the EnD regularizer , which entangles target correlated features and disentangles biased features . Several authors ( Alvi et al. , 2018 ; Kim et al. , 2019 ; Teney et al. , 2021 ) have designed DNNs as a shared feature extractor and multiple classifiers . In contrast to the shared feature extractor methods , ( McDuff et al. , 2019 ; Ramaswamy et al. , 2021 ) constructed a classifier and conditional generative adversarial networks , generating test samples to check whether the classifier is biased . ( Singh et al. , 2020 ) proposed a new overlap loss defined by a class activation map ( CAM ) . The overlap loss reduces the overlapping parts of the CAM outputs of two bias labels and target labels . Different approaches ( Geirhos et al. , 2018 ; Wang et al. , 2018 ; Lee et al. , 2019 ) have assumed that the characteristics of bias features are known and prevent learning the implicit bias features . To avoid the known texture bias of ImageNet ( Deng et al. , 2009 ) , ( Geirhos et al. , 2018 ) generated stylized ImageNet , and ( Lee et al. , 2019 ; Wang et al. , 2018 ) inserted filter in front of the models so that the influence of the backgrounds and colors of the images can be removed . Mitigating bias without human supervision . To reduce human intervention , recent research ( Clark et al. , 2019 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Nam et al. , 2020 ; Li & Vasconcelos , 2019 ; Li et al. , 2018b ; Le Bras et al. , 2020 ; Darlow et al. , 2020 ) has used ensemble-based methods , using two separate networks with different purposes . The biased model learns biased information and even empowers it . Then , the biased model is used to train the de-biased model as a pathfinder , which provides information about the bias features . The ensemble idea has been used as a key component in several studies , ( Clark et al. , 2019 ; Cadene et al. , 2019 ; Bahng et al. , 2020 ; Nam et al. , 2020 ) . ( Cadene et al. , 2019 ) proposed a method known as RUBi , which multiplies the sigmoid output of the biased model by the softmax output of the de-biased model . ( Clark et al. , 2019 ) summed the output of the two models to train the de-biased model with an entropy regularizer . ( Bahng et al. , 2020 ) proposed ReBias algorithm using the Hilbert-Schmidt independence criterion ( HSIC ) as an objective of the de-biased model , by independent of the biased model . Learning from failure ( LfF ) ( Nam et al. , 2020 ) used generalized cross-entropy-based training that emphasizing the usual samples but ignoring unusual samples . In contrast , biased and de-biased models are trained sequentially ( Li & Vasconcelos , 2019 ; Li et al. , 2018b ; Le Bras et al. , 2020 ; Darlow et al. , 2020 ) . Researchers computed the weights to reconstruct the training set from the biased model and trained the de-biased network on the reconstructed training set . The REPAIR ( Li & Vasconcelos , 2019 ) and RESOUND ( Li et al. , 2018b ) methods proposed mutual-information-based weights to create a downsampled dataset . In addition , AFLite ( Le Bras et al. , 2020 ) proposed accuracy-based weights from multiple biased models , unlike LAD ( Darlow et al. , 2020 ) and BiaSwap ( Kim et al. , 2021a ) , which generated new images using an autoencoder based on the latent representation of the biased model .

Summary: The paper proposes a de-biasing method for neural network without requiring explicit annotations of biases. The authors propose a resampling-based approach that automatically detects underrepresented samples based on the direction and magnitude of their loss gradients, then balances the dataset by rejection sampling. The paper further proposes a denoising module to identify and filter out noisy samples under the presence of label noise. The gradient-based de-biasing method is empirically shown to outperform prior work on both controlled synthetic and real-world datasets.