M ITIGATING D ATASET B IAS U SING P ER -S AMPLE G RADIENTS FROM A B IASED C LASSIFIER
Anonymous authors
Paper under double-blind review
A BSTRACT
The performance of deep neural networks (DNNs) primarily depends on the conﬁguration of the training set.
Speciﬁcally, biased training sets can make the trained model have unintended prejudice, which causes severe errors in the inference.
Although several studies have addressed biased training using human supervision, few studies have been conducted without human knowledge because biased information cannot be easily extracted without human involvement.
This study proposes a simple method to remove prejudice from a biased model without additional information and reconstruct a balanced training set based on the biased training set.
The novel training method consists of three steps: (1) training biased DNNs, (2) measuring the contribution to the prejudicial training and generating balanced data batches to prevent the prejudice, (3) training de-biased DNNs with the balanced data.
We test the training method based on various synthetic and real-world biased sets and discuss how gradients can efﬁciently detect minority samples.
The experiment demonstrates that the detection method based on the gradients helps erase prejudice, resulting in improved inference accuracy by up to 19.58% compared to the other state-of-the-art algorithm.
I NTRODUCTION
Over the past decade, deep neural networks (DNNs) have been a focus of research owing to humanlike performances in various tasks, e.g., image classiﬁcation (Krizhevsky et al., 2012), object detection (Girshick, 2015), and image generation (Goodfellow et al., 2014).
Despite these impressive results, deploying DNNs directly in real-world problems remains a signiﬁcant challenge due to the difﬁculties in obtaining well-curated training sets.
Speciﬁcally, unintendedly biased information in training sets causes prejudice, resulting in wrong decisions at inference time (Torralba & Efros, 2011;
Shrestha et al., 2021).
For instance, most “ski” images contain “skier.”
This unintended correlation can recommend a wrong shortcut for “ski” by examining the person.
Prior studies have employed human supervision (i.e., providing additional labels related to the bias)
(Kim et al., 2019; McDuff et al., 2019; Singh et al., 2020) or giving a speciﬁc information about a biased domain (Lee et al., 2019; Geirhos et al., 2018), which is very expensive, to reduce the inﬂuence of a “skier” on “ski” class images.
Recently, studies on replacing human labor with
DNNs have been actively discussed (Li & Vasconcelos, 2019;
Nam et al., 2020; Cadene et al., 2019;
Bahng et al., 2020; Clark et al., 2019; Le Bras et al., 2020).
Typically, two separate networks have been used: the biased network and the de-biased network.
A biased model learns to replace human intervention and teaches a de-biased model using the prejudiced knowledge of the biased model.
Previous studies based on the two mentioned networks approaches have had two directions: adjusting objectives and resampling .
Adjusting the objective refers to a method that obtains weighted loss or additional regularizers for each sample differently, where the weights or regularizers are obtained from the biased model (Nam et al., 2020; Cadene et al., 2019; Bahng et al., 2020; Clark et al., 2019).
Methods that adjust the objective have been proposed frequently due to their several advantages, such as the implementational simplicity.
However, according to (An et al., 2020), adjusting-objective approaches suffers instability with stochastic gradient descent (SGD) type optimizers owing to the high variance of learning weights per sample.
Alternatively, resampling methods could reconstruct balanced sets.
Previous research (Le Bras et al., 2020; Li & Vasconcelos, 2019; Li et al., 2018b; Kim et al., 2021a) has focused on which samples should be oversampled to build a balanced set.
However, almost all prior works has focused on expensive human labor to tune hyperparameters related to the resampling densities of data points.
For instance, it is difﬁcult to set the amount of resampling of each sample without knowing how much biased the training set is.
The de-biasing methods, belonging to both adjusting-objective and resampling approaches are designed to emphasize samples that are hard for a model to learn.
However, a few noisy labels (i.e., mislabeled samples) in the training set could signiﬁcantly interfere with such de-biasing because it might be difﬁcult to discriminate between two types of hard samples: noisy samples that the model should not learn and rare samples that have to learn.
Although training sets have noisy labels in practice (Natarajan et al., 2013), to the best of our knowledge, no study considers the de-biasing problem under noisy label cases.
Contribution.
In this study, we propose a score-based resampling scheme for de-biasing, which does not require hyperparameters related to training set reconstruction.
Instead of hyperparameters, we construct two types of scores that leverage gradients of the biased model trained on the given biased set.
The proposed scores determine the required proportions for each sample in training set reconstruction.
Before designing scores, we explain our hypothesis that gradients have remarkable differences between samples generating prejudice and the others , and check it empirically.
The hypothesis is based on the following observations.
First, the gradient magnitudes (especially the Euclidian norm) of the samples which contributes to the prejudice are relatively smaller than those of the others.
Second, samples that make prejudice and the others do not have signiﬁcantly different gradient directions, measured by the estimated likelihood of the von Mises-Fisher (vMF) distribution (Banerjee et al., 2005; Lee et al., 2018).
Based on these observations, we propose two types of scores and a resampling-based de-biasing method.
The proposed method consists of three steps similar to previous resampling studies (Li & Vasconcelos, 2019; Le Bras et al., 2020; Kim et al., 2021a).
The ﬁrst step is to train a biased model based on a biased raw training set.
Afterward, the scores are obtained by the biased model.
The mini-batch sampler generates balanced mini-batches using the scores, and it feeds them into an ultimate model for de-biased training.
In addition, we investigate the side effects of de-biasing methods when incorrectly labeled samples are in the training set.
We observe signiﬁcant performance degradations of de-biasing methods with a small portion of noisy data.
To alleviate the negative side effects of noisy data, we ﬁrst de-noise the training set using loss values from the our observation that noisy label samples have higher losses than clean samples and then run de-noising algorithms.
The de-noising step successfully protects the de-biasing performance.
Finally, we demonstrate the effectiveness of our method using various biased benchmarks (Colored MNIST (Nam et al., 2020;
Kim et al., 2019; Li & Vasconcelos, 2019; Bahng et al., 2020),
Watermarked MNIST, Cartoon (Royer et al., 2020), CelebA (Liu et al., 2015), Biased action recognition (Nam et al., 2020), and ImageNet / ImageNet-A (Bahng et al., 2020; Deng et al., 2009; Hendrycks et al., 2019)).
In most experiments, the proposed method outperforms the current state-of-the-art methods.
In particular, the proposed method improves the accuracy of the unbiased test by 68 .
% from the baseline with a small performance degradation of the biased test of 0 .
Moreover, we compare the proposed method with the previous resampling method, REPAIR (Li & Vasconcelos, 2019).
The proposed method improves the average accuracy by 71 .
04% → 98 .
R ELATED W ORK
Mitigating bias with human supervision.
In (Goyal et al., 2017; 2020), generated a de-biased dataset using human labor.
A collection of studies (Alvi et al., 2018;
Kim et al., 2019; McDuff et al., 2019; Singh et al., 2020; Teney et al., 2021; Ramaswamy et al., 2021; Tartaglione et al., 2021;
Geirhos et al., 2018; Wang et al., 2018; Lee et al., 2019) aimed to mitigate bias based on one of two types of human supervision: explicit bias labels and implicit bias information .
Other studies (Alvi et al., 2018;
Kim et al., 2019; McDuff et al., 2019; Singh et al., 2020), have used bias labels for each sample to reduce the inﬂuence of the bias labels when classifying target labels.
(Tartaglione et al.,
2021) proposed the EnD regularizer, which entangles target correlated features and disentangles biased features.
Several authors (Alvi et al., 2018; Kim et al., 2019; Teney et al., 2021) have designed
DNNs as a shared feature extractor and multiple classiﬁers.
In contrast to the shared feature extractor methods, (McDuff et al., 2019; Ramaswamy et al., 2021) constructed a classiﬁer and conditional generative adversarial networks, generating test samples to check whether the classiﬁer is biased.
(Singh et al., 2020) proposed a new overlap loss deﬁned by a class activation map (CAM).
The overlap loss reduces the overlapping parts of the CAM outputs of two bias labels and target labels.
Different approaches (Geirhos et al., 2018; Wang et al., 2018; Lee et al., 2019) have assumed that the characteristics of bias features are known and prevent learning the implicit bias features.
To avoid the known texture bias of ImageNet (Deng et al., 2009), (Geirhos et al., 2018) generated stylized
ImageNet, and (Lee et al., 2019; Wang et al., 2018) inserted ﬁlter in front of the models so that the inﬂuence of the backgrounds and colors of the images can be removed.
Mitigating bias without human supervision.
To reduce human intervention, recent research (Clark et al., 2019; Cadene et al., 2019; Bahng et al., 2020;
Nam et al., 2020; Li & Vasconcelos, 2019;
Li et al., 2018b; Le Bras et al., 2020; Darlow et al., 2020) has used ensemble-based methods, using two separate networks with different purposes.
The biased model learns biased information and even empowers it.
Then, the biased model is used to train the de-biased model as a pathﬁnder, which provides information about the bias features.
The ensemble idea has been used as a key component in several studies, (Clark et al., 2019; Cadene et al., 2019;
Bahng et al., 2020; Nam et al., 2020).
(Cadene et al., 2019) proposed a method known as
RUBi, which multiplies the sigmoid output of the biased model by the softmax output of the de-biased model.
(Clark et al., 2019) summed the output of the two models to train the de-biased model with an entropy regularizer.
(Bahng et al., 2020) proposed ReBias algorithm using the Hilbert-Schmidt independence criterion (HSIC) as an objective of the de-biased model, by independent of the biased model.
Learning from failure (LfF) (Nam et al., 2020) used generalized cross-entropy-based training that emphasizing the usual samples but ignoring unusual samples.
In contrast, biased and de-biased models are trained sequentially (Li & Vasconcelos, 2019; Li et al., 2018b; Le Bras et al., 2020; Darlow et al., 2020).
Researchers computed the weights to reconstruct the training set from the biased model and trained the de-biased network on the reconstructed training set.
The REPAIR (Li & Vasconcelos, 2019) and RESOUND (Li et al., 2018b) methods proposed mutual-information-based weights to create a downsampled dataset.
In addition, AFLite (Le Bras et al., 2020) proposed accuracy-based weights from multiple biased models, unlike LAD (Darlow et al., 2020) and BiaSwap (Kim et al., 2021a), which generated new images using an autoencoder based on the latent representation of the biased model.
BIAS PROBLEM FOR TRAINING A CLASSIFIER 3. U NINTENDEDLY BIASED TRAINING
Suppose that a training set D is composed of images x , as illustrated in Figure 1.
Each image can be described by a set of attributes { a 1 , ..., a k , ... } , (e.g., {“digit 0”, “digit 1”,... “red”, “green”, ... , “thick”,...}).
The goal of the training classiﬁer is to ﬁnd a model f
θ that correctly predicts the
C target attributes “digits,” y
= { a t 1 , ..., a t C } = {“digit 0”, “digit 1”,...}.
Remark that target attributes are also interpreted as class .
However, we focus on the case in which another attributes exists, “colors,” where b = { a b 1 , ..., a b C } = {“red”,“green”, ...} (cid:54) =
y , which is highly correlated to the target “digits” (i.e., H ( a t 1 | a b 1 ) ≈ 0 ).
Such an correlated attributes and training set are denoted as the bias attributes and biased training set , respectively.
In addition, the “color-digit” correlated and uncorrelated samples, in the top two rows and the bottom row in
Figure 1, are called the majority and minority .
We use the notations M and m for the majority and minority sets, respectively.
For example, “red-digit 0” image is majority sample, while “orange-digit 0” one is minority sample.
When we train a model based on the biased training set D , the model may infer the bias attributes b , not target attributes y , which is problematic because the target is y , not b , and we refer to this as an unintended bias problem .
For example, if the model trained on the images in Figure 1 suffers from this problem, this model outputs 4 when orange, 0 image is given.
3. W HEN DOES THIS PROBLEM HAPPEN AND INTENSIFY ?
[Figure] [Figure]
(Nam et al., 2020) argued that this problem arises when two conditions are met simultaneously: (C1) bias attributes b is easier to learn than target y , and (C2) highly correlated b to y .
We focus on observing the effect of (C2) on the unintended bias problem among the two conditions when (C1) holds.
To understand (C2) precisely, we check the effect of the minority ratio | m | / | D | on the accuracy.
A detailed description of the training setting is provided in Appendix A.2.
We investigate the test accuracy of the samples in the majority and minority sets.
As in Figure 2, the average performance drop of the minority samples intensiﬁes when the minority ratio | m | / | D | decreases.
On the other hand, the accuracy of the majority samples for all cases are not dropped.
More precisely, we plot the confusion matrix to see how poor the performance is when the bias attributes b and the target attributes y are not aligned according to the minority ratio | m | / | D | .
As in
Figure 3, the increments of the minority ratio does not make the performance change of the aligned samples located in the diagonal, but other entries change a lot.
These results can be interpreted as when the ratio | m | / | D | becomes small, the model highly likely infer the bias attributes b rather than target attributes y .
D E - BIASING USING PER - SAMPLE G RADIENT I NFORMATION
When a given training set is biased, the model learns b over the intended target y , and it is strengthened when | m | / | D | approaches zero.
If this is the case, it is desirable to modify the training set to a larger | m | / | D | so that the model learns the target y rather than the biased attribute b .
To do so, we considered constructing a new training set D (cid:48)
= { ( x (cid:48)
i , y (cid:48) i ) } Ni =1 such that | m (cid:48)
| / | D (cid:48) | increases.
Similar to prior resampling-based work (Li & Vasconcelos, 2019), we sequentially train two separated networks: biased model f θ b and de-biased model f θ d .
The biased model, f θ b is trained based on the raw training set D .
After sufﬁciently converging to point θ b → ˆ θ b , the model computes gradients ∇ θ i = ∇ θ
L CE ( x i , y i ; θ ) of all the samples ( x i , y i ) ∈ D .
Then, we compute the sampling probability ( p s ( i ) ) of the i th sample using the per-sample gradient ∇ θ i , to break (C2) by balancing between M and m .
Finally, we train the de-biased model f θ d using a rejection sampler that creates a balanced training set D (cid:48) based on the sampling probability p s ( i ) .
4. P ER - SAMPLE G RADIENT -B ASED S CORES
As the ﬁrst step of de-biasing, we train the biased model without modifying training set D .
The converged model f ˆ θ b ( · ) contains in-depth information about the distribution of the training set in its model (especially, through the per-sample gradient).
Therefore, we leverage the per-sample gradient, ∇ θ i =
∂
L CE ( x,y ;ˆ θ b ) ∂θ | x i ,y i of ( x i , y i ) , which is dissolved in the biased model.
Why the per-sample gradient?
The majority and minority samples exhibit a clear difference in the per-sample gradient for the model f ˆ θ b owing to the following reasons.
(R1)
Each sample has its own loss landscape, and similar representation samples have similar loss landscapes (see Figure 4(a)).
In other words, the minority samples have different loss landscapes from those of the majorities, which implies that a local minimum cluster is composed of similar representation samples.
(R2)
[Table]
.||
∇θ
|| = 0.
[Table]
[Table] [Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table] [Figure]
[Table]
[Figure]
The values are gradient direction and loss difference (Li et al., 2018a).
See Appendix detailed setting the ratio of majority and minority samples at each respect to the directional and
From the averaging property of the batch-based optimization methods, the inﬂuence of the minority samples is diluted of according to their ratio.
Thus, the model trained on the biased training set converges to a proximity point of the minimum point of the majorities and it is reﬂected in the gradient of each sample.
The red arrows in Figure 4(a) and 4(b) indicate that the majority samples (top) have shorter gradients than the minorities (bottom) for both per-sample and aggregated cases.
Similarly, the direction of both cases is signiﬁcantly different .
Additionally, as seen in Figure 4(c), the portion of minority samples in the lower directional likelihood and higher magnitude bins is much larger than the majority samples.
Gradient-based score.
Based on the intuition of per-sample gradient and empirical results, we design a gradient-based score S ( i ) =
S (cid:16) ( x i , y i ); ˆ θ b (cid:17) , which is computed for each class separately.
The score is computed using two base scores, the magnitude score M ( i ) and direction score D ( i ) , which are deﬁned as follows:
[Equation]
(1) Magnitude score M ( i ) :
The contribution to prejudicial training is inversely proportional to the magnitude of its gradient ( ∝ 1 / (cid:107)∇ θ i (cid:107) ).
We used Euclidian distance to measure the magnitude.
(2) Direction score D ( i ) :
The contribution to prejudicial training is also proportional to the directional likelihood ( ∝ p dir ( i ) ).
For the directional likelihood, the estimated von Mises-Fisher (vMF) distribution is used to understand the concentration of the directional distribution.
The vMF distribution is
deﬁned as
follows
Deﬁnition 4.1 (von Mises-Fisher Distribution) .
The pdf of the vMF ( u, κ ) is given by
[Equation]
on the hypersphere S d − 1 ⊂ R d , where κ determines how much samples are concentrated based on the distribution along the mean direction u , and C d ( κ ) is a normalization constant determined by dimension d and the concentration parameter κ .
To estimates the parameters of the assumed vMF distribution, we use approximated maximum likelihood estimates (MLE) solutions ˆ u and ˆ κ .
For normalized gradient vectors ∇ θ i = ∇ θ i
[Equation]
θ i (cid:107) , the MLE solutions ˆ u and ˆ κ can be obtained from ˆ u =
[Equation]
i (cid:107) and ˆ κ ≈ ¯ r
( d − ¯ r 2 ) (1 − ¯ r 2 ) , where ¯ r = (cid:107)
[Equation]
Ni =1 ∇ θ
i (cid:107) N (see Appendix G).
Then, the normalized directional score D ( i ) is deﬁned in (1), where p dir ( i ) =
f d ( ∇ θ i ; ˆ u, ˆ κ ) is the approximated vMF distribution.
To relax the computational constraint, we select k -dominant sub-dimensions, denoted by the set
K = arg max K ⊂ N, | K | =
k (cid:80) j ∈ K (cid:0)(cid:80)
i |∇ θ j | i (cid:1) , where ∇ θ j corresponds to the gradient of the j -th dimension.
4. D E - BIASING USING G RADIENT - BASED SCORES
Balanced batch selection.
By the deﬁnitions of the scores, the sample in the majority set M (or the minority set m ) must have a higher (or smaller) M ( i ) and D ( i ) , respectively.
Each sample must interact inversely with the two scores to treat the samples in the minority set m similarly to the samples in the majority set.
To do so, we design aggregated score S ( i ) based on the harmonic mean, and sampling probability p s ( i ) as follows:
[Equation]
where λ is the balancing hyperparameter to emphasize a speciﬁc score between M ( i ) and D ( i ), and
C is the normalization constant deﬁned as C =
min
i S ( i ) , which makes p s ( i ) =
1 for the smallest
S ( i ) sample.
From the harmonic-mean, S ( i ) has a higher value when one of the scores is high.
[Table]
[Table]
To construct a balanced mini-batch, our algorithm use a rejectionsampling-based uniform sampler, Algorithm 1.
Each sample i is rejected with a probability 1 − p s ( i ) , which implies that the rejection is proportionally to the scores.
From the property of rejection sampling, such a sampler generates uniform batches.
[Table]
mini-batch ( X,Y )
= {
( x,y ) } ki =1 from D 5:
Update θ b ← θ b − η | D | ∇ θ (cid:80) ( X,Y )
L GCE ( x,y ; q ) 6: end for 7: /** STEP 2: Calculate p s ( i ) **/ 8: Calculate p s ( i ) for all i ∈ D 9: /
** STEP 3: Train f θ d **/ 10: for R do t=1,2,...,T 11:
Draw a mini-batch { ( x,y ) } ki =1 from the Alg.
1: S ( p s ( i )) 12:
Update θ d ← θ d − η | D | ∇ θ (cid:80) ( X,Y )
L CE ( A ( x ) ,y ) 13: end for
Gradient-based de-biasing.
The ultimate debiasing algorithm consists of three steps.
First, the biased model f θ b is trained on the raw data samples with GCE loss (Zhang & Sabuncu, 2018; Nam et al., 2020).
The GCE loss controlled by q guides the biased model to ignore the minority samples in the training set.
Then, all raw samples are input into the biased model to compute p s ( i ) .
The last step is to train the debiased model f θ d using a balanced mini-batch sampler.
This sampler generates balanced minibatch based on the sampling probability p s ( i ) .
We use the typical image data augmentation method A ( · ) (e.g., rotate, resize) to avoid over-ﬁtting.
D E - NOISING A SMALL PORTION OF THE NOISY LABELS
[Figure]
Why is noisy label a problem for de-biasing algorithms?
All de-biasing algorithms work by emphasizing rare samples.
For example, the proposed method operates by reconstructing minibatches in proportion to the rarity.
However, when a small portion of incorrectly labeled samples, called noisy labels , is in the training set, the de-biasing algorithms accentuate the noise labels due to their mechanism of enlarging the inﬂuence of the rare samples.
Therefore, as presented in Figure 5, all de-biasing algorithms suffer a performance decline.
Distinguishing minority and noisy labels via compensated loss.
Recent de-noising methods (Yu et al., 2019; Yi & Wu, 2019;
Li et al., 2019;
Kim et al., 2021b) have mainly focused on ignoring unusual sample, which may be noisy labels.
However, by comparing de-noising mechanism with (Kim et al., 2021b), previous de-noising algorithms also ignores minority samples which must be highlighted not discarded, see Appendix I for details.
Therefore, we design a module to delete noisylabels in which the minority samples are preserved.
To do so, we observe the loss statistics of the minority samples and noisy labels under various Colored MNIST settings with
( | m | / | D | , noisy label ratio ) =
(0 . 01 , 0 . 1) and (0 . 1 , 0 . 1) (see Appendix B.3).
As depicted in Figure 6, noisy labels have a higher loss than clean labels, regardless of whether the sample is in the majority or the minority set.
The trained model that ignores rare samples, outputs the correct results, whereas the given label for calculating the loss value is incorrect.
However, the loss of the clean minority samples increases, owing to the rarity.
To compensate, we suggest the compensated loss ¯ L i , reducing the inﬂuence of the rarity, and the splitting criteria to distinguish between the clean samples and noisy labels:
[Equation]
¯ L
[Equation]
=
[Equation]
Note that H ( f θ ( x i )) represents the softmax entropy of sample x i , and δ = (cid:80) i ≤ N ¯ L i / (cid:80) i ≤ N H ( f θ ( x i )) .
In Figure 6, the black lines represent the threshold δ , and the lower part of this line represents D clean , whereas the upper part represents D noise , respectively.
Other discrimination results are depicted in Appendix E.
[Table]
De-noising module.
We design a de-noising module that is applicable in front of all prior de-biasing algorithms from the proposed criteria.
As presented in Algorithm 3, this module comprises three simple steps.
First, a noise model parameterized by ˆ θ n is trained on the raw biased and noisy training sets.
Then, we split D clean and D noise according to the criteria (2).
After splitting, we run the de-biasing algorithm using D clean .
E XPERIMENTS
We evaluated the proposed model for two types of benchmarks: synthetically generated bias (referred as controlled biased benchmarks ) and raw bias ( real-world biased benchmarks ).
We compared the results with the ofﬁcially available recent de-biasing methods, RUBi (Cadene et al., 2019), LearnedMixinH (Clark et al., 2019),
ReBias (Bahng et al., 2020), AFLite (Le Bras et al., 2020), LfF (Nam et al., 2020), REPAIR (Li & Vasconcelos, 2019) and the vanilla model.
Each baseline was reproduced using the ofﬁcial codes for each algorithm (see Appendix C).
Moreover, we optimized all algorithms based on the 10% of the biased training set.
For the experiments, three types of convolutional neural networks were used with various regularization methods (batch normalization (Ioffe & Szegedy, 2015), dropout (Srivastava et al., 2014), and weight decaying (Moody & Hanson)).
We report the detailed settings in Appendix A.1.
In addition, we determined the test under ﬂipped labels in MNIST variants with various noise ratios of 10% , 5% , and 0% to verify the side effects from noisy labels and check the protecting performance of the de-noising module (see Appendix A).
6. B ENCHMARKS 6.1. C ONTROLLED B IASED B ENCHMARKS
To precisely examine the de-biasing performance, we generated controlled biased benchmarks using well-known data: MNIST (LeCun et al., 2010) and Cartoon (Royer et al., 2020).
Various bias ratios, | M | / | D | ∈ { 0 . 995 , 0 . 99 , 0 . 95 } , were used for all the controlled tests.
All benchmarks were divided into a biased training set D tr , biased validation set D val ( 10% of D tr ), and test set D te comprising the majority M and minority m (see Appendix B.1 for detailed data construction rules.)
MNIST Variants.
The MNIST dataset comprises a grayscale digit data with a handwritten style.
We injected two different biases, color and different objects (speciﬁcally, Fashion MNIST (Xiao et al., 2017)), which are colored MNIST (CM) and watermarked MNIST (WM), respectively.
This main task is classifying the shape of the digit without examining the biased attribute color of the
[Table]
digit or Fashion MNIST objects .
As an example of the CM, class 0 is divided into major and minor sets colored in red and the others.
Similarly, in WM, the major set of class 1 contains trouser , whereas the minor set contains other objects as watermarks at the top-left corner.
Examples are depicted in Figure 7.
Cartoon.
A Cartoon (CT) (Royer et al., 2020) is an avatar image with 18 corresponding attributes (e.g., hair style ).
We select two of them, hair color and face color , for the target and biased attributes.
To create a bias, the major set is whose two attributes are bonded was subsampled.
For example, in Figure 8, the face color of blond hair is black , except for a few.
6.1. R EAL -W ORLD B IASED B ENCHMARKS
We used three well-known biased benchmarks to determine the de-biasing performance on complex benchmarks: Biased Action Recognition (BAR), CelebA, and ImageNet/ImageNet-A (see
Appendix B.2 for details).
Biased Action Recognition.
Biased action recognition (Bahng et al., 2020; Nam et al., 2020) is a motion classiﬁcation set with a background bias.
For example, in the class climbing , major and
Minor samples have rockwall and ice cliff backgrounds, respectively.
CelebA. (Nam et al., 2020; Liu et al., 2015)
CelebA has facial images with 40 binary attributes.
Among these, we used heavy makeup and gender attributes for target and biased attributes.
In the heavy makeup class, major is the female attribute, whereas male is the minority attribute.
ImageNet/ImageNet-A. ImageNet (Deng et al., 2009) is a well-known image classiﬁcation dataset.
However, (Hendrycks et al., 2019) claimed, a model trained on the ImageNet data easily fails to classify unusual backgrounds.
This is because ImageNet has a biased background (e.g., almost all Frog images are considered in the swamp , which means major sample).
As a de-biasing performance evaluation benchmark, we used ImageNet for training and to test ImageNet-A images with unusual backgrounds.
6. E VALUATION
Controlled benchmarks without noisy labels.
The accuracy values of two major and minor sets are reported in Table 1.
In particular, we ﬁrst examined the accuracy of the vanilla algorithm, the ultimate baseline.
Overall, the vanilla algorithm achieved nearly 100% accuracy for the major case, the vanilla failed to infer with an accuracy similar to that of the major case.
[Table]
De-biasing algorithms correctly classiﬁed the major case, similar to the vanilla case.
However, for the minor case, these algorithms obtained higher accuracy values than that of the vanilla results (at most 31 .
78% ↑ ).
Despite these de-biasing performances, they have a higher gap between major and minor cases (at least 36 .
69% under a 99 . 5% bias ratio).
In contrast, the proposed method obtained the highest accuracy value for the minor case (at most 69 .
92% ↑ )
compared to the vanilla results.
These were obtained the
Controlled benchmarks with noisy labels.
In Table 2, almost all algorithms failed to improve the minority accuracy, when labels are ﬂipped, compared to the 0% noisy label case.
For example, the proposed model’s average performance declined from 98 .
50 % to 84 .
05 %, and the majority performance declined from 99 .
24% to 80 .
60 % for the CM case with 5% noisy labels.
However, these performance decrements were recovered from the de-noising module for all cases.
For the 5% noise CM case, all cases obtained close to 100% accuracy for the majority case and improved the performance for the minority case.
Among all methods, the proposed gradient-based method obtained 92 .
94% on average of the majority and minority test sets for the 5% noise of CM case, and improved by 7.83% in performance from the state-of-the-art algorithm, LfF at 85 .
Moreover, we checked from the 0% noise case that the adverse effect of the de-noising module on de-biasing is − 3 .
50% → 95 .
60% ) on CM case for the proposed method.
[Table]
Real-world benchmarks.
In Table 3, we report that real-world benchmarks can be biased by observing the lousy performance of the vanilla algorithm for all cases.
The BAR case indicates that the proposed method gained the best performance on average among classes (a 5 . 06% increase from the vanilla algorithm and 2 .
53% increase from the second-best accuracy).
As listed in other columns, the proposed method has helped improve vanilla by 3 .
64% and 2 .
87% for the CelebA and ImageNet/ImageNet-A cases, respectively.
It also outperforms other methods by up to 1 .
11% and 3 .
% , respectively.
A detailed analysis is provided in Appendix F.
Analysis.
We report the detailed various studies in Appendix H.
C ONCLUSION
We presented a novel de-biased method by generating a mini-batch so that the trained model does not have prejudice in the inference.
The proposed sampling metric is based on the gradient of each sample, which is mainly correlated with its familiarity.
Through extensive experiments using various biased datasets, we demonstrated the effectiveness of the proposed method in several cases.
Furthermore, we proposed a de-noising module that is easily applicable to all de-biasing algorithms and protects from the side effects of noisy labels.
We hope this study proposed exploration to understand the feature bias problem, especially for resampling-based approaches and gradient-based data valuation areas of machine learning.
A E XPERIMENT D ETAILS A. B ASIC SETTINGS
[Figure]
Network architecture.
We use three types of convolutional neural networks (CNNs) (see Figurefor two of them and the other is ResNet-18 (He et al., 2016)) for all benchmarks.
For MNIST variants, three convolution layers with ﬁlter size 4 and average pooling with size 2 are used (see
Simple ConvNet-1 ).
For Cartoon, 5 convolution layers with ﬁlter size [6 , 6 , 5 , 5 , 4] for each layer with average pooling size 2 are used (see Simple ConvNet-2 ).
The real-world benchmarks are tested under pre-trained ResNet-18 provided by Pytorch torchvision , without freezing feature extractor.
We apply batch normalization (Ioffe & Szegedy, 2015), dropout (Srivastava et al., 2014) for regularization, and utilize SGD optimizer with momentum and weight decay (Moody &
Hanson).
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
Hyperparameters.
The hyperparameters that we used are summarized in Table 4.
We share basic hyperparameters (including Network Type, Learning rate, Learning rate decay,
Learning rate decay epoch, Momentum, Weight decay
[Figure]
and Batch size and Number of Epoch ) for all algorithms.
Precisely, for the proposed method, we set GCE parameter q and Dimension size K for our method, as in Table 4.
Validation.
We optimize hyperparameters using a validation set of 10% of the training set and unseen at the training phase.
Reported our results are obtained from the hyperparameters which shows a small loss for the validation set.
Data augmentation.
Image data augmentation operations have been used frequently to improve generalization performance.
We utilize three operations: RandomResizedCrop with scale (0 . 9 , 1 . 1) ,
ColorJitter with 0 .
05 hue and 0 .
05 saturation, and RandomRotation with ( − 10 , 10) degree using NEAREST ﬁlling criterion.
These augmentation operations are from transforms module in torchvision .
A. T EST SETTINGS IN S ECTION 3, 4, AND
Ratio of minority samples analysis.
To analyze the correlation between | m | / | D | and accuracy statistics, we generated a bias training set based on CM data generation in Appendix B.1.
This tests are conducted on the experiment with difﬁculty 0 . 0001 .
Gradient property test To test our intuitions about gradients, we utilize part of CM data (includes class 0 and 1 ) with a bias ratio of 99 . 5% and difﬁculty 0 . 0001 .
Finally, we draw red arrows using the lowest point and center point.
This center point is the convergence point, and other points are random permuted loss values.
All loss landscape ﬁgures are plotted by following (Li et al., 2018a).
B B ENCHMARKS B. C ONTROLLED B IASED BENCHMARKS
MNIST Variants.
MNIST variants are modiﬁed data from gray-scaled hand-written digit MNIST images (LeCun et al., 2010).
To inject biased attributes into the MNIST, we use two different attributes: color and object.
• Colored MNIST (CM) has been frequently used as a benchmark due to their simplicity (Kim et al., 2019;
Bahng et al., 2020; Nam et al., 2020).
The target of this benchmark is classifying the target attributes (shape of digits) not the biased attributes (color of digits).
To make biased training set, all images are colored after up-scaling dimension from gray R 28 × 28 to 3D RGB R 28 × 28 × 3 .
Uniformly sampled 3-dimensional 10 colors C c
= { R c , G c , B c } 9 c =0 are allocated to each class c .
The samples in class c of set M are colored by their allocated color C c .
The other samples are colored with the color vector C c (cid:48) (cid:54) =
c where c (cid:48) (cid:54) =
c is uniformly sampled.
Furthermore, to deviate correlation between the target and biased attributes, color for each sample ( x i , y i ) is deviated using 3-dimensional Gaussian distribution C
i
=
N
( C y i , αI ) .
This deviation is controlled by the difﬁculty parameter
α ∈ {
• Watermarked MNIST (WM) has a different type of bias.
The target of this benchmark is the same with generic MNIST, classifying hand-written digits, while the unintended object (Fashion MNIST (Xiao et al., 2017) data) is located as a biased attributes.
To locate Fashion MNIST, all images from MNIST are size-up scaled from R 28 × 28 to R 56 × 56 .
Set M
and m includes samples ( x i , y i ) where image x i is made with digit image x target and fashion object x bias whose label indices follow y i =
y target =
y bias and y i =
y target (cid:54) =
y bias , respectively.
To make deviation, we put images from δ pixels far from top and bottom of the canvas.
δ is sampled from uniform distribution δ ∼ U (0 , α ) .
We test two cases, α ∈ { 8 , 16 } .
Cartoon.
Cartoon data composed of synthetically generated face image samples which have 18 attributes (e.g., hair color , face color ) with multiple labels (e.g., hair color = { 1 , ..., 10 } ).
Among 18 attributes, we choose hair color and face color attributes as the target attributes and the biased attributes.
For simplicity, the size of each label is reduced fromto 4 and 10 to 4 by mapping 2 classes (similar colors) into 1 class, (e.g., original classes 1 , 2 are mapped to the new class 1 ).
We make set M whose new target class and biased attributes following c target =
c bias .
Also, set m are sampled among images following c target (cid:54) =
c bias .
B. R EAL - WORLD BIASED BENCHMARKS
Biased Action Recognition.
Biased action recognition data has 1 , 941 motion images for training with 6 -motions { Climbing, Diving, Fishing, Racing, Throwing, Vaulting } on various types of backgrounds (e.g., (Climbing, Rockwall) ).
The images in this benchmark include the frequent background of that action as a biased attributes for set M and rare backgrounds for set m (e.g., (Climbing, Rockwall) for M and (Climbing, Ice cliff) for m ).
This benchmark is originated from (Bahng et al., 2020;
Nam et al., 2020; Kim et al., 2021a).
CelebA. CelebA has 202 , 599 face images with 40 binary attributes from 10 , 177 identities.
Among 40 attributes, we select two attributes ( HeavyMakeup , Male ) for target and biased attributes.
The size of training subsets M and m are 130 , 410 , and 32 , 360 , respectively.
Remains are used for evaluation.
Splitting training and evaluation sets follow (Liu et al., 2015).
This task reﬂects that a real-world benchmark is easy to be biased in some speciﬁc attributes, unintendedly.
This benchmark is originated from (Nam et al., 2020).
ImageNet/ImageNet-A. ImageNet data has 1 , 000 classes, and we utilize a subset of ImageNet, which has 9 super-classes for scalability (Bahng et al., 2020).
Also, we evaluate using ImageNetA (Hendrycks et al., 2019), which gathers the failure cases of ImageNet from web.
As described in (Bahng et al., 2020; Hendrycks et al., 2019) dataset, “frequent background can be a biased attributes.”
For example, frogs in the swamp have a higher proportion than other backgrounds (e.g., underwater).
It indirectly represents that a de-biased model trained under ImageNet has better de-biasing performance when it has higher accuracy on ImageNet-A samples.
B. N OISY LABELS
We inject 10% noise labels into the basic conﬁguration conducted in Section 5.
Noise labels are uniform randomly ﬂipped by following rules.
At ﬁrst, we randomly sampled a set of noise label candidates regardless of whether the source of the data samples are majority set or minority set.
Flip the label y i
→ ˆ y i uniform randomly, so that the label is not equal to the ground truth
ˆ y i (cid:54) =
y i .
Three noise label ratios, { 90% , 95% , 1%0% } , were used.
In order to train noised model f ˆ θ n , we utilize cross-entropy loss (not generalized cross-entropy loss).
Also, to avoid over-ﬁtting, we utilize three data augmentation operations: RandomResizedCrop, ColorJitter, and Random Rotation which are parameterized by (0 . 9 , 1 . 1) , (0 . 05 , 0 . 05) and ( − 10 , 10) , respectively.
C B ASELINES
Vanilla As an ultimate baseline, which means without de-biasing module, we train a network called vanilla.
As summarized before, we utilize frequently used regularization techniques and simple data augmentation operations (jitter, crop, and rotation).
LearnedMixinH (Clark et al., 2019)
LearnedMixinH was proposed by Clark et al for VQA task.
This method of de-biasing is composed of biased (on the question) and unbiased (on both the image and the question) networks.
We generate a biased network in our experiment by simply training on the biased data set.
This is based on the argument in LfF (Nam et al., 2020), which states that biased models learn biased information ﬁrst, because it is easier to learn than the target attribute.
The biased model outputs two values: its softmax output and a bias prediction value for each given sample.
Ultimately, the de-biased model trained under the cross-entropy loss with regularizer of two above values from the biased model.
We implement this algorithm by inheriting the authors’ ofﬁcial code and unofﬁcial code made by the authors (Bahng et al., 2020).
RUBi (Cadene et al., 2019)
Similar with LearnedMixinH, RUBi also targets to solve VQA tasks.
We also utilize the argument from LfF (Nam et al., 2020), to train the biased network.
Instead using two values of the biased model, the authors of this paper utilized sigmoid outputs of the biased model and give weights to the de-biased model’s output, when calculating the cross-entropy loss.
This weighting mechanism gives weighted punishements by followings.
The biased model outputs highly and lower biased sigmoid output when major and minor samples are given.
By multiplying sigmoid values to the softmax output of the de-biased model, the de-biased model suffers higher loss from the minor samples.
We implement this method using the authors ofﬁcial code and unofﬁcial code from (Bahng et al., 2020).
ReBias (Bahng et al., 2020)
ReBias is a bi-level optimization-based de-biasing algorithm proposed by Bahng et al.
ReBias trains the biased and de-biased model simultaneously.
The de-biased model updated its cross-entropy loss and additional regularizer using Hilbert-Schmidt Independence
Criterion (HSIC).
Here, the biased model is composed of large convolutional ﬁlters to capture the bias information better, but we ignore this for a fair comparison with same network.
We utilize ofﬁcial code from the authors.
LfF (Nam et al., 2020)
The main module of this algorithm is two-fold: (i) The biased model trained on Generalized cross-entropy (GCE), which highlights the major samples.
(ii) The de-biased model trained on the weighted loss where weight is computed by the cross-entropy outputs from each.
From
GCE, the cross entropy loss loss of the minor samples increases.
We utilize ofﬁcial code from authors.
AFLite (Le Bras et al., 2020)
AFLite is another type of resampling approach.
AFLite trains the biased model multiple times and selects samples whose accuracy is low.
At the de-biasing phase, the de-biased model is trainend on the sub-sampled new training set.
Because authors offer the ofﬁcial code for a synthetic data, we manually reproduce their idea.
REPAIR (Li & Vasconcelos, 2019) REPAIR is a re-sampling-based de-biasing algorithm proposed by Li et al.
REPAIR reconﬁgure the training set based on the weight parameter w obtained by the adversarial training.
As a ﬁrst step, the biased model and the adversary are trained simultaneously.
The adversary computes per sample weight w so that the biased model gets a larger value for the minorities.
REPAIR utilized w to reconstruct the repaired training set.
The authors proposed four types of re-conﬁguring methods with given parameter k : REPAIR-T (Thresholding, sampling above k ), REPAIR-R (Ranking, Sampling top-
k % )
, REPAIR-PR (Per-class Ranking), REPAIR-S (Sampling, reject with probability 1 − w ).
We utilize REPAIR-S among these variants to fairly compare our sampling mechanism (without hyperparameter k case).
We implement REPAIR using the ofﬁcial code from the authors.
Additionally, note that we obtained w by training the classiﬁer at the adversarial training phase without using biased information, such as RGB values in the case of CM, where as the ofﬁcial code used the biased information.
This is enabled by the LfF (Nam et al., 2020) argument, which state that the network is biased toward a biased attribute that is easier to learn than the target.
D R ESULT : C ONTROLLED B IASED BENCHMARKS (O RACLE VALIDATION )
These results are obtained from the tuned hyperparameters, which are optimized by using the labels related to the biased attributes b .
As we can see, the proposed method obtains the best performance in terms of the average accuracy of majority and minority cases.
Moreover, other works also successfully improves their results.
However, it is difﬁcult to obtain a validation set which is discriminated between majority and minority samples.
Therefore, we can conclude that the proposed method obtains when we can access to the discriminated dataset between majority and minority cases, which is difﬁcult in practice.
[Figure]
[Table] [Table]
[Table]
E S PLITTING PERFORMANCE OF DE - NOISING MODULE Discriminativeness
power of the de-noising module between majority and minority cases on various noisy label ratio and bias ratio | m | / | D | .
As we can see for all cases, our de-noising module with compensated loss criterion can distinguish the minority samples and noisy labels.
In addition, when there is no noisy cases, ( 0% cases) the proposed de-noising module lose some valuable clean minority samples.
All cases are run with difﬁculty parameter 0 . 0001 and 8 for CM and WM cases, respectively.
[Figure] [Figure]
F R EAL - WORLD BENCHMARK : D ETAIL ACCURACY
As in all cases, the proposed method obtains the best performance on average of all cases, e.g., CelebA on average of majority and minority cases.
Also, our BAR results fails to obtain state-of-the-art performance in some classes, but get the best on average of all classes.
[Figure]
[Table]
The best accuracy is reported in bold font .
[Figure]
[Table] [Table]
[Table]
G VON M ISES -F ISHER DISTRIBUTION FOR GRADIENTS
The von Mises-Fisher (vMF) distribution is a probability distribution on the hypersphere S d − 1 ⊂ R d .
The probability density function of the vMF distribution for the unit vector v is given by:
[Equation]
where κ and u are called concentration parameter and mean direction , respectively.
The normalization constant C d ( κ ) is equal to C d ( κ ) =
[Equation]
p
[Equation] [Equation]
) p/ 2 I p/ 2 − 1 ( κ ) , where I t denotes the modiﬁed Bessel function of the ﬁrst kind at order t .
As in (Banerjee et al., 2005), when N samples from the vMF distribution are given as v i , the maximum likelihood estimates solutions ˆ u and ˆ κ can be obtained from ˆ u = (cid:80)
Ni =
[Equation]
i
[Equation]
=
[Equation]
i
[Equation]
and ˆ κ ≈ r ( d − r 2 ) 1 − r 2 , where
[Equation]
To interpret gradient vectors as vMF distribution, given vectors have to be unit vectors: ∇ θ i = ∇ θ
i (cid:107)∇ θ i (cid:107) .
Therefore, the approximated vMF distribution of the gradients can be obtained as follows:
f d ( ∇ θ i ; ˆ u, ˆ κ ) where ˆ u = (cid:80) Ni =1 ∇ θ
i (cid:107) (cid:80) Ni =1 ∇ θ i (cid:107) , ˆ κ ≈ ¯ r
( d − ¯ r 2 ) (1 − ¯ r 2 ) , and ¯ r = (cid:107) (cid:80) Ni
=1 ∇ θ
i (cid:107)
N .
H A NALYSIS
We conducted a few studies to analyze our method: hyperparameter sensitivity, (target,bias)-pairwise accuracy, pairwise sampling probability, ablation study on λ , adjusting objective results based on our metric, class activation map, unbiased case result, training time, results on naive metrics, and impact of GCE.
Controlled benchmarks were used for analysis.
[Figure]
Hyperparameter sensitivity.
In Figure 12, accuracy results of the models trained on the various learning rate η ∈ { 0 . 005 , 0 . 01 , 0 .
and epochs e ∈ { 50 , 100 } were depicted.
It shows that the prediction performance of all algorithms has higher accuracies on biased validation sets and majority set for various learning rates and epochs.
However, in minority case 12(c), ours have a less sensitive result on minority cases by seeing error bar’s length.
[Figure]
(Target, Bias)-pairwise accuracy.
In Figure
[Figure]
pairwise accuracy matrices were plotted using various de-biasing methods.
This represents the prediction result of 10 Major (diagonal) and
Minor pairs.
First, vanilla fails to infer Minor pairs, but it infers Major accurately.
Similar to vanilla, other algorithms, including ours, achieve almost accurate results for the Major pairs.
On the other hand, based on the perspective of minority , other algorithms fail to infer.
Among the de-biasing methods, our method has been approximately 100% accurate for almost all the pairs.
Pairwise sampling probability.
To understand the previous pairwise result, we checked the sampling probability of our method.
In Figure 14, each subﬁgure indicates pairwise sampling probability for all the metrics: p M ( i ) =
1 /M ( i ) (cid:80) i (1 /M ( i )) , p D ( i )
= 1 /D ( i ) (cid:80) i (1 /D ( i )) , and p s ( i )
= 1 /S
( i ) .
In Figure 14(a), the raw case is highly bonded between the shape and color (see diagonal).
In p M ( i ) and p D ( i ) cases, both are well mixed compared to without case.
In Figure 14(d), p s ( i ) has the most uniformized sampling probability.
Red and yellow indicators denote complementary effect of both scores.
[Table]
Ablation study on λ .
In the deﬁnition of the score S ( i ) , we set the hyperparameter λ to make balance between source scores M ( i ) and
D ( i ) .
In order to conﬁrm the effect of λ , the experimental results
[Figure]
are reported from various values of λ = { 0 , 1 , 10 , 100 } on CM with 99% bias without noise.
As in Table 11, there is no remarkable performance difference depending on the λ , so it could be observed that it is robust to the hyperparameter λ .
[Figure]
Adjusting objective vs Resampling.
As an ablation study, we clariﬁed the power of our gradient-based metric by reporting the results of adjusting objective leveraged by our p s ( i ) .
We conducted tests under two controlled biased benchmarks, colored MNIST and watermarked
MNIST with difﬁculty 0 . 0001 and 8 .
p s ( i ) , normalized by the mean value, p s ( i ) (cid:80)
Ni p
s ( i ) /N
, is used for the weights.
Our gradient-based score outperforms others in the adjusting objective (dagger mark † ) and resampling cases.
Compared with our gradient-based ways, the adjusting objective methods achieve lower performance.
This implies that, as argued in (An et al., 2020), the adjusting objective-based approach suffers from training sensitivity.
(a) Vanilla, M (b) LfF, M (c) Repair
[Figure]
[Table]
What de-biased model learn.
The main goal of de-biasing algorithms is to make the de-biased model not learn the biased feature.
To check whether the biased models satisfy this goal, we evaluate two tests: (1) random color test for CM, and (2) CAM for WM.
We construct test dataset by uniformly randomly sample colors for all samples RGB ∼ U (0 , 1) 3 .
As in Table 12, the result of ours show that it outputs based on the shape whether the training set is biased or not.
Additionally, we conduct CAM test for WM dataset.
CAM (Selvaraju et al., 2017) has been used as a tool to explain “where a model is focusing on?”
In Figure 15, we state the CAM results for watermarked MNIST samples obtained from the Major ( Left ) and Minor ( Right ) sets.
In the top row (for Major ), our model better ignores the biased object (fashion object) than the other methods.
Even for Minor , only our model determines the target digit , while the rest focuses on the biased object ( Ankle boots).
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
Unbiased / Rarely biased case.
To analyze the general side effect of additional modules, performance degradation, we check the accuracy of unbiased (especially, 10% = 1 / ( # of classes ) biased case) and rarely biased (70%, 80%, 90%) cases, under MNIST variants with difﬁculties of 0 . 0001 .
As listed in the table, all models have similar accuracy to the vanilla case.
This means that, re-
gardless of whether the training set is biased, all models, including ours, do not suffer from severe side effects.
It shows that our algorithm can be applied without performance loss.
In short, our algorithm needs not to utilize human knowledge about bias in the training set, an on/off knowledge.
[Table]
Training Time.
To see the drawbacks of training time for de-biasing, we measure the training time of 6 related works on WM case.
As in Table 14, the proposed model spends × 1 . 45 time compared to RUBi which is the fastest de-biasing method, while × 0 .
72 faster than the other re-sampling algorithm, REPAIR.
[Table]
Other metrics.
This study shows other metrics-based resampling results, (e.g., softmax response, loss, and logit value), based on sampling probability.
These results show empirical justiﬁcation of our gradient-based scores for de-biasing.
All scores are deﬁned to make the minority samples (unfamiliar samples) have a higher sampling probability.
Each score is deﬁned as follows:
[Equation]
where S ( · ) is softmax response and m = max k ∈ D
( f ( x k , y k )) .
As depicted in Figure H, our gradientbased approach has the highest performance on the average accuracy of majority and minority results.
Moreover, the gradient method has a short error bar than the others, which works reasonably (softmax and loss case).
[Table]
Impact of GCE.
We checked the proposed method without GCE (trained with CE loss) under 99 . 5% WM case.
The proposed method w/o GCE achieved ( M / m ) 99 .
72% , which outperforms the others, while lower than the proposed method without GCE case ( 99 . 35% / 83 .
I D E - NOISING
[Table]
In recent noisy label cases, there have been two major trends: (1) constructing robust objective functions or regularizers (Cao et al., 2020;
Yi & Wu, 2019), and (2) cleaning the noisy data (Li et al., 2019;
Kim et al., 2021b).
Those strategies are mostly used to reduce the inﬂuence of rare samples, such as out-of-cluster samples in (Kim et al., 2021a).
However, in the case of dataset bias problem, minority samples can be omitted when rare samples are discarded, resulting in de-biasing algorithms’ performance degrading.
As a result, erasing noisy label samples while maintaining minority samples is crucial in the dataset bias problem with noisy labels to preserve the de-biasing performance.
Table 16 shows the number of preserved samples after cleansing steps.
We compare the proposed de-noising module to the state-of-the-art method, FINE.
It cutting-out noisy labels by using eigenvectors.
As shown in Table 16, while the proposed de-noising module does not perfectly eliminate noisy samples, it does preserve clean minority samples, whereas FINE does not.