Learning Debiased Representation via Disentangled Feature Augmentation
Authors: Jungsoo Lee
Eungyeup Kim
Juyoung Lee
Jihyeon Lee
Jaegul Choo
Venue: NeurIPS 2021 Oral (2021)
Keywords: Debiasing
Disentangled representation
Feature-level augmentation
Image classification

Abstract: Image classification models tend to make decisions based on peripheral attributes of data items that have strong correlation with a target variable (i.e., dataset bias). These biased models suffer from the poor generalization capability when evaluated on unbiased datasets. Existing approaches for debiasing often identify and emphasize those samples with no such correlation (i.e., bias-conflicting) without defining the bias type in advance. However, such bias-conflicting samples are significantly scarce in biased datasets, limiting the debiasing capability of these approaches. This paper first presents an empirical analysis revealing that training with "diverse" bias-conflicting samples beyond a given training set is crucial for debiasing as well as the generalization capability. Based on this observation, we propose a novel feature-level data augmentation technique in order to synthesize diverse bias-conflicting samples.  To this end, our method learns the disentangled representation of (1) the intrinsic attributes (i.e., those inherently defining a certain class) and (2) bias attributes (i.e., peripheral attributes causing the bias), from a large number of bias-aligned samples, the bias attributes of which have strong correlation with the target variable.  Using the disentangled representation, we synthesize bias-conflicting samples that contain the diverse intrinsic attributes of bias-aligned samples by swapping their latent features. By utilizing these diversified bias-conflicting features during the training, our approach achieves superior classification accuracy and debiasing results against the existing baselines on both synthetic and real-world datasets.

Learning Debiased Representation via Disentangled Feature Augmentation
Jungsoo Lee" Eungyeup Kim 112 Juyoung Lee² Jihyeon Lee' Jaegul Choo
"AAIST AI, KKakao Enterprise, South Korea
(bebeto, eykim94, gima3833, jchoo))kaist. ac.kr, "(bebeto. lee, josh.ey, ey, michael. m hatee jythokkkaaenterprise..omm
Abstract
Image classification models tend to make decisions based peripheral attributes data items that have strong correlation with a target variable (i.e., dataset bias). These biased models suffer from the poor generalization capability when evalu- ated on unbiased datasets. Existing approaches for debiasing often identify and emphasize those samples with no such correlation (iee. bias- thas-conflicting) without defining the bias type in advance. However, such bias-conflicting samples are significantly scarce in biased datasets, limiting the debiasing capability of these approaches. This paper first presents an empirical analysis revealing that training with "diverse" bias-conflicting samples beyond given training debiasing as well as the generalization capability. Based on this observation, we propose novel feature-level data augmentation technique in order to synthesize diverse bias ctuusconfiitiin samples. To this end, our method learns the disentangled representation of (I) the intrinsic attributes (i.e., those inherently defining certain class) and (2) bias attributes (i.e., peripheral attributes causing the bias), from a large number of bias-aligned samples, the bias attributes of which have strong correlation with the target variable. Using the disentangled representation, we synthesize confficnni samples that contain the diverse intrinsic attributes bias-aligned samples by swapping their latent features. By utilizing these diversified bias-conflicting features during the training, our approach achieves superior classi- fication accuracy and debiasing results against the existing baselines on synthetic and real-world datasets.
crucial for
Introduction
Despite the recent advancement of deep neural networks, they often rely overly on the correlation between peripheral attributes and labels, referred to dataset bias [1, especially when such strong bias is found in given dataset. A majority of samples in the biased dataset exhibit visual attributes that are not innate but frequently co-occur with target labels (i.e., bias attributes). For example, most of the bird images in the training dataset may contain the background as the blue sky, while the birds may still be found in different places. Thus, the model trained with such biased dataset is likely to learn the bias attributes more than intrinsic attributes, the innate visual attributes that inherently define a certain class, e.g., the wings birds. This causes the model to learn shortcuts for classification (2), failing to generalize the test phase. Throughout the paper, bias-aligned samples correspond to data items containing a strong correlation between bias attributes and labels (e.g., birds in the sky), while bias-conflicting samples indicate the other cases that are rarely found (e.g., birds on grounds).
the images with such correlations (e.g., birds grounds grass) during
To tackle such a task, previous studies often define specific bias type (e.g., color and texture) in advance 3 4. 7 8 10], which enables them to design debiasing network tailored for the predefined bias type. For example, Bahng et al. 66 leverage BagNet [II. which has limited size
indicates equal contribution. The order of first authors was chosen by tossing coin.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

of receptive fields, to focus on learning color and texture. However, defining a bias type in advance 1) limits the capability of debiasing in other bias types and 2) requires expensive labor to manually identify the bias type. To handle such issue, a recent approach [12] defines a bias based on an intuitive observation that the bias attributes are often easier to learn than the intrinsic attributes for neural networks. In this regard, they re-weight bias-conflicting samples while de-emphasizing the bias-aligned ones. However, we point out that the reason behind the limited generalization capability of existing debiasing approaches lies in the significant scarcity of bias-conflicting samples compared to the bias-aligned ones in given training set. In other words, it is challenging to learn the debiased representation from these scarce bias-conflicting samples because the models prone to memorize (thus being overfitted to) these samples, failing to learn the intrinsic attributes. Therefore, we claim that neural network can learn properly debiased representation when these data items diversified during training.
We conduct brief experiment to demonstrate the importance of diversity in debiasing, Diversity in our work indicates the different valid realization of intrinsic attributes in a certain class (e.g.- thick, narrow, tilted, and scribbled digit shapes in MNIST [3). Our observation is that training a model with diverse bias-conficcinng samples beyond a given training set is crucial for learning debiased representation (Section B.2). In this regard, synthesizing bias-conflicting samples is one of the straightforward approaches to increase the diversity of such samples. In fact, a large amount of bias-alige blas-aligned samples in a given training set already contain diverse intrinsic attributes, which can work as informative sources for increasing the diversity. However, as bias and intrinsic attributes are highly entangled in their embedding space, it is difficult to extract the intrinsic ones from these bias-aligned samples. Therefore, disentangling these correlations enables to synthesize diversified bias-conflicting samples that originate from bias-aligned samples.
In this paper, we propose a novel feature augmentation approach via disentangled representation for debiasing. We first train two different encoders to embed images into the disentangled representation of their intrinsic and bias attributes. With the disentangled representation, we randomly swap the latent vectors extracted from different images, most of which are bias-aligned samples in our training set. These swapped features thus contain both bias and intrinsic attributes without the correlation between them, which, in turn, can work as augmented bias chasffonnii samples in training. These features include intrinsic fcatures of bias-aligned ones, increasing the diversity of a given training set, especially for bias-conflicting data items. Furthermore, enhance the quality of diversified features, we propose scheduling strategy of feature augmentation which enables to utilize the representation disentangled to certain degree. In summary, the main contributions of our work include:
Through our preliminary experiment, we reveal that increasing the diversity of bias- conflicting samples is crucial for debiasing.
Based on such an observation, we propose a novel feature augmentation method via disen- tangled representation for diversifying the bias-conflicting samples.
We achieve the state-of- oftt-h--th performances in two synthetic datasets (i.e., Colored MNIST and Corrupted CIFAR-10) and one real-world dataset (i.e., Biased FFHQ) against existing baselines.
2 Related Work
Debiasing predefined bias Several existing approaches mitigate the bias by pre-defining a certain bias type, either explicitly 345 5 or implicitly 7 8 9 I0, [4]. For example, Bahng el al. (6] and Wang et al. [7] design a color- and texture-oriented network to adversarially Iearn a debiased model against the biased onc. However, these methods still require specific bias type such as texture in advance, they lack the general applicability to the datasets where the bias types are demanding to recognize.
Instead of defining certain types of bias, recent approaches 12 15 16 rely on the straightforward
assumption that networks are prone to exploit the bias when it acts as shortcut [2] i.e., easy to learn
in the early training phase. Nam el al. [12 emphasize the bias-conflicting samples during training
by using generalized cross-entropy loss [17] Darlow et al. [5] and Huang el al. [166 presume that
high gradient of latent vectors accounts for the shortcuts that model learns. In the line with the recent
studies, we tackle debiasing without pre-defining a certain bias type.

---

Dataset
Diversity ratio
Sampling ratio
Accuracy (%)
5%
50%
83.77
Colored MNIST
1%
50%
07.19
5%
1%
77.97
1%
1%
49.91
5%
50%
46.99::082
Corrupted CIFAR-10
1%
50%
33.08
5%
1%
36.66+0055
1%
1%
23.98
Table I: The classification accuracy on the unbiased test sets. The diversity ratio indicates the ratio of bias-conflicting samples in the dataset pooled for each experiment. The sampling ratio refers to the ratio of bias-conflicting samples included in each mini-batch. We report the averaged accuracy over three independent trials with the standard deviation. both datasets, we observe that the bias can be mitigated with diverse bias-conflicting samples even with small sampling ratio. Bold and underlined values indicate the best and second best accuracy, respectively.
Data augmentation for debiasing Geirhos et al. [10 mitigate the texture bias by utilizing additional training images with their styles being transferred by adaptive instance normalization (AdaIN) (18). Minderer et al. [9] train an image-to-image translation network for removing shorteut cues in the self-supervised task. However, such image-level data augmentation is limited to resolving the predefined texture bias which can be adopted other general types of bias.
One alternative is to exploit the latent space for data augmentation. For example, Darlow et al. 15 adversarially perturb the latent vectors corresponding to the high gradients generate the samples against bias. Zhou et al. 20 mix the style of different source domains by AdalN [8] to increase the domain generalization ability. Despite the effectiveness of the augmentation in the latent space, the strong unwanted correlation between bias attributes and labels prevents from obtaining the desirable intrinsic features. We resolve this issue by leveraging the disentangled representation in debiasing, which is widely used in image-to-image translation task [21 22 233. To the best of our knowledge, no previous work in debiasing leverage this disentangled representation for the purpose of feature augmentation. For the rest of the paper, we elaborate how we perform the feature augmentation based on the disentangled representation.
3 Importance of Diversity in Debiasing
This section deseribes the details of toy-set experiment in which we observe the importance of diversity in learning debiased representation. In Section 3.1 we first introduce the two synthetic datasets, Colored MNIST and Corrupted CIFAR-10, that we utilize for the observation. Then, elaborate the results of the experiments in Section 3.2
3.1 Dataset
Colored MNIST is modified MNIST dataset 13 with the color bias. We select distinct colors and inject each color on the foreground each digit to create color bias. By adjusting the number of bias-conflicting data samples in the training set, we obtain four different datasets with the ratio of bias-conflicting samples of 0.5%, 1 1%, 2%, and 5%.
Corrupted CIFAR-10 has ten different types of texture bias applied in CIFAR-10 (24 dataset, constructed by following the design protocol of Hendrycks and Dietterich (25). Each class is highly correlated with certain texture (e.g., frost and brightness). Corrupted CIFAR-10 also has four different datasets with their correlation ratios as in Colored MNIST.
3.2 Increasing diversity outperforms oversampling
To confirm the significance of diversity of bias-conflicting samples in debiasing, we train four different
settings: oversampling bias-conflicting samples by 50% each mini-batch (i.e., 128 from a batch size of 256), from the pool of 5% dataset and ii) 1 1% dataset, sampling bias-conflicting samples by 1% each mini-batch (i.e., from batch size of 256) from the pool of iii) 5% dataset and iv) 1% dataset. Oversampling provides the same amount of bias-conflicting samples the aligned ones to

---

the model in every training step. Bias- Bias-conflicting images sampled from the pool of 5% dataset have more diverse appearances of bias-conflicting samples compared those from 1 1% dataset.
Table D shows the image classification accuracy of each setting validated on the unbiased test images. Apparently, oversampling diverse bias-conflicting samples (first row) outperforms the other three methods. Similarly, sampling a small amount of bias-conflicting samples with the least diversity (fourth row) shows the lowest classification accuracy. The interesting finding that sampling fewer but diverse conflicting samples in each mini-batch (third row) outperforms oversampling bias-conflicting samples with limited diversity (second row). These results lead the conclusion that the diversity of bias conffiiifi samples is more crucial factor for learning debiased representation than the ratio of sampling in the training. As the diversity limited (the latter case), the model can be easily overfitted to the given bins-conf samples, thus less likely to learn the generalized intrinsic attributes. With the Colored MNIST as an example, the shape of digits may vary. To be more specific, the digit shape may be thick, narrow, tilted, scribbled, and etc. If the biascconflicting samples do not include certain visual facets (e.g., not including scribbled digit images) due to the limited number of samples, the model may imperfectly learn the intrinsic attributes of digit shapes. On the other hand, in the former case (third row), the model can learn multiple facets of intrinsic attributes when they are sampled from the diverse pool of datasets, resulting in learning intrinsic attributes even without oversampling the bias-cooflicting images.
Debiasing via disentangled feature augmentation
Motivated by such an observation in Section 3.2] we propose feature-level augmentation strategy for synthesizing additional bias-conflicting samples, as illustrated in Fig. [1] First, we train the two separate encoders which embed image into disentangled latent vectors corresponding the intrinsic and bias attributes, respectively (Section 4.1). Swapping these feature vectors among training samples enables augment the bias-conflicting samples which no more contain correlation between two attributes (Section (.2). To further enhance the effectiveness, we schedule the feature augmentation after the representation is disentangled at a certain degree (Section 4.3)
4.1 Learning disentangled representation
In contrast to the bias-conflicting samples, a large amount of bias-aligned images have diverse appearances of their intrinsic attributes. By leveraging these attributes for augmentation, we can naturally obtain the diversified bias-conflicting samples containing the diverse intrinsic attributes. However, it remains challenging in that these attributes are strongly correlated with the bias attributes in the bias-aligned samples. Therefore, we propose to design two encoders with their linear classifiers to extract the disentangled latent vectors from the input images.
As shown in Fig. D encoders E; and F& embed an image x into intrinsic feature vectors 2; = F3 (a) and bias feature vectors 2n Ex(x), respectively. Afterward, linear classifiers C, and C. take the concatenated vector 2 = [2;; 26] as input to predict the target label y. To train E; and C; as intrinsic feature extractor and E, and C2 bias extractor,
utilize the relative difficulty score of each data
sample, proposed in the previous work of Nam el al. [12.. More specifically, we train Es and C, to be overfitted to the bias attributes by utilizing the generalized cross entropy (GCE) [17] while E, and C; are trained with the cross entropy (CE) loss. Then, the samples with high CE loss from C. can be regarded as the bias-confiicting samples compared to the samples with low CE loss. In this regard, we obtain the relative difficulty score of each data sample as
CE(Co(2), 8) ((2) = CEE(C.(E) + CE(C)(2),
(I)
As bias-conflicting samples obtain high values of W, we emphasize the loss of these samples for training E; and C₁ enforcing them learn the intrinsic attributes. Therefore, the objective function for disentanglement can be written as
Laks = W(E)CF(C(().)) + GIF
(2)
To ensure that C₁ and C, predicts target labels mainly based on and 24. respectively, the loss from C is not backpropagated to Eg,, and vice versa.

---

Minibatch
E
C La
R
Zo
E
Co Lox
Permuit Original
Concat Augmment
weiggiingg
The overview of our proposed debiasing approach. (E, C2) and (Es, Co) are pairs of an encoder and linear classifier trained to learn the disentangled representation of intrinsic attributes and bias attributes, respectively. With the disentangled features 2, and the feature augmentation is performed by swapping these latent vectors among different training samples, after certain iterations of training. R refers to the re-weighting algorithm which implicitly differentiates bias-aligned samples and bias-conflicting samples. Each color indicates the different data samples.
Figure 1:
Algorithm Debiasing with disentangled feature augmentation
Input: image z, label iteration t, augment iteration Eswap Initialize two networks (E,, C.), (E,, Cb)
while not converged do
Extract 2i, 2b from Fx(x)), (1) Concatenate 2 = 22; 20
Update (i), (Es, Cb) W (()))E(C(())) (CC
if tswap:
Randomly permute z = [aa, =b] into Sswap [=s;
Calculate Lswap = (()))) (zswap); GCE((C (=swap) 17)
Update (E,, C.), (Eg, Cp) with Louul = Lais + Aswap Issaap
end
4.2 Feature swapping for augmentation
While such architecture disentangles the intrinsic features and bias features, Eg and C₁ still mainly trained with an excessively small amount of bias-conflicting samples. Therefore, E, and C fail to fully acquire the intrinsic representation of a target class. To promote further improvement in learning intrinsic feature vectors, we diversify the bias- chas-coofficting samples by swapping the disentangled latent vectors among the training sets. In other words, we randomly permute the intrinsic features and bias features in each mini-batch and obtain Sswap = [2;; 6o] where 2 denotes the randomly permuted bias attributes of 24. As the intrinsic and bias attributes in Sswap are obtained from two different images, they certainly have less correlation compared to 2 = [2 23] where both are from the same image. Since the biased dataset is mostly composed bias-aligned samples, these vectors are likely from the bias-aligned samples, highly diversified compared to the bias-conflicting ones. Then, Zewap [24; -o] act as augmented bias-conflicting latent vectors with diversity inherited from the bias-aligned samples. Along with d we add the following loss function to train two neural networks with the augmented features
Lswap ()))CE((( + Aswap, GC y)
(3)
where i denotes target labels for permute bias attributes ž. Thus, total loss function is described as
Liotal Lais + Aswap Luwap
(4)
where Aswap is adjusted for weighting the importance of the feature augmentation.
4.3 Scheduling the feature augmentation
While training with additional synthesized features helps to mitigate the unwanted correlation, utilizing them from the beginning of training does not improve the debiasing performance. To be more
5

---

2
3
4
5
7
8
2
3
S
6
7
8
9
2
3
4
5
6
6
9
(a) Colored MNIST
(b) Corrupted CIFAR-10
(c) BFFHQ
Figure 2: Example images of datasets utilized in our work. In each dataset, the images above the dotted
line indicate the bias-aligned samples while the ones below the dotted line the bias-conflicting samples. For Colored MNIST and Corrupted CIFAR-10. each column indicates each class. For BFFHQ, the group of three columns indicates each class.
specific, in the early stage of training, the representations of 2; and 2n are imperfectly disentangled to be used as the sources of feature augmentation. Feature augmentation should be conducted after two features are disentangled at certain degree. Without the disentangled representation, the augmented features work as noisy samples which may aggravate the debiasing performances. We verify the importance of scheduling the feature augmentation in Table 3 Our approach can be summarized with Algorithm
Experiment
This section demonstrates the effectiveness of feature augmentation based disentangled represen- tation in debiasing with both quantitative and qualitative evaluation. We compare our method with the previous approaches in debiasing with three different datasets with varied bias ratios. Then, conduct the ablation study which demonstrates the importance 1) learning disentangled representa- tion, 2) feature augmentation, and 3) scheduling feature augmentation. For the qualitative evaluation, we verify how our approach disentangles the intrinsic features and bias features by visualizing them on 2D embedding space via I-SNE [26) and reconstructing images from them.
5.1 Experiment details
Baselines Our baselines consist of vanilla network, HEX 77, EnD 27), ReBias 6 and LfF [2] Vanilla denotes the classification model trained only with the original cross-entropy (CE) loss, without any debiasing strategies. EnD explicitly leverages the bias labels (c.g., the color label in Colored MNIST) during the training phase. HEX and ReBias explicitly presume the texture of image bias type, while LfF requires no prior knowledge on it.
Datasets As shown in Fig. 2 we use two synthetic datasets (Colored MNIST and Corrputed CIFAR- 10) and one real-world dataset (Biased FFHQ) evaluate the generalization of debiasing baselines over various domains. Biased FFHQ (BFFHQ) is curated from FFHQ dataset (28) which contains human face images annotated with their facial attributes. Among the facial attributes, we select age and gender as the intrinsic and bias attribute, respectively, and construct the dataset with images of high correlation between them. More specifically, most the females are 'young' (i.e., age ranging from 10 to 29) and males are 'old" (i.e., age ranging from 40 to 59). Therefore, bias-aligned samples which compose the majority of the dataset are young women and old men.
For each dataset, we set the degree of correlation by adjusting the number of bias bias-confficting samples among the training dataset. The ratio of bias- samples are 0.5%, 1%, 2% and 5% for both Colored MNIST and Corrupted CIFAR- 10, respectively, and 0.5% for BFFHQ. For the evaluation of Colored MNIST and Corrupted CIFAR-10, we construct an unbiased test set which includes images without the high correlation existing in the training set. For the BFFHQ, we construct bias-conflicting test set which excludes the bias-aligned samples from the unbiased test set. The reason is following. The bias-alignce images consist half of the unbiased test set in BFFHQ which

---

Vanilla 29
HEX 7
EnD 27
ReBias 6
LfF 12
Ours
Dataset
Ratio (%)
X
X
X
0.5
35.19 +5.49
30.33++076
34.28=1.20
70.47
52.50=243
65. 25.424
1.0
52.09
43.73+5550
49.50=2.1
87.4=078
61.89=497
81.73++2.34
Colored MNIST
2.0
65.8 39.86.13.9
56.85+2288
68.4522.16
12.91
71 71.03=2.44
29.79
5.0
82.17
74.62+3.2
81.15=143
96.96=004
80.57=3384
89.66+1.00 1.09
0.5
23.08:1125
13.87:2006
22.89
22.27:2041
28.57=1.30
29.95
Corrupted CIFAR-10
1.0
25.82+0033
14.81+0042
25.46=0044
25.72=020
33.07 =0.7
16.49911.79
2.0
30.06+071
15,20+054
=0.3
31.66+0.43
39.91 =0.30
41.78+2.29
5.0
39.422064
16.042063
40.26
43.43=041
50.27 1:56
11.13
BFFHQ
0.5
56.87=269
52.83+a000
56.87=1.42
59.462064
62.2+10
63.87
Table 2: Image classification accuracy evaluated on unbiased test sets of Colored MNIST and Corrupted CIFAR-10, and the bias-conflicting test set BFFHQ with varying ratio of cins-conn samples. We denote whether the model requires bias type in advance by cross mark (i.e., not required), and check mark (i.e., required). Best performing results are marked bold, while second- best results are denoted with underlines.
may still be correctly classified by the biased classifier. This inflates the accuracy of the unbiased test set which is not our original intention. Therefore, we intentionally use the bias-conflicting test set for BFFHQ.
Implementation details We use multi-layer perceptron (MLP) with three hidden layers for Colored MNIST, and ResNet-18 [29] for the remaining datasets. To accommodate the disentangled vectors, we double the number of hidden units in the last fully-connected layer of each network. During the inference phase, we use (z) for the final prediction, where z = [zi; 2o]. For the training, we set the batch size of 256 for Colored MNIST and Corrupted CIFAR- 10, respectively, and 64 for BFFHQ. Binf-conn augmentation is scheduled be applied after 10K iterations for all datasets. We report the averaged accuracy of the unbiased test sets over three independent trials with the mean and the standard deviation. We include the remaining implementation details Section in the supplementary material.
5.2 Quantitative evaluation
Comparison on test sets T the comparisons of image classification accuracy evaluated on the test sets. In general, our approach demonstrates the superior performance in both synthetic and real-world datasets against the baselines with large gaps. Especially, compared to the baselines which do not define the bias types in advance (vanilla [29) and LfF [12), our approach achieves the state- of-the-art performance across datasets. This indicates that utilizing the diversified bias-conflicting samples through our augmentation plays pivotal role in learning debiased representation regardless
of the bias types.
Regarding the real-world dataset, our approach also outperforms HEX 77] and ReBias 6 which utilize the tailored modules for a specific bias type (e.g., color and texture), and EnD that uses the explicit bias labels. We even show superior performance compared to HEX in Colored MNIST without defining the bias type beforehand. While ReBias achieves the best accuracy in Colored MNIST, they utilize BagNet [1]] in order to focus the color bias. Even without using such an architecture, we achieve the second best performance which comparable ReBias.
Ablation studies Table 3 demonstrates the importance of each module in our approach through ablation studies: disentangled representation learning, 2) feature augmentation, and 3) scheduling feature augmentation. We set the ratio of bias-conflicting samples to 1 5% for Colored MNIST and Corrupted CIFARIO, and 0.5% for BFFHQ We also compare each module with the vanilla network (first row). We observe that performing the scheduled feature augmentation shows the best classifica- tion accuracy on the test sets across all datasets. We also show that performing feature augmentation at the early stage of training does not guarantee the effectiveness of debiasing. Performing feature augmentation at the beginning of training rather aggravates the performance. That is, when the repre- sentation of intrinsic attributes and bias attributes are not disentangled at certain degree, augmented features may act as noisy samples. Training with these additional noisy features prevents models from achieving further improvement.
7

---

Scheduled Augment
Colored
Corrupted
Disentangle
Augment
BFFHQ
MNIST
CIFARIO
52.0912.8
25.82
56.8712.69
74.03+2.40
27.73++102
59,4=2.46
72.29:1382
32.47
61.27
81.73++2.44
52.31+1.00
63.87=031
Table Ablation studies on 1) disentangled representation learning, 2) feature augmentation, and 3) scheduling feature augmentation. Each row indicates the different training settings with check mark denoting the setting applied. We average the accuracy each training over three independent trials.
5.3 Analysis
2D Projection of Disentangled Representation Fig. shows the projection of latent vectors 2; and 2y extracted from the intrinsic encoder E; and bias encoder Eg, respectively, on 2D space using Colored MNIST. We show projection of 2; and in Fig. 3(a and Fig. (b). respectively. The colors of projected dots in the first row (i) and the second row (it) indicate the target labels and bias labels, respectively. We observe that 2; are clustered according to the target labels while 26 are clustered with the bias labels. The results represent that our method successfully learns the disentangled intrinsic and bias attributes.
Prediction with Disentangled Representation In Table report the 1) original and 2) swapping accuracy of C; and C₁ the linear classifiers of the intrinsic and the bias encoder, respectively. To be specific, for the original accuracy, we extract the two disentangled vectors, 2; and zb. from the same image, concatenate them to make = [2s 2o], and forward them into each linear classifier. For the swapping accuracy. however, we first permute 26 and concatenate 2, with the permuted 26 (i.e., denoted as 26 in Section 4.2) to make 2swap [2y; 26.. Then, we pass these concatenated latent vectors each linear classifier Afterward, we evaluate the accuracy of predicted labels of 1) (2) and Cz(Eswap with intrinsic labels and 2) Cz(z) and Cb(Eswap) with bias labels. The Intr ittrinsic and Bias columns in Table 4 denote the accuracy with respect to the target labels and bias labels, respectively. Even the feature vectors of bias attributes are randomly swapped, our method maintains a reasonable classification accuracy. This indicates that our model well disentangles between ž; and 2b. and C; robustly utilizes to predict target labels even when 24 is taken from the different image, and vice versa. Note that we utilized the parameters the model trained on each dataset after converging at certain degree.
(a)
(b)
(i)
D78
180
three
191r
fime
rihht
(ii)
grecn
pupfe
Figure 3: Each row (1 and 11) include 2D projection of 2; and ET with the colors encoded by their labels (i.e., groundtruut labels in row and bias labels row ii) in Colored MNIST. We observe that 2; and 2n are well clustered according to the target and bias labels, respectively.

---

Colored MNIST
Conrupted CIFARIO
BFFHQ
Accuracy(%)
Intrinsic
Bias
Intrinsic
Bias
Intrinsic
Bias
Original
76.08
98.07
35.63
74.16
57.40 58.40
49.00
Swapping
71.40
94.29
35.14
76.46
51.60
Table 4: Accuracy from disentangled representation. The ratio of bias bias-cooflicting samples in Colored MNIST, Corrupted CIFAR- 10, and BFFHQ are 1 %%, 1%, and 0.5%, respectively.
1
2
3
4
8
9
Q
O
/
0
1
1
y
7
7
7
7
7
7
Figure 4: Reconstructed images from disentangled representation in Colored MNIST. Fach column and row indicate the samples where the bias attribute (color) and the intrinsic attribute (digit) are extracted, respectively, By swapping the bias features with given intrinsic feature, we observe that the color changes while maintaining the digit.
Reconstruction of Disentangled Representation Fig. 4 shows the reconstructed images of Colored MNIST by using the disentangled representation of intrinsic features and bias features. Images in the first row and column indicate the images used for extracting the bias attribute (i.e., color) and intrinsic attribute (i.e., digit), respectively. We train an auxiliary decoder by providing the latent vector from our pre-trained models as input in order to visualize the disentangled representations the pixel level. By changing the bias attributes (as the column changes), the color of digit changes while maintaining the digit shape. This demonstrates that the bias features and intrinsic features independently contain color and digit information, respectively. Note that the reconstruction loss for updating the decoder is not backpropagated to our pre-trained classification models. Due to this fact, the reconstructed images may lack qualities such showing blurry images. Further implementation details are included in Section D in the supplementary material.
Conclusions
In this work, we propose feature augmentation method based on the disentangled representation of intrinsic and bias attributes. The main intuition behind our work that increasing the diversity of bias-confie binffcoofii samples beyond a given training set is crucial for debiasing. Since the biased dataset strongly correlates the bias attributes and labels, we intentionally train two different encoders and extract bias features and intrinsic features. After the representations degree, we proliferate the bias bias-conflicting samples by randomly swapping the vectors. We demonstrate the effectiveness of feature augmentation via extensive experiments, ablation studies, and qualitative evaluation the disentangled representation. We believe our work inspires the future work of learning debiased representation with the improved generalization capability.
disentangled to certain
Ackmowledgements This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded the Korean gOSI) (No. Intelligence Graduate School Program(KAIST). No. 2021-0-01778, Development of human image synthesis and discrimination technology helow the perceptual threshold), the Force Research Laboratory, under agreement number FA9550- 18550018-S--00 8--S00033 and Kakao Enterprise. This material is based on research sponsored by The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.
Artificial

---

References
[1] A. Torralba and A. A. Efros. Unbiased look dataset bias. CVPR ',, 2011.
[2] Robert Geirhos, Jörn-Heriik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665 673, 2020.
[3] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[4] Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling, In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 9572-9581, 2019.
[5] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv: 1.007333), 2019.
[6] Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In International Conference Machine Learning (ICML). 2020.
[7] Haohan Wang, Zexue He, Zachary L. Lipton, and Erie P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019.
[8] Remi Cadene, Corentin Dancette, Hedi Ben younes, Matthieu Cord, and Devi Parikh. Rubi: Reducing unimodal biases for visual question answering. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[9] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't 't take the easy way out: Ensemble based methods for avoiding known dataset biases. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),, pages 4069-4082, Hong Kong, China. November 2019. Association for Computational Linguistics.
[10] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenct-trained CNNs are hiased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
[11] Wieland Brendel and Matthias Bethge. Approximating cans with bag-of- -ocallllll-------- fff-lolllffttrree features models works surprisingly well imagenet. International Conference Learning Representations, 2019.
[12] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jacho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
[13] Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010.
[14] Aishwarya Agrawal, Dhruv Batra, and Devi Parikh. Analyzing the behavior visual question answering models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1955-1960, Austin, Texas, November 2016. Association for Computational Linguistics.
[15] Luke Darlow, Stanislaw Jastrzebski, and Amos Storkey. Latent adversarial debiasing: Mitigating collider bias in deep neural networks. arXiv preprint arXiv:2011. 2020.
[16] Zeyi Huang, Haohan Wang, Erie P. Xing, and Dong Huang. Self-challenging improves cross-domain generalization. In ECCV, 2020.
[17] Zhilu Zhang and Mert R Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. arXiv preprint 1xXxv18805007836 2018
[18] Xun Huang and Serge Belongie. Arbitrary style transfer in ccal-time with adaptive instance normalization. In ICCV, 2017.
[19] Matthias Minderer. Olivier Bachem, N. Houlsby, and M. Tschannen. Automatic shortcut removal for self-supervised representation learning. In ICML, 2020.
[20] Kaiyang Zhou, Yongxin Yang. Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. In ICLR, 2021.
10

---

[21] Hsin- Ying Lee. Hung- Yu Tseng, Jia-Bin Huang. Maneesh Kumar Singh, and Ming-Hsuun Yang, Diverse image-lo-image translation via disentangled representations. In European Conference Computer Vision, 2018.
[22] Xun Huang, Ming- Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-imas translation. In ECCV, 2018.
[23] Taesung Park, Jan-Yan Zhu, Oliver Wang, Jingwan Lu, Eli Shechtman, Alexei A. Efros, and Richard Zhang. Swapping autoencoder for deep image manipulation. Advances in Neural Information Processing Systems, 2020.
[24] A. Krizhevsky and Hinton. Learning multiple layers features from tiny images. Maxter's 's thesis, Department of Computer Science, University of Toronto, 2009.
[25] Hendrycks Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[26] Laurens van Maaten and Geoffrey Hinton. Visualizing data using t-SNE, Journal of Machine Learning Rexearch, 9:2579-2605, 2008
[27] Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR). pages 13508- 10588-111,,,,,1 June 2021.
[28] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv preprint arXiv: 1333v.15 2015.


Reviews:
[{"review_id": "s8R9s8AaVeR", "reviewer": "Reviewer_iKKw", "summary": "Dataset bias significantly degrades the generalization performance of deep neural networks. To alleviate this problem, the authors propose a novel approach for developing the debiased representation, where the model learns the disentangled representation and synthesizes diverse bias-conflicting samples via feature-level data augmentation. They demonstrate its effectiveness through extensive experiments on the image classification task.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Strengths]\n\nThe idea is simple and interesting. They propose a straightforward and effective debiasing approach via disentangled feature augmentation. Also, the proposed method handles debiasing without presuming a certain bias type, which is advantageous for a practical setting.\nThey synthesize diverse bias-conflicting samples with feature-level data augmentation. And via a toy-set experiment, they show that the diversity of bias-conflicting samples is an important factor in debiasing.\nThey demonstrate the effectiveness of the proposed method through extensive experiments on the image classification task with synthetic and real-world datasets. (i.e., Colored MNIST, Corrupted CIFAR-10, BAR and bFFH)\n \n\n[Weakness]\n\nIt is questionable whether the experiment on LfF [1] was performed properly. There is a significant gap between the reproduced performance of LfF in Table 2 and the reported performance in LfF. On corrupted CIFAR-10, the reported performance in LfF outperforms the proposed method. For validating a fair comparison, the authors should clarify this.\n\nEnD [2] also handles debiasing based on the disentangled representation. Thus, at least the discussion with EnD is necessary; (conceptually and if possible empirically) comparing  the proposed method with it.\n\nTo show the effectiveness of debiasing, it should evaluate the performance on bias-conflicting samples as LfF [1] and EnD [2]. However, evaluations and comparisons are conducted only on unbiased samples in experimental results. By observing and evaluating on bias-conflicting samples, it would be clear where the effectiveness of the proposed method comes from.\n\nUnlike LfF [1], the objective function of Equation (2) does not include the relative difficulty score W. Hence, it is not clear how E_i and C_i can learn intrinsic attributes. It should be clearly explained whether missing W in Eq (2) has any meaning (or reason)-- if so, how E_i and C_i can learn intrinsic attributes.\n \n[Minor]\n\nFor readability, it would be better to add explanations for (i) and (ii) in Figure 3.\nIt would be great to show the effect of the proposed method by visualizing before and after applying the proposed method using CAM (or Grad-CAM) in the real-world dataset, BAR.\nFor example, in a bias-aligned sample such as ‘birds on the sky’, the proposed method may lead the model focus on the birds, not the sky.\n\n[1] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. “Learning from failure: Training debiased classifier from biased classifier.” In Advances in Neural Information Processing Systems, 2020.\n\n[2] Tartaglione, Enzo, Carlo Alberto Barbano, and Marco Grangetto. \"EnD: Entangling and Disentangling deep representations for bias correction.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes. ", "review_text": "", "extra_scores": {"ethical_concerns": "no", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "JgdWd4364kl", "reviewer": "Reviewer_8Ai1", "summary": "The paper proposes a novel feature-level data augmentation technique to train image classifiers in a debiased manner. The approach consists of disentangling “intrinsic” and “bias” attributes from training set images and then synthesizing new “data points” by combining intrinsic and bias features from different images. Since, a lot of image datasets contain a large amount of bias-aligned data points (high correlation between intrinsic and bias attributes), the proposed feature-swapping approach’s aim is to generate bias-conflicting data points. The approach results in better classification performance on 2 synthetic and 2 real-world datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper tackles an important problem — spurious correlations in image datasets that result in poor generalization in real-world applications. \n- The proposed approach is simple yet effective.\n- The paper presents promising quantitative results for 4 different datasets, supporting the robustness of their approach.\n\n\nWeaknesses:\n- I think the paper needs much more explanation in lines 159-160 about what GCE is and how it enables learning bias attributes as compared to intrinsic attributes? Similarly, I think the paper should additionally discuss how cross-entropy loss enables learning intrinsic attributes? If CE is enough to learn intrinsic attributes, any typical image classification systems trained with CE should be able to learn disentangled representations and there wouldn’t be any need for debasing?\n- Figure 3: Could the authors please include the visualization of z_b for BAR? Since the target labels are available, its coloring with target labels would still make sense. Also, I disagree that this figure (in its current form) shows that the proposed approach successfully disentangles the intrinsic and bias attributes for BAR dataset (referring to line 269).\n- Figure 4: are these examples random or cherry-picked? Could the authors please include a randomly chosen 10x10 matrix?\n- How are the ratio of bias-conflicting samples determined for each dataset?\n- Lines 228: Both encoders and classifiers are MLPs?\n- Lines 50-51: The thickness or the degree of tilt are mentioned as intrinsic attributes. How is that? Shouldn’t they be bias attributes? They should not be contributing to the “digit” classes. This raises a bigger question — how should intrinsic and bias attributes be determined for each dataset? By humans? I understand that the proposed approach doesn’t require it but its qualitative evaluation does require it.\n- Section 3 shows that 1) larger diversity ratio is better and that 2) larger sampling ratio is better. Why is there a need to compare these two very different things with each other (lines 129-131)?\n\nOverall, although the experiment section shows promising quantitative results, I believe that the paper lacks enough explanation of how their approach works. Therefore, I am a bit hesitant in recommending the paper for acceptance.\n\n\n-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\nI thank the authors for their response. It answers all my concerns/questions. Therefore, I am increasing my rating to accept.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors discuss their work’s limitations and societal impact. ", "review_text": "", "extra_scores": {"ethical_concerns": "I do not think that there are any ethical concerns related to this work.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "4hDJsazkzJ", "reviewer": "Reviewer_3RPm", "summary": "This paper proposed a feature-level augmentation method to disentangle the bias and intrinsic features for debiasing image classification. This paper pointed out that the diversity of augmented bias-conflicting (i.e., few-shot, tailed) data is crucial for debiasing. The proposed framework consists of two encoders that encode the image into disentangled intrinsic and bias features. (1) To achieve the disentanglement, the modules for intrinsic and bias features are trained using cross-entropy (CE) loss and generalized cross-entropy (GCE) loss, respectively. (2) To generate augmented data, the intrinsic and bias features are swapped in each mini-batch. Experiments conducted on two synthetic datasets Colored MNIST and Corrupted CIFAR-10) and two real-world datasets (BAR and Biased FFHQ) demonstrate that the proposed method achieves the best performance compared to those that do not require a bias type. The quantitative evaluation further illustrates the disentanglement.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "(+ Strengths, - Weaknesses, * Typos/Comments)\n\nOverall, I like and value the research topic and motivation of this paper and lean positive. However, some details are not clear enough. I would update my rating depending on the authors' feedback. The details are as follows.\n\n+ \\+ Interesting and important research problem. This paper focuses on how to obtain disentangle representations for feature-level augmentation. This topic is interesting and important, and will attract many interests of the NeurIPS community.\n\n+ \\+ Good quality of writing and organization. Overall, the writing quality is good and the paper is well organized. It is comfortable to read this paper, although some details are not clear.\n\n+ \\+ Comprehensive experiments. Experiments are conducted on two synthetic datasets Colored MNIST and Corrupted CIFAR-10) and two real-world datasets (BAR and Biased FFHQ). \n\n- \\- Relative difficulty score and generalized cross-entropy (GCE) loss. It is not clear how the relative difficulty score $W(x)$ in Eq. (1) is used in the pipeline. W(x) is not mentioned again in both the overall objective functions Eq. (2) or Algorithm 1. Since readers may not be familiar with the generalized cross-entropy (GCE) loss, it is encouraged to briefly introduce the formulation and key points of the GCE loss to make this paper more self-contained.\n\n- \\- How bias-conflicting samples and bias-aligned samples are selected. This weakness follows the first one. It seems that the \"bias-conflicting\" is determined based on the relative difficulty score, but the details are missed. Also, the ablation study on how the \"bias-conflicting\" is determined, e.g., setting the threshold for the relative difficulty score, is encouraged to be considered and included.\n\n- \\- Disentanglement. It is not clear how disentanglement is guaranteed. Although \"Broader Impacts and Limitations\" stated that \"Obtaining fully disentangled latent vectors ... a limitation\", it is still important to highlight how the disentanglement is realized and guaranteed without certain bias types.\n\n- \\- Inference stage. It is not clear how the inference is conducted during testing. Which encoders/decoders are preserved during the test stage?\n\n- \\- Figure 1 is not clear. First, it seems that the two $y$ towards $L_{\\text{CE}}$ are the outputs of $C_i$, but they are illustrated like labels rather than predictions. Second, the illustration of the re-weighting module is not clear. Does it represent Eq. (4)?\n\n- \\- Table 4 reported a much lower performance of \"swapping\" on BAR compared to the other three datasets. Is there any explanation for this, like the difference of datasets?\n\n- \\- Sensitivity to hyperparameters. The proposed framework consists of three important hyperparameters, $(\\lambda_{\\text{dis}},$ $\\lambda_{swap_b},$ $\\lambda_{\\text{swap}})$. It is not clear whether the framework is sensitive to these hyperparameters and how these hyperparameters are determined.\n\n* \\* (Suggestion) Illustration of backpropagation. As introduced in  Line 167-168, the loss from $C_i$ is not backpropagated to $E_b$. It would be clearer if this can be added in Figure 1.\n\n* \\* Line 280. Is \"the first row and column ... respectively\" a typo? It is a little confusing for me to understand this.\n\n* \\* Typos in Algorithm 1. Are $\\lambda_\\text{dis}$ and $\\lambda_{swap_b}$ missed in $L_{\\text{dis}}$ and $L_{\\text{swap}}$?\n\n* \\* Typo in Line 209. Corrputed -> Corrupted.\n\n\n\n\n\n============================= After rebuttal ===================================\n\nAfter reading the authors' response to my questions and concerns, I would like to vote for acceptance.\n\nThe major strengths of this paper are:\n\n* The research problem, unbiased classification via learning debiased representation, is interesting and would attract the NeurIPS audience's attention.\n* The proposed method is simple but effective. The method is built on top of LfF [12] and further considers (1) intrinsic and bias feature disentanglement and (2) data augmentation by swapping the bias features among training samples.\n* The paper is clearly written and well organized.\n\nThese strengths and contributions are also pointed out by other colleague reviewers.\n\nMy main concerns were:\n\n* Unclear technical details of the GCE loss and the relative difficulty score. This concern was also shared with Reviewer 8Ai1 and iKKw. The authors' response clearly introduced the details and addressed my concern well.\n\n* Sensitivity to hyper-parameters. The authors' response provided adequate results to show the sensitivity to hyper-parameters.\nOther details of implementation and analysis of experimental results. The authors' responses clearly answered my questions.\n\nConsidering both strengths and the weakness, I am happy to accept this paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "4LkftcbcvRI", "reviewer": "Reviewer_ub8A", "summary": "The paper presents a way to generate bias-conflicting samples at feature level, which help de-bias the model. For this, the authors first propose learning two separate (disentangled) representations: a) representation of implicit attributes (i.e., necessary to predict the target) learned through the normal cross-entropy loss and b) representation of biased attributes learned through generalized cross entropy loss (also used by LfF for bias amplification). The empirical results demonstrate that generating such samples outperforms oversampling/reweighting.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed idea is a very neat adaptation of the code swapping technique [1] applied for bias mitigation. This is a good contribution to the field, as it provides a simple way to generate bias-conflicting samples at the representation level without relying on generative models or annotations for the bias attributes/labels.\n\nOverall the paper is very well written and properly structured. I like the upfront analysis on ColoredMNIST and CIFAR10, motivating the need to for diversity. Ablations and comparisons against other methods e.g., LfF clearly demonstrate that training with the bias-conflicting samples generated in the proposed way improves over simply up-weighting the bias-conflicting/down-weighting the bias-aligned samples.\n\nThe 2D visualizations and reconstructed samples do a good job of showcasing the disentanglement (though not always perfect) between the intrinsic/bias attributes.\n\nNext, I have a few comments/suggestions:\n\n[C1] The main weakness is that the benchmarks used in the work seem to contain only one type of spurious correlation. I wonder what happens if there are multiple types of biases. Would the same GCE-based biased model capture all of those biases? Would one need to create multiple bias models and tune the GCE loss individually to capture those biases?\n\n[C2] Following up on [C1], it would have been a huge plus to have stronger benchmarks with different kinds/levels of biases e.g., GQA-OOD [2].\n\n[C3] Fig. 3 was a bit confusing and could be labeled/captioned better. I think both rows (target type) and columns (representation type) can be labeled to make things clearer. The BAR plot was a bit confusing -- I suppose the bottom right for column ‘c’ is all green due to the unavailability of bias labels?\n\nOther Minor comments:\n\n[MC1] I would prefer the phrase: ‘intrinsic attributes’ to be replaced by something more obvious e.g., ‘signal’ or ‘task-relevant attributes’ or perhaps ‘non-spurious attributes’. However, this is not a major concern, since it is well defined in the abstract/introduction.\n\n[MC2] Line 175/Algorithm 1: The phrase ‘Randomly shuffled bias attributes’ was slightly confusing, as it could also mean shuffling the dimensions within the representation itself as opposed to swapping the entire ‘code’ between samples. Perhaps the text could say something like: \"replace $z_b$ in $z$ with  $\\tilde{z_b}$ from a randomly chosen sample in the batch to create the swapped representation: $z_{swap}$ ...\"\n\n\nHaving said this, all in all, this is a good paper, with potential for high practical impact.\n\n[1] Park, Taesung, et al. \"Swapping autoencoder for deep image manipulation.\" arXiv preprint arXiv:2007.00653 (2020).\n\n[2] Kervadec, Corentin, et al. \"Roses Are Red, Violets Are Blue... but Should Vqa Expect Them To?.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper acknowledges the imperfect disentanglement, but could also mention that the work does not study more realistic cases where biases can stem from different sources and the correlations between bias attributes and target labels may vary, which may pose greater difficulty for disentanglement.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers are in agreement that the paper tackles an important problem in a simple yet effective, and enthusiastically recommend acceptance. The authors provided extensive experiments and explanations in their rebuttal which should be incorporated into the final version. 
Decision: Accept (Oral)