Self-supervised debiasing using low rank regularization
Authors: Geon Yeong Park
Chanyong Jung
Jong Chul Ye
Sang Wan Lee
Venue: Submitted to ICLR 2023 (2023)
Keywords: Debiasing
spurious correlation
self-supervised learning

Abstract: Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. While most of existing debiasing methods require full supervisions on either spurious attributes or target labels, training a debiased model from a limited amount of both annotations is still an open issue. To overcome such limitations, we first examined an interesting phenomenon by the spectral analysis of latent representations: spuriously correlated, easy-to-learn attributes make neural networks inductively biased towards encoding lower effective rank representations. We also show that a rank regularization can amplify this bias in a way that encourages highly correlated features. Motivated by these observations, we propose a self-supervised debiasing framework that is potentially compatible with unlabeled samples. We first pretrain a biased encoder in a self-supervised manner with the rank regularization, serving as a semantic bottleneck to enforce the encoder to learn the spuriously correlated attributes. This biased encoder is then used to discover and upweight bias-conflicting samples in a downstream task, serving as a boosting to effectively debias the main model. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines and, in some cases, even outperforms state-of-the-art supervised debiasing approaches.

Under review as a conference paper at ICLR 2023
SELF-SUPERVISED DEBIASING USING LOW RANK REG- ULARIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. While most of existing debiasing methods require full su- pervisions either spurious attributes or target labels, training debiased model from a limited amount of both annotations is still an open issue. To overcome such limitations, we first examine interesting phenomenon by the spectral anal- ysis of latent representations: spuriously correlated, casy-to-learn attributes make neural networks inductively biased towards encoding lower effective rank repre- sentations. We also show that rank regularization can amplify this bias in a way that encourages highly correlated features. Motivated by these observations, we propose a self-supervised debiasing framework that is potentially compati- ble with unlabeled samples. Specifically, we first pretrain biased encoder in self-supervised manner with the rank regularization, serving as a semantic bot- tleneck enforce the encoder to learn the spuriously correlated attributes. This biased encoder is then used to discover and upweight bias-conflicting samples in a downstream task, serving as a boosting to effectively debias the main model. Remarkably, the proposed debiasing framework significantly improves the gen- eralization performance of self-supervised learning baselines and, in some cases, even outperforms state-of-the-arr supervised debiasing approaches.
INTRODUCTION
While modern deep learning solves several challenging tasks successfully, series of recent works (Geirhos et al. 2018, Gururangan et al. 2018 Feldman et al. 2015) have reported that the high accuracy of deep networks on in-distribution samples does not always guarantee low test error on spurious correlations. AAjovssy jet al. (2019); Nagarajan et al. 2020); Tsipras al. (2018) suggest that the deep networks can be potentially biased to the spuriously correlated attributes, or dataset bias, which are misleading sta- tistical heuristics that are closely correlated but not causally related to the target label. In this regard, several recent works explain this phenomenon through the lens of simplicity bias (Rahaman et al.) 2019 Neyshabur al. 2014 Shah et al. 2020) of gradient descent-based deep networks optimiza- tion; deep networks prefer to rely on spurious features which are more "simpler" to learn, e.g., more linear.
out-of-distribution (OOD) samples, especially in the context
The catastrophic pitfalls of dataset bias have facilitated the development of debiasing methods, which can be roughly categorized into approaches (1) leveraging annotations of spurious attributes.
ie., bias label (Kim et al., 2019) Sagawa et al.) 2019 Wang et al. 2020 Tartaglione et al., 2021
(2) presuming specific type of bias, e.g.. color and texture (Bahng et al. 2020; Wang et al.
2019
Ge et al. 2021) or (3) without using explicit kinds of supervisions on dataset bias (Liu et al.
2021
Nam et al. 2020 Lee al. 2021 Levy et al. 2020 Zhang al.) 2022).
While substantial technical advances have been made in this regard, these approaches still fail to address the open problem: how to train a debiased classifier by fully exploiting unlabeled samples lacking both bias and target label. More specifically, while the large-scale unlabeled dataset can be potentially biased towards spuriously correlated sensitive attributes, e.g., ethnicity, gender, or age (Abid et al.) |021 Agarwal al. 2021), most existing debiasing frameworks are not designed to deal with this unsupervised settings. Moreover, recent works on self-supervised learning have reported that self-supervised learning may still suffer from poor OOD generalization (Geirhos al.

---

Under review as a conference paper at ICLR 2023
2020 Chen et al. 2021 Robinson et al. [002 Tsai et al. 2021 when such dataset bias still remains after applying data augmentations.
To address this question, first made series of observations about the dynamics of representations complexity by controlling the degree of spurious correlations in synthetic simulations. Interestingly, found that spurious correlations suppress the effective rank (Roy & Vetterli (007) of latent representations, which severely deteriorates the semantic diversity of representations and leads to the degradation of feature discriminability. Another notable aspect of our findings that the intentional increase of feature redundancy leads to amplifying "prejudice" in neural networks. To be specific, as we enforee the correlation among latent features to regularize the effective rank of representations (i.e.. rank regularization), the accuracy on bias- coafflecnii samples quickly declines while the model still performs reasonably well on the bias-aligned samples.
Inspired by these observations, we propose self- superviue debiasing framework that can fully utilize potentially biased unlabeled samples. We pretrain (1) a biased encoder with rank regulariza- tion, which serves semantic bottleneck limiting the semantic diversity of feature components, and (2) the main encoder with standard self-supervised learning approaches. Specifically, the biased encoder gives the leverage to uncover spurious correlations and identify bias-conflicting training samples in a downstream task.
Contributions, In summary, the contributions of this paper demonstrate the inductive bias of neural networks favor of low rank representations in the pres- ence of spurious correlations. Based on these observations, we propose novel rank-reullaiization debiasing framework that fully exploits unlabeled samples that do not contain annotation for bias and target label. Various experiments on real-world biased datasets demonstrate that retraining lin- ear classifier in the last layer with upweighting of identified bias-conflicting samples significantly improves the OOD generalization in the linear evaluation protocol (Oord et al.) 22018) even without making any modifications on the pretrained encoder. Our approach improves the accuracy on bias- conflicting evaluation set by 36.4% + 59.5%, 48.6% + 38.4% on UTKFace (Zhang et al. 2017) and CelebA (Liu et al.) (015) with age and gender bias, respectively, compared to the best self- supervised baseline. Moreover, we found that the proposed framework outperforms state-of-the-ee supervised debiasing methods in semi-supervised learning problem with CelebA.
follows: First, empirically
LOW-RANK BIAS OF BIASED REPRESENTATIONS
2.1 PRELIMINARIES
Preliminaries. To evaluate the semantic diversity of given representation matrix, we introduce effective rank (Roy & Vetterli 2007) which is a widely used metric to measure the effective dimen- sionality of matrix and analyze the spectral properties of features in neural networks (Arora et al. 2019 Razin & Cohen, 2020: Huh et al. 2021 Baratin et al. 2021):
Definition 2.1 Given the matrix X E Rm and its singular values {o,) min (m.n). the effective rank p of X is defined as the shannon entropy of normalized singular values:
min (m,
p(X) = Σ
og log o.,
o.,
(I)
where o, - Eg is i-th normalized singular value. Without loss of generality, we omit the exponentiation of P(X) as done Roy & Vetterli (2007).
Effective rank is also referred to as spectral entropy where its value is maximized when the singu- lar values are all equal, and minimized when top singular value dominates relative to all others. Recent works (Chen et al.] (2019ha) have revealed that the discriminability of representations re- sides on wide range of eigenvectors since the rich discriminative information for the classification
'The bias aligned samples refer to data with a strong correlation between (potentially latent) spurious fea- tures and target labels. The bins-conft samples refer to the opposite cases where spurious correlations do exist.
2

---

Under review as a conference paper at ICLR 2023
(c) Color bias
(d) Digit bias
(a) Unbiased correlation
(b) Biased correlation
accuracy
Effective rank
Rato algace samples
Rallo tits-llii samples
10%
C.0
0.4
95%
95%
4.0
022
0.0
98%
90%
1/3
3,0
cubast Cuassr
CIaRRIDC "Nasel CIAARD (unbisees)
99%
E.4
99%
0.5
cypdST (unthasdd)
95.5%
3,4
0% 50% 75% 90% 92%
0%
50%
73%
90% 35%
Sutsampling ratio
Subsampling reti
20
60
100
Index
(e) Subsampling results
(f) Spectral analysis
Figure 1: Empirical analysis rank reduction phenomenon, (a, b): Hierarchically clustered auto- correlation matrix of unbiased and biased representations (Bias ratio=99%). (c. d): Effective rank with treating color digit dataset bias, respectively. 'Unbiased' represents the training model with perfectly unbiased dataset, i.e., assign random color for each training sample. (e): Unbi- ased test accuracy (left) and effective rank (right) measured with subsampling bias-aligned samples. Subsampling ratio denotes the ratio of removed samples among the total bias-aligned samples. (f): SVD analysis with max-normalized singular values. Top 100 values shown the figure (Total: 256). All the analyses use the output Z of the encoder (see notations in section 2.3
task cannot be transmitted by only few cigenvectors with top singular values, Thus from a spec- tral analysis perspective, effective rank quantifies how diverse the semantic information encoded by each eigenfeature are, which is closely related to the feature discriminability across target label categories. In the rest of paper, we interchangeably use effective rank and rank by following prior works.
2.2 SPECTRAL ANALYSIS OF THE HASKRANN RELATIONSHIPS
Degree of spurious correlations. We now present experiments showing that the deep networks may tend to encode lower rank representations in the presence of stronger spurious correlations. To arbitrarily control the degree of spurious correlations, introduce synthetic biased datasets, Color- MNIST (CMNIST) and Corrupted CIFAR-10 (CIFAR-10C) (Hendrycks & Dietterich, 2019), with color and corruption bias types, respectively. We define the degree of spurious correlations as the ratio of bias-aligned samples included in the training set, or bias ratio, where most of the samples are bias-aligned in the context of strong spurious correlations. Figure Ic shows that the rank of latent representations from penultimate layer of the classifier decreases as the bias ratio increases in CMNIST. We provide similar rank reduction results CIFAR-I0C in the supplementary material.
We further compare the correlation matrix of biased and unbiased latent representations in the penul- timate layer of biased and unbiased classifiers, respectively. Figure Taa and [bb we observe that the block structure in the correlation matrix is more evident in the biased representations after the hierarchical clustering, indicating that the features become highly correlated which may limit the maximum information capacity of networks.
We also measure the effective rank by varying the subsampling ratio (Japkowicz & Stephen 2002) of bias-aligned samples. Subsampling controls the trade-off between the dataset size and the ratio of bins-conficttin samples to bias-aligned samples, i.c., conflict-to-align ratio, where subsampling of bias-aligned samples reduces the dataset size but increases the conflict-to-align ratio. Figure
3

---

Under review as a conference paper at ICLR 2023
1e shows that the effective rank is aligned well with the conflict-o--lign ratio generalization performance, whereas it is not along with the dataset size.
Simplicity bias. Here, argue that the rank reduction is rooted in the simplicity bias (Shah al, 2020 Hermann & Lampinen 2020) of deep networks. Specifically, we reverse the task where the color is now treated as target variable of networks, and the digit is spuriously correlated to the color, as done in (Nam et al. 2020). Digits are randomly assigned to color in an unbiased evaluation set. Figure [d] shows that the rank reduction is not reproduced in this switched context where the baseline levels of effective rank are inherently low. It intuitively implies that the rank reduction is evidence of reliance on easier-to-leare lo-learn features, where the rank does not decrease progressively if the representation is already sufficiently simplified.
Spectral analysis. To investigate the rank reduction phenomenon in-depth, we compare the normal- ized singular values of biased and unbiased representations. Specifically, we conduct singular value decomposition (SVD) on the feature matrices of both biased and unbiased classifiers and plot the singular values normalized by the spectral norm of the corresponding matrix. Figure shows that the top few normalized singular values of biased representations are similar to or even greater than that of unbiased representations. However, the remaining majority of singular values decay signif- icantly faster in biased representations, greatly weakening the informative signals of eigenvectors with smaller singular values and deteriorating feature discriminability (Chen et al. 2019b a).
2.3 RANK REGULARIZATION
Motivated from the rank reduction phenomenon, we ask an opposite-directional question: "Can we intentionally amplify the prejudice of deep networks by maximizing the redundancy between the components of latent representations?". If the feature components are extremely correlated, the corresponding representations may exhibit most of its spectral energy along the direction of one singular vector. For this case, effective rank may converge to O. In other words, our goal is to design a semantic bottleneck of representations which restricts the semantic diversity of feature vectors. To implement the bottleneck in practice, we compute the auto-correlation matrix of the output of encoder. Throughout the paper, we denote x E I and y E y as m-dimensional input sample and its corresponding predicting label, respectively. Then we denote X {=+)ū=1 as a batch of samples from dataset which is fed to an encoder fe m R4, parameterized by 0. Then we construct a matrix Z E xndd where each ith row is the output representations of the encoder fo(x;)" for x; E X. Let Z denotes the mean- mean-centered Z along the batch dimension. The normalized auto-correlation matrix C' E Rddd of Z is defined as follow:
En Zo, Zo.
C23
VI n S d,
V VE 2,, /mm
22
where b is an index of sample and í,j are index of each vector dimension. Then we define our regularization term as negative of a sum of squared off-diagonal terms in C:
Breg (x:0) ΣΣ Ch
i 379
where we refer to it as rank loss. Note that the target labels on X is not used at all in formulation.
To investigate the impacts of rank regularization, we construct the classification model by combining the linear classifier fw Rd + I parameterized by W E W on top of the encoder fe, where c= is the number of classes. Then we trained models by cross entropy loss lcg combined with AregEreg. where Aveg > 0 is Lagrangian multiplier. We use CMNIST, CIFAR- and Waterbirds dataset (Wah et al., (201T), and evaluate the trained models on an unbiased test set following Nam et al. (2220); Lee et al. (2021). After training models with varying the hyperparameter Xreg. compare bias-aligned and bias-conffict accuracy, which are the average accuracy on bias-aligned and bias-confficting samples in the unbiased test set, respectively, for CMNIST and CIFAR-10C. Test accuracy on every individual data group is reported for Waterbirds.
Figure 2 shows that models suffer more from poor OOD generalization trained with larger Xreg- The average accuracy on conffiiiig groups is significantly degraded, while the accuracy on bias-aligned groups is maintained some extent. It implies that rank regularization may force
4

---

Under review as a conference paper at ICLR 2023
100
100
80
RccUPaaY (5)
Accuraty (56)
00
90-
60
40
40
Day Ban AArr
50
6609 co
(kx eFgred
20
Oric Boo-celle
0.0 25
12.5
15.0
10
Aray
Amg
Ane
(a) CMNIST
(b) CIFAR-I0C 10C
(c) Waterbirds
Figure (a, b): Bias-conflict and Bias-aligned accuracy on CMNIST and CIFAR-10C (Bias
ratio=95%). (c): Group accuracy on Waterbirds, Detailed simulation settings are in the appendix.
Precision (%)
Recall (%)
Precision (%)
Recall (%)
ERM
85.59
19.76
ERM
52.03
0.06
Rank reg
98.83
95.91
Rank reg
71.39
51.43
(a) CMNIST
(b) CIFAR-10C
Table 1: Precision and recall of identified bias-confiictin samples in enror set of ERM model trained
with and without rank regularization. Bias ratio=95% for both dataset. Areg = 35 and Aveg = 20 used for CMNIST and CIFAR-10C, respectively. Capacity control techniques (e.g., strong & regularization, early-stopping, Liu et al. (202T); Sagawa et al. (2019) are not used to emphasize the contribution of rank regularization. Detailed simulation settings are the appendix.
deep networks to focus on spurious attributes. Table supports that the biased models with strong regularization effectively probe out the bias-confficting samples in the training set. Specifically, we train a biased classifier with rank regularization and distill an error set E of misclassified training samples bias- binscconf samples proxies. As reported in Table 1 empirically observe that our biased classifier is relatively robust to the unintended memorization of bias-conflicting samples (Sagawa al. 2020 in contrast to the standard models trained by Empirical Risk Minimization (ERM).
DEFUND: DEBIASING FRAMEWORK WITH UNLABELED DATA
Motivated by the observations in Section we propose a self-supervised debiasing framework with unlabeled data, coined DeFund. The most important difference from prior works is that the proposed framework can intentionally learn biased representations without human supervision. Recent meth-
ods (Bahng et 2020 Nam et al. 2020 Liu et al.) 2021| Lee al. 2021 I Zhang al. 2022 train a biased model to uncover the spurious correlations and guide the main model to focus on samples that the biased model struggles to predict, which are seemingly the ones conflicting with the bias. While these methods require a bias label or target label to train biased representations, we obtain such biased representations for free using self-supervised learning and rank regularization.
The proposed framework is composed of two stages: We first train the biased encoder, which can be potentially adopted detect the bias-conflccing samples in a downstream task, along with the main encoder by self-supervise learning, both without any labels. After pretraining, we identify the bias-confficting samples the downstream task using linear evaluation protocol (Oord et al. 2018 Chen et al. 2020). This set of samples serves boosting debias the main model.
We denote phius : X + 2d and fmain + Rd as biased encoder and main encoder parame- terzied by 8 E and & θ, respectively, where the dimensionality of latent representations. Then we can compute the rank loss in (3) with introduced encoders and given batch {2x J. with size N. Let rele Rd BCC be single-layer classifier parameterized by W,, E W which is placed top of biased encoder rhias, where C | is the number of classes. We similarly de- fine the linear classifier fW for the main encoder. Then refer to fbsos X RCC biased
5

---

Under review as a conference paper at ICLR 2023
model, where (x) JW, (1)a((x)))) Vx E X. We similarly define the main model fmain as
fmaiin (x) fale (z)ai Vx E X. While the projection networks (Chen et al. 2020 Khosla
W.
al. 2020) are employed as well, we omit the notations because they are not engaged in the linear evaluation after pretraining encoders.
Stage 1. Train biased encoder. To train the biased encoder shin, we revisit the proposed rank reg- ularization term in (3) which can control the effective dimensionality of representations for instance discrimination task. We conjecture that the scope of captured features may be restricted to the easy- to-learn ones if the maximum information capacity of the encoder is strongly suppressed. Based on these intuitions, we apply rank regularization directly to the output of the base encoder, which encourages each feature component to be highly correlated. A simple simulation on the synthetic dataset conceptually clarifies the validity of our intuition, where we figured out that the representa tion becomes more biased as it is trained with stronger regularization, by measuring the bias metric, which quantifies how much the encoder focus on the short-cut attributes (Details provided in sup- plementary material). Morcover, while the overall performance may be upper-bounde boeudbd due to the constraint on effective dimensionality (Jing et al. (021), we observed that the bias-conflict accuracy is primarily sacrificed compared to the bias-aligned accuracy (Related experiments in Section 4
Stage 2. Debiasing downstream tasks. After training the biased encoder, our next goal is to debias the main model. pretrained on the same dataset with standard self-supervised learning approaches, e.g.. CCenn et al. (000)); Chen & He (2021). To achieve this, we recall the recent work which explains the contrastive learning as a protocol inverting the data generating process: Zimmermann iet al. (2021) demonstrates that the pretrained encoder with a contrastive loss from the InfoNCE family can recover the true latent factors of variation under some statistical assumptions. That being said, imagine that we have an ideal pretrained encoder of which each output component corresponds to the latent factor of data variation. Then one may expect that this encoder perfectly fits downstream classification tasks, where the only remaining, job is to find out the optimal weights of these factors for prediction. However, if most samples in the downstream task are bias-aligned, these samples may misguide the model to upweight the spuriously correlated latent factors. In other words, the model may reach a biased solution even though it encodes well- gelleeeeerlized representations.
The above contradiction elucidates the importance of bias-confficting samples, which serve as coun- terexamples of spuriously correlated feature components, thereby preventing the alleged involve- ment of such components in prediction. Based these intuitions, we introduce novel debias- ing protocol that probes and upweights bias-confticting samples to find and fully exploit feature components independent of spurious correlations. We evaluate our framework on two scenarios: linear evaluation and semi-supervised learning. First, following the conventional protocol of self- supervised learning, we conduct linear evaluation (Zhang et al. 2016: Oord et al.) 2018 which trains a linear classifier on top of unsupervised pretrained representations by using target labels of every training sample. After training a linear classifier fals with pretrained biased encoder f fhias given the whole training set D = [ } with size N, an error set E of misclassified sam- ples and corresponding labels is regarded as bias-conflicingg pairs. Then we train a linear classifier file on freezed representations of main encoder fmaai by upweighting the identified samples in E with Aup > 0. The loss function for debiased linear evaluation is defined as follow:
Ldehias (D; W.)) = Xup
Σ (a.))
E(I, 3: Wm) +
Σ
((x, y: Win).
(4)
(2.D)EDAE
where we use cross entropy loss for E X X'Xx W R- Note that the target labels are only used in training linear classifiers after pretraining.
While linear evaluation is mainly opted for evaluating self-supervised learning methods, we also compare our method directly to other supervised debiasing methods in the context of semi- supervised learning. Here we assume that the training dataset includes only a small amount of labeled samples combined with large amount of unlabeled samples. As in linear evaluation, we train a linear classifier on top of the biased encoder by using labeled samples. After obtaining an er- ror set E of misclassified samples, we finetuned the whole main model by upweighting the identified samples in E with Aup- Note that supervised baselines are restricted to using only a small fraction of labeled samples, while the proposed approach benefits from the abundant unlabeled samples dur- ing training of the biased encoder. The pseudo-code of DeFund is provided in the supplementary material.

---

Under review as a conference paper at ICLR 2023
Table 2: (Linear evaluation) Bias-conflict and unbiased test accuracy (%) evaluated on UTKFace and CelebA. Models requiring information on target class or dataset bias in (pre)training stage are denoted with v column Y and B, respectively. The results are averaged on random seeds.
UTKFace (age)
UTKFace (gender)
CelebA (makeup)
Model
Y
B
Conflict
Unbiased
Conflict
Unbiased
Conflict
Unbiased
LNL
45.8
72.6=0.3
+3.
84.9
55.9
76.0+0.6
EnD
/
/
45.3_0..
72.2
75.5
85.5
57.3
76.4+1.4
JTT
V
63.8
69.4 1.3
71.2
77. 77.6
62.4
74.7+0.8
CVaR DRO
V
45.7 =2.0
=0.4=0.3
68.6+1.0
81.0=0.8
58.0+1.7
76.5
ERM V
X
45.4
71.0
65.7
79.5
54.2.0.2
74.1
SimSiam
X
28.2
62.6 20.7
48.5
69.8
39.940.6
66. 10.6
VICReg
X
32.3-0.6
64.6
51.0+1.4
71.340.7
48.640.6
71.940.2
SimCLR
X
36.4=1.5
66.3
56.3+0.2
74.2=0.2
46.9+1.0
69.8+0.4
DeFund
X
59.5 0.8
70.6 0.8
63.7
74.9 10.9
58.4
73.1 11.0
4 RESULTS
4.1 METHODS
Dataset. To investigate the effectiveness of the proposed debiasing framework, we evaluate several supervised and self-supervised baselines UTKFace (Zhang et al. 2017) and CelebA (Liu et al.) 2013) in which prior work has observed poor generalization performance due to spurious correla- tions. Each dataset includes several sensitive attributes, e.g., gender, age, ethnicity, etc. We consider three prediction tasks: For UTKFace, we conduct binary classifications using (Gender, Age) and (Race, Gender) (target, spurious) attribute pair, which refer to UTKFace (age) and UTKFaae (gender), respectively. For CelebA, we consider (HeavyMakeup. Ma falee and (B1 (Blonde Ha r, Male) as (target, spurious) attribute pairs, which are referred CelebA (makeup) and CelebA (blonde), respectively. The results of CelebA (blonde) are reported in appendix. Following Nam al. (2020); Hong & Yang (202T), we report bias-conffict accuracy together with unbiased accu- racy, which is evaluated on the explicitly constructed validation set. We exclude the dataset in Figure 2 based on the observations that the SimCLR models are already invariant w.r.t spurious attributes.
We mainly target baselines consisting of recent advanced self-supervised learning meth- ods, SimCLR (Chen et al., 2020), VICReg (Bardes et al., (02T), and SimSiam (Chen He| (021)) which can be categorized into contrastive (SimCLR) and non-contrastive (VICReg, SimSiam) meth- ods. We further report the performance of vanilla networks trained by ERM, and other supervised debiasing methods such as LNL (Kim et al. 2019), EnD (Tartaglione et al. 202T), and upweighting- based algorithms, JTT (Liu et al. (021) and CVaR DRO (Levy et al.) 2020), which can be catego- rized into methods that leverage annotations
Baselines.
dataset bias (LNL, EnD) or not (JTT, CVaR DRO).
Optimization setting, Both bias and main encoder is pretrained with SimCLR (Chen et al. 2020) for 100 epochs on UTKFace, and 20 epochs CelebA, respectively, using ResNet- 18, Adam opti- mizer and cosine annealing learning rate scheduling (Loshchilov & Hutter 2016). We use a MLP with one hidden layer for projection networks as SimCLR. All the other baseline results are reproduced by tuning the hyperparameters and optimization settings using the same backbone ar- chitecture. We report the results of the model with the highest bias-conflicting test accuracy over those with improved unbiased test accuracy compared to the corresponding baseline algorithms, i.e., SimCLR for ours. The same criteria are applied supervised baselines, while JTT often sacrifices unbiased accuracy for highly improved bias-conflict accuracy. More details about the dataset and simulation settings are provided in the supplementary material.
4.2 EVALUATION RESULTS
Linear evaluation. The bias-conffict and unbiased test accuracy summarized in Table 2 We found that DeFund outperforms every self-supervised baseline by large margin, including Sim- CLR, SimSiam and VICReg, with respect to both bias-conflict and unbiased accuracy. Moreover, in
7

---

Under review as a conference paper at ICLR 2023
Table 3: (Semi-supervised learning) Bias-conflict and unbiased accuracy (%) evaluated on CelebA (makeup). Label fraction is set to 10%. Each first and second / marker represents whether the model requires information on target class or dataset bias in pretraining stage, respectively.
LNL
EnD //
JTT
CVaR DRO Vx
ERM Vx
SimCLR
DeFund
Accuracy
Vx
XX
XX
Conflict
55.7+1.4
55.3
51.5+199
55.641.5
51.5+11
50.5
60.5
Unbiased 75.6
76.2
71.4
75.7+1.0
73.1
71.6
75.6
some cases, DeFund even outperforms ERM models or supervised debiasing approaches regarding bias-conflict accuracy. Note that there is an inherent gap between ERM models and self-supervised baselines, roughly 8.7% on average. Moreover, we found that non-contrastive Icarning methods generally perform worse than the contrastive learning method. This warns against training the main model using a non-contrastive learning approach, while it may be a viable option for the biased model. We provide results of the proposed framework implemented with non-contrastive learning methods in the supplementary material.
Semi-supervised learning. To compare the performance of supervised and self-supe methods in a more fair scenario, we sample 10% of the labeled CelebA training dataset at random for each run. The remaining 90% samples treated unlabeled ones and engaged only in pretraining encoders for self-supervised baselines. Labeled samples are provided equally to both supervised and self-supervised methods.
Remarkably, Table 3 shows that the proposed framework outperforms all the other state-of-the- art supervised debiasing methods. Notably, only about 16 samples remain within (Gender=1, (eavyMakeup=)) group after subsampling. Thus it is almost impossible to prevent deep net- works from memorizing those samples even with strong regularization if we train the networks from scratch, which explains the failure of existing upweighting protocols such JTT. In contrast, the proposed framework can fully take advantage of unlabeled samples where contrastive learning help prevent memorization of the minority counterexamples (Xue et al. 2022 It highlights the im- portance of pretraining using unlabeled samples that most prior debiasing works do not consider. Moreover, such implicit bias of deep networks towards memorizing samples may seriously deterio- rate the performance
existing coafflcifi sample mining algorithms (Kim et al.) [202]] Zhao al., 2 Nam et al.) (2000) when the number of labeled samples is strictly limited. However, such failure is unlikely to be reproduced in the proposed framework since we only train a simple linear classifier on top of a freezed biased encoder to identify such bias-confiictnng samples.
Table 4: Ablation study introduced modules. Accuracy is reported in (%).
UTKFace (age)
UTKFace (gender)
CelebA (makeup)
Method
Conflict
Unbiased
Conflict
Unbiased
Conflict
Unbiased
SimCLR
36.4
66.3
56.3
74.2
46.9
69.8
+ Rank reg
26.6
61.3
50.9
70.3
43.9
68.3
Upweight
53.0
64.6
58.3
74.5
50.1
70.4
DeFund 59.5
70.6
63.7
74.9
58.4
73.1
Table 5: Precision and recall (%) of bias-conflicting samples identified by SimCLR and our biased model. Both case used linear evaluation.
UTKFace (age)
UTKFace (gender)
CelebA
Method
Precision
Recall
Precision
Recall
Precision
Recall
SimCLR
68.31
44.63
33.36
39.59
52.25
28.23
DeFund
68.67
75.94
29.98
50.93
55.29
32.46
8

---

Under review as a conference paper at ICLR 2023
Ablation study. To quantify the extent of performance improvement achieved by each introduced module, we compared the linear evaluation results of (a) vanilla SimCLR, (b) SimCLR with rank regularization, (c) SimCLR with upweighting error set E of the main model, and (d) Full model DeFund. Note that (c) does not use a biased model at all. Table I shows that every module plays an important role in OOD generalization. Considering that the main model is already biased to some extent, we found that bias-conflict accuracy can be improved even without biased model, where the error E' of the biased model further boosts the generalization performance. We also quantify how well the biased model captures bias-conflicting samples by measuring the precision and recall of identified blas-conflicting samples in E. As reported Table [5] the biased model detects more diverse bias-conflicting samples compared to the baseline for free or with affordable precision costs. While the improvement of recall CelebA may seem relatively marginal, a large quantity of bias-conflicting samples is additionally identified in practice considering that CelebA includes much more samples than UTKFace.
DISCUSSIONS AND CONCLUSION
Contributions. In this paper, we (a) first unveil the catastrophic adverse impacts of spurious cor- relations on the effective dimensionality of representations. Based on these findings, we (b) design a rank regularization that amplifies the feature redundancy by reducing the spectral entropy of la- tent representations. Then we (c) propose debiasing framework empowered by the biased model pretrained with abundant unlabeled samples.
Comparisons to related works. Our observations are in line with the simplicity bias of gradient descent-based optimizations, where many recent studies (Rahaman et al. 2019, Shah et al., 2020) have revealed that networks tend to exploit the simplest feature at the expense of a small margin and often ignore the complex features. Similar observations have been made confined to self-supervieed learning named feature suppression, where the encoder may heavily rely on the attributes that make the instance discrimination tasks easier. While these existing works often focus on the innate pref- erence of models on input cues (Hermann & Lampinen 2020); Scimeca al., 202T), provide novel perspective on the practical impacts of spurious correlations on deep latent representations: reduction of effective rank.
Robinson et al. (2021) proposes an opposite-directional approach compared to our framework to improve generalizations of self-supervised learning. aims to overcome the feature suppression and learn wide variety of features by Implicit Feature Modification (IFM), which adversarially perturbs feature components of the current representations used to discriminate instances, thereby encouraging the encoder to use other informative features. We observed that IFM improves the bias-conflict accuracy by about 1% on UTKFace (age) in Table which is roughly consistent with the performance gains on the standard benchmarks, e.g., STLTO, reported in the original paper. However, its performance gain is relatively marginal compared the proposed framework.
Table 6: Results of Implicit Feature Modification (Robinson et al. 2021 with SimCLR UTKFace (age). we denote as the adversarial budget of feature modification as in the original paper.
Accuracy
SimCLR
0.05
= 0.1
= 0.5
Bias-conflict (%)
36.4
37.5
36.4
33.7
Unbiased (%)
66.3
66.5
66.2
64.6
Future directions. While this work has focused on intentionally encoding biased representations, we argue that more advances should be concurrently made in learning both biased and debiased representations, as partially discussed above (Robinson al., (021). The interplay between those bidirectional modules may further improve generalizations. We also note that the proposed rank reg- ularization is one possible implementation of the semantic bottleneck. While we explicitly control the feature correlations, we believe such design can be employed more implicitly. We provide exper- iments examining the potential of existing hyperparameters, e.g., the temperature in InfoNCE loss, etc., bias controller in the supplementary material. Lastly, we believe that semi- semi-supervised learning scenario should be part of a standard evaluation pipeline where many supervised basclines may fail due the inductive bias of networks towards memorizing a few counterexamples.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Abubakar Abid, Maheen Farooqi. and James Zou. Large language models associate muslims with violence. Nature Machine Intelligence, 3(6):461 4631 2021.
Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. Evaluating clip: towards characterization of broader capabilities and downstream im- plications. arXiv preprint arXiv:2108. 02818, 2021.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:/977002S90 2019.
Sanjeev Arora, Nadav Cohen, Wei Hu, and Yuping Luo. Implicit regularization in deep matrix factorization. Advances in Neural Information Processing Systems, 32, 2019.
Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Scong Joon Oh. Learning de-biased representations with biased representations. International Conference on Machine Learning. pp. 528-539. PMLR, 2020.
Aristide Baratin, Thomas George, César Laurent, R Devon Hjelm, Guillaume Lajoie, Pascal Vin- cent, and Simon Lacoste-Julien. Implicit regularization via neural feature alignment. In Interna- tional Conference on Artificial Intelligence and Statistics, pp. 2269-2277. PMLR, 2021.
Adrien Bardes, Jean for self-supervised learning. arXiv preprint arXiv:2105.044906 2021.
Jean Ponce, and Yann LeCun. Vicreg: Variann--ivaaraccccovvvaann reg.roooo
Tiffany Tianhui Cai, Jonathan Frankle, David Schwab, and Ari S Morcos. Are all negatives created equal in contrastive instance discrimination? arXiv preprint 2:X1::2010.06682, 2020.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning. pp. 1597-1607. PMIR, 2020.
Ting Chen, Calvin Luo, and Lala Li. Intriguing properties of contrastive losses. Advances in Neural Information Processing Systems, 34:1 1834-11845, 2021.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750-15758, 2021.
Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long, and Jianmin Wang. Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning. Advances in Neural Information Processing Systems, 32, 2019a.
Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. Transferability vs. discriminabil- ity: Batch spectral penalization for adversarial domain adaptation. International conference on machine learning, pp. 1081-1090. PMLR, 2019b.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 259 2689266,111 2015.
Songwei Ge, Shlok Mishra, Chun-Liang Li, Haohan Wang, and David Jacobs. Robust contrastive learning using negative samples with diminished semantics. Advances in Neural Information Processing Systems, 34:27356-2758 2021.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis. Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trainee trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. arXiv preprint arXiv: 2018.
10

---

Under review as a conference paper at ICLR 2023
Robert Geirhos, Kantharaju Narayanappa. Benjamin Mitzkus, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. On the surprising similarities between supervised and supervise mod- cls. arXiv preprint arXiv:2010.08377,, 2020.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R Bowman, and Noah A Smith. Annotation artifacts in natural language inference data. arXiv preprint arXiv: 1803.02300 2018.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common ruptions and perturbations. arXiv preprint arXiv: 12261, 2019.
Katherine Hermann and Andrew Lampinen. What shapes feature representations? exploring datasets, architectures, and training. Advances in Neural Information Processing Systems, 33: 9995-10006, 2020.
Youngkyu Hong and Eunho Yang. Unbiased classification through bias-contrastive and bias- balanced learning. Advances in Neural Information Processing Systems, 34:26449- 26461, 2021.
Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, and Phillip Isola. The low-rank simplicity bias in deep networks. arXiv preprint arXiv:2103.10427 2021.
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive coatt-rouuaacccacyy.. In Conference on Causal Learning and Reasoning, pp. 336-351. PMLR, 2022.
Nathalic Japkowiez and Shaju Stephen. The class imbalance problem: A systematic study. Intelli- gent data analysis, 6(5):429-449, 2002.
Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse in contrastive self-supervised learning. arXiv preprint arXiv:2110.093445 2021.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 33:18661-18673, 2020.
Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 9012-9020, 2019.
Eungyeup Kim, Jihyeon Lee, and Jaegul Choo. Biaswap: Removing dataset bias with bias-tailored swapping augmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14992-15001, 2021.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.02937, 2022.
Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William Freeman, Phillip Isola, Amir Globerson, Michal Irani, et al. Explaining in style: Training a gan to explain classifier in stylespace. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 693-702, 2021.
Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning debiased representation via disentangled feature augmentation. Advances in Neural Information Processing Systems, 222222333,,,,, 2021.
Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distribution- ally robust optimization. Advances in Neural Information Processing Systems, 33:8847-8860, 2020.
Zhiheng Li and Chenliang Xu. Discover the unknown biased attribute of an image classifier. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14970-14979, 2021.
11

---

Under review as a conference paper at ICLR 2023
Zhiheng Li, Anthony Hoogs, and Chenliang Xu. Discover and mitigate unknown biases with debi- asing alternate networks. In European Conference on Computer Vision, pp. 270-288. Springer, 2022.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang.
Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 9730-3738, 2015.
lya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint (00i::/600800982 2016.
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur. Understanding the failure modes of dfittr--iitt generalization. arXiv preprint arXiv:2010.15775 2020.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De- De-biasing classifier from biased classifier. Advances Neural Information Processing Systems, 33:20673-20684, 2020.
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. In search of the real inductive bias: On the role of implicit regularization in deep learning. arXiv preprint arXiv: 1412.661,, 2014.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. Xiv preprint arXiv: 2018.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In International Confer- ence on Machine Learning, pp. 5301-5310. PMLR, 2019.
Noam Razin and Nadav Cohen. Implicit regularization in deep learning may not be explainable by norms, Advances in neural information processing systems, 33:21 174-211877 2020.
Joshua Robinson, Ching Yao Chuang. Suvrit Sra, and Stefanie Jegelka. Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592. 2020.
Joshua Robinson, Li Sun, Ke Yu, Kayhan Batmanghelich, Stefanie Jegelka, and Suvrit Sra. Can contrastive learning avoid shortcut solutions? Advances neural information processing systems, 34:4974-4986, 2021.
Olivier Roy and Martin Vetterli. The effective rank: A measure of effective dimensionality. In 2007 15th European signal processing conference, pp. 606-610. IEEE, 2007.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. arXiv preprint arXiv: /9/ 119//.000 1.08731, 2019.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterization exacerbates spurious correlations. In International Conference Machine PMLR, 2020.
Learning. pp.
Luca Scimeca, Scong Joon Oh, Sanghyuk Chun, Michael Poli, and Sangdoo Yun. Which short- cut cues will dnns choose? study from the parameter-space perspective. arXiv preprint orXiv:2/00.030 2021.
Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. Advances in Neural Information Processing Systems, 33:9573-9585, 2020.
Afrina Tabassum, Muntasir Wahed, Hoda Eldardiry, and Ismini Lourentzou. Hard negative sampling strategies for contrastive representation learning. arXiv preprint arXiv:2206.011 197, 2022.
12

---

Under review as a conference paper at ICLR 2023
Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentangling deep representations for bias correction. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 13508-13511 2021.
Yao-Hung Hubert Tsai, Martin Q Ma, Han Zhao, Kun Zhang, Louis-Philippe Morency, and Rus- lan Salakhutdinov. Conditional contrastive learning: Removing undesirable information in self- supervised representations. arXiv preprint arXiv:2106.02555 2021.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint a 12152, 2018.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
Feng Wang and Huaping Liu. Understanding the behaviour of contrastive loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2495-2504, 2021.
Haohan Wang. Zexue He, Zachary Lipton, and Eric P Xing. Learning robust representations by 2019.
projecting superficial statistics out. arXiv preprint arXiv: 1eXivv1903.000
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In International Conference on Machine Learning, pp- 9929-9939 PMLR, 2020.
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky, Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp-
2020.
Yihao Xue, Kyle Whitecross, and Baharan Mirzasoleiman. Investigating why contrastive learning benefits robustness against label noise. arXiv preprint arXiv:2201.12499 2022.
Yadollah Yaghoobzadeh, Soroush Mehri, Remi Tachet, Timothy Hazen, and Alessandro Sor- doni. Increasing robustness to spurious correlations using forgettable examples. arXiv preprint arXiv:1911.03861 2019.
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning, pp. 12310- 12320. PMLR, 2021.
Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, and Aaron Courville. Can subnetwork structure be the key to out-of-distribution generalization? In International Conference on Machine Learning, pp. 12356-12367 PMLR, 2021.
Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Ré. Correct-n- contrast: A contrastive approach for improving robustness to spurious correlations. arXiv preprint arXiv:2203.01517. 2022.
Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In European confer- ence on computer vision, pp. 649-666. Springer, 2016.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/reereesiinn by conditional adversarial autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5810-5818, 2017.
Bowen Zhao, Chen Chen, Qi Ju, and Shutao Xia. Learning debiased models with dynamic gradient alignment and bias bias-cooflicting sample mining, arXiv preprint arXiv:2111.13108, 2021.
Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In International Conference on Machine Learning, pp. 12979-12990. PMLR, 2021.
13

---

Under review as a conference paper at ICLR 2023
Appendix
The supplementary material is organized as follows. We begin with providing the algorithm of DeFund, followed by more discussions the related works. Then we provide additional results and analyses section C Optimization setting, hyperparameter configuration and other experimental details provided in section
A PSEUDOCODE
Algorithm Debiasing Framework with unlabeled data
Input: Dj {ar b=) D., { for semi-supervised learning (N2 N1), 2 for linear evaluation, D Di U Du, batch size n, structure of fbios and fmain,
2:
Stage Pretraining encoders
4: for subsampled minibatch X (Ix ) from D do
5: Update 0 of phias with SimCLR NT-Xent loss and Xreg (X;
6: Update o of smain with SimCLR NT-Xent loss.
7: end for
8: Obtain pretrained parameters & and क.
9:
Stage Downstream task
gets 11: Freeze fhins and train JW, with D;. Identify the error set E C Dr with trained fhins
12: if Linear evaluation then Freeze fmein and train SW with Edebias ( (D;; Wm)
14: else if Semi-supervised learning then
15:
Finetune fmaii with Edetias (Dis Wm, Φ) where & is initialized with G.
16: end
B SUPPLEMENTARY INFORMATION RELATED WORKS
Here, have detailed discussions about related works.
Discovering bias without supervision. In practice, several limitations exist against gleaning more labeled samples: labeling budget, expert-level knowledge required for labeling, data privacy, In this regard, most training samples lack annotations on the spuriously correlated attributes.
To mitigate these problems, several works aim discover biases without bias annotations, Liu al. (202T) reveals that the standard ERM model may serve as a bias-capturing model if one trains with strong capacity control. Yaghoobzadeh et (2019) shows that forgettables, or examples that have been forgotten at least once, contain more minority examples, and proposes a novel robust learning framework by fully exploiting the identified forgettable examples. & Xu (20TT) obtains a biased attribute hyperplane of the generative models, which can help identify semantic biases by generating bias-traversal images. [i et al. (2022) introduces the discoverer model, which uncovers multiple unknown biases such that the difference of averaged predicted probabilities the target attribute in two groups is maximized, [ang et al. (202I) proposes novel framework, StylEx, which trains styleGAN to specifically visualize multiple attributes underlying the classifier decisions.
While substantial advances have been made discovering the unknown biases neural networks without bias labels, these works still inevitably require target labels. In contrast, consider very challenging scenario that has received little attention so far: self-supervised debiasing. this regard, work addresses the following open problems/questions::
Can we learn biased/debiased representations by using unlabeled samples?
What is the fundamental difference between biased and debiased representations?
14

---

Under review as a conference paper at ICLR 2023
Is supervised debiasing robust despite decreasing the number of labeled samples?
How can bias-conflicting samples be discovered by leveraging information from unlabeled samples?
Many recent works have reported the limitations of self--upprvised learning (SSL) in OOD generalization. How can we overcome such limitations?
Mitigating bias with reweighting. Recently, Kirichenko et al. (2022) have reported an intriguing observation: Simple last layer retraining, so-called Deep Feature Reweighting (DFR), can match or outperform state-of-the-un approaches on spurious correlation benchmarks. Kirichenko et al. (2022) shows that biased classifiers still often learn core features associated with the desired attributes of the data. Based these observations, they probe invariant features for the reweighting by leveraging explicit group-balanced dataset 1)
We compare the proposed framework with DFR as follows. First, while DFR and the proposed framework can mitigate the bias in representations by retraining the last linear layer, our method is not restricted to such last-layer retraining. Instead, the semi-supervised learning scenario is a more practical application of the proposed method. Specifically, we can fine-tune representations by fully exploiting both unlabeled and labeled samples, which improves the performance compared to the last layer retraining in Table 3 In contrast, DFR trains linear classifier while freezing the pre- trained representations as-is. More importantly, DFR requires pretrained networks or fully labeled datasets where we consider a more challenging scenario without such assumptions. Moreover, DFR does not use mining bias-conflicting samples in the training set. Specifically, DFR trains new clas- sification head from scratch on the available group-balanced data D. In Kirichenko al. (0022), the reweighting dataset D often consists of random group- balanced subset of the training valida- tion data. In other words, DFR is not designed to identify the bias-conflicting samples but exploits the existing group annotations. Considering practical situation with several limitations against col- lecting more labeled samples, it remains unclear how to obtain the group-balanced dataset Á with sufficient number of samples in the absence of prior information on the dataset bias. In contrast, the proposed framework can leverage the explicit set Đ if accessible, as well as identifying the unknown bias-conflicting samples in the training set.
C ADDITIONAL RESULTS
Our additional results can be roughly categorized into: (1) more observations related to the rank reduction, (2) rank regularization in self-supervised learning, and (3) examination of the poten- tial of existing hyperparameters as a bias controller. Our observations include the visualization of reconstructed images with biased representations, rank reduction trends in CIFAR-IOC and Vision Transformer (ViT, Dosovitskiy et al. (2020)), minority mining performance in supervised settings, and rank regularization with a moderate level of bias. Then we present a simple synthetic simula- tion on the behavior of rank-regularized encoder, followed by additional results on non-contrastive methods and CelebA (blonde). Then the potential of using shallow networks as the bias-cap model will be discussed. Lastly, we provide additional analysis relations between existing hyper- parameters of self-supervised learning and effective rank.
C.1 MORE OBSERVATIONS
Reconstruction of biased representations. To understand the relationship between rank regulariza- tion and spurious correlations more deeply, we visualize the pretrained representations with varying degrees of bias. We first trained deep networks on: (a) unbiased CMNIST (random background color), (b) biased CMNIST (bias ratio=95%) without rank regularization and (c) with rank regular- ization (Arey 50). Then, we train the auxiliary decoder, which reconstructs the bias-conflictnng images from freezed latent representations of each pretrained network.
Figure 3 shows that the reconstructed results evidently different for each case. First, the decoder successfully reconstructs the foreground digit from the (a) unbiased representations, while the back- ground color is completely changed in some cases. It implies that unbiased representations may
15

---

Under review as a conference paper at ICLR 2023
2
4
4
3
9
3
9
7
7
3
3
0
3
4
6
2
6
4
3
3
6 6
9
4
7
3
2
4
4
3
9
3
9
7
7
3
3
0
3
4
6
2
6
4
7
9
?
2
2
9
3
7
6
5
5
/
3
/
8
1
6
8
7
0
5
2
8
7
3 9
8
2
7
9
9
5
/
3
/
8
7
7
5
(a) Unbiased
(b) Biased
(c) Biased with rank reg.
Figure 3: Randomly selected reconstructed images from representations with varying degrees of bias. First and third row correspond to the imput bias- conffliifii images. Second and fourth row correspond to the reconstructed images. Reconstructed from (a) unbiased representations, representations, and (c) biased representations with rank regularization (bias ratio=95% in b, c).
(b) biased
lack information on spuriously correlated attributes, i.e., background color. However, both digit and color are well reconstructed in (b) biased case, implying that the biased model encodes both spurious and invariant features. Intriguingly, the decoder fails to reconstruct bias-conflicting images from the (c) biased representations pretrained with rank regularization. Specifically, the foreground digit is blurred, and its class is often changed following the color-digit assignment Figure
Based on these observations, we summarize some key insights: First, the rank-regularized representation may lose its infor- mation on harder-to-leane invariant features. While the recon- structed images (a) or (b) preserve the detailed class, shape, and style of the foreground digit, such properties are deterio- rated in (c), implying the loss of feature discriminability and informative signals. Second, with limited semantic diversity, identify the true underlying independent generative factors for multidimensional data; it may rather encode feature components entangled with both spurious and invariant attributes. In other words, the proposed low-rank regularization prevents features from encoding discriminative information independently. It is indirectly reflected in (c) that the digit class of the reconstructed image is erro- neously determined by the spurious attribute, i.e., background color. The experiments showed that rank regularization encourages the network to focus more on spurious correlations in way that minimizes semantic diversity.
2
3
4
5
7
9
Figure 4: Examples of bias-aligned CMNIST images.
the rank-regularized model fails to
Table Unbiased test accuracy (%) on CMNIST and CIFAR- 10C measured with varying bias ratio The model trained with unbiased dataset (r 10%) serves as baseline.
Dataset
Unbiased r= 95%
T= 98%
99%
r 99.5%
CMNIST
99.87
88.27
68.13
36.21
13.61
CIFAR-10C
78.71
46.15
34.18
26.76
20.94
Rank reduction. Figure 5a shows that the rank of latent representations from penultimate layer of classifier decreases as the bias ratio increases in CIFAR-IOC: In Table 7 we supplement the unbiased test accuracy of CMNIST and CIFAR-10C from the experiments presented in Figure Ic and 5a respectively. Moreover, similar rank reduction trends are observed in Vision Transformer (ViT. Dosovitskky et al. (2020). We train ViT on CMNIST and CIFAR-IOC for 2000 and 10000 iterations, respectively, with Adam optimizer of learning rate 0.001, patch size 4, dimension of output tensor 128, number of transformer blocks 6, number of heads in multi-hear Attention layer 4, dropout rate 0.2 and dimension of the MIP (FeedForward) layer 1024. Figure 5b 5c show that the effective rank of the output of the Transformer encoder z! (notation follows original paper) decreases as bias ratio increases.
Minority mining supervised setting. Following the official implementation of JTT (Liu et al. 2021), we compare the quality of bias-conflicting samples identified by JTT and the rank-regularized model Waterbirds dataset. Following the official command of JTT, Aeg = 1 is used for JTT, while Xe 1e 4 is used for ours to focus on the contribution of rank regularization. The number of
training epochs is selected by tuning T E {20,40 40, 60} for both JTT (T = 60) and ours (T = 40). All the other experimental settings are fixed, including backbone architecture (ResNer-50) and optimizer
16

---

Behavior of rank-regularized encoder. Here, we present simple simulation which conceptually clarifies the impacts of rank regularization in self-supervised learning. Inspired from Chen et al. (2020) Robinson al. (202T), we create a DigitsOnSTL10 dataset in Figure ba where MNIST images are randomly selected and placed on top of the STL10 images. After self-supervised repre- sentation learning. we train two independent linear classifiers on top of the freezed representations, where we provide label of foreground MNIST digit for one classifier, and label of background STL.0 object class for the other. After training linear classifiers, we measure the ratio of MNIST classifier test accuracy to STL10 classifier test accuracy, which we treat as proxy of ratio of spu- riously correlated features to invariant features, i.c.. degree of bias in representations. Intuitively, the short-cut attribute, i.e., MNIST Under review as a conference paper at ICLR 2023
4.45
Effect-on tark.
Effective rank
K.02
Effective
6:35
6.30
1.5
Bles retlo
(a) CIFAR-10C
(b) CMNIST (VIT)
(c) CIFAR-IOC (VIT)
Figure 5: Effective rank measured with (a) CIFAR-1OC (ResNet-18), (b) CMNIST (ViT) and (c) CIFAR-10C (VIT).
(Adam), etc. Table shows that the proposed rank regularization improves the precision and recall of identified bias-conflicting samples by using sufficiently large Areg- Even though these results imply that rank regularization may be viable option for the minority mining algorithms in a supervised learning setting, we defer extensive discussions about this possibility to focus on the unsupervised learning settings.
Table 8:
Precision and recall of bias-conflicting samples identified by ERM, JTT and the proposed biased model Waterbirds dataset.
Metrics
ERM
JTT
Ours (Areg 5)
Ours (Areg 10)
Precision (%)
37.84
48.95
48.91
54.77
Recall (%)
11.67
48.75
46.67
55.01
Rank regularization with moderate level of bias. To study the compatibility of rank regular- ization with weak spurious correlations, we apply the rank regularization the moderately biased CMNIST, i... bias ratio=60%. Table[]show that the rank regularization works well in this natural setting. This implies that the rank regularization can be leveraged reveal the moderate level of bias embedded in the representations, which is supported by the empirical results of other general datasets, e.g.. Waterbirds, UTKFaaee or CelebA.
Table 9: Ablation study of rank regularization weakly biased CMNIST (Bias ratio=60%). Our rank-regularized model trained with Areg = 50. For a fair comparison, all the other experimental settings are fixed. Bias-aligned accuracy, bias-conflict accuracy, precision and recall of identified bias-conflicting samples are reported.
Methods
Align (%)
Conflict (%)
Precision (%)
Recall (%)
ERM
99.49
97.81
79.55
0.87
Ours
96.25
38.15
91.56
60.97
the proposed bias metric increases the encoder focus digit.
We measure the bias metric on the representations of ResNet-18 encoders trained by SimCLR (Chen ct al. 2020) together with rank regularization loss Areg&regs where Areg > is a balancing hyper- parameter. As denoted in the main paper, apply regularization not the output of projection
17

---

Under review as a conference paper at ICLR 2023
9
7
7
0
0
0
3
3
3
9
9
9
0
0
0
3
3
3
Acc ratio ST71O
4
4
4
0
0
0
3
333
3
7
7
7
9
9
9
STL1O 8
3 3
3
9
9
9
9
8
9
0.1 Ave
1.0
(a) Sample images
(b) Evaluation results
Figure 6: (a) Sample images from DigitsOnSTLIO dataset. (b) Test accuracy of STL10 classifier and bias metric.
networks but directly on the output of base encoder, which makes it fully agnostic to networks ar- chitecture. Figure 6b shows that the rank regularization exacerbates the "feature suppression" phe- nomenon revealed by Chen et al. (202I). The representation becomes more biased as it is trained with stronger regularization. While the overall performance self-supervised learning may be opper-bounded due to the constraint effective dimensionality (Jing et al.) (022), we observe in Figure (6 that the bias-conflict accuracy is primarily sacrificed compared to the bias-aligned accu- racy. Coupled with results in section 4 this result implies that rank regularization can amplify bias in self-supervised encoder.
C.2 ADDITIONAL RESULTS ON CELEBA
We report the results of CelebA (blonde) in here due to the limited space. Detailed information on the dataset and simulation settings is provided in the section [ Following (Sagawa al. (2019);
Liu et al.] (202T). we report worst-group and average accuracy because CelebA (blonde) includes abundant samples in (B onde Ha i r=0, Ma e=0) bias-conflcting group. The number of training
samples in each group is provided in Table 17
Table 10 shows that DeFund outperforms not only every self-supervised baseline, but also ERM, CVaR DRO, and LIF Nam et al. (2020) in linear evaluation. Table IT shows that DeFund outper- forms all the other baseline methods in semi-supervised learning. which is consistent with Table of the main paper.
Moreover, recent works unveil that CelebA (blonde) exhibits a large class imbalance which in turn correlates with a large group imbalance. Hong & Yang (202I) Idrissi et al. (2022) found that both target classes are biased toward a non-Male bias class in CelebA (blonde) which obfuscates whether the dataset is indeed biased. In this regard, Idrissi et al. (2022) observed that the simple class balancing serves as powerful baseline due to the class imbalance. This directly motivates us to alleviate the class imbalance and focus on the dataset bias itself. Following Hong & Yang (2021), randomly subsample images from (Blonde Hair=0, Male=0) group so that two target classes are biased toward different bias classes. The number of training samples before and after subsampling provided in Table TTd and [2b respectively. Table [2a]shos that DeFund outperforms JTT with respect both worst-group and average accuracy. These additional results imply that the proposed framework ensures reliable performance the presence of strong spurious correlations.
C.3 NON-CONTRASTIVE METHODS
We provide the results of proposed framework implemented based non-contrastive methods. Specifically, we leverage SimSiam (Chen & He 2021) and VICReg (Bardes et al. 2021 as base- lines. Table [3 shows that the generalization performance of both baselines can be improved with the proposed debiasing framework.
18

---

Under review as a conference paper at ICLR 2023
Table 10: (Linear evaluation) Worst-group and average accuracy (%) evaluated on CelebA (blonde). Results of ERM, CVaR DRO, LfF (Nam al. 2020) and JTT come from Table of the original JTT paper (Liu et al. 202T). Each first and second / marker represents whether the model requires information on target class or dataset bias in pretraining stage, respectively.
ERM
CVaR DRO
LIF
JTT
VICReg XX
SimSiam XX
SimCLR Xx
DeFund XX
Accuracy
Vx
/x
Vx
/x
Worst-group
47.2
64.4
77.2
81.1
10.2
1.1
17.1
77.9
Average
95.6
82.5
85.1
88.0
89.0
89.0
88.9
89.0
Table 11: (Semi-supervised learning) Worst-group and average accuracy evaluated on CelebA (blonde). Label fraction set to 10%. Each first and second /marerr represents whether the model requires information on target class or dataset bias in pretraining stage, respectively.
LNL
EnD
JTT
CVaR DRO
ERM
SimCLR
DeFund
Accuracy
//
//
/x
/X
/X
XX
XX
Worst-group (%)
40.3
41.5
79.2
49.1
30.8
12.8
80.8
Average (%)
91.1
91.0
91.0
91.0
89.1
89.1
90.0
Ma l e
Methods
Worst-group (%)
Average (%)
JTT
70.6
86.6
1558
53483
DeFund
75.1
94.8
Blonde
18417
1102
(a) Accuracy
(b) Subsampled CelebA (blonde)
Table 12: (Semi-superrised learning) (a) Worst-group and average accuracy evaluated on subsam-
pled CelebA (blonde). Label fraction is set to 10%. (b) Number of training samples for each group in subsampled CelebA (blonde). (Original dataset in Table [7d
Conflict
Unbiased
Conflict
Unbiased
SimSiam
28.15
62.63
VICReg
32.33
64.58
+ Rank reg
23.40
59.65
Rank reg
29.73
62.08
Upweight
56.12
65.44
Upweight
51.19
63.41
DeFund
60.37
67.78
DeFundvic
53.93
66.31
(a) SimSiam
(b) VICReg
Table 13: Bias-conflict accuracy and unbiased accuracy evaluated on UTKFaae (age). Last row corresponds to the full version of proposed framework which upweights misclassified samples iden- tified by biased model. Results are averaged on different random seeds. Accuracy is reported in (%).
Table 14: Comparison study the depth of biased networks. Both networks are trained with target labels on CIFAR-IOC (Bias ratio=95%).. For UTKFace (age) and CelebA (makeup), both networks pretrained with SimCLR followed by last linear layer training. Precision and recall are reported
in (%).
CIFAR-10C
UTKFace (age)
CelebA (makeup)
Networks
Precision
Recall
Precision
Recall
Precision
Recall
Shallow
64.73
59.50
55.68
69.98
27.49
33.79
ResNet-18
71.39
51.43
68.67
75.94
55.29
32.46
19

---

Under review as a conference paper at ICLR 2023
C.4 SHALLOW NETWORK
Considering the inductive bias of neural networks towards encoding low effective rank representä- tions in this paper, one may ask whether the shallow neural networks can easily learn such simple inductive bias and serve bias-capturing network. In this regard, we observe some pros and cons of using a shallow network as the bias model throughout experiments. Specifically, we use a simple convolutional network with three convolution layers counterpart of ResNet- 18, with feature map dimensions of 64, 128 and 256, each followed by ReLU activation and a batch normalization.
In the labeled setting, CIFAR-IOC in Table 14 shows tradeoff between precision and recall of the shallow network: The shallow network improves the recall of identified hard samples, i.e., the fraction of the bias-conflicting samples that are identified, because it is robust the unintended memorization due to their fewer number of hyperparameters. However, it sacrifices the precision, i.., the fraction of identified samples that are indeed cias-conflicting because its performance on the bias-aligned samples is degraded due to the low expressivity.
While the shallow network shows promising results with a simple dataset, the tradeoff worsens in the self-supervised setting with a larger dataset. Table shows that the shallow network may suffer from bad precision. is conventional wisdom that unsupervised learning benefits more from bigger models than its supervised counterpart (Chen al. 2020 Considering this, the general performance of shallow networks may deteriorate in a large-scale self-supervised learning scenario. In this case, the identified error set E contains too many false- positoositt bias-conflicting samples. While one may improve the performance with good care of hyperparameter tuning, e.g.. depth networks, learning rate, etc., it may be more laborious compared to the proposed framework, which has only few scalar hyperparameters, e.g., Areg.
C.5 HYPERPARAMETER ANALYSIS
While rank regularization biases the representations effectively, we do not argue that it is the optimal form of semantic bottleneck but rather that it highlights the unrecognized importance of controlling effective rank in encoding biased representations. In this regard, we examine the impacts of existing optimization hyperparameters on the effective rank and degree of bias in latent representations. Specifically, we investigated four candidates of bias controller through the lens of effective rank and generalizations: hardness concentration parameter 3 of hard negative sampling (Robinson et al.] 2020 temperature in InfoNCE (Oord et al. 2018) loss, strength of l2 regularization X and the number of training epochs T.
Hardness concentration parameter. Recent works (Robinson et al. 2020 Cai et al. 2020 Tabas- sum et al. 2022) stress out the importance of negative examples that are difficult to distinguish from an anchor point. Several recent works propose algorithms selecting informative negative samples, often controlled by hardness concentration parameter /3 (Robinson et al., 2020) coupled with importance sampling. [Robinson al. (2021) conducted synthetic simulation showing that increasing 3 makes instance discrimination tasks more difficult, thereby enforcing the encoder to represent more complex features. Thus we aim examine whether 3 can contribute to learn debiased representations with real-world dataset.
Temperature. A recent work on contrastive loss (Wang Liu 2021 have revealed that temperature can also control the strength of penalties on hard negative samples. Contrastive loss with high temperature turns out to be less sensitive the hard negative samples (Robinson al. 2020 202T), thereby encouraging representations to be locally clustered while the uniformity of features on the hypersphere decreases (Wang& Isola, 2020). That being said, we hypothesized that the temperature may indirectly affect the effective dimensionality of representations, where large r may decrease the effective rank.
2020p underlines the importance of la regularization and early-stopping. Sagawa et al. 2019
regularization for worst-case generalization where the naive upweighting strategy may fail it is not coupled with strong regularization that prevents deep networks from memorizing upweighted bias- caafssfoffc samples. In this regard, Liu et al. (202T) leverages capacity control techniques, e.g., strong & regularization early-stopping, to train complexity-constraie We investigate whether such regularizations can serve as a bias controller in self-supervised learning well.
bias capturing models.
20

---

Under review as a conference paper at ICLR 2023
Accuracy
0.01
0.05
0.15
SimCLR
/=0.1
Conflict
35.8
36.3
37.5
37.6
36.6
Conflict
62.0
64.2
Unbiased 65.6
65.6
66.6
66.5
66.0
Unbiased
78.9
80.7
(a) Biased linear evaluation
(b) Debiased linear evaluation
Table 15: Results of controlling concentration parameter on UTKFaae (age). Accuracy is reported in (%). (a): Accuracy of linear evaluation without upweighting bias-conflicting samples. Each value in top row indicates 3 used in pretraining. (b) Accuracy of linear evaluation with upweighting ground-truth bias-conflicting samples. Both models use Xup 10.
Table 16: Results of early-stopping UTKFace.
We denote T as the number of training epochs.
Attribute
Accuracy
T
5
T
10
T ==
15
T
20
T
25
Bias-conflict (%)
31.6
33.0
32.4
32.8
32.8
Age
Unbiased (%)
63.3
64.1
63.6
63.7
63.7
Bias-conffict (%)
54.6
54.0
53.5
53.4
54.5
Gender
Unbiased (%)
72.1
72.0
71.8
72.2
72.7
Results. We evaluate each knob on generalizations with SimCLR. Table 16 and 15a show that im- pacts of both early-stopping and concentration parameter on generalizations marginal, in trast to the observations reported in supervised learning or synthetic simulations (Robinson et al.] 2021]. However, it still remains unclear whether the debiased representations can be encoded by controlling 3. It is because the model may reach a biased solution even though it encodes debiased representations, most samples in linear evaluation are bias- aligned discussed in the main paper. To preclude such confounding relationships, we conduct debiased linear evaluation with upweight- ing ground-truth conffciifi samples. Table T5a and [5b show that there was no significant difference in the performance gain of 3 in biased and debiased linear evaluation, which implies that not enough fully debias representations.
Despite the failure of learning debiased representations with controlling 8, biased representations can be learned by controlling temperature т, and strength of & regularization in some cases. Figure 7a 7b and 7c show that effective rank, temperature and bias-conflicting accuracy are highly corre- lated each other in both UTKFaae and CelebA. It implies that the effective rank can serve as a metric of generalization performance and degree of bias in representations. While temperature control can- not be generalized to several non-contrastive learning methods (Chen & Hel 2021 Bardes et al.) 2021 Zbontar al., (02T), this results imply that the temperature may serve
effective bias &
controller for contrastive learning methods using InfoNCE loss. Moreover, stronger-than regularization also limits the effective rank and bias-conflict accuracy some extent UTKFace
(Figure 7d and 7c while it fails to do so in CelebA.
This series of observations afford us novel insight that both explicit (rank regularization) and implicit (temperature control, strong & regularization) methods offer way to train biased repre- sentations. However, still remains unclear how to directly learn debiased representations. While increasing temperature or reducing effective rank bias representations, inverse does not always hold; Abnormally small temperatures cause the contrastive loss only focus on the nearest one or two sam- ples, which heavily degenerates the performance (Wang & Liu 2021). Moreover, we found that explicit decorrelation of feature components in SimCLR does not lead to debiased representations (not shown in figure).
To sum up, we provide useful recipes on learning biased representations, where rank regularization is mainly discussed in the main paper due its intuitive insights, good performance and broad applicability. We hope these discussions facilitate in-depth studies about advanced algorithms on learning both biased and debiased representations in unsupervised manner.
21

---

Under review as a conference paper at ICLR 2023
20.
Blac-ca col--cofffi accurary (5)
42
404
60
06
27-
06
10
Effective rank
Lffective rank
42
(a) on UTKFaae (age)
(b) UTKFace (gender)
(c) on CelebA (makeup)
Bios-cortlit accuracy (56)
accuracy
45
47
44
46
47
Effective rank
Effective rank
(d) Ar2 on UTKFace (age)
(e) Xr on UTKFace (gender)
Figure 7: Analysis on temperature and strength of fa regularization λe.- Effective rank and bias- conflict accuracy are measured with varying T for (a, b. c), and Ae for (d, e). Standard deviation of bias- aligned accuracy each experiment is 1.0%. 2.8%, 0.3%. 1.3% and 1.7% in order. Perfor- mance become quickly degenerated as λę increases over 0.005 in CelebA (makeup).
D EXPERIMENTAL SETUP
D.1 DATASETS
We mainly evaluate our debiasing framework on UTKFaae (Zhang et al. 2017) and CelebA (Lu al. 2015) in which several prior works has observed poor generalization performance due to spurious correlations. Example images are presented in Figure
UTKFace. We first consider UTKKaaee dataset which is consist of human face images with varying Race, Gender and Age attributes. For each sensitive attribute, we categorize all samples into two groups Specifically, for label associated with age, we assign I samples with age 10, and 0 to samples with age 2 20 following (Hong & Yangl 2021). For label associated with race, we assign to samples with race # white, e.g., Black, Indian and Asian, and 0 to samples with race white. For label associated with gender, we assign 1 to female, and 0 to male. Based on this settings, we conduct binary classifications using (Gender, Age) and (Race, Gender) as (target, spurious) attribute pairs. Following Hong & Yang (202T), we construct biased dataset by randomly truncating portion of samples, where roughly 90% of samples are bias-aligned in our setting. Pixel resolutions and batch size are 64 x 64 and 256, respectively.
CelebA. For CelebA, we consider (HeavyMakeup) Male) and (Blonde Hair, Male) as (target, spurious) attribute pairs, following (Nam et al. 2020 Hong & Yang 2021 Sagawa et al. (019). Pixel resolutions and batch size are 256 x 256 and 128, respectively. The exact number of samples
for each prediction task is summarized in Table 17
0
$229
822
4354
534
0
25789
54460 163
57214 18117
53483
134
1346
435
5341
49601
1102
(a) UTKFace (age)
(b) UTKFacc (gender)
(c) CelebA (makeup)
(d) CelebA (blonde)
Table 17: Number of training samples for each prediction task. A for Age, for Gender, for Race, M for Male, H for HeavyMakeup and B for B1 Blonde Hair.
22

---

Under review as a conference paper at ICLR 2023
(a) UTKFace (age)
(b) UTKFace (gender)
(c) CelebA (makeup)
Figure 8: Example images of datasets. The images above the dotted line denote the bias-aligned samples, while the ones below the dotted line are the bias-conflicting samples. The group of two columns indicates each class of target attribute, i.c., Gender, Race and HeavyMakeup, respec- tively.
D.2 RANK REDUCTION ANALYSIS
For CMNIST, we use simple convolutional network with three convolution layers as a counterpart of ResNet-11, with feature map dimensions of 64, 128 and 256, each followed by a ReLU activation and a batch normalization. For CIFARIO-C and Waterbirds, we use ResNet- 18 and ResNet-50 with pretrained weights provided
PyTorch torchvision implementations, respectively. The convolu- tional network is trained for 2000 iterations using SGD optimizer with inital learning rate 0.1 and decaying by 0.1 for every 600 iterations, following Zhang et al. (202I). For CIFAR 10-C, ResNet-18 is trained for 10000 iterations using the Adam optimizer with learning rate 0.001. After training, misclassified training samples are identified as the bias- confficttin samples as Table For Water- birds in Figure 2c following the official implementation of JTT, ResNet-50 is trained for 300 epochs, and carly-stopped with referring to the validation accuracy, using SGD optimizer with learning rate 0.0001.
D.3 DEBIASING EXPERIMENTS
Architecture details. We use ResNet-18 back-bone architecture with pretrained weights provided in in PyTorch torchvis s i on implementations. For projection networks in SimCLR, we use the MLP consists of one hidden layer with feature dimension of 512, followed by ReLU activation. We employ a single linear classifier in downstream tasks for all self-supervised learning methods.
Training details. Both biased and main encoders are pretrained for 100 epochs on UTKFace, and 20 epochs on CelebA, by using Adam optimizer with learning rate of 0.0003. Cosine annealing scheduling (Loshchilov & Hutter 2016) is leveraged with warmup for the first 20 epochs on UTK- Face, and 4 epochs for CelebA.
For biased encoders, apply rank regularization with using Aveg of 0.3, 0.5, 0.01 and 0.03 for UTKFace (age), UTKFaae (gender), CelebA (makeup) and CelebA (blonde), respectively. This values are selected by tuning Xreg {0.0, 0.1, 0.3, 0.5, 1.0} for UTKFace and Aveg (0.0, 0.02, 0.03, 0.05] for CelebA. Specifically, we report the results of model with highest worst-group accuracy (for CelebA (blonde)), bias-conflicting test accuracy over those with im- proved unbiased test accuracy compared to the SimCLR baseline. Same values are consistently used for upweighting ablation study (Table 4 To emphasize the contribution of rank regularization, we do not control any other parameters, e.g.. strength of l2 regularization, temperature т, number of training epochs. Specifically, we fix r = 0.07 and λτ₃ 0.0001 for every experiment.
After pretraining, we conduct either linear evaluation or finetuning with using Xup of 10, 5, 8 and 15 for UTKFace (age), UTKFaae (gender), CelebA (makeup) and CelebA (blonde), respectively. For UTKFace and CelebA (makeup), these values are selected by tuning Aup E {5, 10) using the above-mentions decision rules, where Aup E {5. 8, 10, 15) is compared for CelebA (blonde). Same values are consistently used in ablation study (Table 4). For linear evaluation, train linear classifier on top of pretrained main encoder for 3000 iterations on UTKFace, and 5000 iterations on CelebA, with using learning rate of 0.0003 and upweighting identified bias-conflicting samples.
23

---

Under review as a conference paper at ICLR 2023
For semi-supervised learning, finetune the whole main model for 5000 iterations, with using SGD optimizer, momentum 0.9, Λe, = 0.1, learning rate of 0.0001, and Aup = 8, 15 for CelebA (makeup) and CclebA (blonde), respectively.
Data augmentations. Following SimCLR, we generate multiviewed batch with random augmen- tations of (a) random resized crop with setting the scale from 0.2 1, (b) random horizontal flip with the probability of 0.5, (c) random color jitter (change brightness, contrast, and saturation) with the probability of 0.8 and scale of 0.4, (d) random gray scaling with the probability of 0.2. In linear evaluation and finetuning, we only apply random horizontal flip. Same augmentation pipeline applied to both SimSiam and VICReg.
Baselines. For a fair comparison, we tune hyperparameters of other baselines using the same ResNet-18 back-bone architecture. We use the official implementation JTT which also includes that of CVaR DRO. Other baselines are reproduced by ourselves with referring to original papers. LNL trained for 20 epochs on UTKFaae, and 40 epochs on CelebA, with using Adam optimizer and learning rate of 0.001. For EnD, we set the multipliers for disentangling and 3 for entangling to For JTT, we tune the upweighting factor Xup E {20, 50, 80} and number of training epochs T E {30, 40, 50). following the original paper. For CV&R DRO, we tune the size of the worst-case subpopulation a E {0.1, 0.2, 0.5}. For SimSiam and VICReg, the architectures for the additional layers followed the official implemenation of each method, where the hyperparameters for the train- ing is identical to the SimCLR case. For DeFundvic-
Xreg = 0.001 for DeFund Siam and Areg = 0.1 for
24


Reviews:
[{"review_id": "pt1Mk7EYZQ", "reviewer": "Reviewer_datJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies debiasing spurious correlations and finds the inductive bias towards encoding low effective rank representations. The paper proposes a low-rank regularization method to debias. First, one bias model learns representations with low-rank regularization to amplify the bias. Then, we use the bias model to find the biased conflicting samples. Finally, use larger weights on biased conflicting samples in training the main models. Their proposed methods do not require information on target class or dataset bias information. The authors evaluate their methods on UTKFace and CelebA, and the results outperform most state-of-the-art. ", "summary_of_the_review": "The paper has its novelty, while some questions I mentioned in the Weakness part blocked me. I may adjust my rating based on the rebuttal. ", "strength_and_weaknesses": "Strength:\n1. The paper has a clear motivation and insights. The authors first find an interesting observation that the bias is a low-rank representation, and then provide a trivial but effective solution. \n2. The proposed methods have a good performance on some complex datasets, e.g. CelebA.\n\nWeakness and Questions:\n1. This is pure empirical work. There are some claims and statements that may need analysis support or more empirical support, e.g. (1) Why are Equations (2) and (3) a good way to estimate the effective rank? Why not use the nuclear norm, which is a convex relaxation of a matrix rank? (2) Why does the low-rank regularization guarantee learning more spurious correlations features rather than intrinsic features? If we only constrain the rank, how should we know which feature it tends to learn? Figure 1d is not enough to answer this question from my perspective. I need more evidence empirically (e.g. feature visualization for the bias model) or theoretically. \n2. In synthetic simulation experiments under Section2, the bias correlation is high, (larger than 95%). What will happen in a more natural setting, e.g. 80%? In Table 1, what is the meaning of Bias ratio =1%? Is it the same as the Bias ratio = 99%?\n3. In Figure 1f, the two curves for CIFAR10-c have marginal differences. \n4. From my perspective, the critical part of the method is to find bias-conflicting samples, or we may call them hard samples. I wonder whether low-rank regularization is the only solution. One potential idea is to use a shallow network (e.g. two-layer neural network) as the bias model rather than using low-rank regularization. I am curious about whether the shallow network method work, considering that it may just learn simple inductive bias as the authors claim. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lPkVUyR545", "reviewer": "Reviewer_3q4H", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "NNs are generally victim of spurious features which are strongly correlated with the label in the training set, but might not be robust for unseen samples. This paper proposes a method to reduce the reliance on spurious correlations.\n\nThe strategy is to first find examples that are bias-conflicting with a *bias model* and then train a robust, *main model* by up-weighting those examples when learning the task classifier. The bias model is trained with a rank-regularization loss which shrinks the effective rank of the the representations in the NN: a small effective rank describes a paucity of features that the network capture, and these likely correspond to easy-to-learn features in a spurious correlation setting. The authors test their approach in the unlabeled setting, using SimCLR/SiamSiam/VicREG models, regularized with their rank loss for obtaining the bias model. \n\nSummarizing the contributions, the paper mainly argues two novelties: 1) rank regularization helps in finding biased examples; 2) one can apply rank regularization also to the unlabeled setting, so that one can exploit unlabeled data for a task to prevent spurious correlations.", "summary_of_the_review": "The paper addresses an important problem. The study on the effective rank is interesting and possibly impactful. The unlabeled perspective should be discussed in more details: implications and applicability to standard pre-trained models should be discussed. Overall, the paper feels a bit dispersed at the moment: I am not sure I can confidently state that a particular technique works better than the baselines or take away a clear message from this paper.", "strength_and_weaknesses": "Strengths:\n- *The paper studies an important problem*.\n- *Novel perspective on an existing problem*: it deserves to be stated more clearly in the paper, but the idea of exploiting unlabeled data to find \"easy-to-learn\" features to robustify models in the labeled setting is interesting. Also, the evidence towards establishing a correlation between effective rank and the degree of bias in the dataset is interesting and make sense.\n\nWeaknesses:\n- *Lack of motivation and clarity of exposition*: I found the paper hard to follow. It is quite dense and contributions are hard to parse at the beginning. Let me try to summarize the two contributions. The first is to propose a rank-suppression mechanism to find biased representations. The second is the will to exploit un-labeled data to mine more plausible features that could act as spurious correlations for the task, which could help for example in the absence of a lot of labeled data. I find that these two contributions can be more clearly stated along with the motivations supporting these contributions.\n\n- *A conceptual weakness does not seem to be discussed*: spurious correlations are defined with respect to the label by definition. The paper claims that rank regularization exacerbates \"short-cuts\" or \"easy-to-learn\" features. It is reasonable to think that spurious and easy-to-learn feature match in the labeled setting, but the connection between the two might be more tricky in the unlabeled setting. Is a model capturing an easy-to-learn feature lead to always better performance against spurious correlations? I could think of a synthetic dataset in which an easy-to-learn feature is perfectly un-correlated with the labels? Would your regularization scheme just learn that feature thus making classification and thus bias-conflicting examples meaningless?\n\n- *Findings lack support*: Another main weakness of the paper is that the contributions are a bit \"dispersed\":\na) With respect to the first contribution, I would expect more extensive analysis of how rank-regularization compare (in the labeled setting) to other methods of finding bias-conflicting examples e.g. JTT, and Forgettables examples (Yaghoobzadeh et. al, 2021, \"Increasing robustness to spurious correlations...\"). Right now the authors compare with ERM in Table 1, without giving much details on the experimental setting. Then, the authors could go on in justifying why rank-regularization can be used in the unlabeled setting (differently from other approaches that require labels).\nb) When talking about un-labeled data, it is natural to ask whether standard pre-trained self-supervised/supervised models can be used in this setting. In the paper, the authors limit themselves to the setting where the encoder is trained only on *unlabeled* data for the task, but what if we are starting from a pre-trained model (classical setting of JTT / Deep Feature Reweighting)? How can the approach be applied?\nc) The authors test their model in CelebA and UTKFace. Waterbirds and CelebA are classical settings for this problem, and the classical CelebA split is (hair, gender). Why do the authors skipped these settings?\n\n- *Weak results in Table 2*: In Table 2, the scores for DeFund are bolded but are less than ERM on some columns and JTT on most of the columns. I understand the model is not using Y in the pre-training phase, but the question one could ask is *why not* given that it is using those labels in the fine-tuning stage and thus the exact same information is used to report those numbers?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "hb8Zgzrvwo", "reviewer": "Reviewer_cTbQ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a debiasing framework using low-rank regularization and self-supervised learning techinque. Specifically, Authors find that rank regularization may force deep networks to focus on spurious attributes and the biased models with strong regularization can effectively probe out the bias-conflicting samples in the training set which is robust to the unintended memorization of bias-conflicting samples. Inspired by the obervation, authors desings two-stage training framework to debias the model. Experiments are verified on UTKface (age), UTKFace (gender) and CelebA to verify its effectiveness.", "summary_of_the_review": "- I am not very familiar with this direction, along with the related work. From my perspective, I think this paper has some intersting points. Thus I give the score of 6 at the current time and will make my final decision aftering reading other reviews.", "strength_and_weaknesses": "**Strength:**\n\n- Well motivated and intersting idea. The experiements and ablation studies are designed carefully.\n\n- The improvemnts are significant observed from Linear evaluation and Semi-supervised learning experiments.\n\n**Weaknesses:**\n\n- It would be better to report error bars to show the stableness of the result.\n\n- If this two-stage training framework would increase the training time compared to other approaches?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "PPWB32OV8E", "reviewer": "Reviewer_rRxe", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a self-supervised method for debiasing models by building a highly biased model, then leveraging that model to select the training set to create an unbiased model. The method does not require any labels of the bias attributes, as many methods do, or even the target classes or attributes, which makes it quite novel. The experiments show that the method outperforms other unsupervised algorithms, even those leveraging class labels.", "summary_of_the_review": "The paper presents a novel solution to the difficult problem of self-supervised debiasing, an important problem that has received little attention so far. The writing is clear, and the experiments are thorough and convincing. The paper should generate significant interest for those working in bias mitigation.", "strength_and_weaknesses": "Strengths\n\nThe goal of the work is to produce an unbiased classifier without having any labels of attributes or classes, which are typically required for debiasing methods. In this problem setup, the method must determine the relevant attributes and whether they are biased, jointly and without any labels i.e. fully unsupervised. This is a very challenging and interesting problem setup that builds upon the recent breakthroughs in unsupervised representation learning.\n\nThe method is based on the observation that model bias seems to be inversely correlated with the entropy of the representation learned by the model. The hypothesis is that spurious correlations in training data lead to a concentration of model attention on a few features, leading to poor generalization performance.\n\nThe approach seems quite novel. Other popular methods are based on GANs or other adversarial techniques, rather than unsupervised learning of bias. The idea of creating a highly biased encoder, based on rank analysis, is clever and seems to be supported by the results.\nThe experiments underlying Fig. 1 are very useful to illustrate the insight and motivation underlying the approach. Using controlled settings in simplified datasets is helpful here to illustrate the best-case scenarios that provide evidence of the theory. Many papers lack such intermediate results and rely solely on improved results of the end task to validate elements of the approach.\n\nThe experiments are performed on face datasets using combinations of standard attributes including gender, age, race, heavy_makeup that have been annotated previously. A comprehensive set of baselines are compared against, including self-supervised methods, supervised debiasing methods and algorithms that select training sets for debiasing. The proposed method handily outperforms other unsupervised methods, and supervised methods in some cases, which is impressive. The experimental protocol follows that of recent works for consistency.\n\nWeaknesses\n\nIn fig. 1 the bias ratios seem very high, 95% or greater, and the plot shows that the amount of change from 0 to 95% is only slightly larger than the change from 95 to 98%, i.e. there is little change in the effective rank when the ratio ranges from 0 to close to 95%. At face value this seems to reveal a significant disadvantage in using the effective rank as it is insensitive to the bias ratio until it becomes extreme. Is that the case?\n\nIs the matrix Z in sec. 2.3, formed from the input vectors and their corresponding feature vectors, the matrix that is used in the rank computations in Fig. 1? If so this should be stated earlier, in the discussion of Fig. 1, with a forward reference. As written the paper is unclear on what the rank of a representation might be until sec. 2.3.\n\nThere are important related works in discovering biases without attribute labels that are not cited:\n\nLang, O., Gandelsman, Y., Yarom, M., Wald, Y., Elidan, G., Hassidim, A., Freeman, W.T., Isola, P., Globerson, A., Irani, M., Mosseri, I. “Explaining in Style: Training a GAN to explain a classifier in StyleSpace.” ICCV 2021.\n\nLi, Z., Xu, C. “Discover the Unknown Biased Attribute of an Image Classifier.” ICCV 2021.\n\nAnother one, in ECCV 2022, published after the ICLR submission deadline:\n“Discover and Mitigate Unknown Biases with Debiasing Alternate Networks”\n\nThese papers all rely on GANs or other adversarial techniques, and rely on target labels to some extent. Hence they do not infringe on the novelty of the proposed method. However, the discussion of related work should be expanded to include these, and to better differentiate the novel claims.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject