Group Robust Classification Without Any Group Information
Authors: Christos Tsirigotis
Joao Monteiro
Pau Rodriguez
David Vazquez
Aaron Courville
Venue: NeurIPS 2023 poster (2023)
Keywords: out-of-distribution generalization
robustness
fairness
spurious correlations
systematic generalization
model selection
TL;DR: We use pretrained models to define an entirely bias-unsupervised training and validation methodology for group robust classification

Abstract: Empirical risk minimization (ERM) is sensitive to spurious correlations present in training data, which poses a significant risk when deploying systems trained under this paradigm in high-stake applications. While the existing literature focuses on maximizing group-balanced or worst-group accuracy, estimating these quantities is hindered by costly bias annotations. This study contends that current bias-unsupervised approaches to group robustness continue to rely on group information to achieve optimal performance. Firstly, these methods implicitly assume that all group combinations are represented during training. To illustrate this, we introduce a systematic generalization task on the MPI3D dataset and discover that current algorithms fail to improve the ERM baseline when combinations of observed attribute values are missing. Secondly, bias labels are still crucial for effective model selection, restricting the practicality of these methods in real-world scenarios. To address these limitations, we propose a revised methodology for training and validating debiased models in an entirely bias-unsupervised manner. We achieve this by employing pretrained self-supervised models to reliably extract bias information, which enables the integration of a logit adjustment training loss with our validation criterion. Our empirical analysis on synthetic and real-world tasks provides evidence that our approach overcomes the identified challenges and consistently enhances robust accuracy, attaining performance which is competitive with or outperforms that of state-of-the-art methods, which, conversely, rely on bias labels for validation.

Group Robust Classification Without Any Group Information
Christos Tsirigotis"
Joao Monteiro ServiceNow Research
Pau Rodriguez Apple MLR
Université de Montréal, Mila, ServiceNow Research
David Vazquez
Aaron Courville'
ServiceNow Research
Université de Montréal, Mila, CIFAR CAI Chair
Abstract
Empirical risk minimization (ERM) is sensitive to spurious correlations in the training data, which poses significant risk when deploying systems trained under this paradigm in high-stake applications. While the existing literature focuses on maximizing group-balanced worst-group accuracy, estimating these accuracies is hindered by costly bias annotations. This study contends that current bias- unsupervised approaches group robustness continue rely on group information to achieve optimal performance. Firstly, these methods implicitly assume that all group combinations are represented during training. To illustrate this, we introduce systematic generalization task on the MPI3D dataset and discover that current algorithms fail to improve the ERM baseline when combinations observed attribute values are missing. Secondly, bias labels are still crucial for effective model selection, restricting the practicality of these methods real-world scenarios. To address these limitations, training and validating debiased models in an entirely hias-unsuperrvsed manner. We achieve this by employing pretrained self-supervised models to reliably extract bias information, which enables the integration of a logit adjustment training loss with validation criterion. Our empirical analysis synthetic and real-world tasks provides evidence that our approach overcomes the identified challenges and consistently enhances robust accuracy, attaining performance which is competitive with or outperforms that of state-of-the-art methods, which, conversely, rely on bias labels for validation.
propose revised methodology for
Introduction
Supervised learning algorithms typically rely on the empirical risk minimization (ERM) paradigm minimizing the average loss on training set. The ERM paradigm operates under the assumption that the training data a representative sample of the true data distribution [67]. Consequently, models that achieve low expected loss can still be unfair when the model is tasked to predict outcomes for
underrepresented groups 38 10 66), and prone to rely on spurious correlations between target annotations and generative attributes of the training data [32
62 that do not hold up under more general testing conditions. As automated predictors are deployed failure-critical applications [4 interact with society, where fairness must be guaranteed [13] 36), robustness and faimess become requirements that standard learning strategies do not satisfy.
Work done during internship ServiceNow Research. Author correspondence sstriiigoo@mmil quebec This work was also funded, part, from A. Courville's Sony Research Award. Courville also acknowledges support from his Canada Research Chair.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

These limitations have motivated the search for alternatives that perform uniformly across different data subgroups 669 6T or, equivalently, that rely less on spurious features [2] or "shortcuts" 28 Part of solving this problem includes making the correct assumptions about
data, which should consider spurious statistical correlations due to bias in its generative pro- cess. A popular example of this issue is the cow vs. camel classification problem [2] As one might expect, pictures of cows very often contain grass background, while camels are usually depicted in a desert. As such, a binary classifier that pre- dicts whether there is grass in the
Phase
Phase 2
Biased pred.
Predictions
Ba anoed Acouracy
Blased Logits
Logits
Train
Lineer
Adjust
Linear
Validation
background of an image could achieve high prediction accu- racy on top of natural images of the two animals. However, such a classifier would fail whenever the background changes from the typical occurrences.
Contusion Matix
lahels
Train
Pro-trained SSL Model
SSL Model
Inttal 20
Train
Valiation
Train
Validation
Data
Data
Data
While progress has been made, existing methods to improve ro- bustness the least frequent at- tribute combinations (e.g., cow on sand), require knowledge about the bias attributes (bias-supervision) either during training 61,1, or validation [57 48
Figure 1: Unsupervised Logit Adjustment (uLA), We train linear classifier on top of an SSL pre-trained model to obtain biased predictions. These predictions are then leveraged train debiased model. No bias information is used during training or cross-validation.
50
This limits their applicability in practical scenarios where bias annotations could be too costly or impossible to obtain. Even in cases where they can be obtained somewhat efficiently, human annota- tors might be biased themselves sensitive annotations may not be readily available due to privacy concerns (6).
Moreover, access to group information is rarely satisfiable natural data, where curse of generative dimensionality implics that given the high number of attributes controlling the generative process, no realistic finite sample can cover all possible exponential in number- combinations of such attributes. In fact, many group robustness algorithms that rely on data rebalancing or loss reweighting techniques [61 48 assume that all group combinations are present during training. However, this condition is not satisfied in systematic generalization tasks, where certain data subgroups that are present in test data, are absent from training data. This motivates us to study the extent that training algorithms can successfully operate using group information implicitly present in the training dataset.
Specifically, in the case of these challenging systematically split datasets, study the generalization of models on unseen combinations of attributes that were independently observed during training. This combinatorial type of generalization draws its name from the cognitive property of systematicity 26 In deep learning, this question was introduced for the first time by Lake and Baroni [46), who studied the systematicity of RNN models in sequence prediction tasks. Here, raise the same question classification. To do so, we introduce a benchmark consisting of systematic splits from the MPI3D (thereby named sMP13D) image dataset [29] In particular, we use the 'real' split of MPI3D which consists of photographs of a robotic arm that has a colored rigid object attached to its end effector. The images are captured in way that controls their generative attributes, such as the shape of the rigid object or the position of the robotic arm. We use it to ask the following question: given a 'shape' classifier that has been trained on objects of all possible shapes and colors but only a subset of their combinations (c.g., red cubes and blue spheres), how would it perform for new color-shape combination (c.g.. blue cubes)? In this example, the 'color' attribute plays the role of a bias, obstructing view of all possible colors that cubes could have. In Appendix D we describe in more detail the construction of the SMPI3D task, while in Figure 2d we illustrate an example of a systematic split.
As discussed above, existing state-of-the-art methods suffer from two limitations. The first one, as demonstrated by Asgari et al. 3 Table 1], is that the robust accuracy of many recent training algorithms, which do not demand any bias annotations during training, degrades severely when there is no access to bias labels during model selection. This highlights that robust algorithms should prescribe a way of performing unsupeusuer validation, assuming the same access to i.idd data

---

sphere
3
Pyramndd
03
bexagon
gylinder
0
13
8
8
3
cube
come
8
83
Color
(a) CMNIST
(b) SMPI3D
(c) CELEBA
(d) A split of SMPI3D
Figure 2: Left: Example tasks. Circled in dashed lines are samples from training split, exhibiting statistically major attribute groups. Those outside are example test samples, where all groups are equally considered. A classifier trained on a biased training set may misclassify a bias-conflicting test sample- recognizing 'red three' 'zero' (2a) or 'blue sphere' as 'cone' (2b) or be unfair to the sensitive gender attribute when tasked to classify facial attribute, e.g. hair color (2c). Right: An
example a systematic split. the number of color values per shape. 'Crosses' represent groups
used to sample the training and validation splits, while 'circles' are entirely out-of-distribution.
resources as training. The second, revealed by our study on SMPI3D (Section 4.2p. is that in most cases these methods fail to improve over the ERM baseline, which fails to systematically generalize.
To address these issues, our approach, summarized in Figure I is based on pretraining a base encoder using self-supervised learning 7 to extract proxy for the missing bias labels and to provide an initialization point for finetuning a debiased model. Pretraining a proxy for the bias variable enables us use this network for two purposes: first, to train a group-robust network and, second, define validation criterion for robust model selection. The debiasing training algorithm is based on the logit adjustment paradigm (54). Our entirely bias-unsupervised methodology to group robustness using logit adjustment, which we call ULA, is able to compete with or outperform state-of sitee--f-theeeee art counterparts, that otherwise utilize bias labels during model selection, synthetic and real-world benchmarks. At the same time, it is the only method to consistently offer improvements over the ERM baseline the SMPI3D systematic generalization task; thus effectively tackling the identified challenges about explicit implicit use of group information.
2 Preliminaries
Problem Formulation. Consider a multi-class classification task, where C X' the input variable and Y c V, the categorical target variable with [}*] = K classes. We have access to a dataset of observations D == {I yr 1 sampled i.i.d. from underlying data distribution Paata (X, Y) over X V. The setting above may become problematic once we consider that the deployment data Drest are sampled from different testing distribution: Prest * Pduta- In other words, assume that there are two data generating processes; one which generates development data (D and Daalid, where Dvalid is used for validation) according to Pdaaa and which generates deployment data (Dtest) according to Prest
In further detail, focus particular transfer learning problem from Pian to Prest, which is due to a distribution shift in attributes which participate in the generative process of the input variable X. Our study considers anti-causal prediction tasks for which the target variable Y is of the generative attributee of X, and C 2 another (possibly unobserved) categorical generative attribute with | L classes. Z is marginally independent to Y under Pres. but it might not be under Pdata For this reason, we say that the variables Y and Z are spuriously correlated in training and validation data, and Z will also be referred as the bias attribute. Under this setting, group (y, 2) [K] [Z] defined as a combination of target and bias attribute values, and group robustness can be formulated as transfer learning problem from Pdota Prest under the following two assumptions relating the two joint distributions over X, Y and Z:
Piest (v. 2) xx 1 Supp Pen (Y) (X)x Para (2) (v, 2),
(1)
and Piest (2ly, 2) = Pdata (x) y, 2).
(2)
where ss(s) if E else 0. the characteristic function of a set S, and Supp p denotes the support set of distribution p. Relation asserts that, during test time, the target and bias variables
3

---

are distributed uniformly over the product of their respective marginal supports under Pdata- In other words, all combinations of observed attributes values are considered equally, This also implies that Y and Z are marginally independent under Ptes On the other hand, relation assumes the invariance of mechanism, which an assumption typically found in disentangled causal process literature [64] These two assumptions underlie the group robustness literature, in that they are equivalent to the evaluation of classifiers under a popular robust performance criterion which is described below.
Performance Criteria. Let (a;; f) == argmax, f(z)y be the predictions of a scoring function f:X -X-RP -KK The accuracy of f under Prest corresponds to the group-balances accuracy
Accy (fiPros)
E y.erepest 2trePhod (1212))
1 i(=f( KL
1
Σ v.e
E
1 - j(4; f)),
(3)
x
Duma (v.a))
which is frequently used performance metric in literature 61
48 50 Essentially, this performance
criterion first separately computes the accuracy for samples of each group (v, 2) and then averages them. The averaging operation per individual group accuracy directly stems from the uniformity assumption in Pess (rel. [ and implements group fairness notion: care equally about performing well in all groups. Another popular group robustness criterion is the worst-group accuracy [61 which substitutes the average accuracy over individual groups with the minimum (worst) accuracy.
As described in Section S there exist different approaches to improve group fairesss 6 50 that depend on knowing biases for cross-validation. Here we focus on logit adjustment techniques like the one proposed by Liu et al. (50), which co-train two models where one corrects the biases of the other. In the following sections, we show how, by decoupling the training of the biased model from the bias-corrected model, it is possible to obtain a proxy criterion that can be used for cross- -ros-vvaliattion without knowing the bias attribute.
Logit Adjustment was originally developed as a technique for supervised learning under class- imbalanced 2 53 or
(54) data. For clarity of presentation, we will assume for the moment that we have access to bias labels. We describe SLA, a bias- suaeevude training process with logit adjustment. Let he X' + RD' be the model that we want to train for group robustness, such as a neural network parameterized by @ implementing a function from samples in X' to the unnormalized logit space of Y. We then train by minimizing the average cross-entropy loss for the following logit adjusted model
po(y)ar2) x exp(ho(=)) + log Panta (yl=))
We estimate the conditional Paan 2))2) directly from the available training data, for example via empirical frequencies of the finite number of groups (y, 2) E [K] x [L]. Finally, during inference, we cancel out the contribution of the logit bias term Pdata (y) (2) and predict only according to the optimized neural network hg..
Note that, since Y is categorical variable, log (vl=) takes non- ponitiisiiv values. For this reason, we can intuitively interpret logit adjustment as soft masking operation for outputs of he which are unlikely in the training data when we have observed z. In this way, we account for the dependency of Y to Z which spuriously exists in the training distribution. By fitting the cross-entropy objective under logit adjustment, the network he has to model the remaining relations for Y X that are not spurious since those are already accounted for. An expressive enough model class in Equation (4) can achieve this, assuming further that the likelihood ratio (=)()))).))) Paulyl is independent (a/])
of z. In appendix [ we derive SLA from first principles, bridging the gap between the well- studied application of this technique to the class-imbalance problem with its application to group robustness. Moreover, we demonstrate that SLA is well-justified procedure by proving the following proposition.
Proposition 2.1 (sLA optimizes the group-balanced accuracy). Under the assumption that the hypothesis class pe (yl [r, 2) (eq. 4 contains Paaa (3) 2). the minimizer network hg- of the cross- entropy loss maximizes a lower bound the group- balanebalanced accuracy Accy |x (hg- Puss).
Systematic Generalization. We expect SLA models to systematically generalize, as they optimize for the transfer learning problem defined by Equations (1) and (2p. As Figure suggests, systematic generalization is just an extreme case of such problems, for which samples from some combinations
*Generally, the variables, which are uniform over support which factorizes into cartesian product of their marginal supports, independent: clxxy(æ, v) cl(2. E X y) cl(# X E y).

---

have Py 2) = O during training, however they populate the test set with Presi 2) > 0. As a coumterexample, the setting by Ahmed al. [ does not correspond to a systematic generalization task, as we define it. In what they refer to as systematic splits of colored MNIST, all possible combinations of color and digit are exposed to the model during its training. while in our SMPI3D some color and shape combinations are only revealed during test-time. This detail makes our setting significantly more challenging, as the conditions for applying importance sampling are not met, as Supp Pdota (Y, 2) 2 Supp Prest (Y, Z) 44 Chapter 5]. This means that a simple re- re-weighting of the per-sample loss with (B,3) in order to estimate Bea (y, he (2))], is not appropriate for the task.
Self-Supervised Learning (SSL) refers to a collection of methods 7 in unsupervised representation learning where unlabeled data provides the supervision by defining tasks where model is asked to predict some part of the input from other parts, for example by contrasting independently transformed
versions of data samples [14 35 [5]. Methods like SimCLR [14]. MoCo [35,. B 30 and
Barlow Twins [73] provide solutions for low-data generalization, robustness, as well as transferability of learnt representations for image classification. Although self-supervised learning methods are generally more robust to distribution shifts than purely supervised methods [49]. they can still be affected by significant shifts, for example in long-tailed data learning (5 63 In this work, explore the limits and utility of SSL in group robust classification. An extended preliminary discussion about self-supervised algorithms can be found in Appendix
B
3 ULA: Bias-unsupervised Logit Adjustment
Here, we introduce bias-unsupervised logit adjustment (ULA), a logit correction approach which improves on the work of Liu et al. [50] by removing dependency on explicit bias annotations both during training and validation. For this reason, we create a proxy variable for the bias via the predictions of pretrained network using SSL. The advantage of pretraining, over co-training a bias network (57 17 50], is two-fold: First, we can reuse the fixed bias proxy network to define validation criterion for group robustness. This enables us to perform hyperparameter search, but also informs us about when to stop training the debiased model, which is critical for optimal has demonstrated, in the case of training with bias-supervised data, that a linear classifier on top of pretrained base models provides with substantial group-robustnest improvements. Here, we initialize the debiased model from the pretrained base model and finetune it using logit adjustment. Figure S provides a summary of our approach and its Further training details are discussed in the following.
performance 48 Second, recent literature 39
pseudocode can be found in Appendix C
3.1 Bias-unnu Training
Biased network: pretrain with SSL. We start by training a base model, fusse. using an SSL method on the unlabeled data of the training set D. We decide on the hyperparameterss of the SSL algorithm (learning rate, weight decay, temperature, augmentations and potentially others) by maximizing the ii.d. validation accuracy of an online linear classifier, which probes the representations of the base model. Afterwards, we train a linear classifier Ie on top of a frozen ftase and against target variable labels Y using a vanilla cross-entropy loss, in order to derive proxy for the bias variable. Finally, we retrieve the composite neural network hbias = 96 o frase. and use its predictions g( [2; hbsss) as proxy for the missing bias variable observations. For the same purpose, Nam et al. [57 trained bias-extracting network by employing Generalized Cross Entropy (75). an objective which provides robustness to noisy labels. Since the goal of the proxy network is to predict the spurious attribute, bias-confficting or minority samples can be perceived as mislabeled data points. Here, we follow these observations and leverage the representations learnt with SSL which, when composed with a low-capacity (linear) classifier, provide with a model that is more robust to label-noise [71]. In addition to deriving a bias proxy, we will also use faase as initialization point in the parameter space for training the debiased model.
In this paper, we have chosen to use MoCoV2+ 15 as the SSL algorithm for the image classification tasks we consider. We make this choice since contrastive learning algorithms, like the MoCo family 35 15, , offer relatively stable training since they explicitly prevent representation collapse in their loss function (see Appendix B). However, as we demonstrate in Section 4.3 our method is not restrained to the use of particular SSL algorithm.
5

---

Debiased network: logit adjustment. In the absence of bias labels during training, we need substitute for the estimate Pram (Y, Z) in eq. 4 We use the predictions Ybias of the bias proxy network Ihii to that end. The resulting joint distribution between the target variable and the biased network's predictions, (y Ybiss), can be thought of as a soft confusion matrix of hiias and can be computed using the available training data with
Paatu (y: Mbias)
Σ [D] E'D
Pbias (Yuias (x')1(y - y²),
(5)
where Phias (Ybins x)) x exp(hiias (2) Drine /T) is the biased model conditional. Note that Pbias (Ubias (x) is post-hoc calibrated by temperature hyperparameter т. As we rely the bi- ased network to approximate the spurious correlation structure described in Pdata (Y, 2), it is crucial
that the predicted conditional probabilities of the biased network are calibrated correctly [31 56 53 Afterwards, we are ready to begin training a debiased network with logit adjustment as in Section 2 using only the predictions of hbias as the bias variable. The debiased network is initialized at the composition of random linear classifier with the pretrained network fbase, and during training we finetune while adjusting its output logits by
po(y 2)) xx exp (ho(z) + 1 log Paata (y (a:; Mbbiss)
(6)
where 1) 2 is hyperparameter controlling the strength of the additive logit bias. Notice that for ? we fall back ERM training. By tuning 17 we can mitigate calibration errors of the debiased model [53]. similar to what T does for the bias proxy. Selected hyperparameter configurations can be found in Appendix
3.2 Bias-unsupervisedd Validation
We re-purpose the pretrained biased classifier hpias so that training no longer requires bias-annotated validation data for model selection. Our bias-unsupervised validation criterion calculates balanced accuracy across pairs (y, Ubias) E [K] x [K] of true labels and biased classifier predictions. In practice, we compute
BalAcc(f; hbias)
Σ
= 0(x;;(;;
K2
Sy.bbia
M-Bhiin
Sy.rrim = [(x Vi) E Dealid y/ y and (z;; hbias) = ybias)
where Sy. Sylhis are partitions of Dvalid based on the value of predictions of haias on sample x; and its ground-truth target label ys. This corresponds to a form of group-balanced accuracy. Alternativeyy, we could also calculate a form of worst-group accuracy by taking the minimum across SymMins We find that worst-case validation is more suitable for tasks with small number of classes K.
During training, we evaluate models at every epoch and we select the one that maximizes our validation score across the duration of training trial. In addition, we use this criterion tune hyperparameters. In particular, for each task we tune learning rate, weight decay, logit adjustment strength coefficient 7), calibration temperature and, in addition, the number of pretraining steps for the SSL backhone- whenever it is applicable- and for the linear classification probe of the bias proxy network. Fig. depicts our bias-unsueervised training and validation procedures.
Experiments
Datasets.
The tasks we consider all specific instances of the setup above (see Fig. and
Section 2 This spans group robustness challenges like colored MNIST [57] CMNIST], corrupted CIFAR TO 37 CCIFARI0] and WATERBIRDS (62), fair classification benchmarks like CELEBA [1]] and systematic generalization tasks such as the contributed SMPI3D. Details about their construction can be found in Appendix [
Training Setup. For CMNIST, we train 3-hidden layer MLP, while we use a 18 34 for CCIFAR 10 and SMP13D, and a ResNet50 for WATERBIRDS and CELEBA. For all datasets except WATERBIRDS, we pretrain the base model with the MoCoV2+ [05] process, while training of the linear probe for the bias network and finetuning for the logit adjusted debiased network happen with

---

Bias Labels
CMNIST
CCIFARIO
Train
Val
0.3%
1.0%
2.0%
5.0%
0.5%
1.0%
2.0%
5.0%
61
GROUPDRO
/
v
63.12
68.78
76.30
84.20
33.44
38.30
45.81
57.32
S7I
LFF
x
/
42.50
61.89_ca7
71.03
80.57a
28.57a: 20
33.07
39.91a0.0
50.27
1
DEA
/
65.22s1.4
81.73-234
84.79
89.
39.0641.0
29.95
26.49
41.78a2.
51.13
60 L.C
x
/
71.25ga
82.25g9.1
86.21
91.1610.0
34.5620.c
27.34
47.81gm00
54.55g1
ERM
X
x
35.19
52.09
65.86ga.5e
82.17.0.7
23.08a:::
25.82
30.06.o.7
39.42.0.04
ULA (ours)
X
75.13
81.80-141
81.79-1.00
92.79.a.a
31.39h:.144
62.49
63.88.1.07
74.49
Table 1: Results using datasets from Chu et al. [17] for various % of bins-conf examples in the training set. We report avg. group-balanced test accuracy (%) and std. dev. over 5 seeds. "Results from Chu et al. [7]. "Results from Liu et al. (50)
Bias Labels
WATERBIRDS
CELEBA
Train
Val
i...d.
worst group
i.i.d.
worst group
61
GROUPDRO'
93.5
91.4
92.9
88.9
9
ERM
X
/
97.6
86.7
93.1
77.8
57
LFF"
X
/
97.5
75.2
86.0
77.2
JTT
X
/
93.6
86.0
88.0
81.1
50
LC
X
V
90.5+1.1
88.1 1
ERM
X
X
97.3
72.6
95.6
47.2
Bardenhagen et al. 9
X
X
97.5
78.5
88.0
78.9
MASK TUNE
X
X
93.0+0.7
86.411.9
91.3:20.1
78.0+1.2
ULA (ours)
X
X
91.5+0.7
86.1
93.940.2
86.5+3.7
Table Results on WATERBIRDS and CELEBA. We report avg. test accuracy (%) and std. dev. over 5 seeds. "Results from Liu et al. [88.
AdamW [52] optimizer. For WATERBIRDS instead, we leverage a base model which was pretrained on Imagenet [60]. following baselines in the literature for fair comparison, and we finetune it using SGD, Finally, for CMNIST, CCIFAR 10 and SMP13D we our group-balanced bias-unsupervised validation criterion, whereas for WATERBIRDS and CELEBA the worst-group version. Further details described in Appendix [
Baselines. We compare uLA with vanilla ERM and diverse set of group robustness techniques
described in Section 5 GROUPDRO 66 provides with fully bias-supervised baseline, while
[50 and DFA 17 are bias-unsupervised during training although they require bias annotations during validation to achieve robust optimal performance. We also consider two fully bias- maas-unuupervised methods: Bardenhagen et al. 99 propose early stopping networks to derive proxy for bias unsupersuee validation, and MASKTUNE [3 which provide competitive results under fully bias-unsupervised benchmarks without performing any validation procedure.
LFF [57], JTT [88]
4.1 Results on Benchmarks
In Table D we report the group-balanced accuracy on CMNIST and CCIFAR10. across different percentages of bias-conficctnng examples in the training set. For CMNIST, we observe that our method performs overall competitively against LFF, DFA and LC, even though these baselines use bias annotations during model selection. On the other hand, for CCIFAR 10, we observe significantly improved group robust performance for 3/4 difficulty levels. The highest difference is observed at the 1.0% task, where our method outperforms GROUPDRO by about 24% absolute increase in group-balance accuracy.
In Table 2 we observe worst-group accuracy results in the more challenging WATERBIRDS and CELEBA datasets. In both cases, our approach performs again competitively among unssupusueee training algorithms that leverage bias information during validation, falling slightly behind LC, which the best out of the ones considered. Notably, our approach still the best performing fully unaprrssueer method, outperforming the supervised learning pretraining validation scheme of Bardenhagen et al. [9) and performing on par with MASKTUNE on WATERBIRDS and better than on CELEBA by R=8% absolute gorst-group accuracy.

---

Bias Labels
SMP13D (various C colors shape)
Train & Val
2
3
% i..dd samples
33.33
50.00
66.66
83.33
61 GROUPDRO
/
31.23:
46.0114.18
69.6827.82
82.18
ERM
X
31.91=1.67
47.68
71.94=7.71
83.
63.10+8.64
48 JTT
X
31.89=0.88
48.9312.04
67.78-3.14
83,31
150
LC
X
31.29
45.01
61.67
94.62..0.88
ULA (ours)
X
59.58
80.5313.16
91.11
98.0510.64
Table 3: Results SMPI3D for various numbers of C colors per shape value in the training (see Appendix D.. We report avg. group-balanced accuracy (%) and std. dev. over 5 seeds & dataset generations. "GroupMix augmentation was not used for fairness of comparison.
CelebA
CCIFARIO
sMP13D
20%
0%
from scratch
linear
finetune
from scratch
linear
finetune
from scratch
linear
fineture
Figure 3: Ablations on the influence of SSL pretraining and on the use of VANILLA process, SLA, or uLA for training against the downstream task. For CCIFAR10 (1%) and SMPP3D (C 3) we report the group-balanced test accuracy, whereas for CELEBA the worst-group. We report avg. accuracies and std. dev. over 5 seeds.
4.2 Systematic Generalization
SMP13D is our contributed task which we use to study combinatorial systematicity in classifiers. With this benehmark, we aim to study the ability classifiers generalize samples generated from novel combinations of observed generative attributes values, which under Pdata have O probability. The target task is to classify the shape of an object, which is spuriously correlated with its color. We devise 4 difficulty levels for this task which we denote by C, the number of color values present in the training set per shape. In Fig. 2d we display a possible split between in-distribution and o.o.d.
combinations of attributes for C 4. Details about its construction are given in Appendix D
Results, In 3 we further validate our approach on our contributed systematic generalization task. Under this setting, all bias-unsupervised approaches are evaluated fairly since their models are validated with exactly the same data resources; access bias labels cannot give model selection advantage to any algorithm since there are no o.o.d. samples in the validation (just like in the training) split. Our method is the only one which consistently offers group-balanced accuracy improvements across difficulty levels, demonstrating generalization to o.o.d. samples. On the other hand, GROUPDRO, vanilla ERM, JTT and LC are not able increase group-balanced accuracy over the percentage of i..d. samples present in the balanced test set.
4.3 Ablation Studies
We perform set of studies understand better the efficacy of our approach. In Fig. we ablate the choice of training paradigm and the influence of a pretrained base model using SSL to the robust performance of trained model the target task. The paradigms we choose are among vanilla cross-entropy minimization, ULA and SLA (bias-superrised logit adjustment see Section 2). For this ablation, a bias- supervised validation procedure was used for comparison against the fully bias- superviee SLA baseline. We find that finetuning the pretrained base model gives the best performance across training paradigms and tasks. Second, forn CELEBA and CCIFAR 10, vanilla cross-entropy finetuning does not offer stark performance improvement over vanilla training from scratch. Only when we apply a logit adjustment training procedure, we are able take significant advantage of the learnt representation space. At the same time, the accuracy gaps between ULA and

---

Group-balanced Test Acc.
ERM GROUPDRO
25.82++0.33
38.30
ULA W. McCoV2+
62.49410.0
uLA W. BYOL
59.73+22003
ULA W. BARLOWTWINS
50.08
Table 4: Ablation of SSL pretraining methods for the backbone model. Default rj = 1.0 and T = 1.0 used. We pretrain for 1000 epochs, and train the linear head of the bias proxy for 100 epochs. We report avg. group-balances test accuracy (%) and std. dev. over seeds on CCIFAR10 (1%).
Group Balanced Test Accoracy
Sample Size
60-
Supervised pretrairx 131
Pretraining method
Pearson correlation
SSL pretrain:
End-to-end supervised
0.690 (0.568. 0.770)
SSL + linear head
0.819 (0.721. 0.883)
Table 5:
Effect of pretraining method for a bias proxy network on results of hyperparameter search using our validation criterion. Each train- ing trial's hyperparameters the same prior. Pretraining with SSL results in trials whose best validation criterion correlates better with selected model's 's group-balanced test accuracy on CCIFARIO (1%). In parenthesis, we compute 95% confidence intervals.
10-
sampled from
Best Trial Validation Accuracy
Figure 4: Validation vs test on CCIFARIO (1%).
SLA are small which indicates that we are able to recover correctly the bias attribute with the proxy network. On the other hand, for SMPI3D the gaps between ULA and sLA are large. This shows that, in the systematic generalization case, bias extraction remains an open challenge for future work, and that improvement over baseline procedures in Table 3 due to pretraining with SSL.
In addition, we study the impact of the choice of SSL method that we use to pretrain the backbone. For this reason, we perform an ablation experiment on CCIFAR 10 (1 %% by changing the pretraining strategy from MoCoV2+ to B YOOL 30 or BARLOW TWINS 73 We find that, while B YOL performs within the error margin of the best MoCoV2+ setting, BARLOW TWINS underperforms. BARLOW TWINS seeks to match the empirical (i.i.d.) cross-correlation between features to the identity. Arguably, we expect that the cossecorrelltion is different under the shifted test set. In any case, ULA significantly outperforms the non-ULA baselines with any of the considered SSL methods.
Finally, study how the choice of pretraining paradigm for the bias network influences the quality of hyperparameter search using our proposed validation criterion. In Fig. we present two separate searches the same space of hyperparameters using two different pretraining approaches. In red see our approach of pretraining an SSL base model for the bias network, and in blue we see baseline approach where we pretrain with purely supervised learning. We observe that SSL pretraining enables stronger correlation between the proposed biss-unsupervised group-balanced validation criterion and the corresponding test accuracy on CCIFARIO. is more difficult to tune hyperparameters with a bias network pretrained with supervised learning, because it may fit the training set entirely. In that case, the validation criterion collapses the in-distribution test accuracy which is not indicative of the group-balanced test accuracy. On the contrary, classification with a linear probe on top of SSL representatton prevents from fitting the training set entirely, having small generalization gaps in-distribution [8]. In this way, the validation criterion remains strongly correlated even in larger validation accuracy values, maintaining its utility greater range of hyperparameter configurations.
Related Work
Prior literature can be grouped according to three main strategies that attempt to improve model's robustness to dataset bias [54]. (i) Resampling strategies increase or decrease the frequency of biased

---

Loss reweighting methods Especially relevant to our work are the reweighting methods that improve robustness and generalization when training on biased datasets [6] 57 48 (iii) Post-hoc adaptation methods 644 25|
attributes in the input space (27 55 or latent space [23
74 7
balance class or feature importances during training 74
65
70 correct the biases learned by already-trained models. Most relevant to our work is the logit adjustment technique proposed by Menon et al. [54] for long-tail learning. which leverages class frequencies to rebalance the model predictions after training or to train with loss function which is aware of the class prior. We develop a logit adjustment technique for the problem of learning from biased data, which does not require previous knowledge about the dataset's biases.
Several solutions have emerged for the group robustness problem, especially when bias attribute data is available. For example, GROUPDRO [6]] leverages explicit bias attributes to reduce worst- case group loss. Our method, however, reduces reliance on such data, recognizing that access to them can be impractical. Bias-unsupervised methods like Learning from Failure (LFF) [57] and Disentangled Feature Augmentation (DFA) [17 reweight the loss of the unbiased model using co- trained biased model, removing the need for bias supervision. Conversely, Just-train-twice (JTT) 48]
reweights misclassified samples from an initial biased training. emphasizing worst-performing group data. Similarly, He et al. (33 exemplify the same intuition in that second network is trained on examples that cannot be predicted already using the spurious structure. Liu et al. [50] proposed bias uasuperuueer logit adjustment technique (LC) also based on co-training biased network. Utilizing domain knowledge, Clark et al. [18] describe a bias-supervised logit adjustment approach to debiasing Visual Question Answering (VQA) models by incorporating bias proxy which is trained exclusively on question data. These methods only generalize to seen attribute groups during training and require bias knowledge during validation for optimal performance. We tackle these issues by employing SSL to pre-train a network, deriving a bias proxy for debiased model training and validation, as well using as initialization.
In order to derive bias-unsupervised solutions, literature has proposed to train reference models as proxies for the missing bias labels. Creager et al. [20] optimize reference model for group assignments which maximally violate a relaxation of the Environment Invariance Criterion. Chen et al. [16] seek to establish conditional independence between the predictions of the proxy model and the target variable given the inferred groups Our work follows more closely the approach of [48], in which biased network is simply trained with ERM. As we demonstrate at Figure of Appendix F by utilizing a frozen backbone pretrained with SSL., our approach improves on the sensitivity to the number of training steps for the bias proxy.
Bardenhagen et al. [9] suggested a validation scheme dependent on early stopping of bias proxy network training. Our method, using SSL pretraining, avoids this by treating pretraining steps as tunable hyperparameters, maintaining performance of alternatives which were otherwise tuned with bias information. Chen et al. [16] perform experiments using methodology dubbed as Training Environments Validation (TEV). Similar to us, TEV validates models based on inferred groups from training. however the methodology is unfortunately not well documented in the literature, making its reproducibility difficult. As we show in the ablation study of Figure 4 implementation details can make a large difference in the quality of the criterion. Finally, MASKTUNE [3] eliminates spurious shortcuts by masking input data during a secondary training phase. Despite its resilience in performance without using a hisserusueervised o.o.d. model selection criterion, the need for reliable validation strategy for group robustness remains.
Conclusion
We explored group robust classification in synthetic and real tasks, proposing generalization task with unseen attribute combinations. Current robust classification methods struggle in this setting, motivating our SSL-based logit adjustment approach. Importantly, we introduce methodology for training and validating robust models without group labels. Empirical evaluations on five datasets show our method outperforms existing fully bias- uasuupuuuere approaches and rivals those using bias annotations during validation. In terms of limitations and broader impact of our contributions, as machine learning systems handle high-stakes applications, ensuring robustness to underrepresented samples is crucial. Our work reduces reliance on known data biases, but existing benchmarks differ from real-life scenarios with unknown biased attribute combinations. To bridge this gap, we proposed synthetic benchmark and encourage further research on real data, revealing more system limitations.
10

---

References
[1] Ahmed, Y. Bengio, H. van Seijen, and A. Courville. Systematic generalisation with group invariant predictions. In International Conference on Learning Representations, 2021.
[2] M. Arjovsky. L. Bottou, I. Gulrajani, and D. Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.00 907.02893,, 2019.
[3] S. Asgari, A. Khani, F. Khani, A. Gholami, L. Tran, A. Mahdavi Amiri, and G. Hamarneh. Masktune: Mitigating spurious correlations by forcing to explore. In Advances in Neural Information Processing Systems, 2022.
[4] S. Asmussen and P. Glynn. Stochastic Simulation: Algorithms and Analysis. Stochastic Modelling and Applied Probability. Springer New York, 2007.
[5] M. Assran, R. Balestriero, Q. Duval, F. Bordes, Misra, P. Bojanowski, P. Vincent, M. Rabbat, and N. Ballas. The hidden uniform cluster prior in self-supervised learning. In International Conference on Learning Representations, 2023.
[6] H. Bahng, S. Chun, Yun, J. Choo, and S. J. Oh. Learning de-biased representations with biased representations. In International Conference on Machine Learning, 2020.
[7] R. Balestriero, M. Ibrahim, V. Sobal, A. Morcos, S. Shekhar, T. Goldstein, F. Bordes, A. Bardes, G. Mialon, Y. Tian, et al. A cookbook of self-supervised learning. arXiv preprint 1xXi::3304.12210, 2023.
[8] Y. Bansal, G. Kaplun, and B. Barak. For self supervied learning, rationality implies generaliza- tion, provably. arXiv preprint arXiv:2010.08508, 2020.
(9] V. Bardenhagen, A. Tifree, and F. Yang. Boosting worst-group accuracy without group annota- tions. In NeurlPS 202/ Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
[10] S. L. Blodgett. L. Green, and B. O'Connor. Demographic dialectal variation in social media: A case study of affican-americaa english. arXiv preprint vrXii:1608.000 2016.
[11] K. H. Brodersen, C. S. Ong. K. E. Stephan, and M. Buhmann. The balanced accuracy and its posterior distribution. In International Conference on Pattern Recognition, 2010.
[12] K. Cao, C. Wei, A. Gaidon, N. Arechiga, and T. Ma. Learning imbalanced datasets with label-distributionaawe margin loss. Advances in Neural Information Processing Systems, 2019.
[13] S. Caton and C. Haas. Fairness in machine learning: A survey, arXiv preprint arXiv:0010.04053. 2020.
[14] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, 2020.
[15] X. Chen, H. Fan, R. Girshick, and K. He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003. 04297, 2020.
[16] Y. Chen, R. Xiong, Z.-M. Ma, and Y. Lan. When does group invariant learning survive spurious correlations? In Advances in Neural Information Processing Systems, 2022.
[17] S. Chu, D. Kim, and B. Han. Learning debiased and disentangled representations for semantic segmentation. In Advances in Neural Information Processing Systems, 2021.
[18] C. Clark, M. Yatskar. and L. Zettlemoyer. Don't take the easy way out: Ensemble based methods for avoiding known dataset biases. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP--JCNLP). Association for Computational Linguistics, 2019.
[19] G. Collell, Prelec, and K. R. Patil. Reviving threshold-moving: a simple plug-in bagging ensemble for binary and multiclass imbalanced data. ArXiv, abs/1606,080 086988, 2016.
11

---

[20] E. Creager, J.-H. Jacobsen, and R. Zemel. Environment inference for invariant learning. In International Conference on Machine Learning. 2021.
[21] Y. Cui, M. Jia, T.-Y. Lin, Y. Song, and S. Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattem Recognition, 2019.
[22] V. G. T. da Costa, E. Fini, M. Nabi, N. Sebe, and E. Ricci. solo-learn: A library of self- supervised methods for visual representation learning. Journal of Machine Learning Research, 2022.
[23] L. Darlow, S. Jastrzebski, and A. Storkey. Latent adversarial debiasing: Mitigating collider bias in deep neural networks. arXiv preprint arXiv:2011. /486, 2020.
[24] K. Fykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, Prakash, T. Kohno, and D. Song. Robust physical-worke attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[25] T. Fawcett and F. Provost. Combining data mining and machine learning for effective user profiling. In International Conference on Knowledge Discovery and Data Mining, 1996.
[26] J. A. Fodor and Z. W. Pylyshyn. Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1). 1988.
[27] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. Imagenet- trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2018.
[28] R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2020.
[29] M. W. Gondal, M. Wuthrich, D. Miladinovic, F. Locatello, M. Breidt, V. Volchkov, J. Akpo, O. Bachem, B. Schölkopf, and S. Bauer. On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset. In Advances in Neural Information Processing Systems, 2019.
[30] J.-B. Grill, F. Strub, Altché, C. Tallec, Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to supervuper learning. Advances Neural Information Processing Systems, 2020.
[31] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, 2017.
[32] S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. Bowman, and N. Smith. Annota- tion artifacts in natural language inference data. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018.
[33] H. He, S. Zha, and H. Wang. Unlearn dataset bias in natural language inference by fitting the residual. In Proceedings of the 2nd Workshop on Deep Learning Approaches for cow-Resource NLP (DeepLo 2019). Association for Computational Linguistics. 2019.
[34] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016.
[35] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[36] L. A. Hendricks, K. Burns, K. Saenko, T. Darrell, and A. Rohrbach. Women also snowboard: Overcoming bias in captioning models. In European Conference on Computer Vision (ECCV), 2018.
[37] D. Hendryeks and T. Dietterich. Benchmarking neural network robustness to common corrup- tions and perturbations. In International Conference on Learning Representations, 2019.
12

---

[38] D. Hovy and A. Søgaard. Tagging performance correlates with author age. In Association for Computational Linguistics, 2015.
[39] Izmailov, P. Kirichenko, N. Gruver, and A. G. Wilson. On feature learning in the presence of spurious correlations. In Advances in Neural Information Processing Systems, 2022.
[40] M. A. Jamal, M. Brown, M.-H. Yang, L. Wang, and B. Gong. Rethinking class- methods for long-tailed visual recognition from a domain adaptation perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[41] B. Kang, S. Xie, M. Rohrbach, Yan, A. Gordo, Feng, and Kalantidis. Decoupling representation and classifier for long-tailed recognition. arXiv preprint
2019.
[42] B. Kim and J. Kim. Adjusting decision boundary for class imbalanced learning. IEEE Access, 2020.
[43] P. Kirichenko, P. Izmailov, and A. G. Wilson. Last layer re- re-training is sufficient for robustness spurious correlations. In International Conference on Learning Representations, 2023.
[44] O. O. Koyejo, N. Natarajan, P. K. Ravikumar, and I. S. Dhillon. Consistent binary classification with generalized performance metries. In Advances in Neural Information Processing Systems, 2014.
[45] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[46] B. Lake and M. Baroni. Generalization without systematicity: On the compositional skills of sequence-fo-sqquence recurrent networks. In International Conference Machine Learning, 2018.
[47] Y. LeCun, C. Cortes, and C. J. Burges. Mnist handwritten digit database. In Proceedings of the IEEE, 1998.
[48] E. Z. Liu, B. Haghgoo, A. S. Chen, A. Raghunathan, P. W. Koh, S. Sagawa, P. Liang, and C. Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, 2021.
[49] H. Liu, J. Z. HaoChen, A. Gaidon, and Ma. Self-supervieed learning more robust to dataset imbalance. In NewrIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
[50] S. Liu, X. Zhang, N. Sekhar, Y. Wu, P. Singhal, and C. Fernandez-Granda. Avoiding spurious correlations via logit correction. In International Conference on Learning Representations. 2023.
[51] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, 2015.
[52] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint 1rXiv:1711.05101, 2017.
[53] A. Menon, H. Narasimhan, S. Agarwal, and S. Chawla. On the statistical consistency of algorithms for binary classification under class imbalance. In International Conference on Machine Learning, 2013.
[54] A. K. Menon, S. Jayasumana, A. S. Rawat, H. Jain, A. Veit, and S. Kumar. Long-tail learning via logit adjustment. In International Conference Learning Representations, 2021.
[55] M. Minderer, O. Bachem, N. Houlsby, and M. Tschannen. Automatic shorteut removal for self-supervised representation learning. In International Conference on Machine Learning, 2020.
[56] R. Müller, S. Kornblith, and G. E. Hinton. When does label smoothing help? Advances in Neural Information Processing Systems, 2019.
13

---

[57] J. Nam, H. Cha, S. Ahn, J. Lee, and J. Shin. Learning from failure: De-biasing classifier from biased classifier, Advances in Neural Information Processing Systems, 2020.
[58] A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 1807.03748, 2018.
[59] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin,
N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, nigh-perffrmance deep learning library. Advances in Neural Information Processing Systems, 2019.
[60] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 2015.
[61] S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2020.
[62] S. Sagawa, A. Raghunathan, P. W. Koh, and Liang. An investigation of why overparameteri- zation exacerbates spurious correlations. In International Conference on Machine Learning, 2020.
[63] Y. Shi, I. Daunhawer, J. E. Vogt, P. Torr, and A. Sanyal. How robust is unsupervised representa- tion learning to distribution shift? In International Conference on Learning Representations, 2023.
[64] R. Suter, D. Miladinovic, B. Schölkopf, and S. Bauer. Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness. In International Conference Machine Learning, 2019.
[65] J. Tan, C. Wang, B. Li, Q. Li, W. Ouyang, C. Yin, and J. Yan. Equalization loss for long-tailed object recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2020.
[66] R. Tatman. Gender and dialect bias in youtube's automatic captions. In ACL workshop on ethics in natural language processing, 2017.
[67] V. N. Vapnik. Statistical Learning Theory. Wiley- Inter-enterscceece, 1998.
[68] C. Wah, S. Branson, P. Welinder, Perona, and Belongie. The caltech-uest uesd birds-200-2011 dataset. Technical report, California Institute of Technology, 2011.
[69] H. Wang, Z. He, Z. C. Lipton, and E. P. Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2018.
[70] J. Wei, H. Narasimhan, E. Amid, W.-S. Chu, Y. Liu, and A. Kumar. Distributionally robust post-hoc classifiers under prior shifts. In International Conference on Learning Representations, 2023.
[71] Y. Xue, K. Whitecross, and B. Mirzasoleiman. Investigating why contrastive learning benefits robustness against label noise. In International Conference on Machine Learning, 2022.
[72] Y. You, I. Gitman, and B. Ginsburg. Large batch training of convolutional networks. arXiv preprint arXiv: 1708.03888, 2017.
[73] Zbontar, L. Jing, I. Misra. Y. LeCun, and S. Deny. Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning. 2021.
[74] X. Zhang, 7.. Fang, Wen, Z. Li, and Qiao. Range loss for deep face recognition with long-tailed training data. In International Conference on Computer Vision, 2017.
[75] 7. Zhang and M. Sabuneu. Generalized cross entropy loss for training deep neural networks with noisy labels. In Advances in Neural Information Processing Systems, 2018
14

---

[76] C. Zhao, J. Mangat, S. Koujalgi, A. Squicciarini, and C. Caragea. Privacyalert: A dataset for image privacy prediction. In Proceedings of the International AAAI Conference on Web and Social Media, 2022.
[77] B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 2017.
15

---

Appendix
A Bias-Supervised Training with Logit Adjustment
The goal of this Appendix section is to develop bias-supervised approach for maximizing the group-balanced accuracy (Equation (3), which is going to serve as an initial point the development
of our entirely bias- unsupervsid methodology in Section B
Under the problem formulation we introduced in Section [ the metric that we would ultimately like to
maximize given scoring function Jyjx X - IR* is Accyjx (Jyjx; Ptea), the top-1 1 accuracy under the test distribution Pres- Simple algehraic manipulations reveal that this test accuracy corresponds to the group- group-balancee balanced accuracy:
|ccy|x x ( =
E pyyaauu
1 = argmax fr (z)vr
(8)
y'
E
1 argmax Jy (x)
y.an-past xmmpoen (11.2))
KL E
E S-Ri (-l.))))))
1(v = argmax (fIx(1),))
y'
A related quantity in the literature is the class-balanced top-1 accuracy, which is used in class- imbalanced classification problems [1] [3]. We make the connection by imagining (y. 2) as a multi-label classification target a multi-label, but innalaeeba,, problem. The multi-label class-balanced accuracy of multi-label scoring function fY;zIx X R x R would then be
Xxcy,2xx(UxxxxiPeeee KL
E (x.)) (-ppaul-l
E
2(y, argmax (1) (9)
V
We will set towards building multi-label classifier which maximizes multi-label class-balanced accuracy and we will finally show that its part which only predicts Y from X can be shown to maximize lower bound the group-balanced accuracy.
We assume that D and Dvalid provide us with tuples observations (zr, y, z) distributed according to Pdata- In this case, results in the class-imbalance literature [53] 19] state that the Bayes-optimal
scoring function f for maximizing the class- balanced criterion 9 is given by
argmax fr,z(( D.z
ursmaxPPstll (x y. 2).
(10)
y.z
2 we have that (z y.z) = Pdata (I y,, This allows us compute the mechanism conditional probability in Pdata distribution terms
Note that under the invariance of mechanism assumption
Prest (z y, 2) Pdlatu (y, xx)
Pdata (z)
(11)
Pdara (y: z)
by utilizing the Bayes rule. This way have candidate strategy to maximize multi-label class- balanced accuracy in Equation (9): First, we will estimate P z (.)) and Paata from data, and then we will divide the estimates. Notice that, in order to make prediction about (y,, 2) given æ, we do not need to model Pdata (x) as this quantity is constant in (y, z), and thus does not influence the argmax.
Training process. We can directly estimate Pdata 2) from available training data by computing the empirical frequencies of pairs (y, 2) [4] x [K], by Bayesian estimate with a Dirichlet prior. On the other hand, we can train a parameterized discriminative model po with maximum conditional likelihood estimation to estimate Puaua (v. 2/22)
max
Σ
log Po,, 21x)
(12)
D
2.9.ED
which is equivalent to the minimization of a KI. divergence estimate between the Paata and po model conditionals.
16

---

The method described so far would have sufficed if the goal was to perform multi-label classification, however we need devise a way to overcome the bias-supervision, since we only care about classifying Y without having access to Z annotations. For this reason, we need to proceed to modelling assumptions about po, which will allow us to develop the bias-unsupervised method described in Section 3 First, we recall that Pdata factorizes as
Pdata 2 x) = Pdtaaa (y 2, 2) =) (z xt).
(13)
Let he: : X + RK be parameterized model (such as a neural network) tasked to predict unnormal- ized logits of Y given an observation z. Then, model Pdata (y/ 2z, 22 by
po(y| x, 2) x exp (2a( log Paata (y) (y]=)).
(14)
Unpacking our modelling assumption Equation (14). we have the following: First, we are going to compute Pdana (y/ [2] (2.0) by using our estimated Pa 2) and by marginalizing out y
P=)(2)
for (d) = [ Pdana(U: 2). Given a tuple of observables (ær, y. 2) in a training batch, we use log Pastu (y =) to adjust additively the outputs of our parametric discriminative model he to the observed input x. Notice that since Y is a categorical variable, log Panaa (y [=) E (-o0, 0] for all soft masking operation for outputs of he which are unlikely in the training data when we have observed z. In this way, account for the dependency of Y to Z which spuriously exists in the training distribution. By fitting the cross-entropy objective under logit adjustment, the network hg has to model the remaining relations for Y X that are not spurious since those are already accounted for.
y E [K]. For this reason, we can intuitively interpret logit adjustment
Prediction process. Our modelling assumption enables the prediction of y without computing the maximizer over combinations of (y, =) [K] x [L]. This is an important step towards developping a bias-unsupervised training procedure. This is because assumption 14 allows us to compute the maximizer for y and = separately. We show that under our modelling assumption, if we are only interested in predicting only the target variable y given an observation z, we do not need to model z x at all. To understand how this is possible, we take close look at the total expression for optimal predictor. After fitting the model py using Equation (12), we calculate the estimated scoring function for the multi-label dlss-imbalanced problem as in Equation 111)
Pdaaal (y, 2 x
fr,2(x(2)) (x) Prest (xt 2)
Daata (x))
(15)
Praa 2)
(0(11 (x,2) Pannal (z (x)
Pesna(2))
Poina (y) 2) (z)
exp (ho (2)) + togg (2t 2)))
Peaau (2 (=)x)) (amna())
xT)
Paata (Ir)
Zo(2, xx) frtanty (2)
exp (ho(v))) Paita (2|22) (P)ta))))(
Zo x:) Ddiata (2)
where Zz( = Eye [x] exp (ho(z), + log Paatn =)) is the partition function of ps (y/ 2).
Notice that the expression simplifies into multiplication of two terms; the first, exp (ho(z)y) ). solely
depends on (I, 2). while the second fraction only depends on (a, 2). Then, predicting according to
Equation 10) amounts to computing
Petata (alr) Zo(2, 2)) )
max max (x y. 2) = max exp (ho()),) max V.z
(16)
y
Essentially, information about Z is only need during training time and not during prediction of Y given X. In practice, during prediction we adapt the model by removing the logit adjustment term in order acquire the unbiased network for the task. Consequently, the obtained scoring function for debiased Y predictions is just the trained neural network
frix (a;; 0) ho(x:)
(17)
Notice that our design decisions are well-justified, in the sense that, under our class of models (eq. 14), we have Accy, 2 x (fy. xx: Pres) < Accy|x (frix; This means that by maximizing the multi-labe class-balanced accuracy we indirectly maximize the top-1 accuracy of the candidate scoring function frix under the test distribution Presa, which is the group-balanced accuracy as we have demonstrated.
17

---

Proposition A.1 (SLA optimizes the group- balu-ee accuracy). Under the assumption that the hypothesis class of polyl poly|z,2) ) (eq. 4 contains Pa)ma() , 2). the minimizer network hg. of the cross- entropy loss maximizes a lower bound to the group-baland balanced accuracy Accy xx (ha- Ptest)
Proof. We will start by the quantity
Accy, z|x Preal (tees))
M.y ((y, argmax g v'
(18)
the classification accuracy conditioned on the pair (y, 2). Recall that under our modelling assumption
Equation (14), the estimated Bayes-optimal scoring function for the multi-label imbalanedde classification problem is
frzzix(z) Pasa y, 2) exp (he- ((2,))
(19)
where C(2,z)
| (u) (2.2 (
is constant in y. We then substitute to get
x (fr xi Prea (,y, pplees) xly.=
(g. 2) argmax exp (ha- (z)x)
(20)
However max gets pushed the innermost term since exp (ho- (z))rr is independent of 2', as well as max, C(z', æ:) is independent of y'. This means that can take the argmax of individual expression independently
Accy, z (fx (fr,zix: Presa Prea =))
(21)
p((es) zlp.=
((y = arginax exp (ha- (x))r (= argmax (2(=,
V'
and since (P(AN B) P(A) and exp is strictly increasing function, we get
A (frexxxxxx 2))
P (teat) #ly.= (y argmax he- (z)y).
(22)
"/
We recognise the scoring function frix (z) ha- (x) on the right hand side of the expression.
Finally, by summing up all inequalities for all (y,2) [K] x [L], and dividing by KL, we get
Σ
Accy, (fv.zixi Prest (-18, (1,,2) S
Σ
p((eat) Alp. (v = argmax fx1x(x))))
(23)
KL
KL
8.22
Accy, xccy,zxx(fv.ppxip Press S Accy (Frxx: Press)-
(24)
This shows that the scoring function fy(x(x) = hg- (x) maximizes lower bound to the group- balanced accuracy.
B Self-supervised Learning Algorithms
Target Attribute
Bias Attribute
Pdano
Piest
Paua
Pistt
CMNIST (1%)
98.86
27.01
99.88
96.04
CCIFAR (1%)
98.93
36.53
99.60
82.05
SMP13D (C' 3)
98.19
66.08
99.91
87.22
CELEBA
94.01
75.44
95.74
90.60
Table 6: Downstream online linear classification against the target and bias attributes of each task considered (see Appendix D). We present top-1 (average) accuracy (%) of the corresponding classifiers evaluated at last epoch of MoCoV2+ training, under samples from the i.i.d. validation set (Pama) and the o.o.d. test set (Peas). For CELEBA. since the test split is not controlled to be group-balanced, simulate the accuracy under Press by utilizing the existing bias annotations estimate a group-balan balancd test accuracy.
18

---

In our work, we are using contrastive learning algorithm [58] 14 35 15 to pretrain base model for the bias proxy, as well as initialize the debiased model. In contrastive learning, an instance discrimination pretext task is defined from unlabeled data. In particular, pairs of data points (positives), which are derived by independently augmenting the same observation (views), should have representations that are closer in distance than representations of other samples in the dataset (negatives). To achieve that we optimize a form of the InfoNCE loss (58)
exp(1 T k+ /T)
Elq, kt, (1. HE)
log
(25)
expla
T h+ /T) + E expla
where (q, is the positive pair and (1T))) the negatives. Typically, representations projected to the unit hypersphere for training stability. In the MoCoV2+ framework [15), gradients backpropagated only through the query representation (q) the encoding network, while the key representations (A++ and k-) are extracted by a network derived from the exponential moving average of its parameters. Finally, positive key representations of past batches are kept in a queue in memory to serve as negative key representations for the subsequent batches.
C uLA: Algorithm
In Algorithm | we provide a high-level description of the uLA methodology for training and valida- tion of group-robust models without any bias annotations. In addition, PyTorch (59) implementation is available at the following repository: https:/ www.gi......tttii.....e
D Datasets
D.1 Systematic MPI3D
SMP33D is our contributed dataset which we use to study combinatorial systematicity in classifiers. By this term, we designate the ability of classifiers to generalize to samples generated from novel combinations of generative attributes, which under Pdata have probability. However, all the con- stituent values of the individual generative attributes in the novel combinations have been observed under some combination in the training data. The real split of MPI3D [2]] consists of photographs of robotic arm that has colored rigid object attached to its end effector. The images are captured in way that controls their generative attributes, such as the shape of the rigid object or the position of the robotic arm.
We this dataset create development, and deployment splits test for systematic generalization. In particular, we consider the shape of the rigid object to be the target variable Yshape. while its color to be the bias Zeolor- There are 6 possible shapes for the objects and possible colors, totaling 36 combinations of attributes. Figure 2d illustraates pairs of attributes used to sample an example systematic training split. A training set is created by assigning C' number of colors per shape so that: first, Pama (Zcolor |shsupe) 1 if (Vshape: &color) exists in the elected pairs of attributes for training; otherwise O. Second, we make sure that marginally all colors and shapes are represented uniformly, 6. During testing, we desire to generalize to samples
which means that Piat (y) generated from Prest which distributes pairs (yshape: 2color) uniformly, covering combinations that are entirely missing from the training set. In our benchmarks, we are testing methods for C' E (2, 5JJ, against the same sets of systematic splits which we generate independently per C.
1 and Piata (2) = 1
Systematic split generation. As we described above, SMPI3D is generated using the real "split" of the MP13D dataset (29]. The dataset is distributed as a tensor of shape (6, 6, 2, 3, 3, 40, 40, 64, 64, 3). with the last 3 axis corresponding to pixel values of a 64 x 64 RGB image and the rest to coordinates for generative attribute values. The first two coordinates correspond to the color and shape attributes for the rigid object depicted in the image. The procedure for generating splits of sMPI3D is stochastic, meaning that it depends on random seed set by the user. The reason for this is that we would like to benchmark methods against the possibility that some systematic splits are consistently easier to generalize than others. To generate a main-validation-tess split of SMP13D, we first generate the pattern of included-excluded combinations for the development dataset (training and validation datasets), like the one presented in Figure 2 The pattern depends on the argument C, the number of color values observed per shape value. Then, randomly permute data points across the the first two
19

---

Algorithm 1: ULA: Logit adjustment without bias labels during training or model selection.
Data:
Training split: D (Imm Dm m=1
M
Data: Validation split: Dvalid = {Inz yn 3n
Data: Hyperparameters: SSL checkpoint Tsl. linear probe training steps Tsups logit adjustment strength v and calibration т, and optimization hyperparameters
Result: Debiased model hg- and its validation_sen score
/+ Load pretrained SSL encoder
2 Shase f sTa)
/* Train Linear Head
Define classifier hissa = 96 o fuiss from the composition of pretrained fuase and linear classifier I6 with parameters ø
Predict biased predictions using (a;; hishs) argmax, Abian (z)y
for t Ssoo do
Sample batch B from D
Update so that linear model on top of hisas (a) minimizes average cross-entropy on B
end
10
/* Finetune with logit adjustment
Define calibrated model of biased predictions Phins (Ubius))))))) x exp (libia (2)) vrina
Compute confusion matrix Paum (bias) // See Equation (5)
13 Define ha to be the debiased model Initialize base model in hg from pretrained fiase
15 Predict debiased predictions using j(a;; he) = argmax, ho(x)y
16
17 checkpoints []]
18 for f = to maximum number of iterations do
19
Sample batch B from D
Update so that model po (o(y)a) (r) x exp (ho(x)y y + 1 log Pantu (3/ (xx; Mosss)) minimizes
20
average cross-entropy on
validation.scorz irrriiionncoeeeee
(aalid, Abiass ha) // see
Equation (7)
22
checkpoints checkpoints + [(validation.scoreekke
23 end
Final debiased model, hg- <- maxualidation.scor checkpoints
axis of the data tensor, which correspond the color and shape attributes. Afterwards, split the data tensor according to whether a certain position (color, shape) belongs to the development dataset We subsample the deployment dataset uniformly across i.i.d. combinations for 180k and 18k mutually exclusive data points for the training and validation sets respectively, for all C' E (2, 3, 4, 5}. Finally, the remaining data points are combined and 54k images sampled uniformly from all combinations to create the unbiased test set. We provide code for generating systematic splits on MPI3D in the supplementary material.
D.2 Non-systematic Benchmarks
Colored MNIST. (CMNIST) is an RGB version of MNIST dataset [47), in which the digit target variable Yaigit is paired with color bias variable Zeotor draw an image of that digit using specific color. In this work, we study the dataset which follows the data generation procedure from Nam et (57] Each of the digits
{0. ..., 9} is paired with a distinct color out of choice of ten, with probability Paual Zcoloo k Yaigit k) = 3 of the available training samples for that digit& The rest the probability mass split uniformly across the remaining color
options, @ for * k. Essentially, 9 controls for the percentage of bias- confficting samples in the training set. The goal is to train classifier that performs well under the
Pamaa Zenlor Ydigit = k) = 9
20

---

Learning Rate
Batch Size
Weight Decay
Temperature
CMNIST
1.0
256
le-3
0.1
CCIFARIO
0.9
256
le-4
0.1
SMPI3D
0.9
le-4
CELEBA
0.3
128
36-5
0.1
SSL Augmentations
CMNIST
RRC+ Gaussian Blur
CCIFARIO RRC + Color Jitter + Gray Scale + Gaussian Blur + Horizontal Flip
SMPI3D RRC +
Color Jitter + Gray Scale + Gaussian Blur + Horizontal Flip
CELEBA
RRC + Color Jitter Solarization Horizontal Flip
Table 7: Hyperparameters used during SSL pretraining.
test set in which digits are paired with colors with random chance 1 1/102. We obtain four different tasks by varying 3 E (0.5%, 1%, 2%, 5%
Corrupted CIFARIO. (CCIFARIO) 10) is modification of the CIFARIO dataset [45] by Hendrycks and Dietterich 37 in which images are affected by type of texture noise which correlates with the original target category. Similarly to CMNIST, there are 10 different types of texture noise each of which predominantly paired
unique CIFARIO label. The training set is created under Paata noise = k Yeitar k) 1 - B with the rest of probability spread uniformly to the bias-conflicting options of texture noise. Our goal again is to perform well under the test set which pairs of texture noise and labels are distributed uniformly. Four development datasets are used in the
experiments with (0.5%, 1%, 2%, 5%).
Our study employs the WATERBIRDS dataset, devised by Sagawa et al. 611. This dataset is derived from the CUB dataset's bird images [68], superimposed on backgrounds from the Places dataset [77]. The dataset classifies seabirds and waterfowl as waterbirds, and all other species as landbirds. Ocean and natural lake backgrounds from Places are labeled as 'water background' while bamboo and broadleaf forest backgrounds are termed 'land background'. Four groups emerge: land background with waterbird and landbird, and water background with waterbird and landbird. The first two are minority groups due to fewer examples, while the latter two constitute majority groups. The dataset maintains the original training, validation, and test splits from Sagawa et al. [6] In training, 95% of waterbirds and landbirds are respectively paired with water and land backgrounds, while we care for the worst-group test accuracy.
Waterbirds.
We examine a fair classification task using the CELEBA dataset [51] of celebrity facial attributes. Same Sagawa al. [6]] we consider the BlondHair attribute to be the target variable. Then, a spurious correlation can naturally be observed with the binarized gender attribute Male. Four groups emerge for which the minorities are males with blond hair, followed by females with non-blond hair. We use the standard splits defined in the literature, and we report the worst-group test accuracy.
CelebA.
E Experiment Details
Architecture, Optimization, Augmentation and Validation. For CMNIST, we train a 3-hidden layer MLP with 100 hidden neurons for each layer, while we use ResNet18 34 for CCIFARI O and SMP13D, and a ResNet50 for WATERBIRDS and CELEBA. For all datasets except WATERBIRDS, we pretrain the base model with the MoCoV2+ [15 process, while for WATERBIRDS we use an already pretrained ResNet50 on Imagenet [60]. Training of the linear probe for the bias network and finetuning for the logit adjusted debiased network happen with AdamW (52) optimizer using default momentum hyperparameters 3 (0.9, 0.999), except for WATERBIRDS where we use SGD with 0.9 momentum. The learning rate for all dataset, excluding WATERBIRDS, is scheduled with cosine decaying rule. Batch size is set to 256 for CMNIST, CCIFAR 10, SMPI3D and CELEBA, and 64 for WATERBIRDS. We use minimal data augmentation for training the linear probe, as well as debiased logit adjusted model, following the baselines we compare against for fair comparison. In more details, we use random resized crops (RRC) and random horizontal flipping for all datasets, except CMNIST where no augmentation is used. Finally, training happens for maximum of 100
21

---

Learning Rate
Weight Decay
"
CMNIST
(5e-4, 1e-3, 2e-3, 5e-3)
{0.0, 1e-4, 1e-2, le-1]
[1.0, 1.25
CCIFARIO
(1e-5, 5e-5, le-4, 5e-d, 1e-3)
{0.0, 1e-3, 1e-2, le-1]
(1.0, 1.25, 1.5, 2.7,
SMPI3D
(1e-5, 5e-5, le-4, Se-d, 1e-3)
1e-,
le-1]
1.0
WATERBIRDS
(1o-4, 50-4, 1o-3, 5e-37
{0.0, 1e-4,
1,,0, 2.5, 3.0)
CELEBA
f1o-5, 50-5, le-4, 5c-4]
10.0, 1e-4, le-3, 1e-2)
{1.0, 1.25, 1.5
Tes/Tato
CMNIST
{0.1, 2.0)
{10, 20, 50, 100) epochs (100, 200. 300, 400, 500] epochs
CCIFAR 10
[0.5, 2.0}
SMPI3D
1.0
{10, 20, 50. 100) epochs
WATERBIRDS
{0.1, 0.5, 2.0, (0.5, 2.0}
{50, 100, 200, 500, 1000, 2000, 5000) steps
CELEBA
{10, 20, 50, 100) epochs
Table Search space used to tune hyperparameters for each the datasets considered
epochs for all datasets (except WATERBIRDS which we train for 200), and the best model across the training duration is selected for each trial, by evaluating our proposed validation score at the end of each epoch and monitoring for the maximum value. For CMNIST. cCIFARIO and sMPI3D, we use our bias-unsupervised group- balanced validation score (see Equation 7 whereas for WATERBIRDS and CELEBA the corresponding worst-group version.
Self-supervised Pretraining. For all cases, we adapt self-supervised pretraining recipes from solo-learn software library (2]] In partieular, we optimize the MoCoV2+ loss with a LARS optimizer (72 using TLARS = 0.002. Furthermore, an MLP projector with single hidden layer used project features to a hypersphere in a 256-dimensional space. Cosine scheduling used for the learning rate and for the exponential moving average coefficient of the momentum encoder. We train for a maximum of 100 epochs for the CMNIST, sMPI3D and CELEBA tasks, and for 500 epochs for the CCIFAR 10 datasets. We save periodically checkpoints of base models across training, as we are going to decide which one use as a basis for the bias proxy and for the debiased model as hyperparameter of ULA. For each dataset, we perform small random search over contrastive temperature and data augmentation by maximizing the i.i.d. validation accuracy of an online linear classifier on-top of extracted representations. In general, we fix the contrastive temperature to 0.1 and use default Imagenet augmentations for SMPI3D and CCIFAR 10, while we search for effective variants of those for CMNIST and CELEBA. Details about the hyperparameter combinations used are displayed at Table 7
In all cases, we verify that the augmentations at Table do not cause representations to be invariant to variations of the bias attribute. We take extra care in order delineate that, especially for synthetic tasks, improvements are observed due to our ULA methodology, and not due to the use of data augmentations which nullify the bias attribute; something which, in real-life tasks such as CELEBA, might be impossible to do with hand-crafted transformation. We classify against the bias attribute using a linear classifier on extracted representations, and we find that, for the hyperparameters used, bias information is indeed retained (see T
Hyperparameter Search for ULA. For each dataset and task, we search over the respective spaces defined at Table g8] In particular, we sample uniformly 64 independent configurations (128 for WATERBIRDS) and we choose the one with the best bisseurseeed validation score (Equation (7)). To make the space a bit smaller, we consider the same weight decay for the training of the bias proxy and the debiased logit adjusted network, as well as we consider the same number of training epochs
Learning Rate
Weight Decay
?
T
Tea (epochs)
Tstop
CMNIST (1%)
2c-3
le-3
1.5
2.0
100
100 epochs
cCIFAR 10 (1%)
5e-5
le-4
1.0
1.0
500
500 epochs
sMP13D (C' = 3)
5c-5
lc-4
1.0
1.0
100
100 epochs
WATERBIRDS
lc-3
0.0
3.0
0.1
50 steps
CELEBA
lc-4
0.0
1.5
0.5
10
10 epochs
Table 9: Best hyperparameters selected by our proposed bias-unsupervised validation score.
22

---

for SSL-pretraing base network and the linear head for the bias proxy (which we train online in our implementation). For WATERBIRDS, the linear head trained on top of a pretrained ResNet50 on Imagenet, which is taken from a PyTorch [59] model repository. The pretrained model's parameters remain frozen during the training of the bias proxy, so in this case there is no number of pretraining steps to select. The hyperparameters used for the benchmarking experiments, that we presented in Section 4 can be found at Table
Extended Experiments
L-tnd trnoe (x 100.0); 0.39
Prediction 99.20
Confusion Maini (%)
140.00 a dals ouj
oleed 01
9:70
oc1 06 5.00 000 (. 0.01
0L77 0.55
020 I2 0.00 000 (MS. 0.00 6.00 6.00
0:64 7770 08 o 08.
90000000 00l0 010
IAC IEc
100001 1G 1a0 (M02 1.:0
0-116 m. 9 03
S.I so eememmmmtiii a min
0.05 0:02 0:11
1:000: 000 (. 0.c0 5000
Tagge Venbale
Bia Varaable
0.024 01
0.00 D:1
4:00 10X (0:0
BL232 1S00
SIC
0.00 0:1 6.00 0.00
0.00
0L29 020.06
1100 220.5
500
S10.05
000
2
10 110.0
SU0
05
22
12
00 0.00 0.0010.0.00
0299 0.00
2.01
0
02
IAC DED ILEZ
Biss-Proay
(a) Left: Soft confusion matrix estimate using Polas (aaasyyauu (Ybias (x)) of the bias proxy and the formula Eq. 5 in the main paper. Right: Estimated prior distribution between the target and bias variables using ground-truth bias observations.
(b) Accuracy of predicting the bias vari- able via arg maxy Phias (yhina (x), even though bias proxy was trained on target variables.
Figure 5: Heatmaps displaying the effectiveness of the proposed method derive a proxy network for predicting the bias variable. Backbone is pretrained using BYOL on CCIFAR10 10 (1%).
100
90
Gooup-balanced Teat Age
57.5
88.0
Accurcy
Metric 1..d valedation acc.
cur acc. (Eq.
80.0
group balanced
50
45.0
60
42.5
50
75
100
125
150
175
200
Bias Proxy Epochs
Epach
Figure 6: Left: Ablation of final group-balanced test accuracy (%) on CCIFARIO (1%) with respect to the number of epochs we train the linear head on-top of pretrained backbone with BYOL to produce proxy network for the bias variable. Right: Progression of accuracy metrics (%) during training of debiased network with ULA. Our validation accuracy (orange) tracks better the group- balanced test accuracy (green), than the standard in-distribution validation accuracy. Model selection with our criterion will result in better model (close to the mode of the test curve) compared to the one using i.idd. validation.
23


Reviews:
[{"review_id": "1s3TdoeS9D", "reviewer": "Reviewer_Y39n", "summary": "This paper introduces a new bias-unsupervised method for addressing spurious correlations, encompassing a debiasing training algorithm and a model selection paradigm. Specifically, the method employs a Self-Supervised Learning (SSL) pre-trained model as a bias proxy. The SSL model's fixed predictions are used to adjust the logit during the predictor's training phase. These predictions are also utilized in the model selection stage to adjust the weight of the sample, aiming to approximate group-balanced accuracy. Empirical evidence supports the method's efficacy, as demonstrated on several popular benchmarks, as well as a systematic generalization benchmark.", "questions": "- Why the proposed method can benefit systematic generalization, theoretically or intuitively?", "limitations": "Some theoretical justifications for the proposed method are lacking.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- How to design a proper bias proxy is important for the bias-unsupervised methods.\n- The presentation of the method is clear and generally easy to follow.\n- A variety of benchmarks are considered in the experiments and an ablation study part is included.", "weaknesses": "- Inconsistency. In the introduction part, one of the motivation of this work is to solve the failure of existing methods in systematic generalization. However, though empirically shown, it is not clearly introduced how the proposed method could achieve that improvement.\n- Some important related works are missing. \n   - There are some other existing bias-unsupervised methods, including EIIL[1], PGI[2], and SCILL[3]. Similar to the method proposed in this work, all the 3 methods utilize a frozen reference model as a bias proxy. Especially, PGI is proposed for the systematic generalization task. \n   - The logit adjustment strategy is similar to the strategies in some debiasing training methods, e.g. PoE[4] and DRiFt[5]. \n   - A bias-unsupervised validation paradigm is also proposed in [2].\n\nTherefore, this paper should discuss these related works and reposition its contribution.\n\nReferences:\n\n[1] Creager et al. \"Environment inference for invariant learning.\" ICML, 2021.     \n[2] Ahmed et al. \"Systematic generalisation with group invariant predictions.\" ICML, 2021.     \n[3] Chen et al. \"When does group invariant learning survive spurious correlations?\" NeurIPS, 2022.\n[4] Clark et al. \"Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases.\" EMNLP, 2019.\n[5] He He, Sheng Zha, and Haohan Wang. Unlearn dataset bias in natural language inference by fitting the residual. DeepLo workshop, 2019.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback, recognition of the novelty of our approach and positive remarks on the presentation of our method.\n\n### Weakness 1/Question 1:  It is not clearly introduced how the proposed method could achieve improvement in systematic generalization, intuitively or theoretically.\n\n**Theoretically**, Proposition 2.1 (L156 and Appendix A) justifies why we expect models trained with logit adjustment to systematically generalize: it optimizes for the transfer learning problem defined by Eqs 1 and 2. At the same time, systematic generalization is just an extreme case of this transfer learning problem, for which some combinations of features have 0 probability in training+validation. The scope of our analysis considers anti-causal prediction tasks for which the target variable is a generative factor. This is relevant in practice as Fig 2 suggests. We will add a sentence which clarifies the scope in Sec 2.\n\n**Intuitively** (L63), many of the existing methods will fail in a systematic generalization task, as *there are no rare samples to reweight their loss contributions or to upsample during training*. In contrast to these methods, logit adjustment avoids this pitfall. It adds the spurious $\\log p(y|z)$ to the unnormalized logits of the mode, and downweights the score of the class y that is unlikely given the observed z. That is, as explained in L147-152, this additive operation will mask out the logits that are improbable to prevail under the structure of spurious correlations of the training set.\n\n### Weakness 2: Discussion on important related works and request for repositioning our contributions.\n\nThank you for the important references. All of them will be mentioned in Sec 5 about Related Work. In particular, [1,2,3] will also be discussed in a dedicated related work appendix section about utilizing reference models as proxies for bias information.\n\n**[1]** focus on extracting majority and minority group annotations based on a reference model, which ideally predicts based on spurious features only. In particular, they optimize for group assignments which maximally violate a relaxation of the Environment Invariance Criterion. **[3]** carefully examines various data generation processes and they propose an alternative to [1] for bias-unsupervised group design. More concretely, [3] seeks to establish conditional independence between the predictions of the bias proxy model and the target variable given the inferred groups. Our work follows more closely the approach of [6], in which a biased network is simply trained with ERM. By utilizing a frozen backbone pretrained with SSL, our approach improves on the sensitivity of [6] to the number of training steps for the bias proxy. As we demonstrate at the **left plot of Fig 2 in the rebuttal material**, the test accuracy of a uLA debiased model increases given sufficient training time of the bias proxy and after that it stays within the noise level of optimal performance.\n\n**[2]** introduce experimental settings which highlight difficulties of various training methods. They use [1] to pretrain a bias proxy and they debias with an alternative target-conditioned invariance criterion. **The authors of [2] claim that one of their colored-MNIST settings corresponds to a systematic generalization task. However we find that this is not accurate**; systematicity can only be probed for when entirely novel attribute combinations are used to generate the test samples. In their case, however, the training set contains 80% of bias-aligned samples and 20% of bias-conflicting, exposing the model to all possible combinations. In contrast, in our sMPI3D, there are always some (target, bias) combinations which are not represented in the training set at all (Fig 2d). Finally, [2] present evaluations using different model selection strategies: they use accuracy under an iid validation set, an extra ood validation set and a mixed case. However, unlike our work, they do not propose or evaluate under a specialized robust model selection methodology using bias-unsupervised iid data. \n\n***Thus, we want to underline that to our knowledge we are the first to examine systematic generalization classifiers trained with group-robustness methods.***\n\nFurthermore, the authors of [3] perform experiments using a methodology dubbed as TEV (Training Environments Validation), which is briefly described in Appendix D.1. Iit seems related to our approach as it validates models based on inferred groups from training. The paper cites [7] as a source for TEV, however we are not able to find a methodology which matches the provided brief description in [7], making a detailed comparison and reproducibility difficult. On the contrary, we have shown that our approach often outperforms even alternatives which use group labels during model selection (Table 1,2,3). Also, an extensive search experiment shows that our selection criterion is more reliable than an alternative version of our validation score based purely on supervised learning (Fig 4).\n\n***Overall, we believe that our work addresses explicitly an important gap in group-robustness literature [7] and proposes an effective bias-unsupervised iid robust validation protocol.***\n\nFinally, **[4]** employs a logit adjustment approach to debiasing VQA models by incorporating a bias proxy which is trained exclusively on question data. On the other hand, **[5]** exemplifies the same intuition as logit adjustment in that a second network is trained on examples that cannot be predicted already using the spurious structure. Our work instead utilizes a simple and generic logit adjustment loss, which is derived from first principles as we show in Proposition 2.1 and Appendix A.\n  \n[6] Liu, Evan Z., et al. \"Just train twice: Improving group robustness without training group information.\" ICML. 2021.   \n[7] Ishaan Gulrajani and David Lopez-Paz. “In search of lost domain generalization.” ICLR. 2021. \n"}, {"review_id": "wxHqnml4o6", "reviewer": "Reviewer_knxG", "summary": "The paper works on mitigating the impacts of spurious correlations during the risk minimization. The authors firstly introduce a systematic generalization task and illustrates existing methods implicitly made the assumption that all group combinations are represented within the training procedure. And then reveal that the importance of bias labels in model selection. Hence, the authors leverage pre-trained self-supervised models for bias information extraction, during the training and validating the debiased models. ", "questions": "* (1) The choice of ssl algorithm: I am not asking authors to run additional experiments, just curious about how uLA behaves when different SSL algorithms are utilized, if the authors already have the result.\n\n* (2) Analysis of $\\eta$: are there any analysis on the role of $\\eta$ in eqn. (6), either theoretical or empirical analysis?\n\n* (3) How is the performance of uLA on waterbirds when a SSL pre-trained model is utilized? ", "limitations": "As mentioned in weakness 3, one limitation is **Efficiency of uLA:** SSL algorithms usually require longer training time and more computer resources, compared with classical training algorithms. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "The proposed method uLA trains a linear classifier on top of an SSL pre-trained model. The predictions are hence biased, and could be made use of model debiasing. The key advantage is that uLA does not need any group labels, neither in the training nor in the validation.", "weaknesses": "* (1) The presentation could be better , i.e., (a) in figure 2, it would be much better if authors could give more introduction about MPI3D. datasets; (b) Eqn (5) is a core component of uLA, it would be better if authors could explain more about the the insights of the estimate for $\\hat{p}_{\\text{data}}(Y, Z)$, for example, why this is a reliable estimate? Is it possible for authors to provide some empirical verification on the effectiveness of this estimation?\n\n* (2) Personally speaking, the proposed method is somehow related to a family of distributional robustness approach, that holds the viewpoint that \"fine-tune last layer\" [R1] or \"post-hoc adjust the model prediction\" [R2] can improve the distributional robustness substantially. Such methods also rely on a well-trained model, uLA can also be categorized in this setting since uLA leverages SSL pre-trained models. Thus, it would be better to either compare or discuss the connections between uLA and this family of robust solutions.\n\n* (3) Efficiency of uLA: SSL algorithms usually require longer training time and more computer resources, compared with classical training algorithms. \n\nReferences:\n\n[R1] Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations. [ICLR'23]\n\n[R2] Distributionally Robust Post-hoc Classifiers under Prior Shifts. [ICLR'23]", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed feedback and their questions.\n\n### Weakness 1.a: Presentation of MPI3D\n\nWe update line 73 with: “In particular, we use the ‘real’ split of MPI3D which  consists of photographs of a robotic arm that has a colored rigid object attached to its end effector. The images are captured in a way that controls their generative attributes, such as the shape of the rigid object or the position of the robotic arm.”\nIn addition, we append the following at line 77: “In Appendix D.1 we describe in more detail the construction of the sMPI3D task, while in Figure 2d we illustrate an example of a systematic split.”.\n\n### Weakness 1.b: Is Equation 5 a reliable estimate and when?\n\nWe want to leverage the empirical observation from [1] that a biased model will predict the actual labels using the spuriously correlated (bias) variable, rather than the true feature corresponding to the class label. Using the case of cMNIST for instance, simpler predictive features (like color) are generally learnt faster than more complex predictive features (like the shape of digits). Typically training of deep neural networks will tend to use the bias attribute, when it is a simpler feature which is spuriously predictive of the target variable [2,3]. For example, it could be the case that during training most images that display the digit 5 are green. Predicting “digit 5” whenever the green feature is present in an image is sufficient to achieve minimal training errors.\n\nPlease refer to Figure 1 of the rebuttal material where we display heatmaps comparing the soft confusion matrix computed using Equation 5 versus the distribution of bias and target attributes in the training set for cCIFAR10 (1%). We observe that the soft confusion matrix closely matches the joint distribution between the target and the bias variable.  \n\n[1] Nam, Junhyun, et al. \"Learning from failure: De-biasing classifier from biased classifier.\" NeurIPS. 2020.   \n[2] Geirhos, Robert, et al. \"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.\" ICLR. 2019.   \n[3] Rahaman, Nasim, et al. \"On the spectral bias of neural networks.\" ICML. 2019.\n\n### Weakness 2: Related work\n\nThanks for the pointers. Note that we already acknowledge R1 ([37]) as a bias-supervised method in L177. We update the related work with R2 as an example of bias-supervised post-hoc method. Note that R2 differs from our proposal in that it seeks to adjust logits to align a classifier's outputs with an empirical label marginal distribution given by an **annotated** validation sample. In our case, we focus on situations where such validation data are not available, and uniform accuracy over groups is expected nonetheless. More importantly, the approach in R2 would not cover cases with systematic splits such as sMPI3D since validation data, even if annotated, would still not cover all possible groups if they are indeed i.i.d. to the training set.\n\n### Weakness 3: Efficiency of uLA\n\nWe understand the concern about training an SSL model. However, we are glad to highlight that this model is re-used to initialize the de-biased network and thus, we only need to train a single model (L179). Furthermore, training a linear probe on top of extracted features or finetuning a pretrained network typically require fewer parameter updates than training from scratch.\n\n### Question 1: How does uLA behave when different SSL pretraining methods are used?\n\nIn **Table 1 of the rebuttal material**, we demonstrate results of uLA on cCIFAR10 (1%) with BYOL [3] and BarlowTwins [4] SSL pretraining. In both of these extra cases, uLA outperforms ERM and the SotA bias-supervised baseline. Performance with BYOL is comparable to MoCoV2+ (we have not performed extensive hyperparameter search), while performance with Barlow Twins is inferior. The latter could possibly be explained by the form of the Barlow Twins objective which attempts to match an estimation of cross-covariance between features using (spurious) training statistics to the identity.\n\n[3] Grill, Jean-Bastien, et al. \"Bootstrap your own latent-a new approach to self-supervised learning.\" NeurIPS. 2020.   \n[4] Zbontar, Jure, et al. \"Barlow twins: Self-supervised learning via redundancy reduction.\" ICML. 2021.\n\n### Question 2: Analysis of Equation 6.\n\nProposition 2.1 (L156 and Appendix A) provides a theoretical justification of the objective in Equation 6 with $\\eta = 1$. The concluding result is that by this procedure the optimal model maximizes a lower bound to the group-balanced accuracy, hence the procedure behaves in a desired way.\n\n**For why $\\eta$ should be tuned**, please refer to suggestions by [5] in the last two paragraphs of Section 4.1. What they refer as $\\tau$ is $\\eta$ in our case. In short, calibration errors of deep neural networks [6] need to be minimal (in this case of the debiased network) and tuning this helps in mitigating them. We will clarify its role in the main paper.\n\n[5] Menon, Aditya Krishna, et al. \"Long-tail learning via logit adjustment.\" ICLR. 2021.   \n[6] Guo, Chuan, et al. \"On calibration of modern neural networks.\" ICML. 2017.\n\n### Question 3: Why did we use an ImageNet pretrained model on Waterbirds? How would SSL affect performance?\n\nFor Waterbirds, we abide by the practice followed by the baselines in literature [see references at Table 2] for fair comparison. We will highlight this in the main text. All methods initialize a ResNet50 from PyTorch ImageNet-pretrained weights. A more complete explanation is that Waterbirds’ training set contains only 4795 samples, which are too few for successful supervised or SSL training from scratch. Instead, literature has chosen to test the ability of robustness algorithms in the finetuning setting. We are pleased to find that our method is not hindered by supervised pretraining in this case, as long as we freeze the weights of the backbone when we are training the bias proxy network.\n"}, {"review_id": "Sas0ei6Xnu", "reviewer": "Reviewer_R4tq", "summary": "This work aims to evaluate and introduce a method to make classifiers perform well across subgroups of the data, focusing in particular on “spurious correlations” where one attribute correlates with the output label in the training set, but need not at test time. While most past work doing this requires “bias annotations”, or labels of the potential spurious attribute, some work aims to avoid spurious correlations without group info. This work reveals shortcomings in these methods; they introduce a new dataset (sMPI3D) where the goal is to classify the shape of an object, which is spuriously correlated with its color. They then introduce a new method, “bias-unsupervised logits”, which adjusts the output logits approximating the log probability of the label given the hidden attribute of x, without any access to the bias attribute. To approximate this quantity, the authors take representations trained with some self-supervised learning approach (they use MoCoV2+), then fine-tune these representations towards the target labels and use this as the prediction. The authors find that their method is comparable to existing methods that use bias annotations on Waterbirds, CelebA, cMNIST, and cCIFAR, and find that their method is the only one that reliably outperforms empirical risk minimization on sMPI3D. \n", "questions": "* Could you provide more intuition for why training on the actual labels is a good way to estimate p(y \\mid z); in particular, what properties of the pretrained representation do you need for this to work? \n", "limitations": "Addressed limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper is well written, motivated, and referenced throughout. \n* The dataset sMPI3D seems like a good contribution; in particular, the authors do a good job explaining how the number of subgroups grows exponentially with the number of attributes you care about, that this means you can’t cover all groups in the training set reliably, and this dataset gives a great simple way to test how well models extrapolate to new groups (when the individual attributes are supported, but the direct product is not)\n* The authors setup---no bias annotations during both training and validation---seems realistic and important for subsequent work to follow\n* The results against other methods on existing benchmarks seem promising, though they’re sometimes worse than other methods. \n", "weaknesses": "* There are places where the paper could use more exposition; for example, the current paper does not devote much time to how they extract bias variable observations, or time to how they construct the dataset (the two core contributions)\n* The method, while performing comparably to baselines, only offers significant improvement over previous methods on the sMPI3D, which the authors release (and thus have a lot more control over). This is somewhat ok because the task setup for sMPI3D is different (support over all individual attributes but not the direct product in training), but it would be nice to see additional experiments. \n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful assessment of our paper. We address the raised concerns in what follows:\n\n### Weakness 1: There are places where the paper could use more exposition: how they extract bias variable observations, or time to how they construct the dataset (the two core contributions)\n\nThank you for pointing this out. We estimate bias attributes via a biased classifier. That is, we train a classifier with ERM without any additional approach to avoid spurious solutions that rely on biasing attributes. Since that approach is vulnerable to spurious correlations, predictions will correlate strongly with spurious features if those exist. \n\nTo create the sMPI3D dataset, we use the shape of the object as the target variable, while its color is the bias. Both generative attributes can assume 6 values so that 36 combinations are possible.  A training set is created by assigning a number $C$ of colors per shape. We also make sure that all colors and shapes are represented uniformly. The test split is created by covering the shape/color combinations that **are not** observed in the training set. In our evaluation, we consider different cases with $C \\in \\{2, 3, 4, 5\\}$.\n\nPlease refer to appendices A, C and D.1 for a more in-depth exposition of both the training procedure and the construction of sMPI3D. We finally highlight that, in case of acceptance, the additional page in the camera ready version will be used for more details on the data construction along with further discussion on the effectiveness of the bias estimation approach as evidenced by Figure 1 of the rebuttal material.\n\n### Weakness 2: Significant improvement over previous methods only on sMPI3D\n\nWe would like to highlight that uLA outperforms all other methods in Table 1 in 5 out of 8 setups. Please note that in our assessment we compare against methods which may require group information during training and/or validation in order to perform well. Our method, however, outperforms them without **any access** to privileged information sources. In particular, for the cCIFAR10-1% case we outperform the state-of-the-art approach, GroupDRO (which uses group information during training and validation), with a margin of about 24% absolute accuracy.\n\nEven though these results demonstrate the effectiveness and potential of our proposal across a diverse range of scenarios, we believe our main contribution to be the improvements demonstrated in the systematic generalization case. In fact, it is fair to claim that a considerable chunk of the existing literature on robustness to spurious correlations and group fairness is expected to fail in such a case, as confirmed by our empirical assessment. We hope our contribution motivates further work in this direction given its practical relevance.\n\n### Question 1: Could you provide more intuition for why training on the actual labels is a good way to estimate p(y \\mid z); in particular, what properties of the pretrained representation do you need for this to work\n\nWe simply aim for $p(y|z)$ to be such that a given class will be consistently estimated whenever a spurious feature is observed. Luckily, training a model with ERM typically satisfies that since learners tend to focus on *easy* features. Please refer to Figure 1 on the additional experiments for empirical evidence of that being the case for the cCIFAR-10 dataset.\n\nIn further detail, we want to leverage the empirical observation from [1] that a biased model will predict the actual labels using the spuriously correlated (bias) variable, rather than the true feature corresponding to the class label. Using the case of cMNIST for instance, simpler predictive features (like color) are generally learnt faster than more complex predictive features (like the shape of digits). Typically training of deep neural networks will tend to use the bias attribute, when it is a simpler feature which is spuriously predictive of the target variable [2,3]. For example, it could be the case that during training most images that display the digit 5 are green. Predicting “digit 5” whenever the green feature is present in an image is sufficient to achieve minimal training errors.\n\nIn **Figure 1 of the rebuttal material** we display heatmaps comparing the soft confusion matrix computed using Equation 5 versus the distribution of bias and target attributes in the training set for a cCIFAR10 (1%) experiment. We observe that the soft confusion matrix closely matches the joint distribution between the target and the bias variable.    \n\n[1] Nam, Junhyun, et al. \"Learning from failure: De-biasing classifier from biased classifier.\" NeurIPS. 2020.  \n[2] Geirhos, Robert, et al. \"ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.\" ICLR. 2019.   \n[3] Rahaman, Nasim, et al. \"On the spectral bias of neural networks.\" ICML. 2019.\n"}, {"review_id": "xpvzhpjMeD", "reviewer": "Reviewer_Tfdv", "summary": "The paper identifies that current methods tackling spurious correlations requires group annotation in either the training or validation stage. To address this limitation, the authors propose uLA, a bias-unsupervised method that achieves superior empirical performance without any group annotation.\n", "questions": "1. For the reported baseline, how are the models trained? Do they also go through the MOCOV2+ pretraining and linear probing as uLA?\n2. On line 225, it is mentioned that the model needs to be validated at every epoch, which is uncommon in practice. Could you provide some justification for this procedure? Does it mean that the algorithm is sensitive to hyperparameters?\n3. Why use an Imagenet pretrained model on waterbird? Are there any scenarios where using SSL hurt the performance? I hope the authors can provide more detail about this design choice and more discussion about the selection of the training scheme of the shared backbone.\n", "limitations": "The authors have adequately addressed the limitation.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "The paper studies an important problem: how to tackle spurious correlations when group annotation is unavailable. The proposed algorithm is simple yet effective, demonstrating superior empirical improvement. The performance on the systematic generalization is particularly promising as it is a much more realistic scenario.\n", "weaknesses": "Although the authors show some conceptual and empirical advantage of SSL over pure supervised training, it remains unclear how crucial a role SSL has in the training pipeline. I hope the authors can provide clearer insights about the criterion of selecting the pretraining scheme for the shared backbone model. (Also see question 3)\n\nThe robustness of the algorithm to different hyperparameter settings is questionable (also see question 2). Not only does the biased prediction matter, but the biased logits are also quite important in the training pipeline. However, the logits can take dramatically different values given different hyperparameters. I hope the authors can provide more insights in this regard and perhaps include more principled method for hyperparameter search.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed assessment of our work and for highlighting the merits of our approach, as well as the importance of the problem. We address all concerns below:\n\n### Weakness 1/Question 3.b.: Even though SSL is demonstrated to provide benefits over pure supervised learning, it is not clear how the choice of SSL pretraining affects debiased training.\n\nIn **Table 1 of the rebuttal material**, we demonstrate results of uLA on cCIFAR10 (1%) with BYOL [1] and BarlowTwins [2] SSL pretraining. In both of these extra cases, uLA outperforms ERM and the SotA bias-supervised baseline. Performance with BYOL is comparable to MoCoV2+ (we have not performed extensive hyperparameter search), while performance with Barlow Twins is inferior. The latter could possibly be explained by the form of the Barlow Twins objective which attempts to match an estimation of cross-covariance between features using (spurious) training statistics to the identity.\n\n[1] Grill, Jean-Bastien, et al. \"Bootstrap your own latent-a new approach to self-supervised learning.\" NeurIPS. 2020.      \n[2] Zbontar, Jure, et al. \"Barlow twins: Self-supervised learning via redundancy reduction.\" ICML. 2021.\n\n### Weakness 2.a: Robustness to hyperparameters\n\nBeyond the typical hyperparameters in stochastic optimization of deep neural networks (learning rate, batch size, architecture, augmentation choices and others), we have perceived performance benefits when we search over the logit adjustment strength $\\eta$, biased model calibration temperature $\\tau$ and the number of epochs for training the linear head of the bias proxy network $T_\\mathrm{stop}$. As we present in **Table 7 of Appendix E**, we have found stable search spaces for $\\tau$ (values around 1.0, with default value 1.0) and $\\eta$ (values larger than or equal to 1.0 with default value 1.0) across benchmarks. Furthermore, for $T_\\mathrm{stop}$ we find in **left plot of Figure 2 in the rebuttal material** that the group-balanced test accuracy of a debiased classifier trained with uLA on cCIFAR10 (1%) increases given sufficient training time of the bias network and then it stays within the noise level of optimal performance.\n\n### Weakness 2.b: Request for more principled method for hyperparameter search\n\nWe want to highlight that one of the core contributions of our work is a meticulous methodology for model selection and hyperparameter search for robust/ood classification problems. We have demonstrated the effectiveness of our approach via experimenting against known baselines in the literature, often outperforming alternatives which use privileged group annotation information during hyperparameter search (Table 1,2,3). Extensive ablation experiments show that our hyperparameter search methodology is more reliable than a naive alternative validation objective (Figure 4).\n\nThe choice of hyperparameter search algorithm can be any black-box optimizer, however in this work we opt for random search, which is the most simple, general and yet more effective than grid search [4].\n\n[4] Bergstra, James, and Yoshua Bengio. \"Random search for hyper-parameter optimization.\" JMLR 13.2 (2012).\n\n### Question 1: Are baselines pretrained with MoCoV2+ too?\n\nNo, they do not as the methods described in their respective papers do not utilize any form of pretraining for the debiased network. We compare against accuracies reported by the peer-reviewed published methods in standard benchmarks.\n\n### Question 2: On validating the model periodically during training and model selection\n\nIf we understand the reviewer correctly, there is a concern about the practice of periodic validation of a model during its training. We want to note that this is a well-documented practice, commonly referred to as ``early stopping’’. From the Deep Learning book [8, Chapter 7]: \"The only significant cost to choosing this hyperparameter automatically via early stopping is running the validation set evaluation periodically during training\", referring to how to choose the number of training iterations with early stopping.\n\nWhile in many standardized benchmarks the number of epochs has been already searched for, in ood generalization literature it is often [9] a sensitive factor and early stopping can bring benefits in generalization. Our work addresses the need for bias-unsupervised validation for robust model selection using iid data resources. In the right plot of Figure 2 in the rebuttal material, we demonstrate that our validation accuracy [described in Equation 7] closely follows the group-balanced test accuracy and model selection according to it is likely to be close to the maximum test accuracy. In contrast, if a practitioner performs model selection by maximizing the (average) iid validation accuracy, they will get much worse performance. \n\n[8] Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.     \n[9] Gulrajani, Ishaan, and David Lopez-Paz. \"In search of lost domain generalization.\" ICLR. 2021.\n\n### Question 3.a: Why did we use an ImageNet pretrained model on Waterbirds? How would SSL affect performance?\n\nFor Waterbirds, we abide by the practice followed by the baselines in literature [see references at Table 2] for fair comparison. We will highlight this in the main text. All methods initialize a ResNet50 from PyTorch ImageNet-pretrained weights. A more complete explanation is that Waterbirds’ training set contains only 4795 samples, which are too few for successful supervised or SSL training from scratch. Instead, literature has chosen to test the ability of robustness algorithms in the finetuning setting. We are pleased to find that our method is not hindered by supervised pretraining in this case.\n"}]

Author rebuttal:
We thank all reviewers for their insightful assessment of our work and for providing useful feedback and actionable suggestions. We are glad they found the method we proposed to be simple yet effective (reviewer Tfdv) and efficient since no bias labels are required during training or validation (reviewer knxG), our evaluation broad covering a number of benchmarks and including ablations (reviewer Y39n) and realistic (reviewer R4tq), the sMPI3D data to be a good contribution (reviewer R4tq), and the manuscript to be clear (reviewers Y39n and R4tq).

We addressed each reviewer individually and provided additional empirical analysis as suggested. New results include:   
  -  Evaluations of models using SSL backbones pre-trained with different approaches.
  -  Analysis of the effectiveness of using ERM trained classifiers as estimators of bias attributes.
  -  A comparison of different cross-validation criteria show how i.i.d. accuracy yields to sub-optimality and that our proposal closely aligns with the oracle group-balanced test accuracy, as suggested by our original results.
  -  Plot on how bias proxy training time influences group-balanced test accuracy of a debiased model.

Furthermore, we summarize the edits that we suggest to perform in the paper as a result of reviewer feedback:
  -  Clarification on the scope and relevance to systematic generalization for Proposition 2.1
  -  Updated related work and extra Appendix section which discusses methods to create bias proxy networks
  -  More main paper details on MPI3D and sMPI3D
  -  Extra study in main paper: evidence about Eqn 5 being reliable (Fig 1 of rebuttal material)
  -  Clarification that ImageNet pre-training for Waterbirds is for fair comparison to baselines 
  -  Clarification on why we search $\eta$
  -  Add in Appendix the ablation for different SSL pretraining methods (Table 1 of rebuttal material)
  -  Add in Appendix the sensitivity curve: bias proxy training time vs test accuracy of debiased model (Left plot of Fig 2 in rebuttal material)
  -  Add in Appendix the validation and test curves during training (Right plot of Fig 2 in rebuttal material)


Meta-review: Models trained with ERM can underperform on under-represented subgroups due to the presence of spurious correlations or biases. Existing approaches for tackling this problem assume the presence of all label-bias combinations to be present in either the training or validation set. In contrast, this paper seeks to learn a classifier robust to spurious correlations in an entirely bias-unsupervised manner. They do so by training a proxy model to predict the "bias" attribute and using predictions from this model to formulate a logit-adjusted training loss. 

The reviewers were generally appreciative of the problem setup and recognize the introduction of the MPI3D as a useful contribution.
While there were some initial concerns about the exposition and lack of clarity about how the paper trains a reliable proxy model, these were resolved during the rebuttal.  

However, an important pending concern was about the **paper's claim that it guarantees "systematic generalization"** (Reviewer Y39n), i.e. the ability to generalize to combinations of attribute not seen in the training/validation sample. The authors point to the use of "logit adjustment" to guarantees this (they say that in contrast, prior work are unable to systematically generalize as they instead rely on *re-weighting* rare samples).

It took me quite a while to convince myself of the authors claim (after going through proofs/material in the appendix). While we are happy to accept this paper, we urge the authors to provide a stronger justification for their claim. Please take a close look at the following comments.

**Closer inspection at the claim of systematic generalization:**

At the heart of the author's claim is the assumption in eq. 14, which essentially states that the conditional distribution $p_{\rm data}(y | x, z)$ can be decomposed as:

$$
p_{\rm data}(y | x, z) = p_{\rm data}(y | z) \cdot G(x, y),
$$

for some function $G$ independent of $z$. In the paper, $G(x, y) \propto \exp(h_\theta(x)_y)$. In other words, the authors assume that $\frac{ p\_{\rm data}(x | z, y)  }{ p\_{\rm data}(x | z) } = \frac{ p\_{\rm data}(y | x, z)  }{ p\_{\rm data}(y | z) }$ is independent of $z$. Unfortunately, this assumption was not made very explicit, except in passing in Proposition 2.1. Please state this explicitly in the main text.

I can see how under this assumption, the use of logit-adjustment guarantees systematic generalization. Specifically, due to the independence assumed, we don't need to see all $(y, z)$ combinations to learn $G(x, y)$ (or a function $h_\theta(x)$ that shares the same maximizer over $y$). 

However, I am still unconvinced why re-weighting the loss woudn't have the same effect (assuming eq. 14 holds). More precisely, wouldn't re-weighting by the inverse of $p_{\rm data}(y | z)$ instead of using it as a part of the logit-adjustment also be able to learn $G(x,y)$? For example, if we were to use the following re-weighted loss:
$$
\\mathbb{E}\_{x, y}\\left[ \\frac{1}{\\hat{p}\_{\\rm data}(y | z)} \\cdot \\ell_{\\rm ce}(y, h\_{\\theta}(x)) \\right],
$$
even though the loss will not see some $(y, z)$ combinations, wouldn't it still end up learning $h\_{\\theta}(x) \approx \log\left( \frac{ p\_{\rm data}(y | x, z)  }{ p\_{\rm data}(y | z) } \right)$, given the assumption that the RHS is independent of $z$.

The authors are strongly encouraged to make their assumptions clear and explain why they aid in systematic generalization. If the authors do believe that prior re-weighting style approaches would not guarantee systematic generalization under the same assumptions, they need to explain this clearly in the paper.
Decision: Accept (poster)