Mitigating Biases in Blackbox Feature Extractors for Image Classification Tasks
Authors: Abhipsa Basu
Saswat Subhajyoti Mallick
Venkatesh Babu Radhakrishnan
Venue: NeurIPS 2024 poster (2024)
Keywords: bias;fairness;spurious correlation
Primary area: fairness
TL;DR: Analyses the propagation of biases in presence of blackbox feature extractors and suggests a simple migitation strategy for image classification

Abstract: In image classification, it is common to utilize a pretrained model to extract meaningful features of the input images, and then to train a classifier on top of it to make predictions for any downstream task. Trained on enormous amounts of data, these models have been shown to contain harmful biases which can hurt their performance when adapted for a downstream classification task. Further, very often they may be blackbox, either due to scale, or because of unavailability of model weights or architecture. Thus, during a downstream task, we cannot debias such models by updating the weights of the feature encoder, as only the classifier can be finetuned. In this regard, we investigate the suitability of some existing debiasing techniques and thereby motivate the need for more focused research towards this problem setting. Furthermore, we propose a simple method consisting of a clustering-based adaptive margin loss with a blackbox feature encoder, with no knowledge of the bias attribute. Our experiments demonstrate the effectiveness of our method across multiple benchmarks.

Mitigating Biases in Blackbox Feature Extractors for Image Classification Tasks
Abhipsa Basu
Saswat Subhajyyti Mallick
R. Venkatesh Babu
Vision and Al Lab, Indian Institute Science, Bangalore
Abstract
In image classification, it is common to utilize a pretrained model to extract meaningful features of the input images, and then train classifier on top of it to make predictions for any downstream task. Trained on enormous amounts data, these models have been shown to contain harmful biases which can hurt their performance when adapted for a downstream classification task. Further, very often they may be blackbox, either due to scale, or because of unavailability of model weights or architecture. Thus, during a downstream task, we cannot debias such models by updating the weights of the feature encoder, as only the classifier can be finetuned. In this regard, we investigate the suitability of some existing debiasing techniques and thereby motivate the need for more focused research towards this problem setting. Furthermore, we propose a simple method consisting a clustering-bate based adaptive margin loss with a blackbox feature encoder, with no knowledge of the bias attribute. Our experiments demonstrate the effectiveness of our method across multiple benchmarks. The code is publicly available at httpss //github. com/ abhipsabasu/blllae.aaabbbbbbbbbb hawssu/ubackbbboo bias aitas_mmtine misimiiaiiiiinnn
Introduction
Deep learning models are known to inherit harmful stereotypical biases with respect to the different genders, races, cultures, from the datasets they are trained on 2 3 For example, model trained on a gender-biased dataset with images of people having blond and non-blond hair may wrongly learn correlation between the label 'blond hair' and the gender of the person in the image. Thus the model fails to classify images belonging to minority groups (in this case blond males and non-blond females). These biases can affect the performance of Al systems handling job recruitment, e-commerce, health care, face detection and recognition 4 5 6. Models can also learn spurious correlations between irrelevant training features and the target labels, instead of focusing on the relevant ones [7] g
Several works focus on mitigating such biases from trained models in a variety of tasks 9
In recent times, large-scale models, pretrained on enormous amounts of data, are being used by machine learning practitioners as feature encoders for numerous downstream applications, as their features are shown to be semantically rich [12 13]. However, do these powerful pretrained features themselves exude the harmful stereotypical biases that are known to affect traditional deep learning systems? A previous work [14] sheds light on these questions (when the pretrained model can be fully finetuned on the downstream dataset) - the downstream models finetuned on top of pretrained models can inherit their biases and such biases can be mitigated simply by manipulating the finetuning data.
In this paper, investigate more constrained yet practical problem setting. With billion-panl paarmmeerr models gaining impetus in today's 's world, it may not be feasible to finetune/retrain such models due to T7 [8]
scarcity resources or unavailability of model weights for privacy concerns [13] 12
15
16
Work done while working project assistant the Indian Institute of Science
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Blond
Blased predictions
Frozen Encoder
Adapter
Non-Blond
Mitigation
Blas amplification
Figure 1: Summary our setup. A frozen pretrained encoder used as feature extractor for a downstream task. An adapter is attached on top of the encoder that learns the bias in the downstream dataset. Finally, the bias is mitigated with the help an adaptive margin loss, leading to unbiased predictions. No knowledge of the bias attribute is assumed apriori.
Hence, for downstream classification tasks, simply the classifier layer on top of the pretrained features trained on the corresponding dataset, keeping the rest of the network frozen. We ask, to what extent can such systems be debiased, given that the pretrained model weights are unavailable?
For the scope of this paper, assume that the downstream dataset consists of multiple groups (e.g. blond males, non-blont males, blond females, non-blond females) due to the presence of a certain bias attribute (e.g. gender). We find that the effect of the bias in the downstream model depends on how well the pretrained feature encodes the target attribute as compared to the bias attribute. The system remains unbiased if the pretrained features are highly aware of the target attribute (even when the bias correlation in the downstream dataset is high). However, if the pretrained features instead predominantly encode the bias attribute, the downstream system becomes biased. A simple solution to the problem is to group- balanbe or reduce the bias-correlation of the finetuning dataset, similar to the suggestion of Wang et al. [14). However, as manipulating the bias correlation of a dataset is not straightforward, we advocate for a specific strategy to debias the system in the given scenario.
To simplify the bias-mitigation task, insert trainable adapter module between the pretrained feature extractor and the classifier. With such setting, after evaluating existing methods, we find that large number of these methods do not yield the expected performance gains compared to the ERM-trained model, indicating the challenges in the problem setting as well as the necessity of designing specific debiasing strategies. To aid the mitigation process, forcefully amplify the bias in the downstream model into the adapter (following previous works 19 20)), and then investigate a number of simple debiasing techniques utilizing this biased adapter. Finally propose a method involving clustering-basee
adaptive margin loss which first clusters the biased feature space, and accordingly sets the margin value for given sample such that it inversely proportional to the frequency of the sample's 's ground truth class in the cluster it belongs to. The problem setting is depicted Fig. We summarize our key contributions in this paper below:
We highlight a practical yet under-explored problem setting on how harmful biases creep into a model when it uses pretrained but blackbox feature extractors obtain features.
We explore the scenarios in which biases can propagate from the pretrained features to the down- stream tasks and demonstrate the necessity and fcasibility of debiasing strategies in such cases.
To debias, we first amplify the bias learnt by the model from the downstream data and propose simple mitigation strategy by utilizing the identified biases and applying an adaptive margin loss. Extensive experiments show that the proposed method is effective across multiple benchmarks.
2 Related Works
Mitigation of biases with the knowledge of the bias attributes, as well as their labels, is widely explored problem [21 22 23 24 25, 26, (7 28). Roh et al. [29] propose sampling algorithm by formulating a combinatorial optimization problem for unbiased sample selection. While Sagawa et al. (23 minimize worst-case training loss over the groups formed using the available bias labels and the target class labels, Zhang et al. (30) disentangle the bias and target representations and
Known biases:
debias the network using mutual information estimator. Some methods adopt semi-supervised

---

approach, where bias attributes are annotated for only a few data samples (20 31 Another body of
work assumes knowledge of the bias attribute but not its specific labels [32 33 351
Unknown biases: Recent works consider the more practical scenario of unavailability of the knowl-
edge of the bias attribute as well as its labels 36
43 This area encompasses a wide variety of works, subset of which is described here. JTT 19] debiases models using misclas- sifications from an ERM-trained model. Some approaches like BPA and GEORGE employ clustering to identify the biases in the dataset 44 45.. Certain works employ the Generalized Cross-Entropy
37
38 39 40
19]
(GCE) loss [46 47 to amplify the biases learnt by the network and then debias it. While most of these employ two branches in the network, with one branch over-learning the bias and the other debiasing it, Ahn et al. [40] utilize the GCE loss to learn the bias, and then find the per-sample gradient of the trained model to obtain a balanced dataset. DebiAN 38 identifies multiple biases by having a discoverer model which optimizes an equal opportunity violation loss. Correct-n-Cotrast [4]] identifies training samples having the same class labels but dissimilar bias features via Empirical Risk Minimization (ERM) training and then applies contrastive loss-similar to Contrastive Adapter [50]-to to bring the target features of these same-class samples closer. Qraitem et al (51) propose a sampling method to reduce dataset biases. Kim et al. [52] use a committee of
works, like LfF 20 37 48
auxiliary classifiers to identify the biases in the network, assigning large weights to the identified bias-conflicting samples during the training of the main classifier. Jeon et al. [53] show how bias in a CNN network is more pronounced at the top layers, and hence, leveraging features from the lower layers can help the model to exploit less biased representations. Kirichenko et al. [54] observe that retraining only the last layer enough for having an unbiased model. However, they require bias annotations in the 2nd stage of training. This is avoided by LaBonte et al. [55] by constructing reweighting dataset using model disagreements for the second stage of training.
Biases in pretrained models: Utilizing pretrained feature extractors for downsteam tasks is a com-
mon trend in recent times. Such encoders may carry biases that crept in from the datasets used for training them many recent works focus quantifying the fairness of such pretrained models (56). Srinivasan et al. [57] study biases in multimodal vision-language systems. Goyal et al. (58) shows how pretrained models trained with self-supervision exhibit lesser biases than those trained with
supervision. Recent works show that biases of pretrained models can be reduced by manipulating the fine-tuming dataset in both Computer Vision and NLP [14] [5]. Salman al. [60] show that bias transfer happens from pre-trained models to the downstream tasks, even when the target data is unbiased. Our problem setting on the other hand consumes features from blackbox pretrained encoder and aims to mitigate the biases arising from that encoder. This is challenging, especially when the bias annotations not available.
3 Problem Statement and Methodology
3.1 Preliminaries and Problem Setting
This work focuses the task of image classification. Let X' = F21, be a set of training images of size N, and (y1,
yx) be the corresponding labels, where each y; E C. Each data point (I;; y/) is associated with hidden spurious attribute a; E A, and consequently group 9: E G, where 9 = C x A. We assume that each group E G is present in the training data. In an unbiased dataset, number of training samples in each g remains approximately equal. However, models tend to learn spurious correlations when there is an imbalance in these numbers. We refer to samples favoured most by the ERM trained models as bias-aligned, and the rest as bias-conflicing 20] 38).
The goal is train model to optimize mapping function f X [RI In ERM training, optimize the Cross-Entropy (CE) loss as defined below for a sample (s. y):
[C|
LT
pS log P.
(1)
where [PI, ... Piccl is a one-hot vector representing y. The corresponding predicted probability vector for the same sample is given by (pi,Pi,..... pfal.
3.2 Biases in Pretrained Features
Extracting features from popular pretrained encoders for downstream applications is a common norm, as being trained on large amounts of data equips these models to perform well variety of
3

---

Table I: Performance of Waterbirds on different pretrained encoders. We compare the model performance for a pretrained ViT-H encoder and a pretrained ResNet-18 encoder for three versions of the training data: a) The original, b) By removing all bias pias-cooflicting (Bi-Co) samples and c) Group-Balances Balanced. For all eases, the test set remains the same.
Dataset
ViT-H
ResNet-18
Type
Worst Group
Average Group
Worst Group
Average Group
Original No Bi-Co samples
88.01
94.79
38,90
76.22
80.06
90.44
18.22
68.25
Balanced
92.37
95.47
83.55
85,94
tasks. We freeze these models to avoid backpropagating into their architectures that are generally large-scale [12] and often only accessible through API calls [15 18 16 17 In this subsection, we choose the WaterBirds dataset [23] to analyse the effect of the pretrained model on the downstream performance (see Section for details). We observe two different scenarios here:
Scenario 1:
Pretrained features are target-aware. Mehta et al. [6] find that if one chooses a proper pretrained encoder for their specific downstream task, ERM training
linear classifier attached to the pretrained embedding is enough to obtain unbiased predictions. For instance, using ViT-H 14 encoder pretrained on the SWAG dataset [62] followed by end-to-end ImageNet [63] finetuning can achieve state-of-the-art results on the Waterbirds dataset. We find that even if all the bias-conflicting samples removed from the training data, the worst group accuracies remain sufficiently high for the test set (see Table [ Thus, the pretrained model is highly aware of the downstream target attribute, and no debiasing is required. However, single pretrained model is hardly& panacea for bias mitigation. For instance, the worst group accuracy for the CelebA (64] dataset's 's Blond Hair Classification is merely 6.71%!
Scenario 2: Pretrained features are bias-aware. Contrary to Scenario 1, when we use an ImageNet- pretrained ResNet-18 the pretrained encoder, notice that the worst group accuracies considerably low for the original Waterbirds dataset itself (see Table T An even further drop is seen when the bias-conflicting samples removed. This shows that this feature encoder exhibits the biases present in the downstream dataset. One simple mitigation strategy is to reduce the bias- correlation in the training set. Craun-balancing the Waterbirds dataset leads to considcrably uniform accuracies across all groups, irrespective of the underlying pretrained model, as shown in Table [ This may not always be simple firstly, the bias attribute may be known apriori to a practitioner, and secondly, coal-baaanci or reducing the bias in the downstream dataset may be expensive. Further analyses on these lines are presented in Appendix subsection A.. Thus, for such systems, an explicit mitigation strategy is required so that the model predictions are unbiased.
3.3 Bias mitigation
To mitigate biases this problem setting, we consider an image classification model that consists of 3 primary components (see Fig. []: a) A pretrained feature extractor m to extract the image features plocked = m(x) (m is blackbox, i.e. frozen) b) an adapter consisting of a multi-layer perceptron model h comprising of a single non-linear hidden layer, projecting flocked to new latent space defined as f = h flucked)) c) classifier C, attached to if to obtain the final predictions ý C(f).
Performance of existing methods. A large body of work exists in the domain of bias mitigation. However, most of these works assume fully trainable feature encoder. We pick a few representative ones like DebiaN [38] BPA [45), GEORGE [44], LiF (20). JTT 19 and Contrastive Adapter (Co- Ada) (50 (see Section 2 for details) and explore their efficacy in this setup. We choose three popular
(descriptions available in Sec 4 and use a frozen 18 feature encoder pretrained on the ImageNet dataset for this experiment. We have number of key observations (see Table 2):
benchmarks: Waterbirds, CelebA, and ColorMNIST-O..C
LiF performs well on Waterbirds and ColorMNIST-0.999 9.95, however, it drops by 14% on CelebA. Similar trends are seen for other methods as well. This shows that existing methods exhibit inconsistencies in this constrained problem setting.
Co-Ada, which was designed to improve the zero-shot performance of foundation models (without hackpropagating into the underlying model), works consistently well across all datasets. However, Contrastive Adapter is computationally expensive (Appendix Table 17.

---

Table 2:
Performance of existing methods on the proposed problem setting. We observe that for three different benchmarks, performance of existing methods either close to that of the ERM model (measured by AERM), not consistently high. Co-Ada [50] is one exception among compared methods, having the highest worst-group accuracies for all three benchmarks.
Waterbirds
CelchA
CMNIST-0.995
Backhone
Model
Worst
AZRM
Worst
ABRM
Bi-Co
ATRM
ERM
38.90
27.20
0
49.09
DebiAN
38
58.94
20.04
26.
-1.1
49.82
0.73
BPA
58.70
19.80
66.71
39.51
17.84
-1.25
ResNet-18
LIH
20
66.09
27.19
13:26
-13.94
71.61
22.52
JTT
19
49.84
10.94
56.25
29.05
42.86
- -6.23
GEORGE 44
59.35
20.45
42.22
15.02
48.77
-0.32
Co-Ada
50
67.57
28.67
78.37
51.17
65.48
16.39
CelebA
Waterbirds
ColorMNIST
100
Blas-Alinned Dias-Confictin
75
Training
Accunacy
50
P-NB
L-LB
25
F-8
L-WB
M-B
W-WB
Training Acc
Training Acc
50-2
0.01
0.05
0.7
0.5
1.0
1.5
5e-2
0.01
0.05
0.
0.5
1.0
1.5
56-2
0.01
0.05
01
05
1.0
(
(6)
Figure 2: Effect of increasing weight decay. (a) For CelebA, as À increases, the worst group (Blond Male (M-B)) accuracy reduces, though the accuracies for Blond Female (F-B) and Non-blond Males (M-NB) still remain high. (b) For Waterbirds, we see a fall in both Land-Waterie and Water-Landbinn (W-LB) accuracies with increasing X, though the scores for Land-Landbird (L-LB) and accuracies reduce expected, the training accuracy dips beyond X 0.1.
(L-WBB
(W-WB) remain high. (c) For ColorMNIST, while the bias-conflicting
The above observations serve as motivating principles behind the proposed method for this problem setting that computationally efficient and effective across benchmark datasets.
3.4 Our approach
In this subsection, we present our approach. Inspired by previous works 20 19 38), we first amplify the biases in the system. Following this, we discuss few alternatives for mitigation that eventually lead upto our final method.
Bias-amplified Training. We propose learning features f first using an adapter (i.c. an MLP layer) on top of the pretrained features flacked through ERM training, and then identifying the biases from these learnt features. The goal is to reduce the worst-group accuracies of the training set while maintaining the overall training performance thereby amplifying the bias learnt by the model. As indicated in previous works 65] 23), we find that increasing the weight decay À reduces worst- group performance of the training considerably. We next show the effect of changing À popular datasets: Waterbirds, CelebA and ColorMNíST-0.99) In Fig. 2la), we observe that with increasing weight decay, the worst group accuracies reduce for CelebA's Blond Hair classification 64 (i.e. for Blond Males), and also dips for Non-blond Females, while the other two groups' performances remain high. This points to the amplification of gender bias, i.c.. the model tends to predict Blond Hair for female images and Non-blond Hair for male images. A similar phenomenon is noticed in the case of Waterbirds (23] shown Fig. (b). Although the same pattern exhibited in the case of ColorMNIST-0.999 in Fig. c). an interesting observation also emerges. For À = 0.1, training accuracy drops off quite steeply. So, with the goal of reducing the performance on bias conflicting samples, the model may end up not learning anything meaningful at all (as also seen Fig. a)). This hints at a tradeoff, where we À to a high value while ensuring the training accuracy does not fall drastically. We select the model based on the training accuracy as higher training accuracy ensures optimal learning of the bias-aligned data points even when there less learning of the bias-conflicting samples. To avoid all predictions from collapsing single class,
5

---

Comparison of different alternatives. We compare three methods loss-weighted CE loss (LW), cluster-weighted CE loss (CW), and cluster- based margin loss (CM). While LW leads to improved scores compared to an ERM-trained model, CW further improves upon it for CelebA and ColorMNIST-0.995 Finally, the proposed CM outperforms the above two methods by a large margin. All results are with respect to an Irreeeee ResNet-18 18 model.
Table 3:
Waterbirds
CelebA
ColorMNIST-00999
Method
Worst
Avg
Worst
Avg
Bi-Co
Bi-Al
Avg
ERM
38.90
76.22
27.20
75.43
49.09
100.00
74.55
LW
69.78
83.52
15.56
79.84
50.58
99.90
75.28
CW
70.16
84.21
69.4
85,16
65.00
99.05
82.00
CM
80.29
84.56
81.61
86.04
72.56
96.28
84.42
sample equally from each class for each batch of the training set. We call the obtained bias-amplified model B. and utilize its knowledge to design few alternative mitigation strategies. For debiasing, we use the same architecture as that of the biased model B, and denote by D.
Before delving into the rest of our method, we discuss a caveat here. An attentive reader might enquire to whether the above technique of bias amplification will always work. We reiterate that we consider the bias in the downstream dataset to align with that in the feature encoder. An example the contrary has been discussed in Scenario I (subsection 3.2 This presents an interesting future direction where one might attempt to debias a feature encoder which does not capture the downstream dataset bias.
Technique 1: Loss- weighted (LW) Cross-Entropy Loss. We use a weighted CE loss to train CI
D: L.W = L Σド=1 Pj log p.. where LP is the CE loss computed from B for the sample with respect to its ground truth label. The intuition is that B being biased. it would upweight the CE loss for the bias-confficting sample in the mitigation stage. On implementing this for the Waterbirds, CelebA and ColorMNIST-0.999 we find that even though the mitigation performance increases as compared to the ERM-trained model, the improvement is not satisfactory (see Table (3).
Technique 2: Cluster-weighted (CW) Cross-Entrops Loss. The adapter features f in the biased model B are expected to encode the bias in the downstream dataset. Hence, we explore cluster-based weighting scheme for bias mitigation by clustering the bias-amplified feature space f in B. We define K the number of clusters obtained and m. as the proportion of class within the cluster 2, i.e.,
n + Evec nz. +
where n is the number of times samples of class e have occurred in cluster z. For any sample the training data, we find the cluster it belongs to (say z), and then weight the CE loss for training D E (1 mnf);; )e; log2;- P.-- Intuitively, the loss is upweighted if the
in the following way: L sample belongs to minority class in its cluster, otherwise it is downweighted. From Table we find
that it is a significant improvement over LW. Infact, from Table we find that it performs on par with Co-Ada [50] for Waterbirds and ColorMNIST-0.995.
3.5 Cluster-based Adaptive Margin Loss
In Technique 2, we see that clustering- weighting methods can he effective, where we upweight samples belonging to the minority classes in a cluster, whereas we downweight the others proportion- ally to increase the importance of the sparse samples. We further ask, inside each cluster, can we make the features of the individual classes more discriminative? We conjecture that in such a case, it will be easier for the debiasing model D to distinguish among the samples from the different groups in the dataset (assuming that the clusters are an approximate representation of the true groups in the dataset). To this end, margin losses are an efficient class of loss functions that can reduce intra-class distance and increase inter-class distance the feature space. Enforcing such margin constraints on hyper- spherical feature spaces has been shown to be beneficial in case of deep face recognition 6. 5 66].
long-tailed learning [67], few-shot learning (68) and the language-bias problem [69 70 in Visual
Question Answering (VQA). Especially, inspired by the works of anigtaileddPee learning and VQA,

---

make the margins adaptive to ensure discriminative features among the frequently and infrequently occurring classes within cluster. To achieve this, we utilize the weights used in the cluster- weighted (CW) CE loss (defined above) as the margin values. We begin by defining the following:
Normalized CE loss: First, we reformulate the CE loss cosine loss 5 6 by 1.2normalizing the classifier weight vectors W E C' (recall that C is the classifier) for each class ct E C (k
1, 2, -.. (C]), and the trainable feature f. We define Wx W and foorm [[Fi]* where is TTWLl
f
a scaling parameter. The bias term is set to for simplicity. Let Ox be the angle between f and cos = s cos 0z. The features form are thus distributed on hypersphere with a radius s. This makes the normalized CE loss for single sample
kk- Therefore, the logit for each class Gj becomes: ØK- W T "foorm ||
[0
exp(s cos 0z)
LNS =
Pk log
(3)
Σ/e exp(s cos 0g)
1=1
Adaptive Margin Loss. Inspired by the ArcFace loss 6 used in face recognition, we define the adaptive margin loss here. The loss adds a margin penalty to the angle between the features f, and the classifier weights W& for the hthh class. Since the margin is placed on the angle, it maps exactly to the "geodesic" distance on the hypersphere [6), leading to highly discriminative features. For face recognition, it suffices to have constant value for the margin penalty (0.5 for ArcFace), Since we want discriminative features for the frequently and infrequently occurring classes in a cluster, we ensure that the training samples assigned to a cluster and belonging to the majority class in that cluster are allowed smaller margin than those belonging to the minority class. We assign the loss weights from CW as the margin values (i.e. eq 2). Specifically, for data sample belonging to class with cluster id z, we denote its margin as m = (1 - mā) (see Fig. 3 for an overview of the system). Finally, we define the angular adaptive margin loss for each sample belonging to the tuple t 2c,2)
by combining eq.s 3 and 2
IC
exp(s cOs + +m²,) )) Cl Σ Σ exp(s cos(O; + m
Margin
Pk log
Gaussian Randomization of the margins: While we estimate the margins for the margin loss by clustering the features from the biased model B, the obtained clusters can capture noisy signals, leading to erronous results. Alluding to our previous example of a dataset of people with blond and non-blond hair with gender bias (Section D. since most of the females have blond hair, all such females should be given lesser margin values, whereas blond-haired haired males should be given higher values. But clustering leads to noisy grouping of samples in the dataset, whereby a certain group may include less blond females and more blond males. Therefore, for those female blond samples, comparatively large margin penalty will be assigned, while the blond male samples will get a low margin penalty. Also, the margins being high for sparse classes and low for the frequent ones in cluster, they may over-learn the former, ignoring the latter. In face recognition, Boutros et al. [7]] suggest that in typical margin losses, setting constant margins can limit the generalizability and discriminative power of a model and advise the introduction of stochasticity in the margin values to boost the same. This stochasticity can help smooth out the effect of the noises in clustering on one hand and make the model more generalized to all groups. To enforce this, we use a randomized version of m called Fe. where F N (m r). Recall that m denotes the proportion of samples belonging to class e and cluster z. N is the Gaussian distribution, and a is the standard deviation (a hyperparameter). This impedes the model from overcorrecting with respect to the rare classes in group, thus increasing its generalizability while also compensating for the errors in margin values due to the noisy cluster labels. Finally, we obtain the randomized margin r We then replace m with r in eq. 4 Our overall method is summarized in Fig.
x for each
4 Experiments and Results
Dataset Details. We evaluate our method on multiple benchmarks. Waterbirds 23 is dataset of birds, labeled as waterbird if the bird is a seabird, and landbird otherwise. A spurious correlation exists between these labels and the background-land or water. The dataset has 4795 training samples.
The ColorMNIST dataset (CMNIST) (72) is generated from MNIST [73] where each digit is
7

---

Waterard Landard
Land
Water
Dasi Hair Bleod Hair
Famale
Siage ERM Training
Stge Clustering
Millptioon
Stage 8
Group margins
Waterbird
Margin losa
Blond
Er
7
2
FFizznn
Clustering
Learrable
One
Broodor
Utcappop
Figure 3: Overview our method. There are steps: a) Bias-amplified training of the model
through Cross-Entropy Loss with high weight decay, b) Clustering the biased features f. .) Miti- gating the biases by using the resultant clusters calculate the margins and the corresponding loss, leading to decent performance on the bias-conflicting data points.
Table 4: Performance comparison of method (CM). Worst and Average Group Accuracies for Waterbirds & CelebA (Blond Hair), Bias-Conflicting (Bi-Co), Bias-Aligned (Bi-Al) and Average Accuracies for CMNIST-0.9. Highest accuracies are marked in bold, the 2nd highest ones underlined. All scores for our method are averaged over 3 seeds.
Waterbirds
CelehA
CMNIST-0.9
Backhone
Model
Worst
Avg
Worst
Avg
Bi-C
Bi-AI
Avg
ERM
38.90
76. 76.2241.04
27. 29
75.4340..57
61.7250.00
99. 90.900
65.54
ResNet-18
Co-Ada 50
67.57 1122
28.37
85. 7..7s
80.82
89 63
85,22
CM (Ours)
80.29#2.00
84.56=1.20
81.61 +1.02
86.04 86.04
81.91 #0.40
92.75=00.50
87.33
ERM
67. 1719
81.6540777
36,00 08.0092042 42
79. 5.00000
90,22
99. 18 1.4
91
CLIP RN50
Co-Ada 50
81.95
87 87.35 tt0.as
90.52
91.88
83.81 83,81200.0
95. 68..6
89.74
CM (Ours)
79.28 10.94
86.79
90.4720-43
92.52 1.042
94.07 1:000
96. 40..0.06
95.22
1.000
ERM
59.67
83.31 $3.31=0.80
31.72=0.64
77. 77.82#0.02
88.93
99. 69 to.1s t9.uus
94.31 do0.14
VIT-B
Co-Ada 50
63.71 11:18
80.24
85.87
89 11.17
79.71
2..007
83.28 1.01
(Ours)
74.92
82.85
87.28 10.00
89.31 20.00
91.32-90.1
98.59-40.15
94.96 2:016
associated with one color in its background most of the time (see Fig. 3). It has 50000 training and 10000 test images. We evaluate our margin loss method on two variants of CMNIST: CMNIST-0.9, each digit is associated with one color 90% of the time and other colors only 10% of the time (moderate bias). In CMNIST-0.9955, each digit associated with a single color 99.5% of the time (severe bias). The real-world CelebA dataset [64 consists of 202, 599 face images of celebrities along with 40 attributes. We choose Blond Hair as the target attribute, as it is known to suffer from severe gender bias [28 38 45). For evaluating models on CclebA and Waterbirds. we find the accuracy of each group g = (c, a) in the test set 200 45). and report the worst of all the groups in G and their average. For CMNIST, maximally associated colors) and bias-confi (i.e. images of digits with other colors) accuracies, along with the average of all the groups (38. We also show the results for two more real-life datasets (BAR 200 and UTKFaae (74) in Appendix Table
report the overall bias-aligned (i.e. images of digits with their
Implementation Details. We evaluate our method multiple pretrained encoders: ImageNet- pretrained R8 (75) and ViT-Base (6) encoders, and CLIP ResNet-50 image encoder [12 (pretrained on other datasets). All our implementations use 1-hidden layer of M neurons followed by a non-lincarity in the adapter. and the classifier is linear layer. For clustering, we use the KMeans algorithm. We discuss the hyperparameters and their analyses in Appendix subsection A.3 We assume the availability of a small group-balanced (but unannotated) validation set and calculate the overall accuracy over this dataset for model selection during the bias-mitigan mas-mmitigttion phase.
4.1 Results
We compare our baseline against Contrastive Adapter [50]. given that it performs the best among all other methods across different benchmarks (see Table and the ERM model. The results are presented in Table 4 where we compare our method against the ERM method and Co-Ada. For comparison with other competing methods, see Appendix Table We also present the results for the ViT-H 14 encoder in Appendix Table 13
8

---

Table 6: Ablations of the proposed method: Here show the roles of the different components of our model using ResNet- 18 as the pretrained backbone.
Waterbirds
CelebA
CMNIST-0.9
Model Component
Worst
Average
Worst
Average
Bi-Co
Bi-Al
Average
CM (Ours)
80.20 29
84.56
81.61
86.04
81.91
92.75
87.35
Constant Margin (0.5)
55.76
77.45
34.44
77.63
44.91
100.00
72.45
No Randomization
79.28
82.9
78.51
86.07
82.28
91.23
86.70
Clustering from flacked
74.92
84.95
80.52
85.04
75.96
94.88
85.42
For the Waterbirds dataset, our method far outperforms Contrastive Adapter for ResNet-18 and ViT-B (by 12. 12. and 11.21% respectively), though for CLIP the worst-group accuracy is slightly lesser (by 2.67%). In case of CelebA, for ResNet- 18 and ViT-Base, the margin loss outperforms Contrastive Adapter (by 3.24% and 1.41%). For the CLIP encoder, we observe the two methods to perform similarly. The CMNIST-0.9 dataset is affected by moderate degree of the background color bias, whereas for CMNIST-O. 995 the bias strength is severe. In case of both datasets, our method outperforms Contrastive Adapter for all backbones by a large margin, especially for CMNIST-0.9 (see results for CMNIST-0.99 995 in T Compared to Contrastive Adapter, our method has another discussed in the Appendix Table 17 Our method is effective even
advantage: it is time-efficient when finetuning to the encoder weights is possible (see Appendix subsection A.7
5 Ablations
In this subsection, we discuss the important com- ponents of our approach and the margin loss strategy. We first show what happens when the margin penalty is constant as the ArcFace loss (6) itself. Then, we show the role played by the Gaussian randomization of the group-based margins (read subsection 3.3 and Appendix sub- section ??). Finally, show what happens when we estimate the different bias-groups in the training data by clustering the blackbox features instead of the bias-amplified adapter performed Water-
Table 5: Performance comparison of our method for CMNIST-009955, We report Bins-Conflicting (Bi-Co), Bias- Aligned (Bi-Al) and Average Accu- racies. Highest accuracies are marked in bold, the 2nd highest ones are underlined. All scores for our method are averaged over 3 seeds.
CMNIST-0.9995
Model
Bi-Co
Bi-Al
Avg
ERM
49.0g
100.00=0.70
74.55=0
layer. All evaluations birds, CelebA (Blond Hair classification) and CMNIST-0.9 with ResNet-18 the backbone. The results are shown in Table
RI8
Co-Ada
25
30.42
74.98=0.40
CM (Ours)
72.56-0.85
96 96.28
84.42
ERM
56.87#00.1
100,00-9:
48.41=0.02
CLIP
Co-Ada
77.3
92. 18:0.06
84. 84.76=0.15
(Ours)
80.65=0.05
23.29
$6.97
Constant Margin. We present the results ob- tained by keeping the margin value constant at 0.5 as suggested by ArcFace [6] (no Gaussian randomization is applied here). This study en- ables to judge the utility of the adaptive nature of the margins. While for Waterbirds and Blond Hair, the scores are better than those of ERM, there's a sharp drop in both worst and average group accuracies compared to our method. The trends similar in CMNIST-0.9 as well. Hence, conclude that while margin losses improve model performance, a constant margin is not enough.
59.21
00110
79.61 =0.42
VIF-
Co-Ada
76.88
92.27=0.01
84.58
CM (Ours)
77.9 77.97=07
#0.80
8G.42 #1..4.4
Without Randomization. We remove the Gaussian randomization of the margins, but here the margins are adaptive (as per eq. [ For Waterbirds, we see that both the worst and average group ac- curacies reduce compared to the final scores. Worst group accuracy drops for CelebA well, though the average group score remains similar. CMNIST, the bias-conflicting accuracies slightly improve at the cost of small drop in bias-aligned performance. Thus, overall we find that randomizing the margins for each sample is helpful.
Clustering from plocked Our model identifies biases by clustering the adapter features f. However, the pretrained features flocked can be useful as well, if the pretrained model and the downstream

---

dataset share similar biases. Hence, we cluster the pretrained features flacked instead of f to obtain the margin penalties. The rest of the training pipeline remains the same, as described in subsection [.3 For all 3 datasets, we see a reduction in the worst and average group accuracies, however, the scores are still close to that of the proposed approach. Since clustering the pretrained features does not require an extra stage of ERM training, therefore in presence of time-constraints, flocked can be used as proxy for f. and subsequently, relatively decent model predictions can be obtained.
6 Discussion and Limitations
Our method specifically targeted towards cases where the bias in the downstream dataset is already encoded in the pretrained model. We believe that detecting if this assumption holds apriori is highly challenging in the absence of the bias labels. We put forward few suggestions to identify the scenarios that fit this assumption: a) Obtain bias annotations for the small validation set. If the worst group accuracy of the validation set does not reduce substantially with increasing weight decay, indicates that the features have stronger signals of the target class than that of the bias, making it harder to capture the bias. If the bias annotations of the validation set cannot be obtained due to privacy concerns, the overall validation accuracy can indicate strength of the bias. For example, the difference between the validation and training accuracy 25% for an ERM trained method for the Waterbirds dataset on the ResNet-18 18 backbone. The higher this difference, the more the indication that the model is overfitting to more and more samples. Such overfitting can indicate that the model is learning the bias in the dataset, thus not generalizing on the bias-conflicting samples. While we have mentioned the situation when the pretrained model is aware of the target attribute in Section 3 another potential use case might be when the pre- pre-training data distribution may be different from that of the downstream dataset. We prescribe finetuning the pretrained models to amplify and mitigate the biases in such case. A more practical approach would be to choose another model, as also suggested by Zhang et al. [50].
One limitation of our method is that it currently relies on the existence of biased groups in the training set (i.e., the bias labels are expected to be categorical) as it uses clustering to identify the biases. Furthermore, since approach (along with most competing methods) relies amplifying the bias first and then mitigating the same, it involves a risk wherein if the mitigation module fails, the bias in the system may be exacerbated. With these limitations in mind, we hope this work initiates much required discussion in this direction leading to more sophisticated and targeted solutions in the near future.
Scalability of the proposed approach to large datasets and models. Since our method involves clustering the features, if the dataset is large-scale, one can randomly sample small percentage of it to do the clustering. For example, on clustering only 10% randomly sampled images from the CelebA dataset, we find that the worst group accuracy is 81.11%, whereas the average group accuracy is 85.6% on the R8 backbone. The scores become 80.55% and 85.6% respectively when the clustering is performed only on 1% of the images (with full clustering, the scores are 81.61% and 86.04 04% respectively). On the other hand, for very large models, the requirement is to be able to load the model into GPU memory. Our method adds negligible overhead owing to the addition of only an adapter and a classifier layer.
Conclusion
In this work, we explored the effect of using pretrained but frozen feature extractors on downstream applications with biased datasets and found the need for specific bias mitigation strategies in cases where the biases in the downstream dataset align with the pretrained encoders. Such mitigation is challenging as the encoder is blackbox. While we found many of the existing works be inadequate, we proposed a simple method where we first amplified the biases present in the downstream dataset and then employed a clustering based margin loss to mitigate the same. Our experiments showed this method to be an efficient and effective technique across multiple benchmarks. Lastly, it is our hope that this work will initiate a much-required discussion among the scientific community on encouraging similar works in this highly practical problem setting. We share further studies, detailed hyperparameter analyses and experiments of the proposed method the appendix.
10

---

8 Acknowledgements
This research funded by the Qualcomm Innovation Fellowship. Abhipsa Basu is supported by the PMRF Fellowhsip.
References
[1] Jialu Wang, Yang Liu, and Xin Eric Wang. Are gender-neutral queries really gender-neutral? mitigating gender bias in image search. arXiv preprint arXiv:2 2021.
[2] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint urXiv: 2017.
[3] Danaë Metaxa, Michelle A Gan, Su Goh, Jeff Hancock, and James A Landay. An image of society: Gender and racial representation and impact in image search results for occupations. Proceedings of the ACM on Human- Computer Interaction, 3(CSCW1):1-22 2021.
[4] Erik Hjelmås and Boon Kee Low. Face detection; A survey. Computer vision and image understanding, 83(3):236-274, 2001.
[5] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 5265-5274, 2018.
[6] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690-4699, 2019.
[7] Robert Geirhos, Jörn- Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning deep neural networks. Nature Machine Intelligence, 2(1 1::6656673, 2020.
[8] Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don't just assume; look and answer: Overcoming priors visual question answering. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 4971-4980, 2018.
[9] Schrasing Tong and Lalana Kagal. Investigating bias in image classification using model explanations. arXiv preprint arXiv:2012.05463, 2020.
[10] Vikram V Ramaswamy, Sunnie SY Kim, and Olga Russakovsky. Fair attribute classification through latent space de-biasing. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9301-9310, 2021.
[11] Jean-Rémy Conti, Nathan Noiry, Stephan Clemencon, Vincent Despiegel, and Stéphane Gentric. Mitigating gender bias in face recognition using the von mishe- mixture model. In International Conference on Machine Learning, pages 4344-4369. PMLR, 2022.
[12] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 7768-8863. PMLR, 18 24 Jul 2021.
[13] Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15180-15199 2023.
[14] Angelina Wang and Olga Russakovsky, Overwriting pretrained bias with finetuning data. In Proceedings of the IEFE/CVE International Conference on Computer Vision, pages 3957-3968. 2023.
11

---

[15] Google. Google Vision API.
2004.
[16] Amazon Web services.
Amazon Rekognition
API. https: htte:///wwaaaaaznncooom
rekognition/
[17]
Microsoft. Azure AI Vision.
ai ai-ser snervices/aivviiion
[18] IBM. IBM Watson Visual Recognition.
https://developen www.:eer......o.oo...mo wlm.ccmm/comoonettt..
w wwta---iisaa--eemnttiiiioee
[19] Evan Liu, Behzad Haghgoo, Annie Chen, Aditi Raghunathan. Pang Wei Koh, Shiori Sagawa, Perey Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pages 6781-6792.
PMLR, 2021.
[20] Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: Training debiased classifier from biased classifier. In Advances in Neural Information Processing Systems, 2020.
[21] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9012-9020, 2019.
[22] Yi Li and Nuno Vasconcelos. Repair: Removing representation bias by dataset resampling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9572-9581, 2019.
[23] Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint 19Xiv:/911008/0 2019.
[24] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv: /900/0999....,,. 2019.
[25] Damien Teney, Ehsan Abbasnejad, and Anton van den Hengel. Unshuffling data for improved generalization in visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1417-1427, 2021.
[26] Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentan- gling deep representations for bias correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15508-13517, 2021.
[27] Zeyu Wang. Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. Towards fairness in visual recognition: Effective strategies for bias mitigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8919-8928 8928, 2020.
[28) Zhibo Wang, Xiaowei Dong, Henry Xue, Zhifei Zhang, Weifeng Chiu, Tao Wei, and Kui Ren. aawae adversarial perturbation towards bias mitigation for deployed deep models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 10379-10388, June 2022.
[29] Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. Sample selection for fair and robust training. Advances in Neural Information Processing Systems, 34:815-827, 2021.
[30] Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, and Jiebo Luo. Learning bias-invariant repre- sentation by cross-sample mutual information minimization. In Proceedings the IEFF/CVF International Conference on Computer Vision, pages 15002-15012, 2021.
[31] Sangwon Jung, Sanghyuk Chun, and Taesup Moon. Learning fair classifiers with partially annotated group labels. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10348-10357, 2022.
12

---

[32] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. Advances in neural information processing systems, 32. 2019.
[33] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't 't take the easy way out: Ensemble based methods for avoiding known dataset biases. arXiv preprint arXiv:1909.0366 03683, 2019.
[34] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv: 1.1....1..2211 12231, 2018.
[35] Haohan Wang, Zexue He, Zachary C Lipton, and Eric P Xing. Learning robust representations by projecting superficial statistics out. arXiv preprint arXiv: arXiv: 0et./99330666.6 2019.
[36] Elliot Creager, Henrik Jacobsen, and Richard Zemel. Environment inference for invariant learning. In International Conference Machine Learning, pages 2189-2200 PMLR, 2021.
[37] Jungsoo Lee, Eungyeup Kim, Juyoung Lee, Jihyeon Lee, and Jaegul Choo. Learning debi- ased representation via disentangled feature augmentation. Advances in Neural Information Processing Systems, 14:25123-22133 2021.
[38] Zhiheng Li, Anthony Hoogs, and Chenliang Xu. Discover and Mitigate Unknown Biases with Debiasing Alternate Networks. In The European Conference on Computer Vision (ECCV), 2022.
[39] Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographies through adversarially reweighted learning. Advances in neural information processing systems, 33:728-740, 2020.
[40] Sumyeong Ahn, Seongyoon Kim, and Se-Yonng Yun. Mitigating dataset bias by using per- sample gradient. In The Eleventh International Conference on Learning Representations, 2023.
[41] Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves cross- domain generalization. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pages 124-140. Springer, 2020.
[42] Luke Darlow, Stanistaw Jastrzębski, and Amos Storkey. Latent adversarial debiasing: Mitigating collider bias in deep neural networks. arXiv preprint 1.t:::20/../450000 2020.
[43] Youngkyu Hong and Eunho Yang. Unbiased classification through binscconee and bias- balanced learning. Advances in Neural Information Processing Systems, 34:26449-26461, 2021.
[44] Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Advances in Neural Information Processing Systems, 13:19339-19352. 2020.
[45] Seonguk Seo, Joon- Young Lee, and Bohyung Han. Unsupervised learning of debiased repre- sentations with useudo-attributer In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 16742-16751, 2022.
[46] Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31, 2018
[47] Sheng Liu, Xu Zhang, Nitesh Sekhar, Yue Wu, Prateek Singhal, and Carlos Fernandez-Granda. Avoiding spurious correlations via logit correction, 2023.
[48] Sheng Liu, Xu Zhang, Nitesh Sekhar, Yue Wu, Prateek Singhal, and Carlos Fernandez-Granda Avoiding spurious correlations via logit correction. arXiv preprint arXiv:2212.01433 2022.
[49] Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Fino, and Christopher Ré. Correct-n-contrast: contrastive approach for improving robustness to spurious correlations. 2022.
arXiv preprint
13

---

[50] Michael Zhang and Christopher Ré. Contrastive adapters for foundation model group robustness. Advances in Neural Information Processing Systems, 35:21682-21697, 2022.
[51] Maan Qraitem, Kate Saenko, and Bryan A Plummer. Bias mimicking: A simple sampling approach for bias mitigation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2031 1-20200, 2023.
[52] Nayeong Kim, Sehyun Hwang, Sungsoo Ahn, Jaesik Park, and Suha Kwak. Learning debiased classifier with biased committee. arXiv preprint arXiv:2206. 10843, 2022.
[53] Myeongho Jeon, Daekyung Kim, Woochul Lee, Myungjoo Kang, and Joonseok Lee. conser- vative approach for unbiased learning on unknown biases. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 16752-16760, 2022.
[54] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.029377 2022.
[55] Tyler LaBonte, Vidya Muthukumar, and Abhishek Kumar. Towards last-layer retraining for group robustness with fewer annotations. arXiv preprint arXiv:2309.0853:: 2023.
[56] Priya Goyal, Adriana Romero Soriano, Caner Hazirbas, Levent Sagun, and Nicolas Usumier. Fairness indicators for systematic assessments of visual feature extractors. In 2022 ACM Conference Fairness, Accountability, and Transparency, pages 70-88, 2022.
[57] Tejas Srinivasan and Yonatan Bisk. Worst of both worlds: Biases compound in pre-trained vision-and- lad-uug models. arXiv preprint arXiv:2104.08666 2021.
[58] Priya Goyal, Quentin Duval, Isaac Seessel, Mathilde Caron, Mannat Singh, Ishan Misra, Levent Sagun, Armand Joulin, and Piotr Bojanowski. Vision models are robust and fair when pretrained on uncurated images without supervision. arXiv preprint arXiv:2202.08366 2022.
[59] Ryan Steed, Swetasudha Panda, Ari Kobren, and Michael Wick. Upstream Mitigation Is Not All You Need: Testing the Bias Transfer Hypothesis in Pre-Trained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3524-3542, Dublin, Ireland, May 2022. Association for Computational Linguisties.
[60] Hadi Salman, Saachi Jain, Andrew Ilyas, Logan Engstrom, Eric Wong, and Aleksander Madry, When does bias transfer in transfer learning? arXiv preprint arXiv:2207.02842. 2022.
[61] Raghav Mehta, Vitor Albiero, Li Chen, Ivan Evtimov, Tamar Glaser, Zhiheng Li, and Tal Hassner. You only need good embeddings extractor fix spurious correlations. arXiv preprint arXiv:2212.06254, 2022.
[62] Mannat Singh, Laura Gustafson, Aaron Adcock, Vinicius de Freitas Reis, Bugra Gedik, Raj Pra- teek Kosaraju, Dhruv Mahajan, Ross Girshick, Piotr Dollár, and Laurens Van Der Maaten. Revisiting weakly supervised pre-training of visual perception models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 804-80 814, 2022.
[63] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. leee, 2009.
[64] CelebA dataset. mittp://mmlan ie cuhk. edu. hb//projects/Celebb. html
[65] Sriram Yenamandra, Pratik Ramesh, Viraj Prabhu, and Judy Hoffman. Facts: First amplify correlations and then slice to discover bias. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4794-4804, 2023.
[66] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212-220, 2017.
14

---

[67] Kaidi Cao. Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distributionaawee margin loss. Advances neural information processing systems, 32, 2019.
[68] Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang. Boosting few-shot learning with adaptive margin loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12576-12584, 2020.
[69] Yangyang Guo, Liqiang Nie, Zhiyong Cheng, Feng Ji, Zhang, and Alberto Del Bimbo. Adavqa: Overcoming language priors with adapted margin cosine loss. arXiv preprint arXiv:2105.01993, 2021.
[70] Abhipsa Basu, Sravanti Addepalli, and R Venkatesh Babu. Rmivqa: A margin loss approach for visual question answering with language biases. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11671-11680, 2023.
[71] Fadi Boutros, Naser Damer, Florian Kirchbuchner, and Arjan Kuipper. Elasticface: Elastic margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1578-1587, 2022.
[72] Hyojin Bahng. Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. In International Conference on Machine Learning (ICML), 2020.
[73] MNIST dataset.
http://yann... lecun. com/ htmn/exdb/mmiitt
[74] Zhifei Zhang, Yang Song, and Hairong Qi. Age progressionfregression by conditional adver- sarial autoencoder. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5810-5818, 2017.
[75] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[76] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
A Appendix
A.1 Biases in Pretrained Features - Further Analysis
In the main paper, we looked at two scenarios on how the absence or presence of biases in the pretrained feature encoder affects the downstream model's 's performance on finetuning data. However, for the analysed cases, we did not look into the specific relation between the pretraining data and the downstream data. In the appendix, we perform a controlled and detailed analysis to answer 4 questions this regard:
Q1: What happens the pretraining data is heavily biased and so is the downstream dataset?
To answer this question, pretrain ResNet-18 18 model with the CelebA dataset to make it heavily gender-biased by performing gender prediction itself as the pretraining task [14). During the down- stream step, choose to predict the Blond Hair attribute of the CelebA dataset, which is known to be heavily correlated with gender. We class-balance the finetuning dataset to have equal number of blonds and non-blonds, Finally, we also sample from the data points in the downstream data is approximately 1 1/4th of the pretraining data.
to avoid the downstream predictions from collapsing into single class. downstream data to ensure that the total number of
Q2: What happens if the pretraining dataset heavily biased but the downstream dataset is less biased?
15

---

Table 7: Dependence of downstream model performance on pretraining data. When the pretraining model has high amount of bias, the downstream data performance heavily affected if the latter itself is biased. On the other hand, the performance is more stable if i) the pretraining data has high bias and the finetuning data has low bias, ii) the pretraining data has low bias and the finetuning data has high bias. The downstream predictions get biased even if the finetuning data has biases which are not common with those in the pretraining data, but the performance drops.
Pretraining data
Finetuning Data
Worst-Group
Average-Group
High Bias
High Bias
6.48
53.21
High Bias
Low Bias
43.33
57.97
Low Bias
High Bias
46.28
54.95
High Bias
High Bias (WB)
4.70
49.76
For pretraining, use gender prediction again as mentioned in the above case, but for the downstream dataset, we group-balance the CelebA dataset in terms of Blond Hair and gender, keeping the total number of downstream samples same as described above.
Q3: What happens the pretraining data is less biased but the downstream dataset is heavily biased?
For the downstream dataset, we choose the same one as mentioned in case of Q1. For the pretraining task, instead of predicting gender, we predict the 'SSmiling' attribute, as we find it to be much less biased in terms of gender.
Q4: What happens if the pretraining data is highly biased and so is the downstream dataset, but the two biases are unrelated?
To simulate this scenario, we pretrain the ResNet-18 encoder by predicting the gender attribute again in the CelebA dataset as mentioned in the above cases, and for the downstream task, we choose class-balanced version of Waterbirds, which has completely different bias.
Findings. Note that QI, Q2 and Q3 are related. From our experiments, we observe the following. When both the pretrained encoder and the downstream dataset are heavily biased with common attribute (QI), the worst group performance on the downstream dataset is the least of all the 3 cases. On the other hand, for Q2 (i.e. when the pretrained model is heavily biased but the downstream dataset is not), the worst group performance improves considerably. This corroborates with our findings in the main paper (Sec 3) that group-balancing the downstream task can improve model performance. The best performance is seen for Q3, where the pretraining model is less biased, but the downstream dataset is still highly biased- here we find that the worst group performance is infact higher than the previous case (Q2). This shows that the lesser the bias in the pretraining data, the more unbiased downstream application is going to be.
When the bias in the pretrained model and the downstream task are unrelated, we find that the model indeed affected by the bias in the downstream data, with a drastic drop in overall performance. This indicates that if the pretraining and downstream data distributions do not match, one should choose a different feature encoder achieve better performance on the downstream task. All the above findings are summarized in 7 Note that the overall performance of the downstream task considerably lower in comparison using an Imagenet pretrained encoder for all above cases. This is likely because the pretraining task involves a binary prediction of an attribute that is different from the downstream target attribute- leading to overfitting of the pretrained features to the former.
A.2 Comparison of the Margin Loss-based method with other Methods
In the main paper, we discuss the performance of our proposed baseline and compare it with Co- Ada (50) across various benchmarks and pretrained encoders. In this section, we describe the performance of the other methods in Table 8 We find that for the ResNet- 18 encoder, among all methods, LfF (00 performs well on Waterbirds and both the versions of CMNIST, but the worst group accuracies for the same method is poor for CelebA. On the other hand, while BPA has high worst group accuracy for CelebA, its performance deteriorates for Waterbirds and both the CMNIST versions. This inconsistency is present for other methods as well, for every encoder.
16

---

Table 8: Performance comparison of our method with all other methods. Worst and Average Group Accuracies for Waterbirds & CelebA (Blond Hair), Bias- Conflictnn (Bi-Co), Bias-Aligned (Bi-Al) and Average Accuracies for CMNIST-0.9 and CMNIST-00999, Highest accuracies are marked in bold, while the 2nd highest ones are underlined. All scores with respect to the proposed approach are averaged over 3 seeds. Co-Ada: Contrastive Adapter.
Buakhhne
Waiehinds
CelenA
CMNIST-0S
CMNIST-0965
A0g
Ci
Avg
Bu
Bi-Al
ERM
28.00 A0
76:22-110
100.00 50
18
DeHAN
80.47 4700
36.10
1300
14.17
19.4
80.4 4..0
12.3
74.83 in.r
MPA
42.45
1.:7
05.5 4635
100.00
100.00-6:21
73-12 72:22-222
13.00-3354
23.00
09.9)21.21
77:01-4066
56.25 23:45
07
50.5522.01
803 96
12.292314
61.415946 65
67-32
B0LIGE1 16
65.2g20
50.5251
54.50
$1.955
FRM
67.008
00:2259
91.18
100.00
CLI RN20
DeHAN
63.51
50.e2m
20
91.31
90,0020011
BBA
09.51
85.7
90.00
02.94
100.00
27 17422
61.2841-43
80002
57.57
91.c0mm k7
TL.1g
50.81
75,
GEORGE L4
83.14
66.11
50.61
100.00000 00 +1000
77.
Ca-Ada
4-:15
90.52
05.66
12.18 Rus
70.28
85.70
50.47
94.00
23,22
ERM
29.67 k2.0
83.31 15000
31.72 1244
82 1r
55.00 164
09 15.11
14.21 1a3s
50.21 1245
100.000 00 15:70
75
WTBB
DeNAN
66.82
191 1..7
77.00 150
50.56-0-45 45
14.57 cu:
10
01 14.2
RPA L5
59.01 11:45
H2-03-4-25
103.00 -3.44
84 [4)
50.28-042
09 19.44
01.05
57.45 11.04
100.000
00 -5644
7% 733
20.02-1440
73.65-4000
28.72-2371
T4.65
50.28-9944
41:4
92.81 20040
70.76-3.08
85, 16
19
75.84
75.57
59.28-9937
03.00 +1:90
G8.41
12.07
79:04
GEORGE [
H1.07 surr
$5.00 1.24
$4.14 44m
57.43-9443 4:
Sa 19.s
03.12 66.35
07.25 14:72
100. 00 -c00
7% G2
Co
61.T1 11:15
M0.54- 1115
55.57 12:43
40 11:17
71.71 -945
H3,15 1.01
T0.00-140 13:43
122 07 fus
84 D5
mothed
74.02
82 -6.3
87.25 1::00
00.31 Le06
01.32 19
44.55
96 19.06
77.07 18:53
14. 44.
42 14.85
A.3 Hyperparameter Details and Analysis
As mentioned in the main paper, for model selection, we use the training accuracy in the bias- amplification stage, whereas for the mitigation stage, we use the overall validation accuracy. Hy- perparameter tuning happens via the validation accuracy of the model checkpoint saved during the mitigation stage. The bias-amplification stage has the following hyperparameters: LR (learning rate), BS (batch size), À (weight decay) and number of epochs. In the clustering stage, the number of clusters K a hyperparameter. Finally, in the mitigation stage, we have two hyperparameters specific to the margin loss: the scaling parameter s (i.e. the radius of the hypersphere on which the features are projected) and the standard deviation for the Gaussian randomization, a. We next define the range of each hyperparamter on which we evaluate our model and the other methods. For learning rate LR, we explore in the range 0.0001, 0.0005. ... .0.05 in step sizes of and 2 respectively. Similarly, for batch size BS we evaluate on the range {64, 128, 256, 512]. We explore higher values of weight decay x E (1, 0.1, 0.05, 0.01), for the bias-amplication stage and for the mitigation stage, we search A in the range 10-6,
10 in step sizes of 10 and also consider À = 0. Number of epochs for training is kept in the range {50, 100). For the clustering stage, we choose K E (2, 4, 6. 8) for Waterbirds and CelebA, and explore K- = 10, 20, variants. In the mitigation stage, we select s E [4. 8, 12, 16) and a E {0, 0.05, 0.1, 0.15, 0.2). We show the selected hyperparameters in the bias-amplification stage in Tables and T0 and for the mitigation procedure, the chosen hyperparameters are present in Table [T] For number of neurons M in the MLP layer, fix the value to be 128. For CelebA experiments, we use SGD as the optimizer, whereas for Waterbirds, and both the variants of CMNIST, we use Adam.
60 with step size of 10 for the CMNIST
Finally, we show the changes in scores by varying the number of clusters K, the standard deviation for the Gaussian randomization of margins and the hypersphere radius s for CelebA and Waterbirds (with ResNet-18 as the feature encoder) in Fig. 4 We find that performance remains consistent for K 2 4 for both datasets. Similarly, similar scores are seen with a > 0.1. The best values for s are found to be 8 and 12. We also vary the weight decay X in the bias amplification stage and show its effect on the final model performance. While for CelebA the effect of changing À is minimal, Waterbirds can be seen to be more sensitive towards higher values.
A.4 Performance Other Datasets and Architecture
Here we describe two real-world datasets: UTKFace 77] and BAR 20 and compare our model against the ERM model and the Contrastive Adapter (50].
17

---

Worst- (Waterbirds)
Average (Waterbirds)
Worst (CelebA)
Average (CalabA)
80
80
80
60
80
20
02
8.00
0.05
0.10
0.15
0.20
5.0
7.5
10.0 12.5
15.0
Number of clusters
Sigma
Scale (5)
Figure 4: Hyperparameter analysis. We vary the number of clusters K, the a for the Gaussian randomization of margins and the hypersphere radius and show model performance for Waterbirds and CelebA.
Worst group (Waterbirds)
Worst group (CelebA) Average Group (CelebA)
Average Group (Waterbirds)
Weight decsy A
1e-5
1e-5
1e-4
1e-4
1e-2
1e-2
Accuracy %
Accuracy %
Figure 5: Effect Weight Decay on Downstream Model Performance. We vary the weight decay À and show the final model performance for Waterbirds and CelebA.
A.4.1 UTKFace
In the main paper, evaluate Waterbirds, CelebA (for Blond Hair classification) and CMNIST- 0.9, CMNIST-0.995. Here, we evaluate on UTKFace (74), which is dataset of faces, with gender, race and age annotations. Specifically, we predict race, with gender the bias attribute. The dataset stored in 3 parts with 10137, 10719 and 3252 images respectively; we use the first part as the training set, second part as test, and third part as the validation set. Like all other experiments, manipulate the validation set to contain group-balanced data. There are 5 races White, Black, Asian, Indian and Other, and two perceived genders females and males. Out of the 10 resultant groups, we report the worst group's 's accuracy as well as the average group score. We observe that our method far outperforms the ERM scores (as well as the competing Contrastive Adapter) for the ResNet-18 image encoder, thus further demonstrating the effectiveness of our method. The results are summarized in Table
A.4.2 BAR
BAR (Biased Action Recognition) [20] is an image dataset with actions as the class labels and places the spurious attribute. Note that the test set only consists of bias-confficting samples, hence,
Table 9: Hyperparameters for the Bias-amplification stage for Waterbirds and CelebA for all the pretrained models shown in the main paper.
Waterbirds
CelebA
LR
A
BS
Ep
Opt
LR
A
BS
Ep
Opt
R18
0.0001
1
256
100
Adam
0.01
1
512
50
SGD
VIT
0.05
1
256
100
Adam
0.0001
1
512
50
SGD
CLIP
0.01
0.1
256
100
Adam
0.0001
0.1
512
50
SGD
18

---

Table 10: Hyperparameters for the Bias-amplification stage for CMNIST-0.9 and CMNIST-009955 for all the pretrained models shown the main paper.
CMNIST-0.9
CMNIST-0.995
LR
BS
Ep
Opt
LR
À
BS
Ep
Opt
R18
0.01
0.05
256
100
Adam
0.0001
0.1
256
100
Adam
ViT
0.01
0.01
256
100
Adam
0.0001
0.1
256
100
Adam
CLIP
0.01
0.01
128
100
Adam
0.0001
0.01
256
100
Adam
Table 11: Hyperparameters for Mitigation stage of our approach for Waterbirds, CelebA, CMNIST- 0.9 and CMNIST-0,995.
Datasets
Pretrained Encoder
LR
À
BS
Eps
Opt
K
s
g
R18
0.01
0.01
64
100
Adam
6
8
0.2
Waterbirds
VIT
0.01
0.01
128
100
Adam
4
8
0.25
CLIP
0.01
0.01
256
100
Adam
4
8
0.2
R18
0.0001
0.0001
128
100
SGD
4
8
0.2
CelebA
VIT
0.01
0
128
100
SGD
2
8
0.15
CLIP
0.1
1e
128
100
SGD
2
8
0.2
R18
0.01
128
100
Adam
30
12
0.15
CMNIST-0.9
VIT
0.0001
1e 6
256
100
Adam
20
4
0.15
CLIP
0.0001
1e
256
100
Adam
20
8
0.2
R18
0.01
256
100
Adam
20
8
CMNIST-0.995
VIT
0.005
1e
128
100
Adam
40
8
0.05
CLIP
0.001
0
128
100
Adam
40
12
0.1
Table 12: Performance of our method on the UTKFace dataset and the BAR dataset. In all cases, we find our proposed approach outperforming the worst group accuracies of the ERM method and Contrastive Adapter. The underlying pretrained encoder is the ResNet-18 model.
UTKFace
BAR
Method
Worst Group
Avg Group
Test
ERM
13.6
40.5
63.15
Contrastive Adapter
8.36
31.46
62.54
Our method
32.26
40.65
65.96
report its test accuracies following previous work outperforms both the ERM method and the Contrastive Adapter (see Table 12]
38
As with UTKKaaee, we find that our method
A.4.3 VIT-H 14 as the Pretrained Encoder
In a previous work 61]. it is shown that spurious correlations can be mitigated simply by using stronger pretrained encoder. Further, the authors show that the strength of mitigation depends on the underlying pretraining data. They observe that using a ViT-H 14 encoder pretrained on the SWAG dataset [62] followed by end-to-end ImageNet (63 finetuning can achieve state-of-the-ar results on the Waterbirds dataset. Likewise, to test whether the same holds for other datasets too, we use this same encoder and train for CelebA and CMNIST-0.999 using the ERM method. We observe that while indeed the worst group score for Waterbirds is considerably high, for CelebA and CMNIST-0.995. the performance still weak. This leads to the important observation that bias mitigation strategies are indeed required for all pretrained encoders, as even though they might perform well for some biased datasets, they may not be free of all biases that can affect model predictions. We compare the ERM scores with that of our margin loss approach and Contrastive Adapter, and find that Waterbirds, though the ERM scores outperform the mitigation methods marginally, for the other two datasets, the mitigation scores considerably improve. Upon comparing our method with Contrastive Adapter, we find that the former outperforms the latter in CMNIST-0.995 and Waterbirds. The results are shown in Table I3
19

---

Table 13: Performance on VIT-H 14 (pretrained on SWAG and finetuned on ImageNet). In this table we show the performance of ERM, Contrastive Adapter and our margin loss-based method for Waterbirds, CelebA and CMNIST-0.999 Waterbirds, the same does not hold for CelebA and CMNIST-0.999
We observe that while the ERM scores are high for
showing that single feature encoder, however powerful, may not mitigate all biases. Thus we advocate strongly for bias mitigation
strategies that can debias models presence of untrainable feature encoders.
Waterbirds
CelebA
CMNIST-O.P9
Method
Worst Group
Avg Group
Worst Group
Avg Group
Bi-Co
Bi-Al
Avg
ERM
91.1
95.99
6.71
61.32
68.41
100
71.57
Contrastive Adapter
86.12
93.17
82.23
85.74
78.09
91.97
86.53
Our method
89.17
95.26
62.93
68.61
85.51
95.29
90.4
Table 14: Ablations of our method: Here we further show the roles of the different components of our margin loss- bassd approach using ResNet-18 as the pretrained backbone.
Waterbirds
CelehA
CMNIST-0.9
Model Component
Worst
Average
Worst
Average
Bi-Co
Bi-Al
Average
Our method
80.29
84.56
81.61
86.04
81.91
92.75
87.33
Randomization (margin 0.0)
40.34
76.26
25.56
75.45
55.65
99.68
77.67
Randomization (margin 1.0)
40.34
76.72
30.00
76.78
56.48
99.69
78.09
Randomization (margin 0.5)
43.61
76.78
29.44
76.20
55.61
99.68
77.65
A.5 Ablations
In this subsection, we further discuss the importance of the adaptive margin components of approach and the margin loss strategy. Specifically, we present the results when the randomization is applied to constant margin values. As with Table the main paper, all evaluations are performed on Waterbirds, CelebA (Blond Hair classification) and CMNIST-0.9 with ResNet-18 as the backbone. The results are shown in Table
Observations. Here we show the effects of randomizing a constant margin value to further understand the role of the adaptive margins. We perform this experiment 3 constant margins 0.0, 1.0 and 0.5. We observe that for all three datasets, both worst and average accuracies decrease drastically. This shows the effectiveness of the adaptive margins, i.e., randomizing the margins do not help unless their underlying values represent the frequency of a training sample's class label in its assigned cluster.
We next show further ablations of our method. As described in the main paper, in the bias- amplification stage, we set the weight decay to a high value (weight decay is the weight used for L2 regularization) to ensure amplification of the bias in the model. Instead of the L2 loss, investigate what happens when we instead use L1 and L1 + L2 regularization during this stage. The performances for all these different regularizations appear similar. We also show what happens when we do not use any of these regularizations the scores drop drastically for Waterbirds. Moreover, in many of the previous works like JTT [19], misclassifications in the ERM stage used to identify the biases. When we use the same in our method to calculate the margins (instead of clustering), we find that clustering works better than using misclassifications for bias-identification. Further, we find that using the Generalized Cross- Pross-FEttrpy loss (0) with 1.2 regularization can have similar effects that of our original approach, though the latter outperforms the former for both the datasets. We also investigate what happens when we replace the ArcFace loss ( with another margin loss called CosFace [5]. Here we only change the loss formulation. but calculate the margins in the same way as that our method as described in the main paper. We call this version of our method CM + CosFace, and find that it is indeed effective, however the original margin loss still outperforms the former on both Waterbirds and CelebA. The results are summarized in Table [5
A.6
Comparison with Other Methods
A.6.1 Last Layer Retraining
In DFR 544, Kirichenko et al. show that simply retraining the last layer of biased model can achieve considerably unbiased predictions. However, they utilize group-balanced reweighting
20

---

Table 15: In this table summarize the roles of the different regularization techniques as well as misclassificatios as an alternate for clustering on the ResNet-18 pretrained encoder. We find that using LI regularization happens when we use the GCE loss [20] along with high weight decay for the bias amplification stage. Finally, evaluate the changes in performance when bias in the network.
be good (even better) proxy for weight decay. We also show what
alternate margin loss mitigate the
Waterbirds
CelebA
Method
Worst Group
Avg Group
Worst Group
Avg Group
Regularization
81.80
85.21
81.79
85.50
(I.1+1.2) +1.2) Regularization
78.55
85.66
79.98
85.47
No Regularization
39.88
77.04
78.33
80.71
Misclassification
75.86
82.76
75.56
85.23
GCE L2 Regularization
78.35
83.43
78.33
83.28
CM + CosFace
77.07
82.55
75.56
85.42
Our method
80.29
84.56
81.61
86.03
Table 16: In this table we show the performance comparison of DFR with the proposed approach for ResNet-18 backbone, for Waterbirds, CelebA and CMNIST-0.9
Waterbirds
CelebA
CMNIST-0.9
Method
Worst Group
Avg Group
Worst Group
Avg Group
Bi-Co
Bi-Al
Avg
Our method 80.29
84.56
81.61 73.33
86.04 80.78
81.91
92.75
87.33
DFR
74.68
81.29
91.39
90.84
91.16
dataset (specifically from the validation set) for this retraining. The difference of this setting with ours is that ours does not assume a feature encoder which is already trained on the biased dataset; rather we assume blackbox feature encoder which does not interact with the downstream training data. We summarize the results in Table 16] While we see that the performance of Waterbirds and CelebA are comparable to our method, DFR has superior performance in CMNIST-0.9, However, is to be remembered here that DFR not only uses part of the validation data for training, but also this excess data is group-annotated. This gives additional advantage to DFR, leading improved scores in some datasets.
A.6.2 Contrastive Adapter
Table 17: Difference in training time (in mins) for Contrastive Adapter and the proposed approach for different datasets trained on the ResNet-18 backbone. We find that while our method trained in considerably less amount of time, Contrastive Adapter requires enormous time, rendering it inefficient for time-constrained settings.
Dataset
Contrastive Adapter
Our method
Gap
Waterbirds CelebA
34
1.05
32.95
105
13.39
91.61
CMNIST
213
2.05
210.95
While the problem statement described by Zhang al. [50] is similar to ours, there are subtle differences. E.g., Contrastive Adapter is meant to boost the group robustness of foundation models. uses the zero-shot capabilities of the existing foundation models (as result, utilizes their text modalities) to find positive and negative samples with respect to anchor sample and apply contrastive loss for bias mitigation purposes. On the other hand, our aim is more generic: detect and mitigate biases in case the feature encoder is untrainable (only accessible through API calls) and does not rely on any other modality. That is why, when we compare our method against other methods, we freeze the feature encoder for all of them and investigate how these methods perform when no backpropagation is allowed into the frozen backbone. Moreover, since ResNet- 18 and ViT-Base do not have the text encoder, cluster the backbone features proxy for the zero-shot
21

---

accuracies of Contrastive Adapter using the KMeans algorithm (suggested by the authors in their code submission). Since KMeans is an unsupervised algorithm, the cluster label might not always conresoond to the correct class index. Simple heuristics are employed to properly label each obtained cluster. the binary classification setting (WaterBirds and CelehA), the cluster that has higher number of correct predictions is assigned the label O (since WaterBirds has higher number of landbirds (class 0) and CelebA has a higher number of Non-blonde samples (class O)). This heuristic fails when the number of classes exceeds 2 (e.g. ColorMNIST), and in such cases, we use consensus to handle the prediction. For each cluster, we count the number of predictions for each class (digit), and the cluster label is the class with the largest number. This associates the most probable label with each cluster in the KMaans model - i.e., we take the ground truth class (digit) each sample, find the majority class per cluster and assign that digit as the predicted class.
Impact of number of positives and negatives. Contrastive Adapter requires the creation of similarity matrix, which consists of positive and negative samples for every sample in the dataset. Typically, a reasonable number of positive and negative samples are necessary for the method (contrastive learning) to function properly. The caveat is that when dealing with multi-class clas- sification problems, gathering a sufficient num- ber of negative samples takes a toll on the system memory, and a server of 128GiB can support only about 128 positives and negatives before overflowing. In addition to this, even if a suf- ficiently large server is procured to be able to support a higher number of samples, the train- ing time grows significantly, which is further drawback on the total runtime as well.
Table 18: Effect of full finetuning. Upon fine- tuning the encoders fully, our method achieves decent performance compared the existing meth- ods across two pretrained encoders. The numbers of all competing methods are taken from previous works for ResNet-18 [45) and ResNet-50 (19).
Res- 18
Res-50
Method
Worst
Avg
Worst
Avg
ERM LFF BPA
62.39
84.63
72.6
97.3
68.02
85.48
75.2
97.5
71.39
87.05
JTT
86.0
98.6
Our method
80.82
85.91
88.82
90.62
We show the differences in training times of the proposed method and Contrastive Adapter in Table 17
is to be noted that we perform all our experiments on a single Nvidia-RTX A5000 GPU. a time-constrained setting, this renders Contrastive Adapter inefficient, whereas our method, even with considerable less training time, outperforms Contrastive Adapter in most cases.
A.7 Full finetuning
In this subsection, we discuss the effect of full finetuning on the proposed loss, i.e., when the loss is allowed to backpropagate into the pretrained encoder. We compare our method with LfF (20). BPA 45 and JTT [19.. We report the results for LfF and BPA on the ResNet-18 encoder as per the results shown by Seo et al [45,, whereas those for JTT (along with LfF) are with respect to the ResNet-50 model [19. From Table 18 we note that for ResNet-18, our method outperforms BPA and LIF. For ResNet-50, our method surpasses that of JTT by 3% in the worst group accuracy, In comparison, for our proposed setting, our method outperforms JTT by a huge margin (30.45%) This study shows how our method performs satisfactorily in the setting of full finetuning, in addition to decent and consistent performance on the problem setting proposed in the paper.
A.8 Social Impact
Since our work aims to create unbiased models in a unique problem setting, we believe that it promotes a positive social impact. Pretrained models are not free biases [14], and our method an attempt to generate unbiased predictions without having to finetune these pretrained encoders - we believe that this is a step towards ensuring that future deep learning models are free of the prejudices already present in the society towards marginalized groups, and are fair to all.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
All claims are reflected in Sections
3 and and Appendix Section A.2
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have a thorough discussion and limitations section in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
23

---

Justification:
No theoretical results have been presented in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Along with the method discussed in Section 3 the implementation and com/
dataset details are shared Section 4 The code available at
abbipsabasuublacckeee wraccccooobbiss____e cisssmmmee nitigation
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
24

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All datasets open-source. Code is available htttps://githuui abhipsaaasubbacckoox. bias mitigation
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc
Soblic//guides/Coeeeubbiis
www.iissssooooiiyiooo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wttps/cccppclcccuui.eeaeeecccuea
Sttelee//ooSSSSeesSS.oeaa www.ssssiicggooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All implementation details are present Section 4 and hyperparameter details are in Appendix Section A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: All experiments have been done over seeds, and Tables and have the corresponding standard deviations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Such details are present in the attached code, and Appendix Section A.6.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts are discussed Appendix subsection
A.8
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
26

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The does not pose any such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets used have been cited. Only open-source methods have been replicated and all of them have been cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets/datasets have been proposed in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing is involved in this work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
No crowdsourcing is involved in this work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "TbiozG4FGe", "reviewer": "Reviewer_pwPo", "summary": "The papers address the debiasing problem using pretrained but frozen feature extractors on downstream applications. Then, they propose a clustering-based method that relies on bias-amplified training through cross-entropy loss. After training, they cluster the biased features and mitigate the biases using the resultant clusters. The experimental protocol considers CIFAR, Waterbirds, and CelebA as evaluation datasets. Also, several ablations exist about distinct loss functions and comparison with literature methods under two scenarios.", "questions": "- About Table 8, Is there any explanation about ResNet-18 achieving best performances than a Vit-B on waterbirds? I Would expect that ViT achieving higher performances in this dataset.", "limitations": "Despite the primary focus being on debiasing, one step of the proposed method involves amplifying existing biases. It introduces a significant risk: if the following step, designed to compensate for this amplification, fails to function as intended (which is a possibility in specific scenarios), the method may ultimately exacerbate the very biases it aims to mitigate. This could potentially allow malicious actors to exploit the technique, leading to harmful outcomes.", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The authors build upon debiasing literature, emphasizing using a frozen feature extractor. This approach requires that debiasing interventions are applied post-feature extraction, a strategy that forms the core of their methodology. Their experimental design is iterative, with each step informed by empirical results. Despite its simplicity, the authors' contributions are clear and understandable.\nThe experimental protocol is comprehensive, utilizing diverse datasets, including CIFAR, Waterbirds, CelebA, BAR, and UTKFace, for evaluation. This selection allows for a broad assessment of the proposed method's effectiveness across various contexts. The authors also conduct extensive ablation studies on different loss functions, components of their proposed method, and comparisons with existing literature methods. These studies demonstrate the superior performance of the new ideas in multiple scenarios. Additionally, the authors provide extra details on reproducing the main results in the appendix.", "weaknesses": "In my opinion, the main weakness lies in assuming that the bias aligns with the feature encoder. There are many cases where the practitioner does not know the model's or the dataset's biases. Also, the paper lacks instructions on how to identify scenarios that fit the previous assumption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for recognizing our contributions and appreciating the comprehensive experiments. We address the concerns below:\n\n**Assumption on alignment of bias of the downstream dataset with that in the pretrained model**. Please see the general responses.\n\n**Table 8: Scores of ViT-B on Waterbirds**: Please see the general responses.\n\n**Limitation on exacerbation of biases**. We thank the reviewer for pointing out this limitation, where after bias-amplification, if the debiasing module fails, the method may end up exacerbating the bias instead of mitigating them. We agree that this is a limitation in general for most of the existing debiasing methods, and attackers can leverage this for all methods that involve a bias amplification stage. We will mention this in the camera ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oTmXXivj3K", "reviewer": "Reviewer_qb1H", "summary": "This work explores a problem setting where one wants to train a classifier on top of a large and frozen pre-trained model, while avoiding bias and improving fairness. It proposes a computationally efficient methodology to accomplish this objective, centered around a novel loss function, the Adaptive Margin Loss.", "questions": "My main question is why we do not see standard deviations in most tables, if all experiments have been done over 3 seeds?", "limitations": "Authors have correctly stated the method's limitations.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "-The manuscript is very clear and well-written. \n-Multiple experiments were performed.\n-The proposed methodology seems computationally efficient and effective at avoiding bias. \n-Results are promising, and the objective of adapting large pre-trained models in an unbiased manner will probably be increasingly more important in the future, with the rise of such pre-trained models.", "weaknesses": "My comments here are minor suggestions, as I have no major concerns.\n\n1- Figure 2: this analysis is very interesting. It would be nice to start the weight decay as 0, not 5e-2 in the plots.\n\n2- A question I have is, how many unbiased samples do we need for this methodology (and the alternative ones) to work well?\n\nTable 8 shows some results that help us answer this question, as we can see how performances vary from CMNIST-0.9 to CMNIST-0.995. What would happen for CMNIST-0.999? Where is the limit where the debiasing methodologies fail?\n\nIn summary, it would be great to expand these experiments, varying the proportion of the majority group in all datasets (not only CMNIST, but also CelebA and Waterbirds), until the point where all methods fail. This experiment would show the reliance of each methodology on the number of samples in the minority group.\n\n3- In the tables, we have standard deviations only for the proposed methodology. They should also be reported for the competing methods as well. Moreover, although authors say that “All experiments have been done over 3 seeds”, multiple tables show no standard deviation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comments and feedback and address the specific concerns below. \n\n**Effects of starting from weight decay $\\lambda=0$ in Figure 2 of main paper**: We have shown the effects starting from weight decay = 0 on the different group accuracies for Waterbirds, CelebA and ColorMNIST-0.995 in Figure 1 in the rebuttal pdf, where we see that the lower the weight decay, the higher the training accuracy, because of overfitting to all the groups. \n\n**Performance of different models for varying degree of bias-conflicting samples**\nThis is a great suggestion, and likewise we train our method on ColorMNIST-0.999 (Table 1, rebuttal pdf), and two versions of Waterbirds and CelebA. For Waterbirds, we show performance of different methods with a) no bias-conflicting samples, b) 50% bias-conflicting samples, where the bias-conflicting samples constitute two groups: waterbirds on land and landbirds on water (Table 2, rebuttal pdf). We show the scores for similar versions of CelebA, where the bias-conflicting samples constitute the blond males (Table 3, rebuttal pdf). As predicted by the reviewer, with a decrease in percentage of bias-conflicting samples, the scores of all methods drop. In Waterbirds, for the case of no bias-conflicting samples, the drop in performance is substantial, as is for ColorMNIST-0.999 (see Table 1, rebuttal pdf). For all other cases, we find Co-Ada and our method to perform decently. It is to be noted that our method outperforms all others in all the explored cases. \n\n**Standard Deviation for competing methods and other tables**\nWe will rectify this in the camera-ready version, upon acceptance. We apologize for not showing the standard deviation values in all cases to ensure ease of readability. We show the values for some of the competing methods below (for ResNet-18 backbone).\n\n| Worst Group Accuracy     | ERM               | DebiAN            | BPA               | LfF               | CoAda             | Ours              |\n|:------------:|-------------------|-------------------|-------------------|-------------------|-------------------|-------------------|\n| WaterBirds |   $38.9^{\\pm1.4}$ | $58.94^{\\pm0.97}$ |  $58.7^{\\pm1.67}$ | $66.09^{\\pm0.62}$ | $67.57^{\\pm1.29}$ | $80.29^{\\pm2.5}$  |\n| CelebA     | $27.20^{\\pm0.89}$ | $26.10^{\\pm1.12}$ | $66.71^{\\pm0.44}$ | $13.26^{\\pm0.67}$ | $78.37^{\\pm1.36}$ | $81.61^{\\pm1.02}$ |\n\n| Average Group Accuracy    | ERM               | DebiAN            | BPA               | LfF               | CoAda             | Ours              |\n|:------------:|-------------------|-------------------|-------------------|-------------------|-------------------|-------------------|\n| WaterBirds | $76.22^{\\pm1.04}$ | $80.47^{\\pm0.97}$ | $80.83^{\\pm1.12}$ | $81.39^{\\pm0.47}$ | $80.10^{\\pm0.14}$ | $84.56^{\\pm1.2}$  |\n| CelebA     | $75.43^{\\pm0.67}$ | $75.41^{\\pm0.44}$ | $84.14^{\\pm0.22}$ | $69.42^{\\pm0.36}$ | $85.79^{\\pm0.78}$ | $86.04^{\\pm0.26}$ |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pghKatzxIi", "reviewer": "Reviewer_wKPv", "summary": "In this paper, the authors propose a simple method with a clustering-based adaptive margin loss for debiasing blackbox pretrained models. Whereas prior works have explored settings where pretrained models are tunable, the authors instead explore a more constrained and realistic setting, where a black-box network is frozen and only a classifier is trained on the fine-tuning dataset. The proposed approach involves training an adapter to amplify biases in the frozen encoder; then, a mitigation procedure is executed using a novel cluster-based margin loss. The proposed approach appears to outperform state-of-the-art methods across three datasets.", "questions": "I have listed my questions above in the “weaknesses” section.", "limitations": "Yes, the authors have adequately described the limitations of their work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper introduces a creative approach for addressing a high-impact problem that has been previously under-researched. As large-scale pretrained encoders become more commonplace, it is important to create debiasing methods that can operate in settings where the model weights are frozen.\n2. The proposed approach appears to demonstrate performance improvements over existing approaches while also demonstrating high efficiency.", "weaknesses": "1. **Applicability**: One important weakness of this approach is that it is difficult to know when it can be used. The authors claim that their \"method is specifically targeted towards cases where the bias in the downstream dataset is already encoded in the pretrained model\". However, this is difficult (and in some cases, impossible) to know a priori. \n    - In their evaluations in Table 1, the authors assume that spurious attribute labels are available in order to determine whether a pretrained model encodes the same bias as the downstream dataset. However, these subgroup labels are not likely to be available in real-world settings (which the authors claim as well later in the paper), making it difficult to know when the authors' approach is useful.\n    - The authors also make a note about how their adapter-based bias amplification procedure will consistently work because they assume that the bias in the downstream dataset aligns with that of the pretrained model. As stated above, this is a strong assumption that is difficult to verify a priori, especially since the authors assume that bias annotations are not available for their downstream dataset.\n    - As a result, I am unsure on the real-world applicability of this approach.\n2. Presentation: The organization of this manuscript is somewhat confusing, particularly in section 3.\n3. Additional clarification on design decisions: Some critical design decisions are somewhat vague in the main text.\n    - How was the value of $\\lambda$ selected in practice for each dataset? The paper states that $\\lambda$ was selected as a high value while ensuring that the training accuracy did not drop drastically. Are there specific thresholds at which the training accuracy drop was viewed as \"drastic\"?\n    - The paper could benefit from additional details on the clustering approach. How was the number of clusters selected? In Table 11, it appears that the number of clusters K selected by the authors varies significantly across encoders and datasets. Is this a hyperparameter that needs to be set by the user? How does performance vary depending on the number of clusters selected? It seems that this is a critical design decision that can affect efficacy of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments. We address the concerns below:\n\n**Clarifications about Table 1:** Table 1 is solely meant for analyzing the nature of the different feature encoders, wherein we can see that different feature encoders may have different levels of awareness of the target labels. If a feature encoder is highly target-aware, even removing all bias-conflicting samples from the training data does not hamper the worst group accuracies to a large extent. On the other hand, if the feature encoder is bias-aware, the worst group accuracies are low even for the original dataset -- these are the scenarios that require explicit bias mitigation. Note that the scores in this table are those of the ERM method, obtained by varying the proportion of the bias-conflicting samples, and is not related to the proposed approach.\n\n**Assumption on alignment of the bias in the pretrained features and downstream dataset:**\nPlease see the general response section.\n\n**Clarifications on the presentation in Section 3:** \nWe apologize if the presentation of this section was not clear. We explain the main highlights of the section below and will make the presentation clearer in the camera-ready version upon acceptance.\n\n- We begin by formally defining the existence of biased groups in deep learning datasets, explaining how model behavior differs for different groups (section 3.1).\n- As mentioned above, section 3.2 analyzes the nature of different feature encoders, and highlights that the bias-aware feature encoders require explicit mitigation.\n- In section 3.3, we show how existing methods (designed for full finetuning) perform inconsistently for different datasets in the proposed problem setting, while Contrastive Adapter [50] (designed for such frozen encoders) performs decently, but is highly time consuming. This motivates the requirement of dedicated research in the direction of the proposed problem setting.\n- Section 3.4 describes our method, where we first motivate amplifying the bias, and then mitigating it through the clustering-based margin loss approach, after exploring a number of simple baselines involving weighted losses.\n\nWe can clarify further if the reviewer still has doubts.\n\n**Clarifications on selection of weight decay value:**\nIn the paper, we have shown that high weight decay leads to a reduction in the bias-conflicting group performances in the training set, thereby helping us detect the bias easily. The value of $\\lambda$ is chosen using the validation accuracy in the mitigation stage in practice from a range of high values (0.01-1). Details are present in Appendix A.4. When $\\lambda$ is sufficiently high, we find that the training accuracy collapses (~50% for Waterbirds and CelebA, <30 % for ColorMNIST-0.995). This fall can be clearly seen in Figure 2c in the main paper for ColorMNIST-0.995 for $\\lambda \\geq 0.5$. This helps us eliminate a few values of $\\lambda$ in our specified range during the amplification stage itself. For Waterbirds and CelebA, the fall is typically seen at $\\lambda \\geq 3$ for the ResNet-18 backbone. This happens as the model’s learning is impeded with extremely high weight decay, which is not the goal of our approach. Recall that the purpose of this step is to learn the bias-aligned samples well, not the bias-conflicting samples. Hence, in the paper we suggest picking $\\lambda$ lower than this threshold. We stick to the search space of (0.01-1) as we find that the scores remain similar for $\\lambda \\geq 1$ for both datasets (see below). \n\n| $\\lambda$ | Worst - Waterbirds | Avg - Waterbirds | Worst- CelebA | Avg - CelebA |\n|:---------:|:------------------:|:----------------:|:-------------:|:------------:|\n|     1     |        80.29       |       84.56      |     81.61     |     86.04    |\n|    1.5    |        79.44       |       84.52      |     81.87     |     85.35    |\n|     2     |        81.46       |       84.59      |     80.51     |     85.21    |\n\n\n**Clarifications on the clustering algorithm:**\nPlease see the general response section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "In1iy0ZPWc", "reviewer": "Reviewer_oZ3K", "summary": "The paper tries to address the critical issue of biases in blackbox feature extractors used for image classification tasks. These biases can impact the performance of models when adapted for downstream tasks. The authors investigate existing debiasing techniques and propose a novel method using a clustering-based adaptive margin loss, which does not require prior knowledge of bias attributes. Their experiments demonstrate the effectiveness of this approach across multiple benchmarks, highlighting its practical applicability in scenarios where feature extractor weights are not accessible.", "questions": "(1) How does the clustering-based adaptive margin losses compare to other adaptive loss functions in terms of computational complexity?\n(2) Could the proposed method be applied to other types of pre-trained models beyond image classification tasks? If so, what adjustments would be necessary?\n(3) How sensitive is the performance of the proposed method to the choice of clustering algorithm and the number of clusters?", "limitations": "(1) A more detailed discussion on the scalability of their approach to extremely large datasets and models would be beneficial. \n(2) While the method shows effectiveness in mitigating biases, exploring its impact on model interpretability could provide valuable insights.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The paper's shows originality and practical relevance. It tackles a challenging problem with a novel approach that is both effective and efficient. The clustering-based adaptive margin loss is a creative solution that demonstrates substantial improvements over existing methods. The thorough experimental validation across multiple benchmarks further strengthens the paper's contributions.\n\n(2) The paper is well-written and clearly presented. The introduction effectively sets the stage for the problem being addressed, and the methodology is detailed comprehensively.", "weaknesses": "(1) The paper could benefit from a deeper theoretical analysis of why the proposed method works, which would enhance its scientific rigor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments. We address the concerns below:\n\n**Deeper analysis of the approach.**\nWe refer the reviewer to the ArcFace paper [6] for details on the margin loss. Here, we show how the adaptive nature of the margin loss aids in the learning of the bias-conflicting samples. As shown in [6], the margin loss increases the angle between a sample's MLP feature ($\\hat{f}$) and its ground truth weight vector. The degree of increase depends on which cluster the sample belongs to, and the inverse frequency of the ground truth class in that cluster. Let $x$ be an image in the bias-conflicting group, $m$ be its corresponding margin penalty (eq 2 and L248 in the main paper), and $\\theta$ be the angle between the weight vector corresponding to the ground truth class and the feature $\\hat{f}$. As $x$ belongs to a bias-conflicting group, the penalty $m$ is high (assuming that the clusters obtained from the features are good approximations of the different groups in the training set), which means that the angle between the feature vector and its weight vector increases considerably in the softmax term inside the margin loss in eq 4 (main paper) (recall that $\\cos(\\theta+m) < \\cos(\\theta)$ for $m>0$). Thus the loss value rises for the bias-conflicting samples (owing to the logarithm and the negative sign in eq 4). For the bias-aligned samples, as $m$ is lesser (as it belongs to the majority class in the cluster), the degree of increase in the loss is lower than those for the bias-conflicting samples. This is how the adaptive margins help mitigate biases. \n\nFor further evidence on the usefulness of the adaptive margin loss, we plot the training loss for each group (summed across all epochs and normalized by group size) in Waterbirds and CelebA between an ERM model and our method in Figure 2 in the rebuttal pdf. While both models have higher losses for the minority groups (for majority groups, the loss values are negligible and similar), the proposed method's minority loss values are much higher than that of the ERM model.\n\n**Computational Complexity of competing methods**\nSee the below table for the analysis of computational complexity.\n\n|    Time    |  ERM |  BPA |  LfF |    CoAda    | Ours |\n|:----------:|:----:|:----:|:----:|:-----------:|:----:|\n| WaterBirds |  22s |  48s |  44s |     615s    |  65s |\n|   CelebA   | 114s | 400s | 832s | 127 mins | 803s |\n\n|     RAM    |  ERM  |  BPA  |   LfF  | CoAda |  Ours |\n|:----------:|:-----:|:-----:|:------:|:-----:|:-----:|\n| WaterBirds | 1.28G | 1.34G | 0.890G |  7.5G |  1.3G |\n|   CelebA   | 1.71G | 3.87G |  1.28G |  112G | 1.82G |\n\n|    VRAM    | ERM |  BPA | LfF | CoAda | Ours |\n|:----------:|:---:|:----:|:---:|:-----:|:----:|\n| WaterBirds | 25M |  28M | 20M |  44M  |  20M |\n|   CelebA   | 20M | 752M | 20M |  34M  |  20M |\n\n**Application to other pretrained models**\nIn the main paper, we show the performance of our model on self-supervised pretrained model like CLIP, and demonstrate the effectiveness of our method in Table 4. Our method can further be applied to any classification based task if a pretrained feature encoder is present. For example, if one wants to run it for a text-based classification task, they only need the features from a pretrained model (e.g., BERT). No other adjustment is necessary. It is to be noted that the method only relies on the angle between the feature vector and the ground truth class' weight vector. As an application, we run our method on the CivilComments dataset [a] using the BERT model pretrained on BookCorpus and Wikipedia datasets (https://huggingface.co/google-bert/bert-base-uncased), and report the results below. We find that both Contrastive Adapter and our method far outperform the ERM scores.\n\n|       | Average | Worst |\n|-------|---------|-------|\n| ERM   |   58.14 |  13.5 |\n| CoAda |   78.19 | 67.96 |\n| Ours  |   76.43 | 68.06 |\n\n**Clarification for the clustering task** Kindly see the general response section.\n\n**Scalability of approach to extremely large datasets and models**\nSince the method involves clustering the features, if the dataset is extremely large, one can randomly sample a small percentage of it to do the clustering. For example, on clustering only 10% randomly sampled images from celeba, we find that the worst group accuracy is $81.11$%, whereas the average group accuracy is $85.6$%. The scores become $80.55$% and $85.6$% respectively when the clustering is performed only on 1% of the images (Note that scores reported in the paper are $81.61$% and $86.04$% respectively). On the other hand, for extremely large models, the requirement is to be able to load the model into a GPU memory. Our method adds negligible overhead owing to the addition of only an adapter and a classifier layer. We will add these points to the limitations section in the camera ready version.\n\n**Exploring the method impact on model interpretability**\nWe thank the reviewer for this valuable suggestion. We intend to investigate this as a future work.\n\n\n    [a] Borkan, Daniel, et al. \"Nuanced metrics for measuring unintended bias with real data for text classification.\" Companion proceedings of the 2019 world wide web conference. 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their thoughtful comments, questions, and suggestions. We are pleased that our work has been appreciated and positively rated. The reviewers recognized the significance and under-researched nature of our problem statement (Reviewer *wKPv*), noting its growing importance (Reviewer *qb1H*). They also appreciated the novelty and creativity of our solution (Reviewers *qb1H*, *wKPv*, *oZ3K*), found our experiments to be comprehensive (Reviewer *pwPo*), and considered the paper well-written (Reviewers *oZ3K*, *qb1H*) . We address the common concerns below:

**Assumption on the alignment of the bias in the pretrained model and the downstream task (*pwPo*, *wKPv*)**: While we agree that our method’s efficacy is dependent on this assumption, we would like to highlight that since the feature encoder is frozen and cannot be finetuned based on the downstream dataset, detecting and mitigating biases in such a scenario becomes challenging. As an example, we discuss the case of ViT-B for Waterbirds here (we appreciate Reviewer *pwPo* for raising this question on *Table 8*). We find the ViT-B features to depict the target class more strongly than the bias class. We verify this by clustering the ViT-B features and comparing the Normalized Mututal Information (NMI : https://scikit-learn.org/stable/modules/generated/sklearn.metrics.normalized_mutual_info_score.html) of the cluster labels with the bias (NMI=0.6) and target labels (NMI=0.62) respectively. The worst group accuracy is still low (59%), indicating a weak yet definitive presence of the bias in the model. As a result, all methods suffer, with most methods scoring close to the ERM values, demonstrating this to be a limitation for the compared methods as well. Co-Ada has a worst group score of 63.91%, which is higher than the other competing methods and the ERM. In contrast, our method achieves 74.92%, compared to 80.29% that we obtain from the ResNet-18 backbone. This highlights the difficulties in mitigating biases in different scenarios in the given setup. We find this problem to be interesting, and mention solving this as a future work in the paper. 

We believe that detecting if our assumption holds apriori is highly challenging in the absence of the bias labels. We put forward a few suggestions to identify the scenarios that fit this assumption (*pwPo*, *wKPv*):
- Obtain bias annotations for the small validation set. If the worst group accuracy of the validation set does not reduce substantially with increasing weight decay, it indicates that the features have stronger signals of the target class than that of the bias, making it harder to capture the bias.
- If the bias annotations of the validation set cannot be obtained due to privacy concerns, the overall validation accuracy can indicate strength of the bias. For example, the difference between the validation and training accuracy is 25% for an ERM trained method for the Waterbirds dataset on the ResNet-18 backbone. The higher this difference, the more the indication that the model is overfitting to more and more samples. Such overfitting can indicate that the model is learning the bias in the dataset, thus not generalizing on the bias-conflicting samples.


**Details on the Clustering Approach (*wKPv*, *oZ3K*)**:
A common question that has been raised is on the selection of the number of clusters, K. In the paper, we mention K to be a hyperparameter within a fixed set of values (appendix A.4), selected based on the validation accuracy. We demonstrate in the below table that our method is not highly sensitive to the value of K (*wKPv*, *oZ3K*). Below we show the performance variation for *CelebA* over different values of K. The bold values denote the numbers reported in the paper.

| $K$ | Worst | Avg |
|:---:|:--------------:|:------------:|
|  2  |      81.70     |     86.07    |
|  4  |      **81.61**     |     **86.04**    |
|  6  |      81.36     |     85.96    |
|  8  |      81.08     |     85.93    |

For ColorMNIST-0.995, we show the effect of different K's below (the bold values denote the numbers reported in the paper):

| $K$ | Bias-Conflicting | Avg |
|:---:|:--------------:|:------------:|
|  10  |      73.34     |     84.21    |
|  20  |      **72.56**     |     **84.42**    |
|  30  |      72.17     |     83.59    |
|  40  |      72.54     |     83.49    |

Next, we change the clustering algorithm from KMeans to GMM, and show that the underlying algorithm does not alter the outcome of the method significantly.

| Dataset | Worst/Bias-Conflicting (GMM) | Avg (GMM) | Worst/Bias-Conflicting (KMeans) | Avg (KMeans) |
|:---:|:--------------:|:------------:|:--------------:|:-----------:|
|  CelebA  |      81.11     |     85.19    | **81.61** | **86.04** |
|  CMNIST-0.995  |      73.08     |    84.84    | **72.56** | **84.42** |

**Attached pdf**. We have attached a pdf containing a few tables and plots, based on the questions and comments of Reviewers *qb1H* and *oZ3K*.

Meta-review: The paper introduces a computationally efficient training of an unbiased classifier using a large and frozen pre-trained model, where a new loss function is introduced: the adaptive margin loss. The paper received 4 reviews, where the average score is 6.25 (Min: 5, Max: 8), with the following strengths: 1) original extension from the debiasing literature, 2) clear contributions, 3) experimental protocol is comprehensive with many datasets, 4) extensive ablation studies, 5) paper is well written, 6) results are convincing, and 7) problem is of high impact. As for the weaknesses, the paper lists the following issues: 1) strong assumption on the alignment of the bias in the pretrained model and downstream task, 2) missing details about clustering, 3) missing uncertainty results from competing approaches, 4) missing details about design decisions, and 5) missing deeper theoretical analysis. Most issues have been successfully addressed in the rebuttal, so I’m recommending the acceptance of the paper.
Decision: Accept (poster)