CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?
Authors: Ibrahim Alabdulmohsin
Xiao Wang
Andreas Peter Steiner
Priya Goyal
Alexander D'Amour
Xiaohua Zhai
Venue: ICLR 2024 poster (2024)
Keywords: contrastive language image pretraining
multimodal learning
fairness
bias
data balancing
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We investigate the effectiveness of data-balancing for mitigating biases in contrastive language-image pretraining (CLIP) models, identifying areas of strength and limitation.

Abstract: We study data-balancing for mitigating biases in contrastive language-image pretraining (CLIP), identifying areas of strength and limitation. First, we reaffirm prior conclusions that CLIP can inadvertently absorb stereotypes. To counter this, we present a novel algorithm, called Multi-Modal Moment Matching (M4), designed to reduce both representation and association biases in multimodal data. We use M4 to conduct an in-depth analysis taking into account various factors, such as the model, representation, and data size. Our study also explores the dynamic nature of how CLIP learns/unlearns biases. In particular, we find that fine-tuning is effective in countering representation biases, though its impact diminishes for association biases. Also, data balancing has a mixed impact on quality: it tends to improve classification but can hurt retrieval. Interestingly, data and architectural improvements seem to mitigate the negative impact of data balancing on performance; e.g. applying M4 to SigLIP-B/16 with data quality filters improves COCO image-to-text retrieval @5 from 86% (without data balancing) to 87% and ImageNet 0-shot classification from 77% to 77.5%! Finally, we conclude with recommendations for improving the efficacy of data balancing in multimodal systems.

Published as a conference paper at ICLR 2024
CLIP THE BIAS: How USEFUL IS BALANCING DATA IN MULTIMODAL LEARNING?
Ibrahim Alabdulmohsin', Xiao Wang Andreas Steiner!, Priya Goyal Alexander D' "'Amour,, Xiaohua Zhail Google Deepmind: "Zürich, Switzerland. New York, USA. Boston, USA. [ibomohsin, xzhai @googgll.......o com
ABSTRACT
We study data-bat balancing for mitigating biases in contrastive language-iman -inage pre- training (CLIP), identifying areas of strength and limitation. First, reaffirm prior conclusions that CLIP can inadvertently absorb stereotypes. To counter this, we present novel algorithm, called Multi-Modal Moment Matching (M4), designed to reduce both representation and association biases in multimodal data. We use M4 conduct an in-depth analysis taking into account various factors, such as the model, representation, and data size. Our study also explores the dynamic nature of how CLIP learns/unlearns biases. In particular, we find that fine-tuning is effective in countering representation biases, though its impact diminishes for association biases. Also, data balancing has a mixed impact on quality: it tends to improve classification but can hurt retrieval. Interestingly, data and architectural improve- ments seem to mitigate the negative impact of data balancing on performance; e.g- applying M4 to SigLIP-B//I B/16 with data quality filters improves COCO image-to- text retrieval @5 from 86% (without data balancing) to 87% and ImageNet O-shot classification from 77% to 77.5%! Finally, conclude with recommendations for improving the efficacy of data balancing in multimodal systems.
INTRODUCTION
Recent advances on multimodal systems have been breathtaking, including in zero-shot classifica-
tion (Radford et al. 202T
Zhai et al. 2022b Yu et al. 2022a Jia al. [20T)) text-to-image genera-
tion Saharia et al. 2022 Yu al. [022bb Chang al. 2023 Rombach et al. 2022 Ramesh et al.)
2022
image captioning [Alayrac et al. 2022; Chen et al. 2022) and music generation (Agostinelli
al.
2023 name few. However, such systems
inflict harm if left unchecked, such as by biases, causing performance disparities, or encoding narrow cultural perspectives.
Contrary to the traditional supervised learning setup, which is well-understood, multimodal systems present novel ethical challenges. They typically operate with an oper-vocabbllry, in which the set of input and/or output tokens is unbounded. Hence, statistical definitions for bias such demographic parity (Dwork et al. 2012 Zafar et al.) 2017 Mchrabi et al. 2019) or equalized odds (Hardt et al. 2016 Kleinberg et al., 2016) do not extend easily to such systems and might even yield contradictory results under differrnt setups (Akyürek et al. 2022). Second, externalization in multimodal systems- by allowing users to interact with the system ways that can disclose its internal reasoning along with the open- vocabulcyy nature can expose users to biases in unanticipated ways. Third, data in multimodal systems is potentially biased, perpetuating societal stereotypes (Birhane et al., 202T).
For concreteness, consider the following examples. If we query the text-to-image Imagen (Saharia jtt al. 2022) with the prompt: "clipart picture of a manager dressed in black talking to a secretary dressed in blue," " we get the generated image samples shown in Figure I (left), in which managers are men and secretarics are women. Similarly, if we ask for pictures of pilots and flight attendants, we get the sample of pictures shown in Figure [ (right). Similar examples
reported by Wang et al. (2023), Cho et al. (2022). Tan et al. (2020) and Mishkin et al. (002)) using GANs (Goodfellow et al. 2014 Stable Diffusion (Rombach et al. 2022 and DALL-E (Ramesh et al. 2021).
Moreover, zero-shot classifiers are also susceptible to biases as highlighted by Hall ct al. (2023)
and Birhane et al. (202T), which demonstrate in Figure] [ using contrastive langeglanguage

---

Published as a conference paper at ICLR 2024
p(man)
(man) 13%
pl (wonan))
p(man)
96%
X(man)
96%
wooman
4%
(womaa))
4%
Figure 1: TOP: Text-to-image models prompted for occupations, such as manager / secretary (left) or pilot flight attendant (right) can reflect societal stereotypes. Refer Section the exact prompts. BOTTOM: CLIP can encode societal stereotypes, such as by associating cars with men. See Section
pretraining (CLIP) (Radford et al. 2021). Evidently, CLIP encodes societal stereotypes, such as by associating ties, cars, and tools with men. Such unintended biases can inflict harm by shaping social narratives. In addition, due to their open vocabulary nature, users can apply such models in unanticipated ways, e.g. so-called "corner cases" (Birhane et al. 202T). to promote prejudices.
In this work, we do not claim to offer comprehensive solution the complex issues discussed above. Rather, we examine in depth the effectiveness of one remediation strategy: data balanc- ing. Specifically, we investigate the impact of debiasing data multimodal systems that align embeeddings/reprrsenttttioos across modalities in the contrastive learning setup.
We focus on contrastive learning for the following reasons. First, it captures many of the complexities involved in multimodal systems, including open vocabulary, extemnalization, lack of data distribution at inference time, and lack well-established definitions of bias. Second, such systems exhibit strong societal biases (Hall et al. 2023 Birhane et al. (02T]. Third, they more amenable to analysis than generative models (e.g. when studied in the zero-shot visual classification setting or cross-moda retrieval). Fourth, they increasingly used in critical domains like healthcare (Sellergren al.) 2022; Tiu et al., 2022 Zhang et al. 2023), and are popular ingredients in many models, as seen in Florence (Yuan al. 2021). CoCa (Yu al. 2022a), and OWL-VIT (Minderer al. 2022a). Finally, biases in such models can manifest downstream, c.g. in captioning and retrieval (Berg et al. 2022)
To enable this analysis, we develop data balancing algorithm, called Multi-Modal Moment Matching (M4), and analyze it theoretically (see Section 5). Using M4, we conduct a comprehensive evaluation of the effectiveness of data balancing for obtaining desirable model behavior, identifying areas of strength and limitation, while accounting for factors such as the model, representation, and training data sizes. We also examine how quickly CLIP learns/unlearns biases, among other evaluations. empirical evaluation of this
In total, we train over 150 models. To the best of our knowledge, magnitude for data balancing in multimodal systems has never been conducted before (see Section 6). Broadly speaking, our results are nuanced, and suggest that while data balancing has positive impact on the model bias with a mixed impact on its quality, it is insufficient for obtaining fair behavior downstream. This echoes prior observations in vision; e.g. Wang et al. (019).
2 PRELIMINARIES
For brief overview, CLIP contains two towers for vision and language, which are encoder-only transformers (Vaswani al. 2017). Denoting x = (v, t) for an image-text pair, let (z", z²) e R' x R' be the corresponding outputs of the two towers, each is embedded in R". We refer to as the "representation size" in this paper. Given a fixed image v and a collection of captions (t1..... tw)
whose corresponding representations are (zi..... 21, Rrxn CLIP assigns probability score to each caption by first calculating the logits 1 ((z" >)))) E I and then applying softmax normalization p = SoftMax(I/T)) where T E R is a learnable temperature.
To study the effectiveness of data bias mitigation in CLIP, we introduce data balancing algorithm that tackles biases in first-order statistics (such as perceived gender imbalances) and second-order statistics (such correlating occupations with a particular perceived gender).
2

---

Published as a conference paper at ICLR 2024
Definition 1 (Data Representation Bias). If's D E 10, is a sensitive attribute sampled i.i.d., the representation bias (RB) in D with respect to a target π E [0,1] is: maxse[m] [T Ep [s]]
To keep our analysis general, E {0, 1)th in Definition is a binary vector that encodes all (potentially overlapping) sensitive attribute categories, such as perceived gender and age groups. For instance, the first entry could be binary indicator for the group "perceived women," " the second binary indicator for the group "perceived men," the next ten indicators for the Monk Skin Tone scale (Monk] 2019), and so on. We keep π in our discussion arbitrary because the desired target distribution of categories may vary depending on the context, as discussed in (Berg et al.) (022).
Data representation bias (RB) measures differences in group prevalence: e.g. if (0.5, 0.5) for "men" and "women," " then having only "men" in 80% of the images implies a significant RB in the data. Definition defines RB w.r.t. π to account for overlapping attributes, such as gender and race. Definition 2 (Data Association Bias). If (s. y) S x V, where S {0, 13th and y = {0, f) is sampled i..dd from a joint distribution D, association bias (AB) in D w.r.f. (S, }) is defined as:
Data Association Bias = max rE(c
ED yy Ss = 1] ED y Ss = 0]|
Definition 2 is an extension of the widely-adopted notion of demographic parity (Dwork et al. 2012 Zafar et al. 2017 Alabdulmohsin & Lucic [022]. It captures bias in second-order statistics. For example, if y is the set of occupations and perceived gender, association bias is large when an occupation is more prevalent among "perceived men" compared to "perceived women."
Both types of bias Definitions]] I andd 2 are defined w.r.t. the data distribution. Next, we provide the analogous definitions for the model itself, which is always CLIP throughout our analysis.
Definition 3 (Model Representation Bias). If f : At is a classifier outputting a probability distribution over some sensitive attributes S, the representation bias (RB) in f w.r.t. a fixed target (Tk [x)]] where fx is the probability assigned to attribute k.
[0, 1]th is: maxke[m]
In particular, if и is uniform and the model assigns a higher probability to one group over others, on average, it has larger representation bias. We care about RB because models should follow the principle of indifference (PI) (Eva 2019) if images do not contain relevant evidence about subgroups.
Definition 4 (Model Association Bias). If (x,ss)) s, E X x S y D are drawn i.i.d. where the sensitive atribbte s E S is as in Definition 2
the association bias (AB) in f X + y is:
Model Association Bias
max kE(m).rE(c)
EED [fr (x) ss 1] ED [fr (x:) sy = 0]
(2)
Our goal is to study how such biases in the data transfer to their corresponding biases in the model, and whether data balancing is impactful, especially in light of the bias amplification phenomenon often observed in the literature (Bolukbasi et al. 2016 Hendricks et al., 2018 Zhao al.] 2017). For consistency, we focus on perceived gender as a sensitive attribute s and use occupations as the set of We denote the original data (without intervention) as baseling ine.
labels y in Definition 3
Generally, to mitigate biases in the data, we explore two options. The first option is to tackle the desired constraints directly; i.e. by focusing only on perceived gender and occupations. We refer to this version of the data as balanced. The second option is to include proxies as well. For instance, CLIP may associate "pilots" with "perceived men" even if decorrelated in the data because cockpits resemble machines, and "machines" might be associated with "perceived men" elsewhere in the data. The set of proxies we use, such as "computer" and "briefcase," is suggested by an LLM and listed in Appendix A.5 During training, we balance the marginal distribution of perceived gender and remove correlations with both occupation and proxies. We refer to this version of the data as proxies es. In Appendix A.5.2 we motivate including proxies using the causal formalism in Veitch et al. (202T).
3 SUMMARY OF FINDINGS
Before presenting the detailed experimental results, we first summarize our major findings:
I Proxies mitigate representation biases: In addition to balancing the prevalence of sensitive attribute, decorrelating sensitive attributes against many proxy variables helps minimize the model's RB, thereby preventing the model from favoring certain subgroups in unrelated contexts.
3

---

Published as a conference paper at ICLR 2024
II But proxies hurt association biases: For AB in the correlation between gender and occupation, adding extra proxy variables can negatively affect attempts to reduce AB. This is likely because the added constraints compete with the original ones during data balancing when not all constraints can he satisfied simultaneously.
setting, e.g. removing
III Fine-tuning is effective for representation bias: RB in the model is sensitive to the last distri- bution seen on the training data. So, fine-tuning on balanced data effectively counters it.
IV But fine-tuning is less effective for association bias: The extent of AB in the model varies gradually based on the duration spent training on balanced data, irrespective of whether the balanced data is seen first or last during training.
Mixed impact on model performance: Data balancing changes the distribution of human and non-human images/texts, which leads to improvement visual classification benchmarks but worse performance on image-text retrieval. We explain this in Section 4.4 and Appendix A.6
VI But improving data quality and model architecture helps: Improving data quality and model architecture, such as by filtering out image-text pairs with low similarity scores, seems to mitigate any potential negative impacts of data balancing on the model's performance. See Appendix A.8
DETAILED RESULTS
Setup. We assume a potentially-infinit set of examples (x((), , (()) where x is an image- text pair, y E {0, 1} are some predefined labels and E {0, 13mh encodes sensitive attribute categories. See Section 5 for further details. We focus on perceived gender and use occupations as the set of labels y (see Definitions and 2). both are inferred from image and text separately and then concatenated. For instance, to remove correlation between gender and occupation, we de-correlate all combinations of image- and text-based annotations. Since and y are binary vectors, zero correlations imply independence so, by the data-pree processiocss inequality (Cover & Thomas| [191]) this offers a stronger bias guarantee than logical disjunction, where an attribute is marked present detected in any modality. See Appendix A. for illustrations and additional discussions.
[
In order to explore the dynamic nature of how contrastive language-image pretraining (CLIP) learns and unlearns biases, we split training into two stages. In Stage 1, we train on either the original data without intervention or on the same dataset but after intervention. In Stage 2, we switch to the other version of the dataset. In total, each CLIP model is trained on IB image-text pairs from the WebLI dataset (Chen et al. 2022). We vary the length of Stage in f0%, 10%, 90%}. We use three random seeds for each setup. Appendix provides the full training configuration.
In order to study the architecture's 's impact, we experiment with two model sizes studied in Dosovitskiy jet al. (2020) andZZai et al. (2022a): (1) size and size with, respectively, 30M and 100M parame- ters for each modality. Besides, we also vary the representation size r E (384, 768} (see Section 2} Since we use a patch size of 32 x 32 in our sweep, we also verify key findings on larger sequence lengths using ViT-B/16 with 16 x 16 patch sizes. We conduct meta-analysis on all configurations to determine statistical significance utilizing the Wilcoxon's signed-rank test (Wilcoxon) 1992).
4.1 REPRESENTATION BIAS
To evaluate RB in CLIP according to Definition 3 we use ILSRCV2012 (Deng et al. 2009) and compare the parity Elp( "man") p( ("woman") across random subset of 8K images. Intuitively. models should be indifferent to perceived gender in the absence of evidence, as discussed in Section
Single Distribution. Figure 2 (top) summarizes the results for models trained on a single distribu- tion. First, we observe that the amount of training examples seems to offer little benefit; mean parity cannot be reduced by simply training on bigger datasets. Second, balancing the data with or without proxies - helps in mitigating RB. fact, adding proxies seems particularly beneficial for models with large representations trained on large datasets. Refer to Appendix A.7 for additional figures.
Because debiasing removes about 10% of the examples, models trained on debiased data see some examples twice. However, there is evidence that examples seen twice behave like fresh examples when training is not converged (Alabdulmohsin al. 2022bb so we do expect this difference have an impact on the results.
4

---

Published as a conference paper at ICLR 2024
Data Size 100M
Data Size 1000M
0.4
0,4
Baseline
Balanced
Proxies
Mean Parity
S/52: 384
B/32: 384
S/32: 768
B/32: 768
S/32: 384
B/32: 384
S/32: 768
B/32: 768
Model ps: Representation
Model/ ps: Representation
0.6
Data Size 100M
Data Size IB
balanced
proxies
balanced
proxies
0.2
baseline balanced
0.039
0.461
0.742
Figure 2: TOP: Mcan parity Elp(man) - p( woman)] across images from the ILSRCV2012 better. BOTTOM: On left, parity scores
dataset (Deng al. 2009). Values closer to for ViT-B/16 (longer visual sequence length). On right, values calculated using Wilxocon's signed rank test (Wilcoxon) 1992) for the null hypothesis that column has the same effect row.
Intervention >> No Interveniion
No Intervention >> Intervention
0.4
0.4
10
90
100
Mean Fariy
0.2
0.2
0.0
0.0
-0.3
S/32:
384
B/32: 384
S/32: 768
B/32: 768
-0.2
S/32: 384
B/32) 384
5/32: 768
B/32) 768
Model ps: Representation
Model Representation
Figure 3: CLIP is trained on IB examples split into two stages. On the left, it is initially trained on intervened data with proxies, before switching to the original data. On the right, it trained on the original data before intervening. Legends indicate the fraction of time [%]] assigned to Stage 1.
Learning Dynamics. Figure displays the mean parity but for different durations of Stage 1. To recall, we split training into two stages in order to analyze how quickly CLIP learns and unlearns biases. As shown in Figure the last distribution seen by the model, even if it is meager 10% of the training duration, heavily impacts its parity. So, fine-tuning on balanced data is an effective remedy.
4.2 ASSOCIATION BIAS
Our second set of evaluations examine the association bias (AB) between perceived gender and occupations. We use FairFace (Karkkainen & Joo, (02T). UTKFace (Zhang et al. 2017) and MIAP (Schumann et al. 2021) datasets, all annotated with perceived gender attributes. The first two datasets contain face images while MIAP contains images more representative of real-world scenarios. We include MIAP because cultural biases extend beyond faces to artifacts and social practices (Berg ct al.
2022 To evaluate AB, we calculate the mean absolute parity (1/1D/) Eyey (p(y |man)
ply woman) across all occupations y by providing CLIP with two labels: the occupation's 's name vs. the empty string. Then, average those for all images of perceived men and all images of perceived women. In MIAP, only images containing a single perceived gender are used.
Single Distribution. Figure (top) summarizes the average parity in models trained on a single data distribution. First, training longer increases the level of bias in baseline, likely because longer training allows the model to reflect biases in the data. But, again observe that balancing data helps. Nevertheless, adding proxies seems to hurt, likely because the added constraints compete when not all constraints during data balancing can be satisfied simultaneously.
Learning Dynamics. Pigos AB the time of intervention. Unlike in RB (see Section we now observe gradual increase or decline in AB depending on how long the model trained on intervened data, irrespective of whether the balanced data is seen first or last during training.
5

---

Published as a conference paper at ICLR 2024
Duta Size 100M
Data Size 1000M
0.2
0.2
Balaneed
FairPaze
0.0
0.0
LTK
0.0
0.2
MIAP
0.1
0.1
0.0
0.0
S/32:
B/32: 384
S/32; 768 Representation
B/32: 768
S/32: 384
B/32: 384
S/32 768
B/32;
Model
Model Representation
FaeeFaee
LTT
MIAP
0.20
0.25
0.15
Data Size 100M balanced proxies
Data Size = IB
balanced proxies
vs. baseline
10
10-6
10-6
10-0
palanced
0.056
10
0.05
0.05
0.00
Figure 4: TOP: A comparison of AB (perceived gender against occupation) evaluated in three downstream datasets. BOTTOM: ViT-B/16 results (left) and statistical analysis (right) Figure
Intervention Intervention
0.2
Faifface
0.2
0.3
0.2
0.2
0.1
0.2
MIAP
0.1
0.1
0.0
0.0
B/32:
Model ps: Representation
Model/ Repeesentation
Figure summary of how CLIP learns or unlearns association bias (shown in y-axis) when intervened data comprises different percentages [%] of training duration. Setup is similar to Figure
4.3 RECOGNIZING SENSITIVE ATTRIBUTES
The simplest way for model remove its bias be entirely blind unaware of the sensitive attribute. For contrastive models, such outcome is undesirable since impacts utility; e.g. the model may not distinguish between "father" "mother" in the caption. RB and AB due to data balancing impacts the ability of the model to recognize those attributes, use zero-shot classification setting with the two labels "man" and "woman" for each image in FairFace, UTKFace and MIAP. When aggregating errors using Wilcoxon's 's signed rank test (Wilcoxon 1992) with the null hypothesis that (baseling ine, balanced, proxies) yield the same performance, we have p > 0.05, indicating the absence of statistically significant differences. See Appendix A.7
examine the improvement in
4.4 MODEL QUALITY
Next, compare the quality of CLIP models across three downstream evaluations: zero-shot classification, few-shot image classification using the representation provided by the image tower, and retrieval for both COCO (Lin et al.) 2015) and Flickr YYoung et al. 2014). Here, report results for ViT-B/16 image tower and deter the full set of figures to Appendix A.7] As shown in Appendix A.7 Table T and Figure 6. balancing the data improves classification,, average, but

---

Published as a conference paper at ICLR 2024
Table 1: 10-shot classification results using ViT-B/16 image tower in CLIP,
pretrained on IB ex-
amples. Datasets are ILSRCV2012 (Deng et al. 2009) Colorectal (Kather et al.
2016), Cars (Krause
al. 2013), Birds (Welinder et al. 2010). UC Yang & Newsam 2010). CIFARIOO (Krizhevsky
2009
DTD Cimpoi et al. 2014). Caltech FFei-feei al. 2004) and Pets (Parkhi al. 2012).
INet
Color
Cars
Birds
UC
C100
DTD
Caltech
Pets
basel ine
50.4
75.7
77.5
46.8#.6
91.3*.1
57.8#.0
66.9
88.9
71.9
balanced
50.9
74.9
79.0
47.6
92.2
57. 57.4
67.4
89.2
73.3
proxies
51.1
75.0
78. 78.72.2
47.7
91.5
57.2
67.02-2
89.6
7
0.65
0,66
C16e700
0.00
0.60
0.75
0.615
0.614
0.268
0:396
0,605
0,603
Baseline
Ralanced
Phosioss
Roodline
Ralanced
Prosiee
Reudline
Ralanced
Prosiox
0.90
0.829
0.70
0.55
0.712
0.964
0.97
0.044
2.02
0:75
Figure 6: TOP: shoo classification for ILSRCV2012, CIFAR100, and Pets. BOTTOM: Retrieval results (image-to-text and text-to-image) for COCO and Flickr. See Appendix A.7 7 for full results.
hurts retrieval. In addition, the impact on each metric is statistically significant with p < 10-5, In Appendix A.6 we conduct in-depth analysis, which reveals that the impact on quality attributed to the distribution shift of human and non-human image-text pairs. In particular, debiasing reduces the number of examples containing humans, which improves visual classification since most benchmarks like ImageNet contain few (if any) human images. By contrast, retrieval datasets, such as COCO, contain significant number (> 40%). We analyze and reproduce the effect of debiasing A.6
Data Quality & Architectural Improvement. In Appendix A.8 we show that improving the data quality and architecture, by grounding texts on images and using the recent Sigmoid- loss Language- Image Pretraining (SigLIP) (Zhai et al. (223), helps in mitigating any potential negative impacts of data balancing on the model's performance.
MULTI-MODAL MOMENT MATCHING (M4)
Denote S |S| = m and y | = c for the set of attributes and labels respectively. To mitigate representation and association biases (see Section 2 reweigh examples in the data. It can be shown that finding a set of weights q assigned to training examples, which mitigate the two types of bias, is equivalent to finding a feasible solution q to the following constraints (see Appendix A.2.I
Vk rE[c] ED pq (Sk TT) yrll < eD A
Vue[m] EEp [q (sy, - -m)]]] S CR.
(3)
for some small tolerance levels ep and ER. The intuition behind this follows from the fact that since both y, and Są are binary-valued, zero covariance implies independence. Because finding solution when q E {0, 1} is NP-hard (Mehrotra Celis! 2021), we relax the problem by optimizing within the unit interval [0, 1] and we sub-sample from the data according to it. However, since the same algorithm can potentially be used in- processing, with q taking any value in R++, we make our treatment general by adding the constraints 0 q Q for some Q E g U (20).
In practice, can sometimes be useful to constrain the average size q as well. For example, when subsampling from a dataset, E[q] must be equal to the subsampling rate 1) E (0, 1]. In addition, common assumption in fair subset selection is to allow examples to have different utilities u 2 (Stoyanovich et al.) 2018); e.g. based video engagement text/image quality. Finally, bias constraints should correspond to a soft penalty to accommodate cases when they are not feasible, which can occur even when ép. en 0. We incorporate all such features into the algorithm.
7

---

Published as a conference paper at ICLR 2024
Algorithm 1: Update step per example
Algorithm 2: Bias vector implementation.
Hyperparameters:
def biasvector 2, pi, epsd, absr)
average weight size maximum weight enforeement level
Q>NERt VER+*
Args:
5: sensitive attributes, of length m. labels, of length
TERT+
learning rale bias constraint levels
pi: target stributioon of length m. opsd & cppr: bias constraint levels.
CDICR E 10.
Update:
I: a <- biasVector(s, y, n. EDD ER)
Returns:
b: bias vector of length 2nte+1).
2: 3 + [u=a+ a 1 nu
dp np.einsum wwb.iiinmm"bibboo b"bi,bo-sbio", 8 pi, 2).
3: a < [u(y- - Q) a 14
reshape (batchhssissss -1))
4: q 7- (v"a+ 1 B)/u
b [p
5: u + e+ ga
cp spad, -dp apsd, AB s pi opur, -5 pi -oper RB axis-1) return
6: P + n
Figure 7: Pseudo-code of the data balancing algorithm in Section 5 LEFT: Single update per example
(s, u), where is the example's utility. RIGHT: Numpy-like implementation of the bias vector a.
An overview of the data balancing algorithm is shown in Figure 7 It maintains two optimization variables v E 2m2(()+1) and μ E R, which are used to calculate the sample weight q by solving:
minimize B|g A
ED u (q - m)²]
+ +
Σ AE(m]
Σ
(4)
kE(m), rE(c]
where 1P max {0, [ED (s: - m&) yr EDP} and d = max DE (q (Sx -+)]]l
{ are the violations to the bias constraints in 3). The first term in (4) encourages the weights to be
close to 7 since we have the constraint B[g] = 7 while assigning higher priority to examples with
greater utility u. The second term penalizes biases with V > 0 controlling bias enforcement levels.
Proposition 1. Algorithm terminates with an optimal solution to the optimization problem in 4
The proof is in Appendix
A.2.1
At inference time, the weight q assigned to a new example is:
u + u
[u Q) - o n|
T 1 mu]" )1+
9
Here, a is "bias vector" calculated from the sensitive attributes s and labels y (see Appendix A.2. We provide examples of how the algorithm works and empirical verification in Appendix [A.2.3] We also compare Algorithm Dagainst other debiasing methods for binary classification in Appendix [A.2.4
aggaitt
Proposition 2. Starting from the initial values = 0 and 0, let F, be the dual loss of the optimization problem in ( after & updates of Algorithm 7 and denote Foc for its limiting value. Then, Algorithm| with the learning rate schedule O(Ivvt) satisfies: mine E[F;] Fool S
Qmac 9
loge
Vt
The proof is in Appendix A.2.2 examples to converge. Since m [S] and | are typically small, convergence is fast. Through- out our experiments, we use the weights q to subsample from the original dataset. The subsampling rate (7) in Algorithm| [1 is chosen to be the maximum rate where bias constraints are still satisfi- able, which happens to be 90% in our setup. We use subsampling. instead of reweighting, because subsampling tends to perform more favorably (Sagawa et al. 2020 Celis et al. 2018).
Proposition 2 states that Algorithm needs, at most, O(Qmc/ ")
6 RELATED WORKS
Fairness in Multimodal Systems. Fairness is social construct and important consideration when evaluating machine learning models. Research has shown that in the absence of bias mitigation, machine learning systems can amplify societal stereotypes (Hendricks et al. 2018 Bolukbasi et al.
8

---

Published as a conference paper at ICLR 2024
2016
Caliskan et al., 2017 7 Yang et al. 2020), cause performance disparities (Buolamwini & Gebru
2018 Deuschel et al., )020) and encode cultural biases and perspectives (Hutchinson et al. 2022
De V ries et a 2019). For multimodal systems, in particular, while there is growing interest their
applications and datasets, such as LAION (Schuhmann et al. 2021) and Webl.I (Chen et al. 2022).
recent findings indicate that the use of multiple modalities not only continues to encode societal
biases (Hutchinson et al., 2022. Wang et al., (223) including in CLIP models (Hall et al.) 2023 Wang
al. 2022). but can also amplify them further compared unimodal systems (Booth et al. 2021).
This includes not only text-to-image generative models and CLIP but also image captioning (Zhao
ct al.) 2021: Tang et al.] 202T]. In particular, multimodal datasets, such as LAION, were found to
contain problematic content, such as stereotypes and ethnic slurs (Birhane et al. (021). Few methods
have been proposed for mitigating such biases including adversarial training ( (Yan et al. 2020 Berg
et al. 2022), projection (Wang et al. 2023) and dropout (Wang et al. (02T). Yet, we are not aware of prior works that examine the effectiveness of data balancing, such as in CLIP (Radford et al. 2021))
Reweighting methods. The data balancing algorithm we develop is variant of reweighting methods. Because it does not alter examples (unlike fair representation learning such as Zemel et al. (2013); Feldman et al. )015)): Lum & Johndrow (2016); Calmon et al. (2017) : Madras et al. (2018) and does not alter labels (unlike methods such as Kamiran & Calders (2009) and Alabdulloooh et al.
(2022a)) it serves some skepticism in parts of the literature about the efficacy of reweighting in overparameterized models (Byrd & Lipton 2019). Intuitively, an overparameterrzed model has the potential to perfectly fit all training examples, rendering it optimal regardless of the sample weights used. Nonetheless, in the multimodal setting, the size of the training dataset can be extremely large (i.c.. billions of examples), and the model is only trained for a few epochs. In this setup, re-weighting the data may can be impactful as we demonstrate in our experiments. In the traditional supervised learning setup, there is evidence that reweighting is competitive (Chen et al.] 2018: Idrissi et al.) 2022; Choi jt al. 2020), and that subsampling performs more favorably (Sagawa et al. 2020; Celis et al.] 2018). Analogous techniques have additionally been used in other contexts, including texts (Dixon et al.)
viable baseline for debiasing CLIP-style models. However, there has been
2018) and fine-tuning on balanced data to mitigate spurious correlations (Kirichenko et al. 2022)).
Advantages of M4. Our data diversity algorithm offers more flexibility than prior methods by relaxing many of their constraints. For instance, it accommodates an arbitrary number of overlapping groups and attributes, making it also capable of handling traditional multiclass settings for which few blxck-box debiasing algorithms exist and they have only recently been developed (Alghamdi et al.) 2022; Allbduulmohsin et al.) 2022a). Diversity sampling has well-established history, sometimes referred to as "fair subset selection" (Drosou et al., 2017: Mehrotra & Celis) 202T). It originates from early observations in search that sub-sampling data by maximizing utility leads an under- representation of minorities (Kay et al., 2015). However, prior works, such as Stooynnvich et al. (2018). Yan et al.] (2020) and Mehrotra & Celis (202T), only address representational biases (i.e., first-order statistics) and fail to account for association biases (i.e., second-order statistics). Other data balancing algorithms such as in Mehrotra & Celis (202I) even solve LPs, making them prohibitive for Internet-scale multimodal data. The data balancing algorithm we introduce resolves such limitations.
7 CONCLUSION AND FUTURE WORK
We present data balancing algorithm and use it study the impact of mitigating biases in Contrastive Language Image Pretraining (CLIP). popular training paradigm used in several domains. We define representation and association bias in data and model separately and carefully disentangle the effects of the model, data, and representation size. Our findings suggest that balancing data is impactful but insufficient for obtaining fair downstream behavior so it should be combined with other intervention methods, such as in- and post-processing. In addition, it is recommended that models are trained on balanced data from the outset given that fine-tuning is less effective in removing association biases. Finally, the impact on quality should be assessed for both human-related metrics (e.g. describing actions) and non-human related metrics (e.g. classifying objects) since they can behave differently. Nevertheless, our analysis is necessarily limited since fairness cannot be reduced to statistical metrics. For example, we do not discuss which sensitive attributes are relevant or what acceptable label associations might be, and we only focus on contrastive (not generative) models. Potential future directions include exploring data augmentation methods and validating the effectiveness of our work on datasets such Casual Conversations (Hazirbas et al. 202T), which have self-identified labels.

---

Published as a conference paper at ICLR 2024
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reductions approach to fair classification. In ICML, 2018.
Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank. Musiclm: Generating music from text, 2023.
Afra Feyza Akyürek, Muhammed Yusuf Kocyigit, Sejin Paik, and Derry Wijaya. Challenges in measuring bias via open-ended language generation, 2022.
Ibrahim Alabdulmohsin and Mario Lucic. A near optimal algorithm for debiasing trained machine learning models. In NeurlPS, 2021.
Ibrahim Alabdulmohsin, Jessica Schrouff, and Oluwasanmi Koyejo. A reduction to binary approach for debiasing multiclass datasets. In NeurIPS, 2022a.
Ibrahim M Alabdulmohsin, Behnam Neyshabur, and Xiaohua Zhai. Revisiting neural scaling laws in language and vision. NeurlPS, 2022b.
Cean-Baptiste Alayrac, Jeff Donahue, Pauline Lue, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35:23716- 23736, 2022.
Wael Alghamdi, Hsiang Hsu, Haewon Jeong, Hao Wang, P Winston Michalak, Shahab Asoodeh, and Flavio Calmon. Beyond adult and compas: Fairness in multi-class prediction. In NeurIPS, 2022.
Hugo Berg, Siobhan Mackenzie Hall, Yash Bhalgat, Wonsuk Yang, Hannah Rose Kirk, Aleksandar Shtedritski, and Max Bain. prompt array keeps the bias away: Debiasing vision-language models with adversarial learning. arXiv preprint arXiv: 11933, 2022.
Lucas Beyer, Xiaohua Zhai, and Alexander Kolesnikov. Big vision. https:://gihubb.com
google rerreearcch hwtpchhchhciccccccc...eee vivisiiivvviio vissin) 2022.
Jeffrey P Bigham, Ryan S Kaminsky, Richard E Ladner, Oscar M Danielsson, and Gordon L Hempton. Wehinsight: making web images accessible. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 181-188, 2006.
Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe, Multimodal datasets: misogyny, pomography, and malignant stereotypes. arXiv preprint arXiv:2110001963, 2021.
C. Blake and C. J. Merz. UCI repository of machine learning databases, 1998.
Tolga Bolukbasi, Kai- Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NeurIPS, 2016.
Brandon M Booth, Louis Hickman, Shree Krishna Subburaj, Louis Tay, Sang Eun Woo, and Sidney D'Mello. Bias and faimess in multimodal machine learning: A case study of automated video interviews. Proceedings of the 2021 International Conference on Multimodal Interaction, pp. 268-277, 2021.
Stephen Boyd and Almir Mutapeic. Stochastic subgradient methods. 2008. URL https:://///.. /see stanford.
subgrad notes. pdf
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge University Press, 2004.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- and Qiao Zhang. JAX: composable transformations of Python+NumPP programs, 2018. URL
http::/gitt com goog/
10

---

Published as a conference paper at ICLR 2024
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In FAccT, 2018.
Jonathon Byrd and Zachary Lipton. What is the effect of importance weighting in deep learning? In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research,
pp. 872 PMLR 09-15 Jun 2019. URL http://://prcceeeeggggg/o/// mwt.ssmmr.pppp...... wtt..mm....pe////////.. Syral9a.html
Aylin Caliskan, Joanna Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 2017.
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre-processing for discrimination prevention. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: / hetps:://rroeeddingg neurips. cc/paper ceeppss. wwreer_ffleeseeeeeeeee fww.sss/pppee//11....... File/ /eLee/94490 922 Saw.aa25554550 5d24 7e3413333
Elisa Celis, Vijay Keswani, Damian Straszak, Amit Deshpande, Tarun Kathuria, and Nisheeth Vishnoi. Fair and diverse DPP-based based data summarization. In Jennifer Dy and Andreas Krause (eds.), ICML, volume 80 of Proceedings of Machine Learning Research, pp. 716-777677 725. PMLR, 10-15 Jul 2018. URL httphhtss////oooceeeeeeoeoo
8a .8tml
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers, 2023.
Irene Chen, Fredrik D Johansson, and David Sontag. Why is my classifier discriminatory? NeurIPS, 31, 2018
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209. 06794, 2022.
Xinlei Chen, Hao Fang, Tsung-Yi Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325. 2015.
Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generative models, 2022.
Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair generative modeling via weak supervision. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1887-1898. PMLR, 13-18 Jul 2020. URL hatpp:://proceee
choi20a. html
M. Cimpoi, Maji, Kokkinos, Mohamed, and Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014.
Thomas M Cover and Joy A Thomas. Elements of information theory. Wiley & Sons, 1991.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009.
Jessica Deuschel, Bettina Finzel, and Ines Rieger. Uncovering the bias in facial expressions. arXiv preprint arXiv:2011.11311, 2020.
Terrance DeVries, Ishan Misra, Changhan Wang. and Laurens van der Maaten. Does object recogni- tion work for everyone?, 2019.
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. In Conference on AI, Ethics, and Society, 2018.
11

---

Published as a conference paper at ICLR 2024
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.
Marina Drosou, Hosagrahar V Jagadish, Evaggelia Pitoura, and Julia Stoyanovich. Diversity in big data: A review. Big data, 5(2):73-84, 2017.
Miroslav Dudik, Richard Edgar, Brandon Horn, and Roman Lutz. fairlearn 0.4.6. 2020. URL http:////pppp...or//ppppeee////s
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Innovations in Theoretical Computer Science, 2012.
Benjamin Eva. Principles of indifference. The Journal of Philosophy, 1667)):90--41,, 2019.
2019.
Alessandro Fabris, Stefano Messina, Gianmaria Silvello, and Gian Antonio Susto. Algorithmic fairness datasets: the story so far. Data Mining and Knowledge Discovery, 36(6):004-2155,, 2022.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. 2004.
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In SIGKDD, pp. 259 268, 2015.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Mclissa Hall, Laura Gustafson, Aaron Adcock, Ishan Misra, and Candace Ross. Vision-language models performing zero-shot tasks exhibit gender-based disparities, 2023.
Moritz Hardt, Eric Price, Nati Srebro, et al. Equality of opportunity in supervised learning. In NeurIPS, 2016.
Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, and Cristian Canton Ferrer. Towards measuring faimess in ai: the casual conversations dataset. IEEE Transactions Biometrics, Behavior, and Identity Science, 4(3):324-332, 2021.
Jonathan Heek, Anselm Levskaya, Avital Oliver, Marvin Ritter, Bertrand Rondepierre, Andreas
Steiner, and Marc van Zee. Flax: A neural network library and ecosystem for JAX, 2020. URL
com www.gooo/ooooooooooooooooooooooo goog e/ flax
Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach, Women also snowboard: Overcoming bias in captioning models. In ECCV, 2018.
Ben Hutchinson, Jason Baldridge, and Vinodkumar Prabhakaran. Underspecification scene
description--todepictii tasks, 2022.
Badr Youbi Idrissi, Martin Arjovsky. Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive arrss-rruup-acurrccy.. 2022.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In /CML, pp. 4904-4916. PMLR, 2021.
Faisal Kamiran and Toon Calders. Classifying without discriminating. 2009 2nd International Conference Computer, Control and Communication, 2009.
Kimmo Karkkainen and Jungscock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 1548-1558, 2021.
Jakob Nikolas Kather, Cleo-Aron Weis, Francesco Bianconi, Susanne M Melchers, Lothar R Schad, Timo Gaiser, Alexander Marx, and Frank Gerrit olllener. Multi-class texture analysis in colorectal cancer histology. Scientific reports, 6:27988, 2016.
12

---

Published as a conference paper at ICLR 2024
Matthew Kay, Cynthia Matuszek, and Sean A Munson. Unequal representation and gender stereotypes in image search results for occupations. In Proceedings of the 33rd anmual ACM conference on human factors in computing systems. pp. 3819-3828, 2015.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXhv:1412.6980, 2014.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness spurious correlations, 2022.
Jon Kleinberg. Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determi- nation of risk scores. arXiv preprint
2016.
Ron Kohavi. Scaling up the accuracy of uaive-Bayes classifier: A decision-tree hybrid. In SICKDD, 1996.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- fine-grained categorization. In 4th International IEEE Workshop 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Taku Kudo and John Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Eduardo Blanco and Wei Lu (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium, October 31 November 2018. pp. 66-71. Association for Computational Linguistics, 2018. doi: 10. 11.18663/////18-20111 URL
http://8o...crr//11166655
2012
Tsung- Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015.
Kristian Lum and James Johndrow. A statistical framework for fair predictive algorithms. arXiv preprint
2016.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations, 2018.
Maggie Makar, Ben Packer, Dan Moldovan, Davis Blalock, Yoni Halpern, and Alexander Amour. Causally motivated shortcut removal using auxiliary labels. In International Conference on Artificial Intelligence and Statistics, pp. 739-766. PMLR, 2022.
Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning, arXiv preprint arXiv: I0Xiv:9008.00 0963,, 2019.
Anay Mehrotra and L Elisa Celis. Mitigating bias in set selection with noisy protected attributes. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 237-248, 2021.
Joel A Middleton, Mare Scott, Ronli Diakow, and Jennifer L Hill. Bias amplification and bias unmasking. Political Analysis, 24(3):307-323, 2016.
Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, al. Simple vocabvocyy object detection with vision transformers. arXiv preprint 22055 062300 2022a.
Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenbom,, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Amab, Mostafa Dehghani, Zhuoran Shen, al. Simple vocabuuoyyylary object detection. In European Conference on Computer Vision, pp. 758-755. Springer, 2022h.
13

---

Published as a conference paper at ICLR 2024
Pamela Mishkin, Lama Ahmad, Miles Brundage, Gretchen Krueger, and Girish Sastry. Dall-e preview risks and limitations, 2022.
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. pp. 220-229, 2019. doi: 10. N0.1145/2277500.28775555 URL
3287560. 3287596
Ellis Monk. The monk skin tone scale, 2019. URL /.KLLittpp////kittoeee.e http:///okkittoe..oogglgg
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. V. Jawahar. Cats and dogs. In IEEE Conference Computer Vision and Pattern Recognition, 2012.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models, 2022.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An investigation of why overparameterizzatonn exacerbates spurious correlations. In Hal Daumé III and Aarti Singh (eds.). Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 8346-8356. PMLR, 13-18 Jul 2020. URL mlr press / 19/ www.awawwaaaaaaaaaaa
https::/prrceediini
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorcalistic text-to-image diffusion models with deep language understanding. 2022.
Christoph
Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs, 2021.
Candice Schumann, Susanna Ricco, Utsav Prabhu, Vittorio Ferrari, and Caroline Rebecca Pantofaru. step toward more inclusive people annotations for fairness. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2021.
Andrew B Sellergren, Christina Chen, Zaid Nabulsi, Yuanzhen Li, Aaron Maschinot, Aaron Sarna, Jenny Huang, Charles Lau, Sreenivasa Raju Kalidindi, Mozziyar Etemadi, et al. Simplified transfer learning for chest radiography models using less data. Radiology, 305(2):454 465, 2022.
Julia Stoyanovich, Ke Yang, and HV Jagadish. Online selection with fairness and diversity constraints. In Proceedings of the EDBT Conference, 2018.
Shuhan Tan, Yujun Shen, and Bolei Zhou. Improving the faimess of deep generative models without retraining. arXiv preprint arXiv:2012.04842, 2020.
Ruixiang Tang, Mengnan Du, Yuening Li, Zirui Liu, Na Zou, and Xia Hu. Mitigating gender bias in captioning systems. In WWW. pp. 633-645, 2021.
Ekin Tiu, Ellie Talius, Pujan Patel, Curtis P Langlotz, Andrew Y Ng, and Pranav Rajpurkar. Expert- level detection of pathologies from unannotated chest x-ray images via self self-supervised learning. Nature Biomedical Engineering, pp. 1-8, 2022.
14

---

Published as a conference paper at ICLR 2024
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017.
Victor Veitch, Alexander D' Amour, Steve Yadlowsky, and Jacob Eisenstein. Counterfactual invariance to spurious correlations: Why and how to pass stress tests. May 2021.
Jialu Wang, Yang Liu, and Xin Eric Wang. Are gender-neutral queries really gender-neutral? mitigating gender bias in image search, 2021.
Junyang Wang, Yi Zhang, and Jitao Sang. Fairelip: Social bias elimination based on attribute prototype learning and representation neutralization. arXiv preprint arXiv:2210. 14562, 2022.
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In ICCV, 2019.
Zihao Wang. Lin Gui, Jeffrey Negrea, and Victor Veitch. Concept algebra for text-controlled vision models, 2023.
Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technology, 2010.
Frank Wilcoxon. Individual comparisons by ranking methods. In Breakthroughs Statistics: Methodology and Distribution, pp. 196-202. Spriner,, 1992.
Shen Yan, Di Huang, and Mohammad Soleymani. Mitigating biases in multimodal personality assessment. In Proceedings of the 2020 International Conference on Multimodal Interaction, pp. 361-369, 2020.
Forest Yang. Moustapha Cisse, and Sanmi Koyejo. Fairness with overlapping groups. arXiv preprint arXiv:2006. 13485, 2020.
Yi Yang and Shawn Newsam. wag-oo-v and spatial extensions for land-use classification. In ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM 2010.
I-Cheng Yeh and Che-hui Lien. The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Systems with Applications, 36(2), 2009.
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67-78. 2014.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini. and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arX1v:2005.01917, 2022a.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Bureu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models for content-rich text-to-image generation, 2022b.
Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, et al. Florence: A new foundation model for computer vision. arXiv preprint I.Xiv22/1..11432 2021.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fair- ness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In International Conference on World Wide Web. 2017.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In ICML, 2013.
15

---

Published as a conference paper at ICLR 2024
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers. In CVPR, 2022a.
Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexander Kolesnikov, and Lucas Beyer. Lit: Zero-shot transfer with locked-image text tuning. In CVPR, pp. 18123-18133, 2022b.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11975-11986, October 2023.
Sheng Zhang. Yanbo Xu, Naoto Usuyama, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, Cliff Wong, Matthew P. Lungren, Tristan Naumann, and Hoifung Poon. Large-scale dopain-specifie pretraining for biomedical vision-language processing, 2023.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progressionregression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017.
Dora Zhao, Angelina Wang, and Olga Russakovsky. Understanding and evaluating racial biases in image captioning. In ICCV, 2021.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint arXiv: 1707:07777,09457, 2017.
16

---

Published as a conference paper at ICLR 2024
Table 2: Dataset examples. The first columns contain the image (resized 224px) and text (translated to English), as seen by the model. The third and fourth column show the profession and gender attributes that extracted from the image (via an image classifier) and text (via regular expression matching), respectively.
Image
Text
y
s' y
Comments
Example of an image where the image signals incomplete, and the text signals provide ad- ditional information would otherwise he lost.
female judge sentencing
judge perceived woman
pexels.com
The gender not men- tioned in the text de- scription, and the regu- lar expression does not trigger for "sport". but the image signal ex- tracts both sensitive at- trihute and profession.
archery bow and arrow
sport
perceived woman
pixahay.com
Here the text introduces an additional associa- tion between the profes- sion "doctor" and the gender, which extracted from the im- age alone.
a male doctor
wearing green scrub sits couch and falling asleep
perceived woman
doctor perceived man
vecteczy.com
In this case, the image signal only points to- wards profession, but the text adds the asso- ciation with "perceived men".
photo of man surfing during daytime
sport
perceived man
pexels.com
A APPENDIX
A.1 SENSITIVE ATTRIBUTE ANNOTATION
We infer attributes from both modalities. Using vision classifier for images and regular expression matching on captions. Image modality is needed because even highly reputable websites have poor alt-text coverage (Bigham et al.) 2006). Using the text modality is needed because images could be interpreted incorrectly noisy annotators. For example, (Birhane et al. 2021 shows how photograph of female astronaut could be misinterpreted by image classifier as picture of "housewife in an orange jumpsuit." Hence, both modalities are used our analysis.
Why do infer attributes from both modalities? As demonstrated in Table 2 the extracted signals from image and text can be incomplete, and in some cases one of the modalities creates an association that is present in the image text alone.
17

---

Published as a conference paper at ICLR 2024
Why do we concatenate them? As explained in Section 4 modalities. For instance, to remove correlation between gender and occupation, we de-correlate all combinations of image- and text-based annotations. Since and y binary vectors, zero correlations imply independence so, by the data processing inequality (Cover Thomas 1991). this offers a stronger bias guarantee than logical disjunction, where attribute is marked present if detected in any modality. We illustrate this with synthetic example next.
concatenate annotations from both
Consider the example shown in the table below, where have examples and the annotations of both and are provided from both image and text modalities.
5 (image)
(imuge)
s (text)
y (text)
8 (image or text)
y (image or text)
1
1
0
2
D
0
0
Ex 3
I
0
Ex 4
1
D
I
D
0
Ex 6
0
0
0
0
0
0
7
0
Ex.
0
0
In this example, if we assume that an attribute present when it is detected by any modality, do not observe any bias in the data (rightmost two columns). In particular, the correlation between s and y is zero and both attributes are statistically independent of each other. However, both the image and text towers see different pictures, where s and are correlated with an absolute correlation coefficient exceeding |p] > 0.5. Hence, by ensuring that (image) decorrelated from both y (image) and y (text) and vice versa, stronger bias guarantee is ensured.
18

---

Published as a conference paper at ICLR 2024
A.2 DATA BALANCING ALGORITHM
A.2.1 PROOF OF PROPOSITION
First, we prove that finding a sample weight function that mitigates both types of bias in Definitions and is equivalent finding solution the set of constraints in (3).
Since Yr and are both binary-valuce random variables, having zero demographic parity (DP) is
equivalent to zero covariance (Alabdulmohsin Lucic 2021) so DP condition is equivalent
E
T&) yrl 0 p (Sk-
because:
Elq sk] Eq
[[q (Sk Th) y.] E [qqq SA yr]
Th B (q y]
E Sk yr!
E
E
By dividing both sides by [[q]], we have:
E [q (Sk T&) yrl
qq St yr]
E
sk]
IE [q y
E[q which is the covariance w.r.t the sample weights Hence, if condition 66 is satisfied, the association bias zero.
F(q
E[q]
E
Second, satisfying the representation bias (RB) constraints is equivalent to setting Elq s by definition, which equivalent to the second condition in
To solve the optimization problem 4 we solve the dual problem. To simplify notation, suppose initially that the dataset is finite and let g E IR" be vector of length n (corresponding to all n examples). The optimization problem is:
min GER" GER",BERRM (mB43((+1)
(g n1) 71) V
Ag b
1 q = rmn
1 5 Q
q. 9.620. 0,
where U is diagonal matrix containing the utilities of all examples and A is matrix formed from constraints in (3) (a Numpy-based implementation of the columns of A is shown in Figure (right)).
We denote the i-th column of A by (()) and refer to it the "bias vector".
The Lagrangian is:
b, B) = 2 (1-
(Ag (-))))))((m)mm)))---- - (Q-Q1) (1-Q1)
subject to v, 3, a 2
Taking the gradient w.r.t. q and setting zero:
q 171 U-1 (AT v + 41 - 8 a))
Setting the gradient w.r.t to zero gives us:
0 V
Plugging the expression for back into L. have the minimization problem:
min Σ inf USrVV 1,
(T + - =)² - ual () TT! -1aa- - B) + Qa)
(8)
1=1 9,020
2u;
We will the following fact:
Lemma 1. wt [[A + is a minimizer to the objective function:
min w20
-) lw
(E
Returning to 88. we have by complementary slackness (Boyd & Vandenberghe 2004 that either
0 or B 0 consider each case separately.
19

---

Published as a conference paper at ICLR 2024
Case I: If 0, the minimizer ( to each of the inner optimization problems is by Lemma
1
3000 = (w(+) () where w(() vf a()) + By plugging this into (8) and using 7 we have:
w( 8
1
u(3
and the stochastic gradients:
9 7
(
(1)
(10)
"
Case II: If 3 = 0, the minimizer (0) to each of the inner optimization problems is by Lemma a (u(9 (n) Q) w01t Note that we indeed have that ( .g(()) expected. Similar to
before, we have:
w (1)
=
(11)
and the same stochastic gradients as in 10 From this, Algorithm follows. By strong duality (Boyd Vandenberghe 2004). the solution returned by Algorithm is the optimal solution the minimization problem in
A.2.2 PROOF PROPOSITION
As shown in Appendix A.2.1 Algorithm minimizes the loss in X via stochastic gradient descent. (1, the stochastic gradient g in (10) is bounded in norm
Writing the optimization variable by:
|||| + (1-2) Q S 7p
Q²
Q2 nT
119122 S
(1 ||ll3)
(12)
because < q S Q in all iterations (see 5), and the second incquality follows from the fact that
Q 2 17 (maximum must be larger than the mean). The bias configuration a satisfies ||a| = 1 +
where e max(ED, ER). Hence: ||a|22 < (2(1 + e)m(c + 1))² because 2m2((+1), Since
< 1 (otherwise the constraints are void) while m, c 2 1, we have the simplified bound:
11 S 5Qm(c+
(13)
Denote the optimal solutions v" and ut and write: R schedule 7, = 1/vl and the well-known convergence rates (Boyd Mutapeic 2008), we have:
|| + (n*)² Using the learning rate
R²
(5mmlct)))
E. =0 ((1))) ((9me) 1
2 logt
min -[F]- -
2 2
vl
where F, is the loss function iteration t.
A.2.3 EMPIRICAL VERIFICATION
To verify the algorithm, run it with the following constraints: (1) balance perceived gender attribute, (2) balance the age group attribute, (3) remove correlations between perceived gender and occupation, and (4) remove correlations between perceived gender and age. The goal is to satisfy all constraints simultaneously. In this illustrative example, however, we consider an attribute present if detected in any modality. Within each sensitive attribute axis, the median distribution across all categories (i.e. 12% for both males and females and 1% for all age both types of bias simultaneously.
the target distribution to be
groups). Figure & displays summary of the results. As shown in the figure, the algorithm mitigates
A.2.4 COMPARISON AGAINST PRE-PRSCGGSING ALGORITHMS
We compare, for reference, the data balancing algorithm against prior preprocessing in the classical binary classification setting.
20

---

Published as a conference paper at ICLR 2024
Perceived Gender
Age Group
Percsived Gender
Age Group
Pescetrod Mine
Persened Wemsin
child
Serrage
addt
sineee
Persowed Men
Perceted Woetes
child
terrage
Absolute Pearson Correlations
perceived gender x occupation
perceived gender x age group median
median
mast
maax
BEFORE
0.010
0,259
0.102
0.218
0.001
0.003
0.002
0.057
Figure 8: TOP:A comparison of the distribution of subgroups before and after applying the data diversity algorithm in Figure which we have four sets of constraints (see Section The target distribution 12% for both perceived gender categories and 1 for all age groups. BOTTÓM: Pearson correlation scores (max and median) between perceived gender and occupation/age. Algorithm mitigates both first-order and second order biases simultaneously.
Table 3: Comparison of the data balancing algorithm against prior bias mitigation methods in the binary classification setting: correlation remover (CR), reduce-to to-tucr---binary (R2B), reduction sensitive rules, threshold-optinzzer (TO), randomized threshold optimizer (RTO). See Section A.2.4 for details. Last row for the proposed data balancing algorithm.
Adult
Default on Credit Card Eroor Ermor
DP
Ervor
Balanced Ermo
DP
Balanced
Baseline
in-
Reduction
01.0 1.00:.0 .0
16.4 11
14.9 + 1
01.4 1
17.9 1.99...
18.2 .1
jostt-
19.9
20.8
.3
18.3
RTO
00.8 22
16.9
16.5
.3
20.7 -
20.7 .2
pre-
CR
18.9
14.7
12.9
00
19.6 .3
20.0
pre-
R2B
08.3
15.1
13.1
00.9
28.8
29.1
pre- Balancing
09.1 1.2
15.6
13.7 1
01.2 1.11
17.9 17.9
18.1 .2
Specifically, we evaluate the algorithm on the Adult income dataset (Kohavi 1996) and the Default of Credit Card Clients (DCCC) dataset (Yeh & Lien 2009). both from the UCI ML Repository (Blake Merz 1998 and are among the most widely used benchmark datasets in the ML fairness literature (Fbbii al. 2022). The Adult dataset contains 48,842 records with 14 attributes and the goal is to predict binary summary of income level. The DCCC dataset contains 30,000 records with 24 attributes, where the goal is predict if client will default on credit card payment. In both datasets, binary sex is used as sensitive attribute.
Besides preprocessing methods, we also include in- in-processing and post-processing our comparison. However, the intent here is not to compete, since our goal is to balance the data prior to training.
We compare against the correlation remover method (Dudik et al.. 2020) and reduce-to tolluc--o--biaryy (Alabdulmohsin et al. 2022a), reduction to cost-sensitive rules (Agarwal et al. 2018). threshold optimizer (Hardt et al. 2016), and randomized threshold optimizer (Alabdulmohsin Lucic (02T)
and use the implementations released by authors in the FairLearn package (Dudik et al. 2020).
hyperparameters are kept to their default setting. The base classifier is a two-layer ML.P with 128 hidden units, ReLU activations (Nair & Hinton 2010), that optimizes the log-loss using Adam (Kingma & Ba 2014) with an initial learning rate 0.001.
For evaluation metrics, report demographic parity (DP), classification error rate, and the balanced етrоr rate (average error rate across both subgroups).
As shown in Table while data balancing mitigation option does not outperform in- and post-pe post-processing methods, the data balancing algorithm performs on par with other pre-processing techniques in mitigating bias without impacting model's quality. Importantly, our algorithm handle a much broader setting than prior methods, including an arbitrary number overlapping attributes and labels.
21

---

Published as a conference paper at ICLR 2024
A.3
MODEL CARD
Model details following Mitchell et al. (2019).
The model contains two towers, which are encoder-only vision trans- formers (ViT) (Dosovitskiy et al. 2020) for image and text modalities. We use architecture sizes ViT-S and VIT-B. For the image tower, we use either a patch size of 32 x 32 or 16 x 16. For the text tower, we use SentencePiece Kudo & Richardson (2018) tokenizer. The two towers are trained to maximize the similarity of image-text pairs via the contrastive loss.
Model Architecture:
Inputs: The model takes both an image and a text as input.
Outputs: The model outputs representations for image and text inputs.
The primary use research on multimodal applications, such as zero-shot classification and retrieval. We use such models to study the effectiveness of data balancing as a bias mitigation strategy.
Intended Use:
Known Caveats: As noted in several prior works, multimodal systems can pick up societal biases. While we demonstrate some of those issues in this work, our analysis is necessarily limited since fairness a societal concept that cannot be reduced to simple statistical metrics.
System Description: The model analyzed in stand-alone setting, and not used as part of a larger system.
Upstream Dependencies: None.
Downstream Dependencies: None.
Hardware Software: The model is implemented using JAX (Bradbury et al. 2018). Flax (Heek et al. 2020), and Big Vision (Beyer et al. (2022). It is trained on TPU v2.
Compute Requirements: Each model trained on x TPU v2 chips on IB seen image- text pairs. A typical training takes 1.3 days.
Model Initialization: The model is trained from random initialization.
Model Size: ViT-S models have approximately 30M parameters per modality. ViT-B models have approximately 100M parameters per modality.
Training Dataset: We
which comprises of images with alt-texts from the public web. See (Chen et al. 2022 for an overview of the data collection process.
English-only subset of WebLI (Chen al.)
2022
Evaluation Datasets: We evaluate the models ImageVet-1SSRC2201 (Deng al. 2009), FairFace Karkkainen & Joo 2021), UTKFace (Zhang et al. 2017), and MIAP (Schu mann et al. 2021)
22

---

Published as a conference paper at ICLR 2024
A.4
TRAINING CONFIGURATION
input
batch_si2e 16_384
shuf=le_buffer_si1 250_000
pp 'decode rescoderesiiae (224) walue_range (-1,1))
model
model. model. www.eeeteeeeeeeeee... - 10.0
optimizer
optax name 'scale bact grad_clip_norm 1.C
ir = 0.001
0.0001
learning rate schedule schedule decay 'ragrt' schedul e. a.timescale 5_000 schedule warmup_steps 5_000
A.5 PROXIES
A.5.1 LIST
We queried a large language model (LLM) to provide suggested list of proxies that might link perceived gender with occupation. The LLM recommended the list shown in Table
bench suit goggles scrubs
briefcase
book
board tray
tools ashtray food
desk
computer
gun
handcuff clipboard
coat
whistle
watch flour gym
sport
microscope
credit card
shopping
sewing machine cabinet
calculator
paint
backpack car
headset mierophone comb
camera
helmet
vacuum
glassware
scissor
toothbrush
phone
Table 4: Full list of proxies used the experiments.
justified each example by linking particular occupation. For example, "bench" might provide a link to the occupation "judge," "briefcase" might provide link to the occupation "manager," "book" might provide a link the occupation "author," and on.
A.5.2 CAUSAL ANALYSIS OF DATA BALANCING APPROACHES
In this section, analyze the properties of the ba 1 anced and prox i datasets from causal perspective, using the formalism presented in Veitch et al. (202I). and make more formal argument motivating the use of proxies. Specifically, highlight a case where balancing labels y on sensitive attributes s alone is not enough because there are unbalanced proxies w.
We assume that the generative process for images is anti-causal with respect to the label y. sensitive attribute s, and proxy variables in our setting. In other words, assume that these variables are underlying causes of the image/text pair x that is taken as input.
Causal Model
In this setting, Veitch et al. (202T) show that the input then be decomposed into three pieces, illustrated Figure a):
x or parts of x that are not causal descendants of the senstiive attributes s. Importantly, X includes portions of x that are causal descendants of but not of s.
Xy+, parts of x that are not causal descendants of y.
XyлS. or parts of x that are causal descendents of both and
The features X,- are often considered "core" features, since they are not causally influenced by the sensitive attributes /eitch al.) (2021) show that predictors that only function of xy4 exhibit desirable counterfactual invariance properties. Correspondingly, the features Xy are often considered "spurious" features, since they have causal relationship to the labels y. Finally, the
23

---

Published as a conference paper at ICLR 2024
5
25
R
My
Kyyns
y
y X
WW
(b)
(a)
Figure 9: DAGs describing causal models for image/text pairs x as a function of sensitive attributes labels y, and proxies w. Solid arrows represent causal relationships; bidirectional dashed arrows represent non-causal relationships introduced by confounding or selection. (a) is a general generative model without proxies; (b) is simplified generative model with proxies.
intersection information Xулх entangles information about both and s; as we show below, this can introduce complications for data balancing mitigations.
Cases Where Balancing Proxies is Needed We now provide formal example highlighting the potential role of proxies in the data balancing algorithm introduced in the paper. For simplicity, in this argument will assume that there is intersection information Xyλs, because it is sufficient to show how proxy variables can complicate data balancing. Veitch et al. (202T) call this the "purely spurious" case.
As shown in Makar et al. 2022). in this setting, data balancing can be sufficient to achieve desirable predictive invariances, there are no uncontrolled confounders. However, if there are proxy varaibles w that are (1) correlated with the sensitive attributes s, and (2) correlated (causally or not) with the "core" features Xgt, these can operate as confounders that introduce correlations between sensitive attributes s and optimal predictors fxx designed to predict labels y from inputs x. Importantly, this can happen even when labels y and sensitive attributes are marginally independent, as we might achieve with data balancing.
To see how this can occur, consider the DAG in Figure Ib) where we introduce proxies w with bidirectional arrow connecting them to Here, we assume that there is no bidirectional edge connecting y to s, or connecting y to w, due to data balancing. Even though these edges are absent, the s-w edge creates are causally influenced by y, but not by s, can nonetheless be correlated with s. Any reasonable fxx) will incorporate information from the core features xg', this correlation implies that f(x) will have some dependence on s. Thus, conditioning on s can change the distribution of x resulting in a different conditional expectation [ff( s) across levels of
open path between and Xg1. This path implies that x,+, the features that
Note that this dependence path is cut off if the edge between s and w is eliminated, e.g., by balancing proxies w with respect sensitive attributes s. as done by proxies. This the primary motivation for this approach at population level. However, there are a number of potential complications to this argument. Chief among these is the need balance on an exhaustive set of proxy variables to eliminate the backdoor path described above. Notably, there is no guarantee that balancing an incomplete set of proxies will reduce dependence; indeed, in some cases the backdoor dependence can increase, phenomenon known as "bias amplification" in the causal inference literature (see, e.g.. Middleton et al.) 2016)). In addition, this argument is made a the population level, but in practice, finite sample complications in optimization or sampling variability may dominate the benefits of proxy balancing in terms of association bias.
'In practice, after data balancing, there can be an edge connecting w. which would imply that there is an open path hetween y and even though there is correlation hetween y and because these correlations are made cancel hy the data balancing algorithm. This complication is out of scope for this motivating example.
24

---

Published as a conference paper at ICLR 2024
Intersilinny
200
Figure 10: The distribution changes of top-50 50 pseudo object labels before and after debiasing.
Table 5: Example COCO captions with human mentions.
Keyword
Caption with human mentions
adult
Group of adults outdoor gathering on clear day.
boy
A little boy sitting on a wooden bench eating half a sandwich.
bride
A bride is standing on the grass under an umbrella.
child
A woman playing catch with her young child.
children
Children are playing saccer field with several adults ohserving nearby.
daughter
The mom is teaching her daughter to play baseball
elderly
A woman and elderly woman sitting a desk together in a classroom setting with notebooks and pencils on the desk.
father
Baby girl feeding her father some pink and white birthday cake.
female
female in a white shirt is playing tennis Two gentleman in formal suits, one of them is adjusting the collar of the other.
gentleman
girl
two girls in shirts grass and a baseball glove
groom
The cat grooms itself atop the workshop table.
lady
lady sitting in a half shell hut A male cook standing by the kitchen sink. man with a hat looks into the air a pop up couple of men that walking around on grass
male
man
mom
A dog and a cal share a moment
mother
A mother elephant and her calf touching trunks in the tall grass some people on some grass playing frishee and trees
people
player
A black and white photo of baseball players looking for a ball
sister
Two sisters watch as their kids decorate cake
son
Mom gives her daughter lesson in using her baseball glove.
teen
Three teenagers are throwing frishee hack and forth in field.
woman
A woman on a tennis court serves the ball.
A.6 MODEL QUALITY ANALYSTS
To understand the discrepancy of model quality on classification and retrieval tasks, we conduct a in-depth study the data distribution before and after debiasing. More specifically, we analyze the distributions on image modality (pseudo object labels, OCRs..), text modality (text length, word frequency, Part-Of-Speech tags...) and image-text cross-modality (image-text similarity). Surprisingly, debiasing doesn't change most distributions, except the pseudo object labels (Figure 10), which are annotated by the open-vocabulary detector OWL-ViT (Minderer al. 2022b) and reduce around 25% "adult" after the debiasing.
We further analyze the test sets of maageNe:-ILSRCV201 (Deng et al. 2009) and COCO Cap- tions (Chen et al. 2013), and find that the majority of ImageNet images are not associated with human-related labels. On the contrary, about 40% COCO images have captions mentioning humans (Table
To verify the hypothesis that the model quality change primarily caused by the human distribution shift in the training data, we randomly drop examples (instead of relying on the debiasing algorithm) which humans occur in either image or text. Figure [I] dropping more human examples leads to higher classification but lower image-text retrieval numbers, which perfectly reproduces the debiasing effect on the model quality.
25

---

Published as a conference paper at ICLR 2024
tast 10shot
INwt D-shot
COCO 21
.* DOCO 91
Flikr St
.. Flickr
65%
60%
80%
9056
drop
90%
0%
(beowlins)
Figure 11: The percentage of human training examples (mentioning human in image text) affeets the model performance on both classification (INet 10-shot and 0-shot) and retrieval (COCO and Flickr O-shot) tasks. All numbers are the mean values of 3 runs.
Furthermore, neutralize the human mentions in text (e.g. by replacing "female"/ "gentlemen" / "she" with "person" "peo- person"), in order to preserve training examples (i.e. avoid dropping human exam-
baseline
neutralize human
+1.13
80%
-0.14
60%
-0.12
+0.22
-0.26
40%
ples) and mitigate the side-effect of debiasing on model quality. Figure [2] demonstrates the fea- sibility of such an approach, and would like to leave this quality.
20%
INet 10-s
INet 0-3
COCO 1t COCO 12) Flickr 12t Flickr 12)
Figure 12: Human neutralization minimizes the influence to model
future work.
A.7 ADDITIONAL FIGURES
A.7.1
REPRESENTATION BIAS
Figures
13 and 14 report the median scores (as opposed to the means) for the analysis of Section 4.1
Data Size 100M
Data Size 1000M
Median Patiy
0.4
0.4
Rosellne
Balanced
Provies
0.2-
0.2
0.0
0.0
-0.2
-0.2
S/32:
384
B/32: 384
S/32: 768
B/32:
S/32: 384
B/32: 381
S/32: 768
B/32:
Model Representation
Model Representation
Figure
13: Median parity E [lp(man)
pl woman)] across images from the ILSRCV2012 Values closer to zero better.
dataset (Deng et al. 2009) See Figure
26

---

Published as a conference paper at ICLR 2024
Intervention No Interveniion
No Intervention Intervention
Median Parity
0.4
0,4
S/52: 384
B/32: 384
5/32: 768
B/32: 768
S/32: 384
B/32: 384
S/32: 768
B/32: 768
Model/ ps: Represcntation
Model/ ps: Representation
Figure 14: Similar to Figure 3 but using the median scores, instead of the mean.
A.7.2 ASSOCIATION BIAS
Figures IS and 16 report the median association bias scores (as opposed to the means) for the analysis
of Section
Data Size
100M
Data Size 1000
Bascline
Balanced
Proxies
FaieFiice
0.1
0.1
0.0
0.0
0.2
LTK
0.2
0.
8.9
MIAP
0.1
0.1
8/32:
B/32: 384
8/32: 768
R/32: 768
S/32: 384
384
8/32: 768
R/32:
Model ps: Represcutation
Model ps: Representation
Figure 15: Reporting the median bias scores instead the means for association bias. Figure and Section 4.2
Intervention >> No Intervention
No Intervention Intervention
0.2
FairPuxe
0.2
0.0
0.0
0.2
0.1
0.1
0.2
0.2
MIAP
0.1
0.1
0.0
384 Model pe: Representation
768
B/32:
384 Model ps: Representation
B/32:
Figure 16: A summary of how CLIP learns unlearns association bias (shown in y-axis) when intervened data comprises different percentages [%] of training duration. Here, we report the median bias scores occupations instead of the mean, which reported in Figure
27

---

Published as a conference paper at ICLR 2024
A.7.3 MODEL QUALITY
Figures 17 18, and 19 report the full model quality evaluation results in of few-shot classifica- tion, zero-shot classification, and retrieval.
Dura Size 190M
Dara Sino ICee
C400
B602
5/50
B632 766
8-51 768
1650 766
Repen-meadion
Regenentation
1..0
Dea Siac 1009
0L:00
Thate Six 3000
0.90
TECH
0.335
555 104
BISC 796
394
B0A0) 304
5/0,
BIA2 116
Data Siice (X0
Data Skie (000M)
5/22 354
B/22: 354
5/50 768
KA5t: 76
5/32
BA12 394 S/22
9692: 765
Thar Sive 191M
Duti/Sie Inees
CTP-MI D0
6/32
564 Reprosnnttoon
500
RH0 Repersnnation
Thra Sine 103M
Daro Si 10000
0.90
0.00
1.755
0:75
3012
5017 Repansenaiion
0.50
Dara Stee 2005
ex6
Dira Stao 10000t
0.115
6:00
10122
5033:
0.25
Thra Sinn 100M
6.45
Dara Si 1000M
502
5032
00
Dura Siee 100M
Daro S0 1000
0.35
C.00
0.35
1012
10:0
Dura Stee J033
0:00
Dire Stao 10000t
0.33
0.00
1012:
5022
Figure 17: Full 10-shot evaluation results across the nine datasets in Table
28

---

Published as a conference paper at ICLR 2024
1623
Dar Size J0M
0:50
Dan Si 100M
0.30
0.55
14.5
1.:00
0.23
Cife
0.30
0.4
0115
Stre
143
02
Medil
Figure 18: Full zero-shot evaluation results across the three datasets in Figure
A.7.4 ENCODING SENSITIVE ATTRIBUTES
Figure 20 reports the full evaluation results of predicting sensitive attributes. See Section 4.3 for details.
29

---

Published as a conference paper at ICLR 2024
Table 6: Association bias (AB) between gender and occupation in SigLIP models when they are pretrained with or without data balancing on high-quality subset of WebLI (Chen et al. 2022 (see Appendix A.
Data
FairFace
UTK
MIAP
Without Intervention
38.8
38.2
28.4
With Intervention 29.9
33.7
20.5
Table 7: Classification and retrieval evaluation results [%] in SigLIP ViT-B/16 with and without data balancing. Data balancing seems to improve the model quality in this setting.
Data
ImageNet
COCO Retrieval
Flickr Retrieval
10-shot
O-shot
i
txt2ing@5
img2 @5
txt2img@5
Without 70.7
77.0
86.0
73.4
98.5
94.3
Intervention
With 71.4
77.5
87.1
73.7
98.8
94.9
Intervention
A.8 IMPACT OF DATA QUALITY AND ARCHITECTURAL IMPROVEMENTS
In this section, investigate the impact of data quality and architectural improvements model performance. Rather than training CLIP on minimally filtered image-text pairs (where filtering primarily addresses personally identifiable information (PII) and inappropriate content described in (Chen et al., 2022), we introduce an additional quality filtering step. This step leverages pretrained models to assess the semantic alignment between image and text content. Image-text pairs with low alignment scores are discarded.
In this evaluation, we the recently-introduced Sigmoid-loss Language Image Pretraining (SigLIP) (Zhai et al.) 2023). We train a two-tower SigLIP with ViT-B/16 image encoder and ViT-B text encoder for IOB examples with and without data balancing (with proxies). Because SigLIP does not provide calibrated probability scores, we evaluate association bias (AB) in the model between gender and occupation by drawing one image of man and one image of woman unifromly at random, and asking the model to predict which of the two images contains person with given occupation (e.g. "nurse"). Parity then defined to the difference in probability that the model chooses one particular gender over the other: [p(man) p( (woman)].
Table 6 provides the maximum observed parity each model across all occupations. Clearly, data balancing has positive impact on the model's bias, in agreement with our earlier results in Section 4 Interestingly. however, we do not observe a negative impact on the model quality this time. In fact, the quality of the model seems improve in all metrics, as shown in Table 7 This suggests that data quality and architectural improvements can mitigate the potential negative impact of data balancing, allowing us reduce biases in the model without impacting its quality.
30

---

Published as a conference paper at ICLR 2024
Duta Stece HMM
Dres Sic 1000
0.93
0:90
0:20
5000 2044
2003: 304
NA5 TM6
5012 Tea
364
B111: 304
5411: 706
B12: 768
Made Reperectation
Masiil pp: Repecintakkn
19M
10301
0.25
0.55
0.35
0:95
6750
503
B/Y2 384
503
Midel (g) Regrosetation
Movill ppe Repessetiioo
Coosträng 401
0:35
0.
0.30
0.55
Than Not 19M
0.55
Tier Sow 193014
0.50
50002 2044
2012: 384
NA2 766
1522 Tea
384
1512: 304
M12: 706
B02 768
Madel
Mopruockation
Madiel (JI Repecentakkn
Dass H10M
1000M
Bade-iuub 4010
11:04
B/92.
855 766
RSS
0.80
592 708 8102
Maskt/p Repessenaiio
Modii (pei Regenentaiin
0L55
0:055
0.00
0.55
0.33
0:75
5532
0L:5
Thua Size H9M
0.33
Ther Sio 1930M
0112
8:20
9130:
364
3030: 364
955 786
502 766
364
BA12 384
992, 766
B/92 768
Madki (m)
Reyroectaiion
Model (p
Repeseetakon
9030M
Ficb-wdiie
0.35
FiLL
0.30-
0.23
5150
0.58
S132
502, 766 Modill Aquesustaton
B/22 384
B/02
Madel Megronctaiion
Figure 19: Full retrieval results across the two datasets in Figure 6
31

---

Published as a conference paper at ICLR 2024
FairFace
0.4
1..0
0.0
S/32:
5/32:
ps: Representation
ps: Reyresenttion
Siee 100M
Data 1000
CL.
0.0
5/50: 1994
IIST: 1944
IM32: 7766
S032 2014
7008
Model psi
Representation
MIAP
Date Stie 100M
Dasa Siaw 1000
0,50
S/92: 364
B/12: 384
S/92: 768
R/St: 768
S190:
R/92: 364
5/90:
B/33: 768
Model / Regresemaatin
Model po: Represematiin
MIAP
100
Daraa 1000M
Wonee
1.0
0.0
0.0
pm: Repreacntaaioon
Medel/ pi Reprosmntaion
UTK
1000
1000M
0.2
0.0
0.0
B/22. 384
B/32,
5/92/ 384
B/32:
ps:
Representation
Data Size 10004
Data Size 1000M
Wenen
S122: 394
B122: 384
5132:
768
B13 768
S02 384
B122 384
5/32: 768
B/32: 768
Modki Representaiion
Maska/ppe Repecsaittion
Figure 20: Full evaluations results for predicting sensitive attributes Fairface, UTKFace, and MIAP, disaggregated by perceived gender. The y-axis the mean log-loss (lower is better). See Section 4.3 for details.
32


Reviews:
[{"review_id": "IeGtUZwJFf", "reviewer": "Reviewer_yZxD", "summary": "This paper identifies the societal bias issue in CLIP models and provides possible explanations and workarounds. Specifically, the representation and association biases are taken into consideration. While these biases can be somehow addressed with the data balancing strategy, other issue emerges. In this regard, this paper discusses in detail the role of data balancing in handling bias issues, and provides useful insights.", "questions": "- To me, the representation and association biases respectively correspond to the distribution shift of marginal output probability p(y) and conditional output probability p(y|x). What about another widely-discussed distribution shift in domain adaptation/generalization literature, i.e., marginal input probability p(x)? Can it also be a significant bias issue in large multimodal models?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well motivated and clearly written.\n- Simple data balancing strategies are proposed to tackle the bias issue, demonstrating promising results.\n- Comprehensive experimental results and analysis are presented, which may benefit the reader in relevant fields.", "weaknesses": "- While AB is relatively easy to mitigate, RB seems much more difficult to remove. In this regard, I would suggest the authors to shed more light on possible reasons and solutions. For example, I assume data augmentation shall be a promising workaround, and encourage the authors to explore more.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "enkj3aPnCs", "reviewer": "Reviewer_5bEw", "summary": "This paper studies two types of biases, representation biases (RB) and association biases (AB), in vision-language models such as CLIP. \n\nRB refers to that the model learns to prefer sensitive attributes (e.g., gender, age groups) in the training data. AB refers to that the model associates certain concepts with sensitive attributes. (e.g. occupations with a specific gender)\n\nStudying such biases is an important problem in the real-world as CLIP-like models are widely used in the industrial applications. In this paper, the authors first investigate the empirical evidence of both biases and how it transfers from data to the model. They show that RB is sensitive to the latest training data distribution thus fine-tuning (FT) is an effective approach to reduce RB. However, FT is weak on AB. They then propose a data balancing algorithm to alleviate both RB and AB by estimating an optimal weight for each data example.", "questions": "1. How exactly the de-correlation between sensitive attributes and proxies is implemented?\n2. In AB experiment, why adding proxies has inconsistent performance?\n3. In data balancing algorithm, how to intuitively interpret $\\alpha$ and $\\beta$? and how $s$ is determined, is it calculating from the dataset over sensitive attributes?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The empirical evidence of RB and AB is well supported in the experiments.\n2. The proposed data balancing algorithm is principled with theoretical analysis.\n3. The paper studies an interesting and important problem which may have a wide impact in real-world industrial applications, such as recommender systems and advertising.", "weaknesses": "1. The number of sensitive attributes in the experiments is limited to only gender and occupation.\n2. Further experiments on proposed data balancing algorithm is lacking in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6R9JabbavR", "reviewer": "Reviewer_yvnK", "summary": "This study delves into the efficacy of data-balancing in reducing biases in CLIP models, highlighting its strengths and constraints. A novel data-balancing algorithm is introduced, showing success in mitigating representation biases, particularly through fine-tuning, but offering lesser impact on association biases, with a variable effect on task performance such as enhanced classification yet impaired retrieval. The research culminates with strategic guidance aimed at optimizing data balancing in multimodal learning systems.", "questions": "1. In Section 4.1, the authors discuss various strategies, including adjusting the training set sizes and fine-tuning with or without dataset intervention. \n\n- While these strategies are standard in model training, the authors' approach underscores a well-known principle: merely increasing the size of a 'corrupted' dataset—say, by tenfold—will not address inherent issues. This scenario is a classic case of \"garbage in, garbage out.\"\n\n- Furthermore, the practice of fine-tuning a model on a specific dataset inevitably alters some pre-existing weights, adapting the model to new data characteristics. Consequently, it is not surprising that models fine-tuned on intervened sets exhibit improved bias metrics.\n\n- In summary of this question: What's new here? \n\n2. This study suggests that balancing our training datasets can mitigate bias. However, it also implies that researchers must painstakingly identify what constitutes bias and determine the features requiring adjustment. I am particularly interested in the authors' insights on how their proposed framework accommodates additional features present in real-world datasets.\n\n\n3. My concern extends to the interrelated nature of certain features, as highlighted in the occupation versus gender discussion in Section 4.2. While it is inappropriate to rely on stereotypes for inferring gender, assuming a 50/50 gender split across all professions disregards real-world disparities. It remains unclear how the proposed methods address \"association bias.\"\n- A link to “U.S. BUREAU OF LABOR STATISTICS” https://www.bls.gov/cps/cpsaat11.htm", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Nice definition of various types of bias in data or model, this is a solid foundation to understand the problem\n\n2. Given a nice definition of bias such as gender, this work shows the effect of model training with various type settings. (More or less data, fine tuning with various length of training time) \n\n3. Tested on various datasets and backbone designs. \n\n4. The proposed balancing algorithm does seem to successfully diminish bias without compromising the quality of the model.", "weaknesses": "1. I hope to see figures that are easier to interpret.\n\n- For Figure 2 (top): It seems you intend to demonstrate that even with extra training data, bias persists. I struggled to determine which bars were being compared. A similar issue occurs with Figure 3.\n- Regarding Figure 4: At first glance, without referring to the captions, all the color bars appear identical. My initial interpretation was that the results were largely uniform across the board.\n\n\n2. I find myself somewhat perplexed by the conclusions drawn from the study's results. This confusion does not necessarily point to a flaw in the research but suggests that further clarification might be beneficial. I recommend referring to the detailed queries I've raised in the question section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)