SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP
Authors: Yusuke Hirota
Min-Hung Chen
Chien-Yi Wang
Yuta Nakashima
Yu-Chiang Frank Wang
Ryo Hachiuma
Venue: ICLR 2025 Poster (2025)
Keywords: Societal bias
CLIP
Debiasing
Fairness
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: We propose SANER, a debiasing method for CLIP that removes societal bias without requiring attribute annotations, while preserving attribute-specific information.

Abstract: Large-scale vision-language models, such as CLIP, are known to contain societal bias regarding protected attributes (e.g., gender, age). This paper aims to address the problems of societal bias in CLIP. Although previous studies have proposed to debias societal bias through adversarial learning or test-time projecting, our comprehensive study of these works identifies two critical limitations: 1) loss of attribute information when it is explicitly disclosed in the input and 2) use of the attribute annotations during debiasing process. To mitigate societal bias in CLIP and overcome these limitations simultaneously, we introduce a simple-yet-effective debiasing method called SANER (societal attribute neutralizer) that eliminates attribute information from CLIP text features only of attribute-neutral descriptions. Experimental results show that SANER, which does not require attribute annotations and preserves original information for attribute-specific descriptions, demonstrates superior debiasing ability than the existing methods.

Published as a conference paper at ICLR 2025
SANER: ANNOTATION-FREE SOCIETAL ATTRIBUTE NEUTRALIZER FOR DEBIASING CLIP
Yusuke Hirota 12; Minninuunn Chen', Chien-Yi Wang', Yuta Nakashima', Yu-Chiang Frank Wang Ryo Hachiuma'
INVIDIA "Osaka University National Taiwan University
fy-hirota, (maassiimaaaa ids. osaka-u ac. jp [minhunge, chienyiw, frankwang, rhachi uma ( wmw.eeiiiiaaccom
ABSTRACT
Large-scale vision-language models, such as CLIP, are known to contain soci- etal bias regarding protected attributes (e.g., gender, age). This paper aims to address the problems of societal bias in CLIP. Although previous studies have proposed to debias societal bias through adversarial learning or test-time project- ing, our comprehensive study of these works identifies critical limitations: loss of attribute information when it is explicitly disclosed in the input and 2) use of the attribute annotations during debiasing process. To mitigate societal bias in CLIP and overcome these limitations simultancously, we introduce simple- yet-effective debiasing method called SANER (societal attribute neutralizer) that eliminates attribute information from CLIP text features only of attribute- neutral descriptions. Experimental results show that SANER, which does not require at- tribute annotations and preserves original information for attribule-specific de- scriptions, demonstrates superior debiasing ability than the existing methods.
INTRODUCTION
Large-scale vision-language models (VLMs), such as CLIP (Radford et al., 2021), have demon- strated remarkable capability multi-modal understanding (Lüddecke Ecker, 2022; Tewel et al., 2022) and generation (Rombach et al., 2022; Tao et al., 2023; Yamazaki et al., 2023), being trained with million-scale image-text pairs. Utilizing these VLMs, recent vision models have achieved sig- nificant performance enhancements across wide range of computer vision tasks (e.g.. captioning (Mokady et al., 2021; Yamazaki et al., 2022; Li et al., 2023b) and object detection (Li et al., 2022; Zhong et al., 2022)), without the necessity for task-specific training (Shen al., 2022).
Despite the success, several works have identified societal bias regarding demographic attributes, such gender and age, in these VLMs (Wolfe & Caliskan, 2022; Hausladen et al., 2024; Alabdul- mohsin et al., 2024), potentially causing unfair or prejudicial decisions by models. Hall et al. (2023) conducted audits on performance disparity, particularly with respect gender, and revealed gender- dependency of the CLIP performance. Qiu et al. (2023) also demonstrated that adopting CLIP for caption evaluation tends favor gender-stereotypical sentences (e.g., preferring "A woman is cook- ing" over "A man is cooking" for images depicting men), highlighting the inherent gender bias. These findings underscore the importance of addressing bias in VLMs.
Some studies have proposed mitigating societal bias in VLMs (Berg et al., 2022; Seth et al., 2023; Dehdashtian al., 2024: Chuang al., 2023). Adversarial debiasing (Berg et al., 2022; Seth al., 2023; Dehdashtian et al., 2024) fine-tunes CLIP to lessen leakage of protected attributes" into the features, while projection- baseed debiasing (Chuang et al., 2023) removes the protected attribute en- coded in CLIP features the inference phase. Our holistic review of these pioneering works (Sec. 2), though, identifies the following potential drawbacks or controversies in their design choices.
Work done intern at NVIDIA.
Project page: https: / rebnej j- gi thub. i lo/saner-c1 wig.t.tu.bcoo. We refer to any demographic variables, like age and gender, protected aftribute (or aftribute in short), based which model's decisions should not he made.

---

Published as a conference paper at ICLR 2025
(a) Text-to-lmagg model using deblased CLIP
[b] Datasets for debiasing
Previous: Lose attribute information
Previous: Face-centric images GT att. annotations
Photo of female doctor
Protected attribute annotations
(e.g). gender. age)
SANER: Retain attribute information
SANER: Natural images Captions
Photo of female docfor
Image-paired caplions
Our debiasing method, SANER, overcomes the limitations in existing methods: (a) at- tribute information retained after debiasing, and (b) protected attribute annotations are not re- quired for debiasing.
Figure 1:
Loss of attribute information explicitly disclosed in the input. Some methods aim to completely remove attribute information by decorrelating the attribute and the features (Dehdashtian et al., 2024) or by squashing the subspace associated with the attribute (Chuang et al., 2023), even when the attribute is explicitly disclosed. This choice can limit the generalizability of VLM's features to spectrum downstream tasks (e.g., Stable Diffusion (Rombach et al., 2022) generates male images for text prompt "a female doctor" when encoded with debiased CLIP shown in Fig. 1 (a) and Sec. 5.2), while it works for attribute-agnostic downstream tasks (Krause et al., 2013).
Use of the attribute annotations. Adversarial debiasing methods (Berg et al., 2022; Seth et al., 2023; Dehdashtian et al., 2024) require protected attribute annotations, as provided in Fair- Face (Karkkainen & Joo, 2021), for fine-tuning (Fig. (b)). Datasets with attribute annotations are still scarce, partly because the annotation process needs ethical considerations (Andrews et al., 2023), limiting their applicability. This dataset scarcity also causes the limited diversity of images and text descriptions used for fine-tuning the VLM, potentially inducing overfilling.
This paper presents simple-yet-effective debiasing approach for CLIP, called SANER (societal attribute neutralizer), that simultaneously overcomes the aforementioned limitations. Specifically, SANER trains a debiasing layer (i.e., a multilayer perception) amend CLIP text feature vectors of attribute-neutral descriptions, given by attribute neutralization, such that they are equidistant those of attribute-specific descriptions using annotation-free debiasing loss. With this, only feature vectors for attribute-neutral descriptions are debiased, whereas the attribut--specffi ones retain the original information. Attribute-spen speeiffc descriptions for all possible attribute groups' can be easily augmented by modifying the attribute-specific words in the original descriptions, directing the training without attribute annotations.
Contribution. Thanks to our annotation-free debiasing pipeline, SANER is designed to be com- patible with any dataset of image-text pairs, such as COCO (Lin et al., 2014). This provides denser guidance for training the debiasing layer compared to the existing methods. Moreover, SANER does not require retraining the CLIP model itself, accessing its original training data, or retraining downstream tasks (e.g., text-to-image generation) when applying the debiased CLIP. Experiments on both discriminative and generative tasks (i.e., text-to-image retrieval (Geyik et al., 2019) and text-to-image generation (Rombach et al., 2022)) show that SANER mitigate gender, age, and racial biases of CLIP. Moreover, we demonstrate that SANER outperforms the existing methods (Berg et al., 2022: Chuang al., 2023), showing that SANER leads to less attributeddeennden of the downstream performance while overcoming the limitations in existing methods.
REVIEW: EXISTING DEBIASING METHODS
Several debiasing approaches for CLIP have been introduced, broadly categorized into two main types: adversarial debiasing (Berg et al., 2022; Seth et al., 2023; Dehdashtian et al., 2024) and
Antribute group is class in a protected attribute (e.g., female and male in gender).
2

---

Published as a conference paper at ICLR 2025
projection-based debiasing (Chuang al., 2023). This section conducts in-depth analysis of these existing debiasing strategies, highlighting their respective limitations.
Notation. Let D denote a dataset, each of whose sample quadruple (v, l, a, d), where v is an image, & is text description, E A is protected attribute annotation from set of all attribute groups, and d is the ground-truth annotation for a downstream task (if any). The CLIP text and image encoders, denoted by fi(t) R and fi(v) R* respectively, take and as input and generate corresponding feature vectors in common space.
2.1 ADVERSARIAL DEBIASING
Adversarial debiasing (Berg et al., 2022; Seth et al., 2023; Dehdashtian et al., 2024) aims to elim- inate protected attribute information in the CLIP features. Specifically, an adversarial classifier is employed to predict and remove protected attribute a from CLIP features.
Prompt tuning-based debiasing (Berg et al., 2022) proposes to use learnable tokens to reduce attribute leakage through the similarity between image and set of pre-defined textual concepts. Concretely, for set of pre-defined concepts (i.e., phrases) that are supposed to be attribute non- specific (e.g., smart and attractive). sequence / of learnable tokens are prepended the sentence template te with concept с E C (e.g., &e = "A photo of a smart person" for = smart) to obtain t [t, te]. where [] represents sequence concatenation. Then, to and arbitrary v E 'D fed into the CLIP encoders to compute similarity sc(v) by
se(v) = fv(o) fv(o)T((Et)) f((t).
(1)
Let s(v) E IR CI denote a vector, each of whose elements is the similarity score se(v) for a concept in C. Due the attribute non-specifeen seen--effiictty of concepts in C, s(v) should not correlate with attribute a of v. However, the CLIP text encoder can embed a into s(v) due to bias, allowing an attribute classifier predict a. We denote the probability of being given s(v) (or a prediction score of the attribute classifier) by m. ms(v))). Prompt luning-based debiasing (Berg et al., 2022) uses for adversarial loss, given by
Ladv = = - log (s(
DED
Minimizing Lady with respect to / reduces attribute leakage through s(v). A contrastive loss between image and text features is also used for regularization.
The experiments (Berg et al., 2022) showed that this method could effectively reduce attribute leak- age through f()., but the limited number of concepts* may limit downstream tasks that enjoy the debiased features because / is learned only through sparse of concepts. Additionally, attribute annotations are necessary for the adversarial loss, resulting in the exclusive use of face-centric image datasets (e.g., FairFace (Karkkainen & Joo, 2021)) D.
Additive Residual Learner (ARL) (Seth et al., 2023) is designed to remove attribute information from CLIP image features. This method assumes that a debiasing layer' can identify vector to additively amend attribute-neutral image feature vector 6(v), i.e.
8(v) = f((v) - (v).
(3)
Similarly (Berg al., 2022), an adversarial classifier is trained to predict from with adversarial loss
Cadv
Σ (o()).
The reconstruction loss between fv(v) and 6(v) regularizes training to preserve the original features.
This method shares common limitation with prompt tuning- based debiasing (Berg et al., 2022), notably requiring attribute annotations. Another limitation is that it tries to remove attribute features when attributes of people in images explicitly disclosed (i.e., when the person is depicted in an image). Consequently, debiased CLIP is ignorant of protected attributes.
*Their experiments used 10 concepts.
A fully-connected layer is used.
3

---

Published as a conference paper at ICLR 2025
Mapper (Dehdashtian et al., 2024) aims to reduce spurious correlations between attributes and task label in D. It applies mappings f. and f( to image and text features, respectively, as zv(v) = (t)) and (t) f((fi(t) for mitigating dependence using a dependence measure Dep(; ) to quantify statistical dependence between features as:
a. The adversarial loss is computed
Ludv - (Dep(zv()) a) Dep(zi(t),a).
(5)
These mapping functioon are also trained to maximize the statistical dependence between the fea- tures after the mapping and task label d, i.e.
(e)( d) (t)(
to retain the predictive power on the downstream task while reducing bias.
Similar to (Berg et al.. 2022; Seth et al., 2023). Mapper relies on attribute annotations. Moreover, it is designed only to address the spurious correlations between the attribute and task labels for specific task but not for different tasks.
2.2 PROJECTION-BASER DEBIASING
rojeection-based debiasing (Chuang et al., 2023) projects CLIP text feature vectors into the orthog- onal complement of the space spanned by a set of CLIP text feature vectors that pertain to protected attribute. Specifically, let H denote set of text descriptions with the target attribute (e.g.. "A photo of a w" U. where w E ("woman". "man") for binary gender), and U be a matrix each of whose column vectors is f((u) with u E U. The projection matrix P into the orthogonal complement for is given by
P= UC UCCTT)-'''
where 7 the identity matrix. can project CLIP text feature vector fi(t) for a text description into the orthogonal complement by P ft().. This process removes attribute information by projecting features into the space orthogonal to attribute-specific directions.
Unlike adversarial debiasing, which requires training by gradient descent update, P has closed- form solution and is computed in the inference phase. However, as with ARL, this method also eliminates attribute information even from descriptions with explicit attributes (e.g.. "A photo of female doctor")..
2.3 SUMMARY OF THE CHALLENGES
The existing debiasing methods, including adversarial and projection- based, reveal several chal- lenges: 1) Loss of attribute information (ARL and projection- even with explicit attribute description narrows down the utility of the debiased CL.P. For example, text-to-image generative model with gender-debiased CL.IP features may not properly depict explicitly specified gender (as shown in Sec. 5.2), 2) Dependency on attribute annotations (prompt tuning, ARL, and Mapper) constrains the range of datasets that can be utilized, often necessitating the use of face-centric image datasets (e.g., FairFace), as opposed to more diverse, natural image datasets (e.g., COCO).
SOCIETAL ATTRIBUTE NEUTRALIZER (SANER)
Our method for debiasing CLIP features, SANER, addresses the limitations of the existing methods identified in Section 2. Notably, SANER 1) retains attribute information in cases where the person's attributes are explicitly described and 2) eliminates the reliance on attribute annotations, allowing the use of any image-text dataset for training the debiasing layer.
SANER comprises 1) attribute neutralization, which eliminates protected attribute information from input text (Section 3.1); 2) feature modification, which removes attribute information from the CLIP text features by amending them with debiasing layer (Section 3.2); attribute annotation- free debiasing loss, ensuring the features are not biased towards any attribute group E A (Section 3.3); and 4) regularization losses, which preserve the original CLIP features and the alignment between image and text features (Section 3.4).
Figure shows overview of SANER. We train the debiasing layer for feature modification an arbitrary dataset D = {(v,t) of image v and text description (e.g., image caption, alt text) pairs, which does not provide attribute annotation a as well target task label d.
4

---

Published as a conference paper at ICLR 2025
2(5,0))
Feature modification
M(..0))
Noutralization
Ldeb
Attribute
Debiesing layer
A woman is eating salad
A person eating salad
ft
encodor
(.
Lrecon
6n(
Leont
Annotation-free debiasing loss
Regularization losses
M(-(0)
h(()
Lee
Lracon
Lcont
((0(0)
MV
Text encoder
Image encoder
Text encoder
woman eating salad
woman eating salad
Aman eating salad
An overview of SANER, exemplified by binary gender. SANER neutralizes attribute- specific text (e.g., "woman" + "person"), modifies features via debiasing layer, and uses three losses for debiasing: Laeb for attribute neutralization, Laccon for feature preservation, and Leont for image-text alignment.
Figure 2:
3.1 ATTRIBUTE NEUTRALIZATION
We first modify text description E 'D that contain person-related words, to remove attribute- specific words. Taking binary gender as protected attribute," i.e. A (female, male), as example, the text description
I = "A woman is eating salad."
contains attribute information (i.e,, woman). We replace attribute-specific terms with the attribute- neutral ones to obtain attribute-neutral text:
(t "A person is eating salad."
where S. denotes a function attribute neutralization. Neutralization can be done for other at- tributes, such as age." We remove age-specific terms (e.g. young and senior) in text descriptions, for instance, "A young woman is cating salad" > "A woman is cating salad". In contrast to the previous approach (Chuang et al., 2023), which optimized not to predict the attribute informa- tion from the original description f, we target the atrribute-neutral descriptions (n()) to preserve the attribute information in the features of attribute-specife descriptions.
3.2 FEATURE MODIFICATION
CLIP text features (()) fi( after attribute neutralization can still convey the protected attribute information due CLIP's bias. To remove such bias, we append learnable debiasing layer r on top of f. inspired by recent CLIP fine-tuning techniques (Seth et al., 2023; Gao et al.,
Person-related words encompass terms that reference individuals (e.g., person, girl, man). The complete list is in the appendix.
FFollowing prior research (Berg et al., 2022; Seth al., 2023; Dehdashtian al., 2024; Chuang et al., 2023; Zhao al., 2017: Garcia et al., 2023; Burns al., 2018), we focus on the binary gender but recognize the importance of inclusivity. SANER applies to non-binary genders.
We use gender words defined (Hirota al., 2023).
"Examples for the race attribute are in the appendix.
10 We define age-specific terms. The list is in the appendix.

---

Published as a conference paper at ICLR 2025
2024). Neutralized i's debiased feature h(En(t)) is given by
(t),(t)) = =(C,(t)) r()((,())))
3.3 ATTRIBUTE ANNOTATION FREE DEBIASING LOSS
To train r to extract attribute information from CLIP features without attribute annotations, we create set T of autribute-specife descriptions for E and for g E A, i.e., T = (tg(t)tt E D, g e A}. where (t) generates description for each attribute group g from t. For binary gender, this involves generating descriptions with female- and male-specific words. For instance, from the text description, "A woman is eating salad.", generate two sentences with female and male attributes:
A woman eating salad. A man is eating salad.
The debiasing loss trains such that h())(t)) is equidistant from (t) for all attribute groups in A. ensuring an impartial representation across the spectrum of attribute groups. We implement this loss as the standard deviation of the cosine similarity between (t))())) and fi(E, (t)). Let sg(() denote the similarity, ie.,
((E,(t)) f((E((t)
$y (t)
(9)
[[A(E(())II (t)(
The debiasing loss Ldeb is defined as
Lateb
Σ - - LED
(10)
D
where s(t) = E, A lower standard deviation means sq is close to 5, leading to (())(t)) being equidistant to (t) for all E A. Notably, this debiasing loss can be computed without attribute annotations.
3.4 REGULARIZZTTION LOSSES
Applying the debiasing loss alone significantly changes original CLIP features, thereby losing mantics (Berg et al., 2022: Seth et al., 2023). To maintain the alignment of resulting image-text features, utilize reconstruction loss (Seth et al., 2023) and contrastive loss (Radford al., 2021). Reconstruction loss Lrecom is the mean squared error between fi(t) and h(t). Contrastive loss Lwom aims minimize the negative log-likelihood of input image-caption pairs, f-(v) and fi((), in com- parison to negative ones. Note that the original description is used for regularization losses.
3.5
TRAINING AND INFERENCE
The overall loss is given by:
L Caaae + 3Lrsso + 7oont,
(11)
where a, B, and y the hyperparameters weight respective losses.
During inference, we apply the trained debiasing layer r and use the modified text features r(f((t) the CLIP text features.
4 EXPERIMENTS: TEXT-TO-IMAGE RETRIEVAL
Following previous studies (Berg et al., 2022; Seth al., 2023; Dehdashtian et al., 2024; Chuang et al., 2023), we evaluate SANER on the toxt-to-image retrieval task regarding gender, age, and racial biases. Further analysis, such the ablation study of the loss components, is in the appendix.

---

Published as a conference paper at ICLR 2025
Table 1: Gender bias, evaluated by MaxSkew @ 1000 (scaled by 100), on FairFace and PATA for the original CLIP (Original), prompt tuning-bawc based debiasing (Prompt), projection- based debiasing (Projection), and our method (SANER). A lower value is better (less gender bias). Bold represents the best across the models.
FairFace
PATA
CLIP Model
Adjective
Occupation
Activity
Adjective
Occupation
Activity
Original (Radford al., 2021)
22.9
33.7
19.5
12.1
18.7
10.7
Prompt (Berg 2022)
12.3
29.9
20.0
16.5
10.2
Projection (Chuang et al., 2023)
37.4
SANER (Ours)
8.9
14.5
7.7
5.4
9.5
3.3
4.1 EXPERIMENTAL SETTINGS
Evaluation metric. We employ the MaxSkew metric (Geyik et al., 2019), utilized in the previous studies (Berg et al., 2022; Seth et al., 2023; Dehdashtian et al., 2024; Chuang et al., 2023), to quan- tify the societal bias in CLIP in the text-to-image retrieval task. MaxSkew measures the disparity between the attribute distribution of |AA| in the top-k retrieved images. Let (ax(g) denote the ratio of images labeled with attribute a in the top-k retrieved images. For attribute neutral query q. Tok (g) should be 1/||A| if the model is unbiased. MaxSkew @ is defined as:
7ak (g)
MaxSkew @k max log
(12)
.EAA 1/1_A|
Ideally, MaxSkew @ is 0 but larger when a model biased.
Evaluation setting. For the attribute-neutral queries, we use template-based queries such as "a photo of person", where is attiibute-neurral concept. Prior work (Berg et al., 2022) has defined a list of person-related adjectives, such as clever and attractive, as attribute-neutral concepts. We extend the list to encompass occupations (e.g., doctor, nurse), and activities (e.g., cooking and cleaning) for a more comprehensive evaluation. MaxSkew @k is computed per concept."
We also evaluate zero-shot image classification accuracy on ImageNet-IK (Russakovsky et al., 2015) to ensure that debiasing does not spoil the original CLIP's performance.
Evaluation datasets. We utilize two datasets, FairFace (Karkkainen & Joo, 2021) and PATA (Seth et al., 2023), which consist of images alongside protected attribute annotations female and male for gender) associated with the person in each image. FairFace consists of 10, 954 cropped face-centric images, while PATA contains 4, 934 natural images with a single person. Most debiasing approaches only report the performance on the FairFace dataset, but we additionally employ PATA to evaluate the debias performance more diverse images.
Methods for comparison. We compare SANER against existing methods, i.e., prompt tuning- based debiasing (Berg et al., 2022) and projection mrojection-based debiasing (Chuang et al., 2023), whose code is publicly available. Unfortunately, we could not reproduce the other methods(Seth et al., 2023; Dehdashtian al., 2024) since sufficient reproduction details are unavailable.
Implementation details. Following the previous works (Berg et al., 2022; Chuang et al., 2023), we employ CLIP (Radford et al., 2021) with ViT-B/16 backbone (Dosovitskiy et al., 2021) model in our experiments. We train the debiasing layer (a multilayer perceptron with two linear layers with ReLU activation (Nair & Hinton, 2010)) appended the model using 170, 624 image- caption pairs, which is a subset of the COCO training set (Lin et al., 2014) with person-related words/phrases (e.g., person and boy). We empirically set a, B. and y to 1.0, 0.1, and 0.0001 (Eq. 11), respectively, and train for 5 epochs. Further details are provided in the appendix.
target
4.2 GENDER BIAS ANALYSIS
Table presents MaxSkew @ 1000 on FairFace and PATA for gender bias, showing SANER mitigates bias the most among all methods. In contrast to the existing methods, this tendeney is consistent across 1) datasets with different image domains (i.e., face-centric and natural images) and 2) concept types (Le., adjective, occupation, and activity). For instance, while the prompt tuning- based method
The complete lists of the concepts in the appendix.

---

Published as a conference paper at ICLR 2025
Table 2: Age bias (left) and racial bias (right), evaluated by MaxSkew @ 1000 (scaled by 100), on FairFace. Bold denotes the best across the models. Results on PATA are in the appendix.
FairFace
FairFace
CLIP Model
CLIP Model
Adjective
Occupation
Activity
Adjective
Occupation
Activity
Original
111.1
121.1
113.0
Original
62.2
57.4
08.3
Projection
107.6
112.8
100.0
Projection
56.9
75.3
67.0
SANER (Ours)
96.0
112.6
101.9
SANER (Ours)
49.3
45.7
46.6
(Berg al., 2022) fails to mitigate bias for activity concepts FairFace (i.e., bias is amplified from 19.5 to 20.0), SANER significantly mitigates bias from 19.5 to 7.7. This verifies the better debiasing performance of SANER compared to the existing methods on diverse concepts and image domains, possibly because SANER is trained with diverse text descriptions (i.e., captions in COCO), which are not constrained like pre-defined concepts required in the previous method (Berg et al., 2022).
4.3 AGE AND RACIAL BIASES ANALYSIS
Tables 2 presents the results of MaxSkew @ 1000 for age and racial biases. We compare SANER with projection-basee based debiasing (Chuang et al., 2023), as the prompt tuning-basee method (Berg al., 2022) does not provide age and race debiasing variants. Similarly to the results for gender bias, SANER surpasses the existing method across the datasets and the concept types. For example, SANER successfully mitigates the racial bias on FairFace. Meanwhile, the projection based method amplifies the bias on the occupation concept (i.e., from 57.4 to 75.3). These results validate the generalizability of SANER bias mitigation across the protected attributes.
4.4 ZERO-SHOT IMAGE CLASSIFICATION
Table 3: Accuracy on Image- Net-1K.
To verify whether debiasing harms the zero-shot image clas- sification performance of the original CLIP, we evaluate the prompt tuning-based method (Berg al., 2022) and SANER on ImageNet-IK in terms of classification accuracy. Projection- based debiasing (Chuang et al., 2023) does not apply to this eval- uation because zero-shot prompts, such as "a photo of car," do not necessarily include person-related words. 12 The results are shown in Tab. 3, showing that applying SANER maintains classification performance, whereas the performance of the prompt tuning-based method slightly degrades.
CLIP Model
Acc. (8)
Original
65.4
Prompt
64.1
SANER (Ours)
65.2
Our method, SANER, which does not require attribute annotations, outperforms previous
Insight:
those that do require annotations-in mitigating gender, age, and racial biases without compromising classification performance.
5 EXPERIMENTS: TEXT-TO-IMAGE GENERATION
We also evaluate SANER on text-to-image generation, for which societal bias is actively investigated (Bansal et al., 2022; Cho et al., 2023; Liu al., 2024b). Specifically, we conduct two experiments from different aspects: 1) gender bias regarding occupations using gender-neutral prompts (Sec. 5.1), and 2) retention of attribute information when prompts explicitly disclose gender (Sec. 5.2).
Image generation settings. We use Stable Diffusion (SD) v2.1 (Rombach al., 2022) as the text-to-image generation model. The CLIP text encoder in SD is replaced with the debiased one for evaluation. Following (Chuang et al., 2023), we gender gender-neutral prompts with specifying occu- pations to analyze gender bias in generated images. These prompts are derived from the template "A photo of o", where is replaced with specific occupation terms, such doctor and teacher." 13 On the other hand, we employ gender-specific prompts to evaluate the capability of attribute informa- tion retention. Concretely, a gender term, either female or male, added just before the occupation
12 Projecttion-assee -hased debiasing requires modified input text to include attribute terms. The list of the occupations is in the appendix.
8

---

Published as a conference paper at ICLR 2025
s0
A photo of designer
Projection
SANER
Figure 3: Generated images for the prompt, "A photo of a designer," by the original Stable Diffusion (SD), projection- basee debiased CLIP (Projection), and our debiased CLIP (SANER). We randomly sample 10 images from generated images. Images framed in green denote those of the minority gender in the generated images (i.e., female).
terms, i.e., "A photo of (female/male) o", to see if generated images specify the gender. We generate 100 images for each prompt with SD's s default hyperparameters.
Evaluation metries. For the bias evaluation for the generative task, we use statistical parity (SP) metric that measures the disparity of attribute groups in generated images (Teo et al., 2023; Chuang et al., 2023). Specifically, we annotate binary gender labels (i.e., = {fema le, ma ma male)) for the generated images with the assistance of human workers. 14 SP is defined as the difference between the empirical distribution Ki of gender a and uniform distribution, given by:
SP- =
(na 1/1 REA
(13)
where Ka ... No with N. being the number of images annotated as For an unbiased text-to-image generation model, SP should be 0 but increases for biased models.
For gender-specific prompts for evaluating gender information retention, we compute the accuracy, i.e, the ratio of the images that contain the same gender as the prompt to all generated images.
5.1 GENDER BIAS ANALYSIS
Table summarizes SP scores, which demon- strate that applying SANER to Stable Diffusion no- tably mitigates gender bias regarding occupations. SANER again outperforms projection-based debi- asing one (i.e.. 0.39 for SANER and 0.47 for projection-based), highlighting the superiority of SANER.
Table 4: Gender bias (SP) and gender infor- mation retention (Accuracy) in images from Stable Diffusion (Original) and the model us- ing projection- baaed debiased CLIP (Projec- tion) and our debiased CLIP (SANER), Re- sults are the mean across occupations. Fe- male and male refer prompts specifying each gender. Bold indicates the best.
We show visual examples where SANER miti- gates gender bias Fig. 3. For the gender- neutral prompt, "A photo of designer," the original SD and projection-basee dominantly generate images of a man. In contrast, ours shows a more balanced gender distribution.
Accuracy 1
Model
SP4
Fomale
Male
debiasing (Projection) pre-
Original
0.51
1.00
1.00
Projection
0.17
0.58
0.79
SANER (Ours)
1.00
1.00
5.2 ASSESSMENT OF RETENTION OF ATTRIBUTE INFORMATION
Table 4 also shows the accuracy of how much the gender of the person in generated images matches the gender specified in the prompts. For both prompts that describe women and men, using debiased CLIP by projection-bast based debiasing leads to losing gender information (i.e., accuracies for projec- tion are much lower than those for the original). Conversely, SANER retains gender information (i.e., accuracies for SANER are 1.00). Figure 4 confirms this, showing that using projection-basee debiased CLIP results in generating male images for the prompt, "A photo of a female doctor".
MDifferent from previous works that pre-trained gender classifiers to assign gender labels, we do not use them due to their bias issues (Ramaswamy al., 2021; Das al., 2018; Dinan al., 2020).

---

Published as a conference paper at ICLR 2025
A phots of female docfor
Projection
SANER
Figure 4: Generated images for the prompt, "A photo of a female doctor," " by the original Stable Diffusion (SD), projection-based debiased CLIP (Projection), and our debiased CLIP (SANER).
Red frame indicates images with incorrect gender (i.e., male).
Insight:
Applying SANER-debiased CLIP to Stable Diffusion generates images with more gender-equal distribution while preserving gender information in input texts.
LIMITATIONS
Further bias mitigation. Our experiments show that SANER noticeably reduces bias CLIP. Nonetheless, the bias is not completely eliminated (e.g., MaxSkew is not zero). A promising direc- tion for further debiasing could involve debiasing the image encoder, specifically training a debiasing layer to remove attribute information from the visual features for images without human subjects.
Intersectional bias analysis. While our experiments focus on gender, age, and racial biases individ- ually, following prior works (Berg et al., 2022; Seth et al., 2023; Dehdashtian et al., 2024; Chuang et al., 2023), SANER can be easily extended to various protected attributes and their combinations. For instance, considering the intersection of binary gender and age, we generate four sentences with (female, young), (female, old), (male, young), and (male, old) for the debiasing loss, e.g., "A young woman is eating salad" for the input text "A woman is eating salad". This potential for addressing complex biases is noted in future research.
Use of pre-defined attribute words. While SANER requires general lists of attributes, creating attribute lists is one-time effort requiring minimal resources, compared to the ongoing cost and complexity of dataset annotation that often needs ethical review and domain expertise.
RELATED WORK
As VLMs like CLIP are applied more tasks, concerns about their social biases have grown (De- houche, 2021; Wang et al., 2021; Ross et al., 2021; Wolfe et al., 2023; Ruggeri et al., 2023; Srini- vasan & Bisk, 2022: Hirota et al., 2024a;b). Hall al. (Hall et al., 2023) examined gender bias in CLIP, uncovering significant discrepancies in object recognition performance based on the gender depicted in images. such as higher accuracy in recognizing an unbrella with women than with men. These biases risk reinforcing discrimination against marginalized groups (Qiu et al., 2023; Tanjim et al., 2024; Wang et al., 2023). Birhane et al. (Birhane et al., 2021) demonstrated that adopting CLIP-based filtering in the dataset creation can select stereotypical images, like labeling a female astronaut as a "smiling housewife," leading to harmful, biased datasets.
CONCLUSION
This paper proposed SANER, a simple-yet-effective debiasing method for CLIP, consisting of at- tribute neutralization and anotation-free debiasing loss. Consequently, SANER can leverage any image-text dataset to train the debiasing layer, outperforming existing methods in both discrim- inative and generative tasks (f.e., text-to-image retrieval and text-to-image generation). We also confirmed that SANER retains attribute information for attribute-specific descriptions through the gender-specified prompts for text-to-image generation.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Ibrahim Alabdulmohsin, Xiao Wang, Andreas Peter Steiner, Priya Goyal, Alexander D Amour, and Xiaohua Zhai. Clip the bias: How useful is balancing data in multimodal learning? In ICLR, 2024.
Jerone Andrews, Dora Zhao, William Thong, Apostolos Modas, Orestis Pupakyriakopoulos, and Alice Xiang. Ethical considerations for responsible data curation. In NeurIPS Datasets and Benchmarks Track, 2023.
Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. How well can text-to-image generative models understand ethical natural language interventions? In EMNLP, 2022.
Hugo Berg. Siobhan Mackenzie Hall, Yash Bhalgat, Wonsuk Yang, Hannah Rose Kirk, Aleksandar Shtedritski, and Max Bain. A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning. In AACL, 2022.
Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: Misogyny, pomoggaahy, and malignant stereotypes. arXiv preprint arXiv:2110001963, 2021.
Kaylee Burns, Lisa Anne Hendricks, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. In ECCV, 2018.
Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In ICCV, 2023.
Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debias- ing vision-language models via biased prompts. arXiv preprint vrXiv:2302.00077 2023.
Abhijit Das, Antitza Dantcheva, and Francois Bremond. Mitigating bias in gender, age and ethnicity classification: a multi-task convolution neural network approach. In ECCV Workshops, 2018.
Sepehr Dehdashtian, Lan Wang, and Vishnu Boddeti. Fai Mitigating bias in pre-trained vision-language models. In ICLR, 2024. URL pttt:::://pperevieee ew. net / fo Forum? d HXog9EqR9e.
Nassim Dehouche. Implicit stereotypes pre-trained classifiers. IEEE Access, 2021.
Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. Multi- dimensional gender bias classification. In EMNLP, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021.
Felix Friedrich, Patrick Schramowski, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302.10893, 2023.
Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang. Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 2024.
Noa Garcia, Yusuke Hirota, Yankun Wu, and Yuta Nakashima. Uncurated image-text datasets: Shedding light on demographic bias. In CVPR, 2023.
Sahin Cem Geyik, Stuart Ambler, and Krishnaram Kenthapadi. Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In SIGKDD, 2019.
Laura Gustafson, Chloe Rolland, Nikhila Ravi, Quentin Duval, Aaron Adcock, Cheng- Hang Fu, Melissa Hall, and Candace Ross. Facet: Fairness in computer vision evaluation benchmark. In ICCV, 2023.
Melissa Hall, Laura Gustafson, Aaron Adcock, Ishan Misra, and Candace Ross. Vision laagun-aanguage models performing zero-shot tasks exhibit gender-based disparities. In ICCV Workshops, 2023.

---

Published as a conference paper at ICLR 2025
Carina Hausladen, Manuel Knott, Pietro Perona, and Colin Camerer. Causal analysis of social bias in CLIP, 2024. URL
net/
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. EMNLP, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. NeurIPS, 2017.
Yusuke Hirota, Yuta Nakashima, and Noa Garcia. Model-agnostic gender debiased image caption- ing. In CVPR, 2023.
Yusuke Hirota, Jerone TA Andrews, Dora Zhao, Orestis Papakyriakopollos, Apostolos Modas, Yuta Nakashima, and Alice Xiang. Resampled datasets are not enough: Mitigating societal bias beyond single attributes. In EMNLP, 2024a.
Yusuke Hirota, Ryo Hachiuma, Chao-Han Huck Yang, and Yuta Nakashima. From descriptive richness bias: Unveiling the dark side of generative image caption enrichment. In EMNLP, 2024b.
Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In WACV, 2021.
Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijaya- narasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint 17Xx::7705.06950, 2017.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCV Workshops, 2013.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. arXiv preprint 1rX1v:2301.12597, 2023a.
Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang. Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training. In CVPR, 2022.
Wei Li, Linchao Zhu, Longyin Wen, and Yi Yang. Decap: Decoding clip latents for zero-shot captioning via text-only training. ICLR, 2023b.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie. James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurlPS, 2024a.
Zhixuan Liu, Peter Schaldenbrand, Beverley-Claire Okogwu, Wenxuan Peng, Youngsik Yun, An- drew Hundt, Jihie Kim, and Jean Oh. Scoft: Self-contrastive fine-tuning for equitable image generation. CVPR, 2024b.
Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In CVPR, 2022.
Ron Mokady, Amir Hertz, and Amit H Bermano. Clipcap: Clip prefix for image captioning. arXiv preprint arXiv:2 1.09734, 2021.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
Haoyi Qiu, Zi- Yi Dou, Tianlu Wang, Asli Celikyilmaz, and Nanyun Peng. Gender biases in auto- matic evaluation metrics for image captioning. In EMNLP, 2023.
12

---

Published as a conference paper at ICLR 2025
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Vikram V. Ramaswamy, Sunnie S. Y. Kim, and Olga Russakovsky. Fair attribute classification through latent space de-biasing. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Candace Ross, Boris Katz, and Andrei Barbu. Measuring social biases in grounded vision and language embeddings. In ACL, 2021.
Gabriele Ruggeri, Debora Nozza, et al. A multi-dimennional study on bias in vision-language mod- els. In Findings of ACL, 2023.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, 2015.
Ashish Seth, Mayur Hemani, and Chirag Agarwal. Dear: Debiasing vision-language models with additive residuals. CVPR, 2023.
Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal, Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, and Kurt Keutzer. How much can clip benefit vision-and-languag tasks? In ICLR, 2022.
Tejas Srinivasan and Yonatan Bisk. Worst of both worlds: Biases compound in pre-trained vision- lan-uang models. In ACL Workshops, 2022.
Md Mehrab Tanji,, Krishna Kumar Singh, Kushal Kafle, Ritwik Sinha, and Garrison W Cottrell. Discovering and mitigating biases clip-based image editing. WACV, 2024.
Ming Tao, Bing-Kun Bao, Hao Tang, and Changsheng Xu. Galip: Generative adversarial clips for text-to-image synthesis. In CVPR, 2023.
Christopher Teo, Milad Abdollahzadeh, and Ngai-Man Man Cheung. On measuring fairness in generative models. In NeurlPS, 2023.
Yoad Tewel, Yoav Shalev, Idan Schwartz, and Lior Wolf. Zerocap: Zero-shot image-to-text genera- tion for visual-semantic arithmetic. CVPR, 2022.
Jialu Wang, Yang Liu, and Xin Eric Wang. Are gender-neutral queries really gender-neutral. miti- gating gender bias in image search. arXiv preprint arXiv:2109.05433; 2021.
Xinpeng Wang, Xiaoyuan Yi, Han Jiang, Shanlin Zhou, Zhihua Wei, and Xing Xie. Tovilag: Your visual-language generative model is also an evildoer. In EMNLP, 2023.
Robert Wolfe and Aylin Caliskan. Markedness in visual semantic ai. In FAccT, 2022.
Robert Wolfe, Yiwei Yang, Bill Howe, and Aylin Caliskan. Contrastive language-vistin viiiin ai models pretrained on web-scraped multimodal data exhibit sexual objectification bias. In FAceT, 2023.
Kashu Yamazaki, Sang Truong, Khoa Vo, Michael Kidd, Chase Rainwater, Khoa Luu, and Ngan Le. Vlcap: Vision-language with contrastive learning for coherent video paragraph captioning. In ICIP, 2022.
Kashu Yamazaki, Khoa Vo, Quang Sang Truong, Bhiksha Raj. and Ngan Le. Vltint: visual- lingu-iin transformer-in--ranffoom for coherent video paragraph captioning. In AAAI, 2023.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In EMNLP, 2017.
Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Lu- owei Zhou, Xiyang Dai, Lu Yuan, Yin Li, and Jianfeng Gao. Regionclip: Region-based language- image pretraining. In CVPR, 2022.
13

---

Published as a conference paper at ICLR 2025
APPENDIX
This appendix includes:
Implementation details for SANER (Appendix A).
Further analysis (Appendix B).
List of person-, gender-, age-, and race-specific terms (Appendix C).
List of concepts and occupations (Appendix D).
Additional visual examples for image generation experiments (Appendix E).
Potential extension (Appendix F).
Potential negative impact (Appendix G).
A IMPLEMENTATION DETAILS FOR SANER
We train the debiasing layer (a multilayer perception with two linear layers with ReLU activation) using 170, 624 image-caption pairs, which is subset of the COCO training set with person-related words/phrases defined in Sec. C. The hidden embedding dimensionality for the debiasing layer is 128. We empirically set a, 8, and y 1.0, 0.1, and 0.0001 (Eq. (11) the main paper), respectively. We set the training epochs, batch size, and learning rate to 5, 128, and 5 X 10 respectively. The training is conducted with machine equipped with single NVIDIA A100 GPU 40GB, and it took five hours to train the debiasing layer. Note that the weights of the debiasing layer are updated, and the weights for the rest of the modules are frozen.
Attribute-specific description generation. To avoid computational complexity, we implement systematic approach using predefined mapping dictionaries between gender terms (e.g.. "woman" + "man", "she" + "he"). Our algorithm carefully identifies and replaces gender-specific tokens while preserving sentence structure, ensuring no ungrammatical combinations (like "A he/she") generated. This ensures efficient and coherent text generation that maintains natural language patterns.
Racial bias mitigation. For race attribute, remove race-specific terms (e.g., African and Asian) in text descriptions, for instance, "An African woman eating salad" > "A woman is eating salad".
B ADDITIONAL EXPERIMENTS
B. COMPLETE RESULTS FOR AGE AND RACIAL BIASES
In Table 5 and 6, we show the complete results of Table 2, including results on PATA. The results further verify that SANER demonstrates superior performance mitigating age and racial biases compared to the existing method.
B.2 LOSS ABLATION
To validate the effectiveness of each regularization loss (i.e., reconstruction loss and contrastive loss), we conduct ablation study by removing one of the losses both losses. Table 7 presents the results of gender bias. The results show that using both reconstruction and contrastive losses yields the best results regarding gender bias mitigation and zero-shot classification accuracy on ImangeNet (Russakovsky et al., 2015). Furthermore, SANER without regularization losses signifi- cantly degrades CLIP's zero-shot classification ability (i.e., from 65.4 to 58.8). These observations confirm the importance of having both reconstruction and contrastive losses for the regularization.
Negative impact on adding only the contrastive loss. When using only the contrastive loss. MaxSkew scores FairFace increase compared to no regularization (18.0/19.4/21.77 vs. 15.7/15.0/15.3 without regularization), indicating less effective debiasing. This occurs because the
We define race-specific terms. The list is in Section C.
14

---

Published as a conference paper at ICLR 2025
Table 5: Age bias, evaluated by MaxSkew @ 1000 (scaled by 100), on FairFace and PATA. Bold denotes the best across the models.
FairFace Occupation
PATA
CLIP Model
Adjective
Activity
Adjective
Occupation
Activity
Original(Radfort al., 2021)
111.1
121.1
113.0
40.4
44.4
39.7
Projection (Chunng et al., 2023)
107.6
112.8
100.0
37.6
45.6
45.5
SANER (Ours)
96.0
112.6
101.9
30.3
36.7
27.5
Table 6: Racial bias, evaluated by MaxSkew @ 1000 (scaled by 100), on FairFace and PATA. Bold denotes the best across the models.
FairFace Occupation
PATA Occupation
CLIP Model
Adjective
Activity
Adjective
Activity
Original(Radlorl al., 2021)
62.2
57.4
68.3
33.4
28.6
31.5
Projection (Chuang al., 2023)
56.9
75.3
67.0
19.9
43.8
26.3
SANER (Ours)
49.3
45.7
48.6
28.9
21.2
20.5
contrastive loss alone focuses on maintaining image-text alignment but does not constrain the de- biased features to remain close to the original CLIP features. As result, the feature modifications may become suboptimal for bias removal. The reconstruction loss plays crucial role in ensuring the modified features preserve essential semantic information while effectively removing unwanted bias.
B.3 ANALYSIS ON THE DATA SIZE
The experiments in the main paper (Sections and 6) verify that SANER outperforms existing debiasing methods, showing better bias mitigation ability in terms of gender and age biases. This superior performance SANER may be because SANER trained with diverse text descriptions (i.e., captions in COCO (Lin et al., 2014)), which are not constrained like pre-defined concepts required the previous method (Berg et al., 2022). In this section, we conduct an experiment to verify this hypothesis. Specifically, caption pairs of COCO) to evaluate the impact of the training dataset size. We use the same settings in Sec. A, but use the different training epochs to align the number of iterations. The results shown in Table 8.
percent of the training samples (i.e., 17, 624 image-
The results validate that as the number of data samples increases, gender bias is reduced. Specifi- cally, while using a part of the training data results in mitigating gender bias (i.e., MaxSkew scores smaller than the original CLIP), using the full training samples (i.e., COCO-100%) gives the best results, showing the importance of the use of more diverse data for debiasing.
B.4 QUALITY OF THE GENERATED IMAGES
We evaluate image fidelity and image-text alignment on the COCO Karpathy test set. Specifically, use FID score (Heusel et al., 2017) and CLIPScore (Hessel al., 2021) to measure fidelity and image-text alignment, respectively. The results show that SANER achieves comparable performance the original Stable Diffusion (FID: 28.1, CLIPScore: 31.2 vs. FID: 28.7. CLIPScore: 31.0 for SANER), confirming that SANER achieves debiasing while maintaining image generation quality.
B.5 EXPERIMENTS ON BLIP
To verify the effectiveness of SANER for VLMs beyond CLIP, we conduct gender bias experiments using BLIP (Li al., 2023a). As shown in Table 9, SANER demonstrates superior debiasing per- formance compared to the original BLIP and the projection- based debiasing.
B.6 EXPERIMENTS ON FACET
In addition to FairFace and PATA, we evaluate SANER on the FACET dataset (Gustafson et al., 2023). Similar PATA, FACET comprises real-world, natural images but includes a broader range
15

---

Published as a conference paper at ICLR 2025
Table 7: Gender bias, evaluated by MaxSkew@l @ 1000 (scaled by 100), on FairFace and PATA for our method (SANER) with different regularization loss combinations. Recon denotes the use of the reconstruction loss, and cont represents the use of the contrastive loss. IN acc is the zero- shot classification accuracy Adjective, Occupations, and Activity, respectively). A lower value better (less gender bias). Bold represents the best across the SANER variants.
ImageNet. Adj, Occ, and Act represent the types of concepts (i.e.
FairFace
PATA
Adj
Oce
Act
Adj
Oce
IN acc (%)
Original (Radlord 2021)
22.9
10.7
SANER
(no regularization)
15.7
15.0
15.3
15.8
14.4
15.4
58.8
Recon
16.7
6.2
12.0
6.2
18.0
7.0
Recon=Coom
8.9
14.5
7.7
5.4
9.5
65.2
Table Gender bias, evaluated by MaxSkew @ 1000 (scaled by 100), on FairFace and PATA for the original CLIP (Original) and our method (SANER) with different data sizes. COCO-n& denotes that we use n% of the training samples. acc is the zero-shot classification accuracy on ImageNet. A lower value is better (less gender bias). Bold represents the best across the SANER variants.
FairFace
PATA
Adj
Oce
Adj
Oce
IN acc (%)
Original (Radford 2021)
22.9
SANER
COCO-50%
15.7
19.4
17.9
10.0
14.1
10.2
65.4 4
C0CO-75%
12.8
17.6
16.1
7.8
13.0
8.3
65.6 4
COC0-100%
7.7
of annotations for protected attributes, such as hair color. To evaluate SANER's effectiveness on FACET, which differs in data distribution from FairFace and PATA, we conducted experiments fo- cusing on gender bias. The results, presented in Table 10, demonstrate SANER's s superior debiasing performance, further supporting its robustness across diverse dataset distributions. Experiments on other attributes, such hair color, are left for future work.
C LIST OF PERSON-, GENDER-, AGE-, RACE-SPECIRES TERMS
The person-related words that are used identify text descriptions that are relevant to humans (in Sec. 4.1 in the main paper) are as below:
actor, actress, adult, architect, artist, associate, aunt, baby, boy, boyfriend, brother, chairman, chairperson, chairwoman, chef, child, coach, colleague, comedian, counselor, cowboy, cowgirl, dancer, daughter, designer, director, doctor, driver, dude, elder, emperor, employee, employer, engineer, entrepreneur, executive, expecting, fneecting.father,, femaer, hairdresser, he, her, hers, herself, him, himself, his, husband, individual, infant, instructor, kid, lady, lawyer, leader, lecturer, male, man, manager, mechanic, member, mentor, mother, musician, neigh- bor, novelist, nurse, parent, partner, people, performer, person. pharmacist, photographer, physi- cian, pilot, player, police officer, policeman, policewoman, politician, pregnant, prince, princess, professor, queen, relative, researcher, royal, scholar, scientist, secretary, server, she, sibling, singer, sister, son, specialist, spouse, sludent, surfer, surgeon, tailor, teacher, technician, teenager, their, theirs, them, themselves, therapist, they, toddler, uncle, veterinarian, volunteer, waiter, waitress, wife, woman, worker, writer, youth, and their plurals.
gentleman, girl, girlfriend, guy,
We list the gender-specific terms that are used to create attribute-neutral text descriptions (t.(t) (in Sec. 4.1 in the main paper); woman, female, lady, mother, girl, aunt, wife, actress, princess, waitress, sister, queen, pregnant, daughter, she, her, hers, herself, man, male, father, gentleman, boy, uncle, husband, actor, prince, waiter, son, brother, guy, emperor, dude, cowboy, he, his, him, himself and their plurals (orange denotes femalc-specific words, and olive represents spec-sce terms). To synthesize attribute-specifie descriptions (iee, T {t)())] D.g E A} in Sec.
16

---

Published as a conference paper at ICLR 2025
Table 9: Gender bias for BLIP, evaluated by MaxSkew 1000.
FairFace Occupation
PATA
BLIP Model
Adjective
Activity
Adjectivv
Occupation
Activity
Original
16.8
15.3
12.8
7.7
7.4
5.5
Projection
14.9
19.8
17.4
12.0
14.7
17.4
SANER (Ours)
15.3
13.9
10.9
6.7
6.7
4.9
Table 10: Gender bias on FACET, evaluated by MaxSkew 1000.
FACET
CLIP Model
Adjective
Occupation
Activity
Original
46.0
37.7
33.4
Projection
37 3
46.3
41.1
SANER (Ours)
31.7
24.7
25.2
4.3 in the main paper) for binary gender, we replace person-specific terms in the attribute-neutral descriptions with their corresponding gender terms (e.g., person + woman and person - man).
We also list the age-specific terms used to create attribute-neutral text descriptions &.. (t) (in Sec, 4.1 in the main paper): elderly bbby, child, kid, teenager, adult, youth, infant, toddler, elder, girl, boy, young, old, teenage, and their plurals. To create attribute-specinn descriptions (i.e., T = [Eg (t)]t E D,g E A} in Sec. 4.3 the main paper) for binary age, we add young or old just before the person-specific terms (e.g., person + young person and person - old person).
Regarding race attributes, we use race-specific terms to create attribute- neutral text descriptions: african, africa, asian, oriental, asia, east asian, south asian, south east asian, black, caucasian, european, hispanic, latino, latina, latinx, white, arab, arabic, middle eastern, native, indigenous, american, african american, usa, united states, chinese, jaianeeepessx. japanese..apan, indian, india, mex- ican, mexico, italian, italy, spanish, german, french. france. english, british, england, russian, swiss, hawaiian, thai, brazil, brazilian, canadian, canada, australian, australia, new zealander, new zealand, korean, korea, filipino. philippines, vietnamese, vietnam, malaysian, malaysia, sin- gaporean, singapore, indonesian, indonesia, thai, thailand, burmese, myanmar, cambodian, cam- bodia, laotian, laos, taiwanese, taiwan, pacific, melanesian, melanesia, polynesian, polynesia, mi- cronesian, micronesia, aboriginal, aborigine. We adopt racial classes used in FairFace (i.e., East Asian, Southeast Asian, White, Black, Hispanic, Middle Eastern, Indian), creating attribute-spccific descriptions. Specifically, we add East Asian for the East Asian class, Southeast Asian for the Southeast Asian class, White or Caucasian for the White class, Black or African for the Black class, Latino, Latina, or Hispanic for the Hispanic class, Middle Eastern or Arab for the Middle Eastern class, Indian for the Indian class before the person-specific terms (e.g., g. person - (East Asian per- son, Southeast Asian person, White person, Black person, Latino person, Middle Eastern person, Indian p)rson))
D LIST OF CONCEPTS AND OCCUPATIONS
We show the concepts used to compute MaxSkew @+ (in Tab. I and 2 in the main paper). The list of the concepts for each category (i.e., adjectives, occupations, and activities) are as follows:
kind, successful, noncriminal, nonviolent, moral, rich, honest, smart, evil, unkind, un- successful, criminal, violent, immoral, poor, dishonest, dumb, clever, stupid, hardworking, lazy, nasty, trustworthy, untrustworthy, caring, dangerous, heroic, villainous, good. We refer these con- cepts from (Berg et al., 2022). Regardig the templates used to create prompts, we use "This is the face of a { person", "This is photo of O person", "This person is {)", "A {} person", "A (} individual", "This individual is ()", "Photo of a ()" "("", "This is a (", "This is the photo of a ()", "A ()", following (Berg al., 2022). The curly brackets replaced with each concept.
Adjectives:
Occupations: doctor, aerospace engineer, computer programmer, electrical engineer, scientist, artist, author, designer, musician, painter, photographer, singer, writer, architect, civil engineer,
17

---

Published as a conference paper at ICLR 2025
engineer. programmer, software developer, childcare worker, coach, dental assistant, dental hy- gienist, dentist, clerk, housekeeper, maid, massage therapist, nurse, therapist. strrappis.pyychologiii social assistant, social worker, teacher, professor. CEO, skateboarder, surfer, baseball player, football player, soccer player, tennis player. For the occupation list, we refer to (Friedrich et al., 2023) with some modifications, such additional occupations for more inclusive list. As for the prompt templates, we select ones that are suitable to occupations (e.g.. "Photo of a ()"). We use this list for the texi-to-image experiments with some modifications. Specifically, we remove the similar occupa- tions (e.g,, remove civil engineer as there is engineer). Additionally, we remove occupations where the text-to-image model (Rombach al., 2022) can not generate images with humans tends to generate multiple individuals.
Activities: arranging flowers, playing tennis, playing skateboarding, playing baseball, playing soc- cer, playing football, playing snowboarding, playing sking, cleaning, dressmaking, tying tie, smil- ing, crying, laughing, cooking, making pizza, dancing, drinking beer, drinking wine, eating hotdog, eating cake, using computer, playing game, gardening, singing, petting dog, petting cat, makeup, shopping, playing piano, playing guitar, carrying baby. For activities, we use a subset of the Kinet- ics dataset (Kay et al., 2017) with some additional activities. For the prompt templates,
use "This the face person who likes ()", This a photo of a person who likes O}". "This person likes ()", "A person who likes ()", "Photo of person who likes ()", "This is person who likes ()"
E ADDITIONAL VISUAL EXAMPLES FOR IMAGE GENERATION EXPERIMENTS
We show additional visual examples for Figures 3 and in the main paper in Figures 5 and 6, respectively.
50
photo of tencher
Projection
SANER
Figure 5: Generated images for the prompt, "A photo of a teacher," by the original Stable Diffusion (SD), projection-based debiased CLIP (Projection), and debiased CLIP (SANER). We randomly sample 10 images from generated images. Images framed in green denote those of the minority gender in the generated images (i.e., male).
A photo of male masiciinn Projectionn
SANER
Figure 6: Generated images for the prompt, "A photo of male musician," by the original Stable Diffusion (SD), projection-based debiased CLIP (Projection), and our debiased CLIP (SANER). We randomly sample 10 images from generated images. Red frame indicates images with incorrect gender (i.e, female).
18

---

Published as a conference paper at ICLR 2025
F POTENTIAL EXTENSION
Additional Attributes. While we have aimed to make the attribute list as comprehensive as possi- ble based on prior work (Berg et al., 2022; Seth et al., 2023; Chuang et al., 2023), we acknowledge that the choice of attribute groups may be subjective and that certain attributes might be missed. However, in contrast previous approaches that require extensive attribute labels for images in the dataset, our method is designed to be flexible and extensible, allowing the attribute list to be expanded or adjusted based on specific needs or ethical considerations of the application domain.
Non-binary gender. While our experiments followed prior work (Berg et al., 2022; Seth et al., 2023; Chuang et al., 2023) in focusing on binary gender, SANER naturally extends to non-binary gender by defining additional attribute-specine terms and their neutral forms (e.g., "non-binary per- son" + "person"). The debiasing loss (Eq. 10) can handle any number of attribute groups, making it straightforward to include non-binary gender categories.
Automated text neutralization. While our current word-level approach is effective for well- defined social biases, it could be extended to handle more complex cases. As potential future direction, embedding-basee
ncutralization or context-aware language models could be explored to automate this process. These methods would enable bias identification and neutralization at mantic level, reducing reliance on predefined attribute word lists and making the debiasing process more robust and adaptable diverse scenarios.
Debiasing LLaVA-like models. SANER can be extended to mitigate societal biases in large vision- language models, such LLaVA (Liu et al., 2024a), by incorporating a debiasing mechanism for the image encoder. Specifically, this could involve training a debiasing layer to remove attribute information from the visual features. particularly for images without human subjects. We leave this extension as an avenue for future work.
G POTENTIAL NEGATIVE IMPACT
Applying SANER to debias CLIP may lead to a potential negative impact where users might over- look remaining biases, assuming the process be fully effective. While SANER performs better in gender and age bias mitigation, as evidenced by the MaxSkew and statistical parity metries, this does not ensure that SANER mitigates all possible societal biases, and there could be dimensions of bias not adequately measured by these metrics. It crucial to acknowledge that SANER, though impact- ful, is not an -ollmascopa solution for removing socictal bias. We notice that researchers must exercise due diligence in evaluating the application of SANER to avoid inadvertently introducing unanticipated biases.
19


Reviews:
[{"review_id": "XAAoqOzOKI", "reviewer": "Reviewer_igmm", "summary": "This paper aims to overcome societal bias present in datasets used to train large scale multi-modal models like CLIP. The authors provide a study of debiasing methods and explain how these struggle with loosing important attribute information, e.g. gender, or depend on attribute annotations which are hard to get and even harder to get in the form of a large diverse dataset. Their contributed method Saner has four components which together aim to remove biases: attribute neutralization, feature modification and attribute annotation free debiasing and regularization losses. The paper compares the results on quite a few tasks such as text-to-image generation, text-to-image retrieval and zero-shot image classification.", "questions": "- The caption in Figure 2 should contain ideally an explanation of what is seen in the figure. At least a reminder of the loss terms would be helpful to avoid having to jump back and fourth through the paper.\n\n- How does Saner apply to non-binary gender? Does it properly use general pronouns like they?\n\n- Conceptually, it seems hard to draw a line between terms related only to the gender and to the gender and a concept. Should de-biasing replace terms like pregnant? There is no male counterpart of this so replacing it will probably remove important information.\n\n- Is the \"multilayer perception\" in line 81 supposed to be a multilayer perceptron?\n\n- The definition of a dataset in Line 104 seems odd in the sense that, if a is a protected attribute then this dataset should only contain data with protected attributes. Is that intended? I would imaging the goal is to train on a large dataset which can have samples with protected attributes but not all samples have to have protected attributes?\n\n- One added sentence for 2.2 on how a orthogonal projection is helping debiasing would be helpful for understanding.\n\n- How hard is it to re-train clip? When the semantics are lost the regularization loss may be high. How computationally heavy is it to get to the same or a similar performance level?\n\n- Table 2's caption should probably read \"racial bias\" and not recial bias.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is very clearly written. Whenever an uncertainty comes up about a term footnotes help understanding.\n\nThe analysis of debiasing techniques is very clearly presented that it could be used for a tutorial. The experimental setup is well done, the evaluation metrics such as measuring the difference between a uniform distribution and a potentially gender biased image generation seems adequate for the task. Showing generated images, i.e. with Stable Diffusion shows also a very immediate practical application of this research.\n\nThe figures help understanding of the method even though the caption could be changed to make them more self-complete.\n\nIn general, this seems a very well written, easy to understand paper addressing a very current need.", "weaknesses": "The attribute groups will be limited to a specific set of defined attributes, which need to be agreed upon and could be debatable, e.g. \"pregnant\" (If I understand list C in the appendix correctly). There may also be missed attributes. This does not seem like a big issue but could be one in an adversarial setting. \n\nIn general, the benefits of not needing a annotated dataset with attributes seems to be bought by needing a general list of attributes. This may not be a weakness per se but it would be interesting to either read about it in the limitations or have the author comment on why this is not a limitation.\n\nThere are general implementation details about the used architecture but none about the training process. Re-training clip with a loss to keep it's performance while having a re-projection that changes the semantics seems complex. Information about the used GPU would help understand how hard training for 5 epochs is.\n\nComparison against other state of the art in this small field is limited but understandably so and the author's did a good job re-implementing relevant approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cIBDjcw3tn", "reviewer": "Reviewer_wrAD", "summary": "This paper aims to address societal biases in large-scale vision-language models, such as CLIP. The authors identify two limitations of existing methods: the loss of attribute information and the reliance on attribute annotations. To overcome these challenges, the authors propose SANER, a simple yet effective debiasing method for CLIP that incorporates attribute neutralization and an annotation-free debiasing loss. Extensive experiments are conducted to verify the method's effectiveness.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The problems of existing methods are straightforward, and the authors conduct several experiments to verify these phenomena.\n- The proposed method has a good performance.\n- The experiments are conducted on both text-to-image retrieval and generation tasks.", "weaknesses": "- Could the authors provide ARL results to verify the loss of attribute information? Besides, do other existing methods, such as Mapper and Prompt tuning-based debiasing, also have the lossing attribute information when debaising?\n- Although effective in debiasing CLIP, the proposed method appears ad-hoc for this specific task and lacks major technical contributions, as most components seem to be existing technologies or tricks. Therefore, this paper may not have significant influence or provide broader insights beyond the CLIP-debiasing task.\n- Why does adding only contrastive losses have a significantly negative impact on FairFace performance (as shown in Table 7)?\n- Is the proposed method only applicable to CLIP? Could the authors test other VLMs to demonstrate its broader effectiveness?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "77Ojv01XT6", "reviewer": "Reviewer_STrF", "summary": "The paper proposed a CLIP debiasing pipeline that aims to address two existing challenges: 1. loss of attribute information and 2. Dependency on attribute annotations. Specifically, the paper focuses on adjusting the text feature by replacing the protected attributes with attribute-neutral terms and further regularizing the adjusted feature by two reconstruction losses. The pipeline is evaluated on Text-to-Image retrieval and generation tasks.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The two identified challenges, especially the loss of attribute information, sound critical and interesting. It seems like the pipeline only involves lightweight training, as only the debiasing layer is trained. The pipeline is evaluated on two different downstream tasks.", "weaknesses": "1. Although replacing the protected attribute words with an attribute-neutral word is a sound neutralization method, it requires a comprehensive list of the protected attributes (as listed in Appx. C). However, in real practice, and especially in non-binary cases, it will be very hard or nearly impossible to have a complete list. Further, since the \"attribute annotation-free debiasing loss\" relies on a set of attribute-specific descriptions, creating the attribute-specific descriptions set may lead to some concerns. For example, if the set is created by iterating all possible words and all possible groups (e.g., man, woman, male, female, he, she, etc), then the size of the set will be overlarge and increase the computation complexity. On the other hand, if only one word is selected for each group (e.g., man + female, woman + male), then how to select matching words from groups may also affect the debias performance. Also, is it possible that the generated description may have unmatched text (e.g., A woman/boy is eating salad) or text that has grammar errors (e.g., A he/she is eating salad, according to Appx. C). I would like to see more discussion on the text generation. \n\n2. The proposed pipeline tried to address the challenge by combining the debiasing loss with two extra regularization losses. The debiasing loss tries to place the adjusted feature of the attribute-neutralized text in the middle point of several features of the attribute-specific text, which seems to be fine. However, the regularization losses try to project the attribute-neuralized feature to both text and visual features of an attribute-specific feature, which seems strange. As demonstrated in Figure 2, the input to the \"Text encoder\" has been neutralized, and the attribute-specific information has been removed. Further, the debiasing layer is also trained to adjust the features to be further neutral. Thus, even there is a residual connection in the \"Feature modification\" process, the output feature should not contain any information regarding the specific attribute. Then, how is it possible that such a feature can be mapped to either text or visual features that contain specific attribute information to avoid \"the loss of attribute information\"? Is it possible that the \"retained attribute information\" is some biased information from the dataset that is implicitly polluted by the regularization loss?\n\n3. For the evaluation of the text-to-image generation, the paper only provides numerical results on debiasing performance and some visualizations but does not provide any image generation imperial results, like FID. Also, only a few SOTA are used as the baseline, which makes the comparison weak. However, this may be understandable as several baselines do not have public implementation, and it seems like the paper prepares all the baseline results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IfvUiKsYsL", "reviewer": "Reviewer_vKXP", "summary": "The paper proposes a method for finetuning a CLIP model to remove social bias by an intuitive training scheme.", "questions": "Have you considered running experiments with LLaVA-like models? I don't think it would be terribly difficult to replace the clip model with the proposed one, run standard llava experiments (there are a few githubs that are easy to setup for such eval), and report the results. As for debiasing experiments, I've seen people use PATA and FairFACE and just look at the next token predicted to get similar scores as the CLIP style. Showing this CLIP replacement method works on text generation as well as image generation would very much increase the impact of the work, in my opinion. However, I realize this experiment may best be left for future work.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "I will start off by saying that the lede of the paper was buried: section 5 as an experiment is astounding. I absolutely love the drag and drop replacement of this debiased model into an existing model. The idea that, 1. you don't have to retrain the text-to-image model and 2. that you don't have to retrain CLIP from scratch is incredibly compelling. I recommend the abstract/intro be rewritten to highlight this experiment.\n\nOn another note, the paper is generally well-written and easy to follow. The problem of bias in VLMs is highly prominent and this work makes an excellent contribution to the field. I am fairly familiar with this field of debiasing clip and I feel section 2.1 does a good job summarizing the various papers.", "weaknesses": "I don't have too many issues with the paper. One thing I would like to have seen is a more automated method for text neutralization. While word level ablation is intuitive and makes sense for the social bias angle, a less hardcoded approach seems like it would be needed for a general-purpose debiasing solution, especially for concepts that are hard to ablate at the word level.\n\nThere's a dataset called FACET that addresses some issues with PATA and FairFace. If there's time, it would be great to run experiments on it as well. This would cement the proposed method as useful and cutting edge in my opinion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)