Association of Objects May Engender Stereotypes: Mitigating Association-Engendered Stereotypes in Text-to-Image Generation
Authors: Junlei Zhou
Jiashi Gao
Xiangyu Zhao
Xin Yao
Xuetao Wei
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Stereotypes
Diffusion Model
Text-to-Image
Primary area: diffusion_based_models
TL;DR: A novel approach to mitigate association-engendered stereotypes in T2I diffusion models.

Abstract: Text-to-Image (T2I) has witnessed significant advancements, demonstrating superior performance for various generative tasks. However, the presence of stereotypes in T2I introduces harmful biases that require urgent attention as the T2I 
 technology becomes more prominent.
Previous work for stereotype mitigation mainly concentrated on mitigating stereotypes engendered with individual objects within images, which failed to address stereotypes engendered by the association of multiple objects, referred to as *Association-Engendered Stereotypes*. For example, mentioning  ''black people'' and ''houses''  separately in prompts may not exhibit stereotypes. Nevertheless, when these two objects are associated in prompts, the association of ''black people'' with ''poorer houses'' becomes more pronounced. To tackle this issue, we propose a novel framework, MAS, to Mitigate Association-engendered Stereotypes. This framework models the stereotype problem as a probability distribution alignment problem, aiming to align the stereotype probability distribution of the generated image with the stereotype-free distribution. The MAS framework primarily consists of the *Prompt-Image-Stereotype CLIP* (*PIS CLIP*) and *Sensitive Transformer*. The *PIS CLIP* learns the association between prompts, images, and stereotypes, which can establish the mapping of prompts to stereotypes. The *Sensitive Transformer* produces the sensitive constraints, which guide the stereotyped image distribution to align with the stereotype-free probability distribution. Moreover, recognizing that existing metrics are insufficient for accurately evaluating association-engendered stereotypes, we propose a novel metric, *Stereotype-Distribution-Total-Variation*(*SDTV*), to evaluate stereotypes in T2I. Comprehensive experiments demonstrate that our framework effectively mitigates association-engendered stereotypes.

Association of Objects May Engender Stereotypes: Mitigating Association-Engendeeee Stereotypes in Text-to-lmage Generation
Junlei Zhou', Jiashi Gao', Xiangyu Zhao?, Xin Yao', Xuetao Weil* 'Southern University Science and Technology, "City University Hong Kong
Lingnan University
12111022,,,,1311111mmmmaaa1a suststc sustech, edu, cn, xy zhao@cityu. edu. hk xinyaoeln.o edu. hk, weixt@sustech. edu. en
Abstract
Text-to-Image (T2I) has witnessed significant advancements, demonstrating supe- rior performance for various generative tasks. However, the presence of stereotypes in T2I introduces harmful biases that require urgent attention as the T2I technol- ogy becomes more prominent. Previous work for stereotype mitigation mainly concentrated on mitigating stereotypes engendered with individual objects within images, which failed to address stereotypes engendered by the association of mul- tiple objects, referred to as Association-Engeeddee mentioning "black people" and "houses" separately in prompts may not exhibit stereotypes. Nevertheless, when these two objects are associated in prompts, the association of "black people" with "poorer houses" becomes To tackle this issue, we propose a novel framework, MAS, to Mitigate Association- engendered Stereotypes. This framework models the stereotype problem probability distribution alignment problem, aiming to align the stereotype probabil- ity distribution of the generated image with the stereotype-free distribution. The MAS framework primarily consists of the CLJP) and Sensitive Transformer. The PIS CLIP learns the association between prompts, images, and stereotypes, which can establish the mapping of prompts to stereotypes. The Sensitive Transformer produces the sensitive constraints, which guide the stereotyped image distribution to align with the stereotype-free prob- ability distribution. Moreover, recognizing that existing metrics are insufficient for accurately evaluating association engendered stereotypes, we propose a novel Stereorype-iitriibttonn
Stereotypes. For example,
pronounced.
ImageeSterrooyppe CLJP (PIS
association-engeeddeee metric, Stereotype Distribution Total- Variation (SDTV), to evaluate stereotypes
in T2I. Comprehensive experiments demonstrate that our framework effectively mitigates association-engenderel stereotypes.
Introduction
Text-to-Image (T21) Nichol et al. [2021) Rombach et al. [2022 Ramesh et al. (202 Saharia et al. (2022)) based on the diffusion model, has achieved significant breakthroughs image generation, showing potential for various downstream tasks such as image creation, editing (Brooks et al. (223)). etc. addition, the scale of T2I applications has expanded impressively. For example, the open- source stable diffusion project (Rombach et al. (2022) has gained the favor of more than 10 million users. As of August 2023, the number images created through T2I has surpassed 15 billion'.
*Corresponding author. http:://wummalleeeeepeellcmamaam
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

When generating images of white people, black people, houses separately, there no stereotypes.
When black people and white people associated with houses, can engender stereatypes.
Previous approaches are unable to mitigate these punciaaioo-engendered stereotypes.
Our MAS approach can effectively mitigate these
X
associatio essoedatoo-engenderee stercotypes.
MEI BIR
Figure 1: Original T2I models can generate stereotypes when given prompts involving multiple objects; when the prompts are "a photo of black/white people" "a photo of house", they do not engender stereotypes. However, if the prompt is "a photo of black/white people and a house", the model may engender stereotypes that the white people's 's house is better than the black people' 's house. noted that previous mitigation approaches have yet to mitigate these association-engendere stereotypes. Our approach, MAS, demonstrates the capability to effectively mitigate these stereotypes.
As images generated by diffusion models become more prevalent, mitigating the stereotypes they engender is increasingly critical (Seshadri et al. 20233 Luccioni et al. 20244). ).
The stereotype in social psychology is mental construct of particular type of person, object, or thing, often based on observing individuals who fit that type. Previous work explored sensitive attributes (such gender, race, region, religion, and age) and corresponding stereotypes present in T2I diffusion models (Wang et al. [2023a], [Esposito et al. [2023]. Bianchi al. (2023]) For instance, when the prompt is "a photo of a politician" the object (politician) in the image generated by the T21 diffusion model always appears male in the sensitive attribute of gender. Previous works (Shen) et al. 2024], Kim et al. [2023] Esposito al. (2023) on stereotype mitigation have been limited www.sciiittionnnnneeeeeee Engendered stereolypes (e.g., only mitigating the stereotype problem in the occupation or gender dimension), and cannot effectively address stereotypes involving the association of multiple objects, referred to as Association- Associaaion-Fingendeee Engendered Stereotypes. For example, as shown in Figure [ when the prompt "a photo of a white people and house" versus "a photo of a black people and a house", T2I diffusion models often generate images where the houses associated with white people appear better than those associated with black people, reflecting an association-engeen -eneedderrdeed stereotype. Therefore, mitigating such sssociation-engendee stereotypes is highly desired for T2I models.
to single object, referred to as Non-Association:
In this paper, our contributions are as follows: To the best of our knowledge, we take the first step towards addressing association-engendere stereotypes in T2I models. We define the stereotype problem in the T2I diffusion model as a probability distribution alignment problem, aiming to align the stereotype probability distribution of the generated image with the stereotype- distribution. @ We present a novel framework MAS to mitigate the association-engenderee stereotypes, which primarily consists of the learns the association between prompts, images, and stereotypes, which can construct the mapping of prompts to stereotypes. The Sensitive Transformer produces the sensitive constraints, which guide the stereotyped image distribution to align with the stereotype-Fin freee probability distribution. @ Given stereotypes.
CLIP (PIS CLIP) and Sensitive Transformer. The PIS CLIP
the insufficiency of existing metrics for accurately evaluating setoociatio-engendeene we propose novel metric for evaluating stereotypes in T2I: Stereotype Distriibbiitiiiioonnn Total- Variation (SDTV). We conduct a comparative evaluation our stereotype mitigation approach on five popular T21 diffusion models and six advanced stereotype mitigation approaches in T2I. Extensive experiments demonstrate the superiority of our approach in effectively mitigating both leesociiatio-eengnderr and association-engendeeee stereotypes.

---

Table 1: Stereotype categories and prompt word templates
Abbreviation
Stereotype Categories
Prompt
S-O & S-SA
Single Object with a Single Sensitive Attribute
A photo of [OBJECT].
S-O & M-SA
Single Object with Multiple Sensitive Attributes
A photo of [SA] [OB/ECT].
M-O & S-SA
Multiple Objects with Single Sensitive Attribute
A photo of [OBJECT 1] and [OBJECT 2).
M-O & M-SA
Multiple Objects with Multiple Sensitive Atrrhhutes
A photo of [OBJECT 1] and [OBJECT 2].
2 Related work
Stereotypes in T2I models. Although the current image quality and capabilities generated by T21
gradually approaching maturity, it is worth noting that various biases and stereotypes may still arise from neutral prompt words (Zhou et al. (2024]). ) Fraser et al. (2023) and Wan and Chang [2024] found that text-generated image models often engender stereotypes of gender, race, and demographics in sociology. These biases can manifest in multiple aspects, such as occupations (Bianchi et al. [2233] Cho et al. [2023]. Luceioni al. 2003)),, )), objects (Mannering (2023) )) and adjectives (Luccioni et al. [2023], Naik and Nushi [2023] ). Additionally, T2I diffusion models may exhibit gender bias when depicting interactions between two more people. Based the above findings, previous researchers proposed stereotype detection frameworks such as Language Agent (Wang et al. (023al] and T21AT detection (Wang al. (2023b). These research results provide practical tools for identifying stereotypes and lay a solid foundation for further mitigating stereotypes.
Stereotypes mitigation in T21 models. Existing approaches to mitigate stereotypes in T2I diffusion models have effectively mitigated some of the stereotypes but still have some limitations. Friedrich et al. 2023 randomly introduced additional text clues by identifying known occupations in the prompts to ensure a fair distribution of the generated images. However, this approach may ineffectively address biases related to occupations that are not predefined. Kim al. [2023] developed a de-
stereotyping loss function and adjusted specific parameters of the soft prompts to balance the sensitive attributes of the generated images but only achieved de-stereotyping in the T2I model containing occupational prompts. CChang et al. 2023) proposed a technique that involves projecting biased directions in text embeddings and debiasing by utilizing text data with calibrated projection matrix. Fine-tuning the diffusion model by modifying the capabilities of the pre-trained model has also proven be an effective strategy. Inspired by Gal al. [2023]. Zhang al. (2023). Brooks al. [2023). Dai et al. [2023), [Shen et al. [2024] fine-tuned the sampling process of the diffusion model and used an optimized loss function align the T2I model with fairness principles directly. Moreover, He et al. [2024] utilized an iterative distribution alignment approach to align biased distributions with a unitied distribution. [Schramowski et al.] [2023] built Safe Latent Diffusion (SLD) capable of suppressing sensitive image portions during diffusion. Previous researchers adjusted the original T21 model and had to consider the image quality and mitigate stereotypes in the original T21 model. stereotypes in the T2I model (such
They mainly focused on mitigating -ww-ssocciationenneeeeee as those engendered by occupation/gender) so it unable to mitigate the stereotypes generated when multiple objects are associated effectively.
3 Our framework: mitigate association-engeederee stereotypes (MAS)
In this section, we construct mathematical model of stereotypes in T2I. Based on this model, we propose a novel metric, Stereotype Distribution Total Variation (SDTV), for evaluating stereotypes. Then, we model the stercotype mitigation problem probability distribution alignment problem and build a new stereotype mitigation framework, Mitigate Association enngadneneeeeeeeeeedd Stereotypes (MAS). Finally, we train PIS CLJP and Sensitive Transformer models construct sensitive constraints. which guide the stereotype probability distribution to align with the stereotype- probability distribution.
3.1 Modeling stereotypes in T2I
Inspired by (Shen et al. 2024), we denote the object of the image as x and the object's sensitive attributes as (such as gender, race, and adjective). We introduce the probability distribution function (PDF) P(s = (())aa)) to quantify the likelihood of the image's 's object exhibiting specific sensitive attributes, where v(s) represents the value of the sensitive attribute. For instance, when
3

---

(1) PIS CLIP model pre-training
(2) Construct the sensitive constraints
T21 Prorpees
Test Encoder
Pronpes
Text Encoder
Text Emheokdings
Query:
Tj
T:
T
1
Mapping prompt
Sensitive Transformer
stereotype
Key+ Value
Inaae Eccode:
(3) Add constraints to T21
Sensitive Coratraints
anage
8
Docription
Stenschpe
Fronder
Text
Deccnder
Strmotype-frr frre imagge
Diflusion Process
Figure 2: Overview of our proposed MAS. (1) In the PIS model pre-training stage, we adjust the traditional CLIP structure from the original "text-image" two-dimensional mapping "text-image- text" three-dimensional mapping, thereby obtaining mapping of prompts stereotypes. (2) In constructing sensitive constraints, we propose a Sensitive Transformer based on the transformer structure to construct sensitive constraints for each prompt. (3) In the stage of adding sensitive constraints to the T2I, we embed the sensitive constraints into the T2I diffusion model to guide the generation of stereotype-fie images.
the generated image's 's object is person and is gender, us)) could be male or female. The PDF represents the probability of person appearing as male female in the sensitive attribute of gender. Furthermore, to distinguish the difference between association association-engendeee engendered stereotypes and
stereotypes explored by previous work (Kim et al. [023], He al. (2024), [hhnn al. (2024)) )
we categorize stereotypes into four distinct categories (Table II based on the association between objects and sensitive attributes: single object with a single sensitive attribute (S-O S-SA, pon-asoociation-engeddeer stereotypes), @ single object with multiple sensitive attributes (S-O & M-SA, aan-aasociaationnneeddeee
stereotypes), multiple objects with a single sensitive attribute (M-0 & S-SA, association-eeggndded stereotypes), ? multiple objects with multiple sensitive attributes (M-O & M-SA, association-eengendrree stereotypes). Here, we utilize a joint PDF to define categories with multiple objects with multiple sensitive attributes.
f(z1, .2....) P(sz, v(s.,) ), 822 v(s2)).... |1 22.
where xj and 22 represent the two objects in the image. For instance, Figure people, and 322 corresponds to house. The variables 8x, and Sx, denote the race and house's description. See Appendix for more PDF definitions of other stereotype categories.
|| I corresponds
new metric to evaluate stereotypes: SDTV
Stereotypes engender when a sensitive attribute in image consistently presents one few states
most instances (Naik and Nushi [2023). Therefore, Saravanan et al.) [2023] and Wang et al.) [023a have utilized various metrics to evaluate the stereotype from the perspective of the overall proportion different subgroups. However, when evaluating eesociatioo-engendeee stereotypes, the proportion of subgroups in the population cannot accurately evaluate association-engendeer stereotypes. To address this shortcoming, we propose new stereotype evaluation metric using total variation distance based the probability distribution model of stereotypes in T2I.
Consider a T2I model, denoted as G, which generates an image, and is an object of the image. We define a set S₂ denotes that object x contains multiple sensitive attributes such that S, (gneder, race, region.... }. We assign the sensitive attributes as v(s2)1 2(,,)),.... ..... v(s,)) ) for each sensitive attribute 82, Vsz E Sn, (e.g. v(8x) femule, v(8z) = black, For object (i.e., x ="doctor" or x ="nurse"), the stereotype-free T2I model should generate images with equal probability across different sensitive attribute values and should avoid significant probability

---

distribution disparities caused by the dependence of sensitive attributes on the object generating. To evaluate the extent of stereotypes, we calculate the difference probability distributions for each sensitive attribute value using the Total Variation distance Chung et al. 1989]: Drv(po =
((x)) ) [rr. Po 2 = "( }) Therefore, based on the PDF definition of stereotype types in Section
3.1
the SDTV value of model & under single-object z with single-sensitiv attribute v(sg) is:
{ Drv Po = ((xx))))))))) po (o = (xs)) (iJ)C
SDTV
max
(2)
In practice, for given prompt text т in T2I models, the sensitive attribute's probability of the generated images under this prompt is represented pa(*, = ((),))))) (a, r). For instance, consider the sensitive attribute of gender (sg ="gender") when the input prompt т is set to "a rich person", the gender-sensitive attributes' probability of the corresponding generated images set to pa( 5 = v(s,)' (x, (where v( represents "male") and pe (S2 = v(s)) [(),)) r) (where v(s,)" represents "female") respectively. We could incorporate the prompt and sensitive attribute and obtain the following:
considered and
(po(=== = u(s)))a po(s= 0(82) (2, 7) r)) (|,,
SDTV = max (ij))))
(3)
where 0 is the parameter of model G. Given input prompt т, po v(x2)* ) T). and (s v(s,)" [x, r) represent the probabilities that the sensitive attribute appears as v(ss)* and ( in the output, where {i,j} S Y and * j. If the value of (G) is considerable, it suggests that the image exhibits very severe stereotypes. Conversely, suppose its value is close to 0. In that case, implies that, under the given prompt conditions, the sensitive attributes displayed in the generated image are relatively evenly distributed, indicating no apparent stereotypes in this sensitive attribute dimension. When calculating the SDTV of eescciatio--engenderedd stereotypes, we deal with multiple objects and sensitive attributes simultancously. so we need to adjust Equation (3). Let's define set of objects
{-x,, where Xn and X2, represent the sets of human and non-human objects, respectively. For human object 2n E Xr with sensitive attributes Sh, and a non- non-human object r%, E X,, with descriptions Sy,, we can adjust Equation (3) accordingly to obtain:
po(sn (s8)) ossh)" (s v(sn)1sh v((h) m)
SDTV(G)
E SA ES * ES
max (UYE
(4)
The sensitive attribute sy and describe sh have y and Z corresponding values, respectively. See Appendix for the detailed proof process of the SDTV.
3.3 Mitigating association-engendeen stereotypes
Our framework aims to alter the stereotype probability distribution in T2I to align it with stereotype-free probability distribution. As shown in Figure 2 we construct MAS to imple- ment this alignment process, which consists of two main network structures, PIS CLIP and Sensitive Transformer. PIS CLIP learns prompts, images, and stereotypes in T2I and con- structs mapping of prompts to stereotypes; Sensitive Transformer builds sensitive constraints based on prompts and mapping of prompts to stereotypes. Finally, we embed sensitive con- straints into the T2I model by utilizing the Sensitive Transformer, which guides the probability distribution associated with stereotypes toward alignment with the stereotype-free distribution. We approach the solution by answering two questions: O How do we learn the sociation between prompts, images, and stereotypes? @ How do we construct constraints?
Algorithm I
CLIP Algorithm
I: Input: Labels - L; Image encoding-> I h,
Stereotype texts encoding - S[n, Prompi texts encoding + P'n, !..
I. +- E norm(] W.) W; leamned proj image to embed
P. f--nomm( P. W2). // W. learned proj of text embed
PIS CLIP. In T21, stereotypes are typ- ically not in prompts but in images. Therefore, we must learn the stereotype in the generated image corresponding to the prompt, which involves mapping the
4: S. <- (ornnonn W.)
To = Cross&FttrpplLoosssese LosteexpL.sssseppLL.. Te P.T, ), L))
Ta. CrossEntroyyoossicppee
-.).
Steaa CrossFatopyyosssfxpp(/ SS)).
8: Lose (Jros Tiass Stare
return Pe. S.
5

---

prompts, images, and stereotype descriptions. Inspired by CLIP (Radford et al. (2021) we construct a novel direesiiiniii mapping approach to learn the features and relationships of prompt- stereotypes. We use a 121-generated image, which contains a stereotype, as bridge to create three pairs: <prompt, image>, <image, stereotype>, and <prompt, stereotype> for training. PIS CLIP learns the potential mapping relationship between the prompt and the stereotype by maximizing the cosine similarity of these three pairs. We optimize a symmetric cross-entropy loss based on these similarity scores. Algorithm presents the process of the PIS CLIP (See Appendix C.I for more detailed experimental settings).
Sensitive Transformer. In PIS CLIP, we learn and establish a mapping of prompts to stereotypes. Then, based on this learned mapping, we can create sensitive constraints for prompts. When given prompt, we aim to provide specific sensitive constraints tailored to that prompt. To achieve this, we approach this as query problem: the aim to construct the value (V: sensitive constraints) by matching a query (Q: prompts) with keys (K: the mapping of prompts to stereotypes). We implement this task based on the transformer architecture (Vaswani al. (017
QKT
Sensitive Matrix (V) softmax(
IV.
Vd2
In Equation ( the input consists of queries and keys of dimension d3. We compute the dot products of the query with all keys, divide each by v /d,, and apply softmax function obtain the weights on the values.
Sensitive Constraints Guide Distribution Alignment. Consider a T2I diffusion model G, which generates images z with a sensitive attribute s. The attribute s can have y different categories and needs to align with a target distribution D that is free of stereotypes. We use a prompt to generate
a batch of images, denoted as = {x; u(*)
JeN. For the PDF of the sensitive attribute of image æ, h(z"()))) let h(2=(*)))) = ps "(*)* [p= [p=())' Pxx e(s)²
.... PP (()) J. E 15% where pz ((*)) represents the estimated probability of generating images with stereotypes using model G. Assume another set of
images, I {x, ( >eE, generated with the same prompt but without stereotypes. The probability
distribution of its sensitive attribute h(("(s)')) = pr (s)' [p" 1p=(*)
.rr v(s)² Pr 1 |}].
u(s)"
and the corresponding probability distribution without stereotypes is denoted as _(s) We compute the distribution distance between pz (()) and Par u(())
o² arg min sup (op(op) 22 r'Sy
66)
where Sy is the constraint space, Sy = (V11),, V(82), ((())) v(s)>), and is set of constraints within this space. The optimal constraint for minimizing the distance between two probability distributions in the constraint space is o². The a contains all possible sensitive attributes S = {s1, S2...., 8n
((Sn)) (((,))
V(s) = [u(s)*, ,o())))(()))...........
Snhne and their value corresponding to the prompts. When a sensitive constraint a is added to the input of the T2I diffusion model, it is combined with the original prompt embedding to form a new input for the T2I sampling process. Since the new input includes the sensitive constraint information, it guides the T2I process to produce a new output v(e);
o (p= (*)) = [x)( Pr ,..., [()))2 J. a(p" is a distribution that closely approximates the
stereotype-free distribution p:(())
Experiment
We conduct experiments on popular T2I diffusion pipelines evaluate the generalizability of our
approach across various T2I diffusion pipelines, which include ① runwayml/stable -iffuuii-ddfisi---v..
5(SD-1.5) (Rombach et al. [2022] stabiliiyailstalle-iifiee sitiiioo------eee xl-base- (SDXL) (Meng et al.
(021)) ByteDance/SDXLL DDL--Lightning (Lightning) (Lin et al. (2024) stabiliryal/ddkl turbo (Turbo) Sauer et al. (202)). ), and stabilityi/staall-cassea
(Cascade) (Pernias et al. (024 )) All these models are openly accessible from Hugging Face"] As mentioned above, we apply our stereotype mitigation approach to the five mainstream T21 diffusion pipelines. We use the optimal sampler and keep the same hyperparameters setting to generate images for all T21 diffusion pipelines (See Appendix for detail experiment setting). We create ten prompts for each of the four categories
'bttss/huuggingfacc.o

---

Table 2: The SDTV value of five mainstream T2I models in four stereotype types. indicates that smaller SDTV values correspond to less severe stereotypes. XX=.XXX represents the optimal result.
S-O & S-SAL
S-O & M-SAL
M-O&S-SAJ
Model
M-O&M-SA
Gender
Race
Region
G.xR.
Gender
Race
Region
SD-1.5
.68
124
811.110
.75+..0
.57::21
19.t.
.511:11
.4:
MAS(Ours)
.17::.14
.21::09
231.11
211.002
.17:.11
.201.09
.2:::002
.161.10
SD XL.
184+.14
.40+229
+59+.20
.61+-11
.74+-11
.8++-11
.8++008
.73+-15
MAS(Ours)
.15+:12
.16+.05
.13+.004
.194-09
.16::110
.20+
.21+
.15+.05
Lightning KAS(Ours)
8111.19
961.02
94:1.02
88:1.09
.86:1.04
.82::.09
.901.04
.78+.09
.18+.11
.16+:009
.17#.04
.154.12
.17+.10
.194:05
.22+.111
.174.08
Turho MAS(Ours)
.2++-08
.89+.10
80-+116
89+-111
.82+-11
.88+-07
.85+007
.72+-08
.164.13
.15+:10
.16:.10
.20-:13
.174:11
.19+t.10
.20+.110
.15+.10
Cascude
96:+.02
.90+.07
88:2.09
.334.05
.90-2.05
.884.07
.89+.06
.814.04
KAS(Ours)
.17:.15
.17+.09
.19+.08
.17+.08
.17+.08
.21.007
.23+-00
.164.04
of stereotypes discussed in Section 3.1
generate 100 images per prompt, and calculate attribute
probability distributions by utilizing 100 x 10 = 1000 images. In addition, for comparison, we test each pipeline using two sets of data, one from the original pipeline and the other from the pipeline with MAS. We utilize the SDTV value to evaluate the severity of stereotypes in T2I models. For four types of stereotypes, we apply the corresponding SDTV to evaluate the severity of each type (See Appendix B for detailed calculation methods for the SDTVs corresponding to different types of stereotypes).
4.1 Mitigation effects
Evaluation of stereotype mitigation effectiveness across different T2I diffusion models. Table 2 reports the performance of our stereotype mitigation approach. Our approach effectively mitigates the pewpiiiattoonnggnndeeee stereotypes in T21 and maintains excellent performance mitigating the non-asooiiaion-eegeenerrd stereotypes. Besides, Table Sihows that the latest T21 models (Lightning. Turbo, and Cascade) display more pronounced stereotypes than their predecessors, SD-1.5 and SD XL. Nevertheless, our approach performs robust generalizability, demonstrating superior performance in mitigating stereotypes across various scenarios, including traditional and the latest T2I models wawpraooonnnnneeddddd/ssnniniiii
involving
stereotypes (Figure 3)
Comparative evaluation with different stereotype mitigation approaches. To validate the effec- tiveness of our approach, we conduct a comparative analysis with state-of-the-art methods. Table shows that our approach outperforms six other solutions, demonstrating fewer stereotypes for all eight single-object and multi-object scenarios. Our approach offers distinct advantage compared to the prompt fine-tuning approach. We directly learn the relationship between prompts, images, and stereotypes, which enables us construct more targeted constraints for the stereotypes in T2I rather than relying on broadly sensitive attributes as supplements. As a result, our approach can maintain an effective mitigation strategy even when faced with more complex and concealed dereottyps. Compared to the model fine-tuning approach (Shen et al. 2024 ), our approach only requires integra-
Table 3: Comparison with Kim. 2023 (Kim et al. 2023 ), Chuang. 2023 (Chuang et al. (0233 )
Gandikota. 2024 (Gandikota al. (2024). Bansal. 2022 (Bansal et al. [002 Wang. 2023 (Wang
et al. (202cc). ), Shen. 2024 (Shen et al. [2024). denotes that this approach is unable to mitigate stereotypes. / indicate that the approach is more outstanding with higher/lower scores.
SAO
S-O&M-SAJ
M-O&SSAA
S.P.1
approach
M-O&M-SAL
Gender
Race
Region
G.x R
Gender
Race
Region
CLIP-T2I
SD 1.5
68+
82 14
$14.10
25:.20
49 25
474.23
194.19
535.17
40= 03
Kim. 2023
.37
.39.03
Chuang. 2023
38=.10
.49 49-00 04
.241.02
37=.04
Gandikota. 2024
.491.33
.43.06
.21:1.03
.38..04
Bansal. 2002
466 4.2
.37-08
.194-00
.36 304.04
Wang. 2023
.471.23
.00.00
.20+1.02
.39 04
Shen. 2024
.20+1.03
.18
.194:006
.39
MAS((Our)
.122.14
21
.33::133
.211:002
.12.11
.011:09
20:100
116:.10
39
tion within the T21 workflow to perform mitigation, eliminating the need to retrain the original T21 model. During the training stage, the fine-tuning mitigation approach must ensure that it cannot affect
7

---

the original T2I diffusion model's 's alignment of text to image. Therefore, not all resources can be utilized to learn stereotypes, significantly limiting the learning of more types of stereotypes. However, our model can fully use computing resources to learn more types of stereotypes. TTii enables our approach to demonstrate a practical mitigation effect covert stereotype scenarios involving multiple objects.
4.2 Generalization
Semanties preservation experi- ment. The primary fundamental of T2I is ensure that the gener- ated image is consistent with the provided prompt. To achieve this, we conduct semantics preserva-
Table 4: Semantic preservation experiment.
SD-1.5
SD XI.
Lightning
Turbo
Cascade
Original
.394.03
.33.02 02
.32
.32:.02
.,440.44
CLIP-T21 f
Ours
.38..05
.33.04 04
.32=
.31..04
422.05
CLIP-121
Ours
801..1
-78-.13
.84
.7.
89 02
tion (S.P.) (Pezone al. (024), Radford al. (202T]) experiments where we encode prompt. text, and images using CLIP's s (Vit-L/I-4 text-image encoder. In Table we report (1) CLIP-T2I: the CLIP score between generated images and prompts; (2) CLIP-I2I: the similarity between stereotype- mitigated T2I and original r2l-generated images for the same prompts and hyperparameters. The sereotype-mitigatee T2I diffusion model can remain consistent with the original T2I diffusion model in semantics preservation.
Generalization non- template prompts. We summarize more than 300 relevant template prompt
Table Non-template prompts evaluation experiment.
S-OU5--AA
S-0LMSA
S.P.
M-ORS-SAJ
M-ORM-A
Gender
Race
Region
CLIP-T21
SD-13
.69..24
843..0
82
J3
48
525.17
403..0
words from the work of
Kim. 2023
.441..1
.381.09
391.00
Chuang et al. [2033 and
Chuang 2004
.3++-11
47++06
24-04
37+-03
et al. [2024] listed in
Gandikota, 2024
.50+:30
44:1:00
22:
40-1 04
Appendix D which con-
Bansal.2022
.491.27
.001.10
.40.1.04
Wing 2023
.491.18
.4.1.10
211:00
.391.03
tains prompt words re-
Shen. 2024
.25::15
44:1.09
17:000
4401 04
lated to occupation, adjec- tives, region, race, gender, etc. To explore generalization to non-template prompts, we randomly select 100 prompt in- stances from the diffusionDB (Wang et al.] (2022) dataset- data that might inadvertently per- petuate stereotypes. Table 5 shows the evaluation results and demonstrates the effectiveness of our approach in mitigating stereotypes. Although we only implement stereotype mitigation for template prompts, the stcreotype-mitigation effect also generalizes to more complex non- templated prompts. We list some images generated with non-template prompts in Appendix F
MAS(Ours)
.20.1.11
.23.1.10
.23..15
20.1.05
22.113
.18...4
.40.1.04
Impact on T2I diffusion models. Since our approach integrates a Sensitive Trans- former model into the original T21 work- flow, could introduce additional over- head to the T2I model or impact the qual- ity of the generated images. To evaluate these potential effects, conduct a se- ries of experiments focusing on the qual- ity of image generation and the efficiency of the generation process. We set four different batch sizes (10, 20, 50, 100) for the five different T2I diffusion models. We evaluate the impact of using MAS
Table 6: Evaluate the impact of MAS on image quality and efficiency generated by the original T21 model.
times/(s)
FDD J
10
20
50
100
SD-1.
25.8
31.2+5.30
125:9.00
19.0
13.5+1.30
MAS(Ours)
29.4 2.80
$8.4 1.90
133.110
270
17.2
XT.
4..9-2.0
8.56-4.50
219+120
429+25.0
16.10
MAS(Ours)
4.47
95.4 15.70
230 13.0
443 1.7.0
16.7..0.80
Lightning
0.30=0.00
129--195
34.0:3.40
64.5+5.10
22.6:1.20
(ASrO)
8,21 0.94
14.7 1.87
39.013.21
1..91
13.1
Turbo
7.30-150
14.5=3.20
35,9-4.20
71.9+5.40
20.6:2.10
MAS(Ours)
10.5-2.10
19.6
43,1 14.40
88.2 14.90
1..00
Cascade
25.9=1.30
7.2:7.20
245
17.0
23.6=1.70
MAS(OUrr)
29.3.
57.3
126. 18.30
267 17.0
24.01.2.20
the quality and efficiency of generated images of original T2I by measuring the Fréchet Inception Distance (FID) (Heusel et al. (017)) value of generated images and the time taken to generate images in different batch sizes. Table 6 demonstrates the evaluation results. Our mitigation approach maintains comparable image quality and generation efficiency to the original T2I diffusion models. Compared to other approaches, although our solution adds sensitive constraints to the original T21 diffusion model, only introduces low additional overhead.

---

(a) Sccnario Single Object with Single Sensitive Attribute. Use prompt: "a photo of professor.
SDTV Gender: 0.91 (original) (de-sterootype): Race: 0.87 (original) - 0.17 (de-stereotype)
Scenario2: Single Object with Multiple Sensitive Attributes. Use prompt: photo of terrorist." SDTV: Racex Region: 0.94 (original) +0.08 (de-steroolype): Race Regiou Gender: 0,53 (original) - 0.03 (de-stervolype) GGnderr
(e) Secnario 3: Multiple Objects with Single Sensitive Attribute. Use prompt: "A photo of doctor the left and a nurse on the right." SDTV: (Gender: 0.87 (original) 00.10 (de-stereotype); Race: 0.83 (original) -+00 19 (de-stereotype)
(d) Scenario 4: Multiple Objects with Multiple Sensitive Attributes. Use prompi: "A photo of whiterblack people and house." SDTV: (Ohjects & Sensitive attributes: 0,76 (original)
- 0.09 (de-stereoypee)
Figure 3: Images generated from the original SD-1.5 (left) and the SD-1.5 with MAS for mitigating stereotypes (right). Use the same prompt and T2I parameter settings for each category to generate 100 batch images and calculate the SDTV value. Compare the changes in SDTV values before and after mitigation. After applying MAS the original model, the stereotypes are significantly mitigated. More images in Appendix
Table Stereotype mitigation experiment in complex T2I scenarios.
Stereotype mitigation ex- periments in more T2I scenarios. This section aims evaluate the ef- fectiveness of MAS in miti- gating stereotypes in complex T2I scenarios.
S-OLS-SA
S-ORM-SA
M.SSS SAL
M O&MMM SA
S...7
Gender
Race
Region
G.xR.
CLIP-TI
R-SD
$4
87
x1=:00
78t.11
31::06
375h11
38..00
MABIOurs)
201.11
221.07
211.009
38.100
R-SD LORA
855.07
80s
79a.13
388.00
356.12
395..04
MAR(Oarr)
21+-11
23+.13
234-06
194.00
214:30
.31.00
38+.04
ConmellNet
$44.06
866
82+
791.10
814:08
384.05
MAS:O
214.30
386..0
In practical T2I genera- tion processes, it is neces- sary incorporate models such as LoRA and ControlNet to control the image's ' style and structure. Consequently, our mitigation approach must be capable of reducing stereotypes multi-model collaboration T2I scenarios. We conduct stereotype mitigation experiments by combining several mainstream models, including the diffusion- retraining model (R- LoRA models, and ControlNet models (Con). Table 7 demonstrates that our stereotype mitigation approach maintains outstanding performance even in T2I generation scenarios involving multiple model combinations. See Appendix G. and Appendix G.2 for detailed descriptions of the experiment and examples of images.
SD LORA + Con
.71.04
861..0
811.10
801.10
331.00
J86.12
38..00
MAS(Ours)
22:1
22:1.3
21:1.09
20:1.07
211.14
.211.09
38
'The most downloaded retraining stable diffusion model the Civitai: https///riiaai.commmoeeelas

---

Conclusion
In this paper, we took the first step toward mitigating association-engenene stereotypes Text-to- Image (T2I) diffusion models. We innovatively modeled stereotypes as probability distribution align- ment problem and constructed a probability distribution model for both non-association enggnodered and sonciiation-nggeddeee stereotypes. Then, proposed the Mitigate Association-engendde Stereotypes (MAS) framework for the first time. MAS learned the mapping of prompts, images, and stereotypes and constructed sensitive constraints guide the T2I diffusion model in generating stereotype-rree images by embedding these sensitive constraints into the T21 diffusion process. Addi- tionally, proposed novel metric to evaluate stereotypes, Stereotype Distribution Total Variation (SDTV). Finally, comprehensive experiments demonstrated that we contribute an effective mitigation approach for association- -eeeiiatoon-ngendereed stereotypes in T2I, establishing more ethical and reliable foundation for future text-to-image generation development.
6 Limitations
This research effectively mitigated both non-association-ennennne stereotypes. While it successfully addressed the identified stereotypes, exploring other subtle stereo- types will be an interesting direction for future research. To our knowledge, modeling the problem as aligning existing distributions with target distributions holds significant potential for further extension into research areas such as debiasing and detoxification in large models. Investigating these avenues could provide deeper insights and more comprehensive solutions to ethical issues generative models.
and association-engendeees
Acknowledgments
This work was supported by Key Programs of Guangdong Province under Grant 1222XX10 Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding parties.
References
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint 12. 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint orXiv:2204.06122) 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. In NewrIPS, 2022.
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In CVPR, 2023.
Preethi Seshadri, Sameer Singh, and Yanai Elazar, The bias amplification paradox in rext-to-image generation. arXiv preprint 2rX8v:2308.007: 2023.
Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jemite. Stable bias: Evaluating societal representations in diffusion models. In NeurIPS, 2024.
Qichao Wang, Tian Bian, Yian Yin, Tingyang Xu, Hong Cheng, Helen M Meng, Zibin Zheng, Liang Chen, and Bingzhe Wu. Language agents for detecting implicit stereotypes in text-to-image models at scale. arXiv preprint ar 1101::2310.11778. 2023a.
10

---

Piero Esposito, Parmida Atighehchian, Anastasis Germanidis, and Deepti Ghadiyaram. Mitigating stereotypical biases in text image generative systems, arXiv preprint arXiv:2310.06904, 2023.
Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng. Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan, Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. FAceT, 2023.
Xudong Shen, Chao Du, Tianyu Pang, Min Lin, Yongkang Wong, and Mohan Kankanhalli. Finetuning text-to-image diffusion models for faimess, In ICLR, 2024.
Eunji Kim, Siwon Kim, Chachun Shin, and Sungroh Yoon. De-stereotyping text-to-image models through prompt tuning. In ICML WorkShop, 2023.
Mi Zhou, Vibhanshu Abhishek, Timothy Derdenger, Jaymo Kim, and Kannan Srinivasan. Bias in generative ai. arXiv preprint arXiv:2403.02726. 2024.
Kathleen C Fraser, Svetlana Kiritchenko, and Isar Nejadgholi. A friendly face: Do text-to-image -mage systems rely on stereotypes when the input is under-specified? arXiv preprint arXiv:2302.071553 2023.
Yixin Wan and Kai-Wei Chang. The male ceo and the female assistant: Probing gender biases in texl-to-image models through paired stereotype test. arXiv preprint arXiv:2402. 10089, 2024.
Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In CVPR, 2023.
Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jemnite. Stable bias: Evaluating societal representations in diffusion models. In NeurIPS, 2023.
Harvey Mannering. Analysing gender bias in text-to-image models using object detection. arXiv preprint arXiv:2307.08022 2023.
Ranjita Naik and Besmira Nushi. Social biases through the text-to-image generation lens. In AIES, 2023.
Jialu Wang, Xinyue Gabby Liu, Zonglin Di, Yang Liu, and Xin Eric Wang. T2iat: Measuring valence and stereotypical biases in text-to-image generation. arXiv preprint arXiv:2306.0000 00905, 2023b.
Felix Friedrich, Manuel Brack, Lukas Struppek, Dominík Hintersdorf, Patrick Schramowski. Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302.1089) 2023.
Ching- Yao Chuang, Jampani Varun, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts. arXiv preprint 2302,00070, 2023.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In ICLR, 2023.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In CVPR, 2023.
Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, et al. Emu: Enhaneing image generation models using photogenic needles in a haystack. arXiv preprint 1iXx::2309.18000 2023.
Ruifei He, Chuhui Xue, Haoru Tan, Wenqing Zhang, Yingchen Yu, Song Bai, and Xiaojuan Qi. Debiasing text-to-image diffusion models. arXiv preprint arXiv:2402. 14577, 2024.
Patrick Schramowski, Manuel Brack, Bjöm Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In CVPR, 2023.
Adhithya Prakash Saravanan, Rafal Kocielnik, Roy Jiang, Pengrui Han, and Anima Anandkumar. Exploring social bias in downstream applications of text-to-image foundation models. arXiv preprint
2023.
11

---

JK Chung, PL Kannappan, Che Tat Ng. and PK Sahoo. Measures of distance between probability distributions. Journal of mathematical analysis and applications, 138(1))280-22) 290, 1989.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748-8763, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurlPS, 2017.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprini arXiv:2108.01073, 2021.
Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation. arXiv preprint arXiv:2402. 13929, 2024.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042, 2023.
Pablo Pernias, Dominic Rampas, Mats Leon Richter, Christopher Pal, and Mare Aubreville. Würstchen: An efficient architecture for large-scale text-to-image diffusion models. In ICLR, 2024.
Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyáska, and David Bau. Unified concept editing in diffusion models. In CVPR, 2024.
Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai- Wei Chang. How well can text-to-image generative models understand ethical natural language interventions? In EMNLP, 2022.
Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch. Concept algebra for score-based conditional model. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling, 2023c.
Francesco Pezone, Osman Musa, Giuseppe Caire, and Sergio Barbarossa. Semantic-preserving image coding based on conditional diffusion models. In ICASSP, 2024.
Jia Li, Lijie Hu, Jingfeng Zhang, Tianhang Zheng, Hua Zhang, and Di Wang. Fair text-to-image diffusion via fair mapping. arXiv preprini arXiv:2311. 17695, 2024.
Zijie J Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: large-scale prompt gallery dataset for text-to-image generative models. arXiv preprint arXiv:22 10. 1222210..4440 14896, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. In NeurlPS, 2017.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Faimess through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226, 2012.
Zonghan Yang, Xiaoyuan Yi, Peng Li, Yang Liu, and Xing Xie. Unified detoxifying and debiasing in language generation via inference-time adaptive optimization. arXiv preprint arXiv:2210.0444 044922, 2022.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. ECCV, 2014.
Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In CVPR, 2021.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In NeurIPS, 2022.
Moin Nadeem, Anna Bethke, and Siva Reddy. StereoSet: Measuring stereotypical bias pretrained language models. In ACL-IJCNLP, 2021.
12

---

A Probability Density Distribution Modeling
A.1 Probabilistic model of four types stereotypes
In our research, we define stereotype as the representation that links objects to sensitive attributes. Specifically, for generated images, we denote the object as J, the subject's 's sensitive attribute as Sa, and the attribute value of as u(xx). Based on this foundation, we introduce the PDF P(sN = v(*))
to describe the likelihood of an image subject displaying specific sensitive attributes. In traditional psychology, a stereotype refers to people's overall view or concept of a particular person, thing, or object. This understanding often arises from individual observations and perceptions within that category. However, defining their stereotypes in artificial intelligence, especially for unconscious systems like AI models, becomes complex and challenging. The lack of clear definitions poses significant challenge for our study. Therefore, our work interprets stereotypes within the T2I model from a probabilistic perspective. By carefully examining different stereotypes, we categorize them into four distinct types:
Single Object with Single Sensitive Attribute
Single Object with Multiple Sensitive Attributes
Multiple Objects with Single Sensitive Attribute
Multiple Objects with Multiple Sensitive Attributes
We utilize random conditional probability distributions for single object with single sensitive attribute to describe their behavior.
f(x) = =(s)))))
(7)
The formula represents the probability density distribution of the attribute value v(s,) of the object concerning the sensitive attribute $.- For instance, consider the prompt phrase "a photo of nurse". In this case, the object depicted in the image is a nurse. If we pay attention to the sensitive attribute of gender, the object in the picture always displays "female" gender-sensitive attribute values (u(*z)).
We utilize multiple iimeension-sessitien attributes of a single object by using probability distributions under various conditional constraints:
f(x) = P(s = u 82 v(52)....(())
i.e. still taking the prompt as "a photo of a nurse" as an example, but this time, in formula R we focus on sensitive attributes such as "s gender", "s² race". then the image always shows "v(s)) = female". u(())) = white race" sensitive attribute value.
For generated images of multiple objects (I), we use a joint probability density distribution to describe the stereotypes of multiple objects.
(Q.... P(s ((,)
The formula 9 describes how multiple objects within the image collectively exhibit the same sensitive attributes. However, these attributes consistently manifest distinct stereotypes due to the presence of different objects. For instance, consider the prompt "a photo of a doctor and a nurse." When the object is a "r1 doctor," it invariably appears "v(s) male". Conversely, when the object is "x2 = nurse", it consistently portrays a "v(s) = female", thus reflecting gender stereotypes.
Likewise, we utilize the joint probability density function:
(2...) = P( = o(5z)), S22 = =((s)) I1, 11, .....
(10)
The formula [0 represents the stereotype probability distribution across multiple objects and multiple sensitive attributes. i.e., when the prompt "a photo of a black man/white man and his house". a common stereotype emerges: the "white man's" 's" house tends to be portrayed as more better than the "black man's house. This stereotype frequently arises when multiple objects and sensitive attributes intersect.
13

---

A.2 Illustrations of four stereotypes
Prompts: photo engineer." "a photo of a driver."
Single Object Single Sensitive Attribute
For the engineer and driver objects in the figures, there are stereotypes that gender always male always white people.
Non-
Only one
mitiguted once.
association- engendered
stereotypes
Prompts: "a photo of a farmer." "a photo of a CEO."
Single Object with Multiple Sensitive Artributes
For the farmer and CEO objects in the figures, there are stercolypes that gender always male and always white people.
Multiple sensitive attributes mittgatted simultaneussy..
Prompts: "a photo boss and employee."
"a photo of professor and teacher"
Multiple Objects with a Single Sensitive Attribute
For these images, there stercotypes that white people always have higher status than other races.
ssnciation -engendered
stereotypes
Prompts: photo of photo a manager secretary:
doctor."
Multiple Objects with Multiple Sensitive Attributes
For these images, there always stereotype that men have higher present white people.
are always
Figure 4: Description the four stereotypes. Previous works have effectively mitigated non- aaxociation-engenddred stereotypes but cannot mitigate association-engender stereotypes effec- tively.
B
SDTV Main Proof.
B.1
SDTV's inference and proof process
In section 3.2
we have explained the variables in the SDTV formula. The single-object with single- sensitive attribute proof process is follows:
DrV Pa(sz v(sx)( Po =(ss)\o)))
SDTV(G) = max (iJ)C)
Drv ( |pols u( T)dT - / po(s= = v(sx)(1, (z, }
max (
Drv Byles = -
B = v(
max (
Ep(e.=u(es)))))00 = 0(82) Eptc, (00o:)/0pppps0000011110)
v(0x)\22,
max (())
(x) 1 12)
max (
Based on the above proof, we consider sensitive attribute set S, = (gneder, race, region, }. which contains different attributes. We evaluate the extent of single-object with multi-eensitive
14

---

attribute stereotypes by calculating the average SDTV values of all attributes.
SDTV(G) = B|s-1
Σ
Drv pol6s - =0(==)))000)))))))))))))))) T). - =(82)) ar 7)
max
BWCC
= =Bis=1 #2
Σ
max (WIE)
(po(s)=e(sa)))
pa(8z = v( u(s,))
Similarly, we expand the object set X = fa1, 22, In IEEN. where the set X contains multiple different objects, and calculate the average SDTV of all objects evaluate the multi-objects with single-sensitive attribute stereotype extent of the image:
X
Drv (po(s= = (((8))a))) = v(*x) (z)) }}
SDTV(g) = EEI
Σ
max (WIE
(po(s= = v(8x)))r, (((n)=1(()))o pa(s= = u((x))
max (1))))
Based on the above process, we simultancously expand the object and attribute dimensions. Let's define set objects X {.,,, }, where Xr, and X;, represent the sets of human and non-human objects, respectively. For a human object 24 E Xa with sensitive attributes Sn, and a non non-human object 2/j X%, with descriptions Sy we can adjust Equation (3) accordingly obtain:
SDTV(G) = E
max (5
po Po = r(s)))16) = o(sh))", po(,,, = v(51)16h = =0sh)m²,
SyES 5 ESS
B.2
Why is it the maximum and not the average?
We compute the distances between the two distributions using the total variation distance (Drv). The evaluation typically utilizes the average in the study by Dyoorkk et al. [2012] and Yang al. (2022). However, our research has revealed that the pair of average values do not accurately depict stereotypes in T2I. Therefore, we will conduct further discussions and analyses on this issue. Ultimately, we demonstrate that utilizing the maximum total variation distance can effectively characterize the extent of stereotypes in T2I. Moreover, we can extend the maximum total variation distance to other studies.
0.2
(a) stereotypical extent descrihed using the maximum and average values when extreme at- tribute values are present in the probability distri- bution of Drv.
(b) The stereotypical extent described using the maximum and average values when the attribute values in the probability distribution of Dry are relatively balanced.
Figure 5: Comparing maximum and average Drv values to evaluate stereotypes in extreme and common T2I scenarios.
As depicted in 5 two scenarios exist for the distribution values of sensitive attributes in T2L. In one scenario, the Drv value is high only for specific sensitive attribute values, while it remains consistently low for other attributes. The diagram in Figure 5[ shows that if we use the average Dry value to represent model G under the sensitive attribute S, it will significantly reduce the extent of stereotype in model G. Conversely, using the maximum value can accurately describe the extent of stereotype in model G under attribute S.
Example. We analyze how model G stereotypes are based on the attribute race (s ruce). The potential values for the sensitive attribute are v(s) (white, black, yellow, brown....) Consider the scenario illustrated in Figure ((a)) where the Dry value for the white and brown categories assumed to be the highest at 0.9. In contrast, it falls between 0.2 and 0.4 for other racial categories.
15

---

When we calculate the average, the SDTV value of model ς is roughly 0.3. Model g has significantly reduced its extent of stereotypes in the attribute dimension of race by almost three times. However, the SDVV value, calculated using the maximum value, is 0.9. This value indicates that the model still has a clear preference for one of the sensitive attribute values related the sensitive attribute of race, thereby displaying severe stereotypes.
Another situation is that the high Dry of some sensitive attribute values maintains relatively balanced state. As shown in Figure [(b) at this time, using the mean SDTV value to describe the stereotype extent of model G will still lower the extent sensitive attributes of the model, and at this time, SDTV will remain at low level. Thus, we mistakenly believe that model G does not exhibit stereotypes.
Example. We analyze how model 9 stereotypes are based on the attribute of race (s = race). Among them, the possible values of the sensitive attribute consider a balanced case, as shown in Figure 5(b) to calculate several races. When looking at the Drv, can be seen that the high Drv value of model & remains in the range 0.5 0.6, while other Drv are around 0.2. When using the average, the SDTV value of model G equals approximately 0.3. The model does not show very obvious stereotype. On the contrary, using the maximum value can show that the model still has stereotypes.
u(s) {white, black, yellow, brown..... We
Some other thoughts from the perspective of mitigation: when use the maximum value Drv description, at the exact moment, we only need to pay attention to the attribute value corresponding to the most severe stereotype under the current SDVV performance. At this time, when facing the extreme situation in Figure (5a) we focus on mitigating the stereotype under this attribute value, which can significantly alleviate the T2I model stereotype; when facing the situation in Figure 5(b) When we mitigate the maximum Drv, the SDTV will still show the maximum Drv among the remaining attribute values, and then continue mitigate the maximum to achieve the step-by-step mitigation of stereotypes in the T2I model. On the contrary, when we use mean SDTV as description, will disperse the focus of mitigation all possible attribute values, and the effect of dispersed mitigation will not be able to solve the stereotype problem effectively.
16

---

C Experiment Setting
C.1 Training the PIS CLIP and the Sensitive Transformer
Dataset. Our training data sources are mainly divided into three channels: (1) The first channel comes from the collected objects and sensitive attributes. We embed objects and sensitive attributes into the prompt word template as the prompts input of the T2I diffusion model and generate 100 batches images for each prompt. (2) The second channel comes from existing datasets, including COCO (Lin et al. 2014)), ), FairFace (Karkkainen and Joo (02T)), and LAION-5B (Schuhmann et al. 2022) ) and extracts the stereotype images in these datasets. (3) The third channel comes from the prompt datasets. We extract prompts with sensitive attributes and objects from the existing prompt datasets. Then, we use the five popular T2I diffusion models (SD-1.5, SD XL, Lightning. Turbo, and
Cascade) to generate images for each prompt. The composition of image data is as follows:
Table The statisties data source in the stereotypes dataset.
Sources
Numbers of Data
Description
prompts templateeee + Objects/SA
2.00K
Prompts data
COCO (Lin et al. [0144]
2.20K
Image data.
FairFace Karkkannen and Joo
202
10.8K
Filter these image datasets
LAION-5B (Schuhmann al.
2022
120K
find stereotypical images.
Stereoset (Nadeem al. 2021
2.12K
Prompts data. Generate stereotypical images.
DiffusionDB (Wang et al.
2022
8.96K
Train setting. We train PIS CLIP for 15K iterations, using the Adam optimizer with learning rate 5e-5 based on CLIP's pre-training model VIT-L/14. The PIS CLIP training takes around 120 hours on 4 NVIDIA A100-80GB GPUs.
Visualization training data
Stereotype deseription of cach image
politician, wale, race....
Prohably distriburion
images' stereotype
[politician always white man.], M-SA]
each image
poor people, male, black racc....
images
[poor people always black race]], [M-O & S-SA]
Figure 6: Training data annotation visualization.
Visualization training loss
12-
$3-
85-
85.
DIEIA R
IIC 18c 1ikccck
10c
15c c 1inck
lik
19k
01533 de
Figure The PIS CLIP training loss. From left right, the curves represent Iloss (<prompt. image>), Tioes (<image, stereotypes>), Stose (<prompt, stereotype>). and the total loss (total loss (4oss Tloss + 3/o)s/3)) as described in Algorithm
C.2 Evaluation and comparison experiments
As shown in Table we set the parameters different T2I diffusion models and generated images prompts in Appendix E For each T2I diffusion model, set 10 prompts for association/non-
17

---

association-engendee stereotypes and generated 100 images for each prompt. We calculate the SDTV value of each T2I model based on 10 100 1000 images. In Tables 7 the "XXX/XX" represents the average value of 1000 images, and "+-XX/+ XX" denotes the error bounds. Addition- ally, Tables 2 [ illustrate that the greater the stability of the T21 diffusion model, the smaller the emror bounds. All evaluation experiments are performed on single NVIDIA A A100-80GB GPU.
Table 9: Parameter settings for images generated by different T21 diffusion pipelines.
Models
Sample Method
Size/w
CFG Scale
Sample Steps
Batch Count
Batch size
SD-1.5 SD XL
DPM++ 2M Karras
512 512
7.0
28.0
1.10,20,50,100
1,10
DPM+ 2M Karras
512
512
7.0
28.0
1.10,20,50,100
1,10
Lightning Turho
Euler
1024
1024
1.0
4,00
1.10,20,50,100
1,10
Euler
512 512
1.0
4.00
1,10,20,50,
1,10
Cascade
cancesanccstral
1024
1024
4.0
25.0
1,10,20,50,100
1,10
C.3 Visualization of training details
Example:
Prompts: Aghato of dostor on the left and mune on riglt.
Sensitive Transformer
Query:
T,
Tg
To
Mapping prompt to stereotype
Sensitive Transformer
Constraints: (typerdict)
(bypeeebbedking)
Vayue
doctor (genderf female. racej black... I [gnner (aa.e.ffmaalllggag racef white black...).....
Sensitive
Eacader
Sensitive
Constraints
Corstraints
Figure 8: The process of transferring prompts sensitive constraints. In practice, there is separate "Constraints" part in sensitive transformer. The direct output of the sensitive transformer is in the form of embeddings. The dictionary format illustration is included here only explain the nature of sensitive constraints.
Random image distribution tensor
Sensitive Constraints Embedding
Image Decoder
INPUT
Guodie
Stenattypo-fize
Diffusion Process
images
Prompts Embedding
Figure 9: Embed the sensitive constraint into the diffusion process. Concatenate the original prompt embedding with the sensitive constraints embedding to construct the new input for the diffusion.
18

---

D
Sensitive attribute values
Objects
Adjectives
Sensitive Attributes (SA)
Gender
Region
Religion
Occupations: [doctorr, nurse, police, boss, nctress, professor, farmer, teacher, student, athlete, builder, senator, vialinist, cooker, ticket taker, baker, barber actor, architett. author, acconntat.Bus butcher, cleaner, computer programmerr criminal, consultant, counselor, CEO. carpenter, dancer, dentist, banker, cconomist, eloctrician, engincer, electrician, flrrsst hairdresser, historian, joumalist, lawyer, librurian, magician, anist, mathematician, footballer, biologist, mechanic, musician, nanny, agent, painter, pastry chef, pedistricion. photographer, plumber, politician, psychologist, receptionist, researcher, sailor, salesperson, surveyor, greengrocer, singer, social worker, sollware developer, president, statistician, surgon. tour guide, translator, governor, waiter, writer, zoologist, biologist, clerk, rogue, firefighter, interpreter, manager, nutritionist, anesthetist, phannacist physicist, pilot, reporter, scientist, poet. narrator, cosmetologist, machine operator, inventor, machinery mechanic, prometer, intetior designer, blaster, custodian, sergeant, executive assistant, anchor, administratur, cardiologist, police officer, therapist, housekerping cleuner. judge, secondary school teacher, gencral internal medicine physician, managing director, markering munager, mechanic, model. printer, secrelary, singer, soldier, tennis player. TV presenter. waitress, detective, fisherman, hunter, acrobat, worker, author, busine nesinessman, surgeon, electrical and electronics repairer, food cooking machine operator, fence erector. manufacture factrred building and mobile home installer. sewer, behavioral disorder counselor, correspondence clerk. construction related worker, legal secretary. marriuge and family therapist, machine off hearer, gas compressor and gas pumping station operator, veterinary techniclan, track switch repairer, influencer. surveying and mapping tochnician, associate dean, public safety te relcommmmicator, accountants and auditor, farm equipmeett service technician, ussociate professor, physical science technician, newscaster, architect. housewife, translator, interpreter, bell boy, boxer, typi st, apprentice, meteomlogist, archucologist] Environments: (car, house, clothes, work space, family, city, street, downtown, alleyway. lawn, bench, skyseraper, park, main street, magnificent architecture, cathedral, castle, turret, bridge, ruins, remains, rubble ruins, pasture. plateau, garden. fields, flower ocean, cafe, castle. concert, stage, harbor, casino, dungoon, grave, anditorium]
beautiful
male
white
Eastern Africa
Christianity
ugly
female
Middle Africa
Islam
rich
buy
yellow
Northern Africa
Judaism
poor
girl
browa
Southern
Hinduism
excellent
man
Western Africa
Buddhism
Carihhean
Atheist
big
unknown
Central America
Agnostic
small
South America
elegant
glanoroos
Central Asia
handsome
Eastern
quaint
Southern Asin
adorahle
South-Fastem Fastem Asia
powerful
Western Asia
famous
Eastern Europe
gifted
Northern Europe
shy
Snuthern Furope
clunsy
Europe
lazy
Ausiralia & New Zealand
mysterious
Mclanesia
repulsive
Micronesia
wretched
Polynesia
hrave
faithful
genile
happy
lively
silly
young
old
teenage
adolescent
youthful
elderly senior prime
middle-aged
E
The template of prompts
In this paper, we use the following prompt template for image generation to evaluate the effectiveness of the MAS method in mitigating stereotypes:
Prompt templates
photo of [Object].
photo a [SA][Object].
photo of [Adjjctivellobjeet]]
[Occupation] in [Environment].
[Adjective]]Occupattoo]] in [Environment].
photo of a [SA][Object] in [Environment].
photo of [Adjective ]][Object 1] and [Adjective 2][Object 2].
19

---

Generating Images by Non-template Prompts
Prompt: "(doctor: 1.2) s.2),.hort foott vew.PPholoograpiccstttdnnnnniiii smile,,e quality.asteriiee.rlliiiie
Original SD-1 model without stereccype mitigation.
SD-1.5 model after stcrontype miligution
LOL
Original sterontype miligationn
steroceppe mitigotion.
alencotype mikigyttoo
Turbo model after sterostype minigation.
Original Lightning model without sterentype mitiationn
Lightning model after stareviype mitigation.
Original Cascade modal without stereatype mirigation.
Cascade medel after stareotype mitigation.
Figure 10: Images generated with non-template prompts, mitigating non-associatiooneeggeeeeedd stereotypes
20

---

Prompts: two peoples, excellent teacher, common teacher,classroom. best qusuuutty.mstrppice.stutmaa Original stereotype mitigation.
sicreutype mntigation.
Original SD XL madel without sterectype mitigation.
SD XL model aña streorype mitigation.
Original Turbo model without stereotype mitigation.
Turbo model afher pntpiattemiiigiiinn
Original Lightning model withour stenestye mitigationn
Lightning model after stereotype mitigation.
Original Cascade model without stereotype mitigation.
Cascade model after stereotype misigation.
Figure 11: Images generated with non-tem template prompts, mitigating non- an-aasoccaationeengndeeeeed stereotypes
21

---

G Details for Stereotype Mitigation Experiments in More T21 Scenarios
G.1 Experiment setting
In this experiment, we utilize three models. Realistic, retrained stable diffusion model known for its realistic style, has been downloaded over 1 million times on Civitaf We use the LoRA model Add More Details to enhance the details in the images generated by Realistic. Additionally, we utilize the ControlNet model to adjust the image structure. the evaluation stage, we calculate the SDTV values in different T21 scenarios with multi-model combinations using the same experiment settings and evaluation methods as Appendix C.2
Negative prompts.
(worst quality, low quality: A)). letterbox (body suit: (worst quality, low
Iuality:1.4)..eeepp field, blurry:1.2). (greesysle, monochrome: (.t.meee............... ptt.:ccroopee..leeeseee.... artifacts, siifacts.signature, watermarkkusername,, username, blurry,artist name, trademara watermark, tiniermmar,,itteemmuutee millm. view. Reference pirett,crry.pllupp.. musculr semale.strabismus.lrr greast, handdneg
ControlNet images.
G.2
Examples of images
Prompts: best quiigyassttmpeee..eiistt...btbta vetw violinist, (female: 1.2), solo, standing,
<loramore details:0.67>>
Prompts: best rewpasterppecceeeaasiiiiiiit. front view,concert, vfolinist,ffemale:113 1.2), solo,standing. <lora:more details:0.67>>; ControlNet parameters: ControlNet Model: control v11p_sd15 5_openpos;; control 1eight:1.0; strating control step: 0.1, ending control step: 0.7; control model: balanced; resize model: just resize.
Figure 12: The stereotype mitigation effectiveness of MAS when simultaneously using LoRA and ControlNet for image control in this scenario. On the left are the original T2I- 721-generated images with stereotypes, and on the right are the images after stereotype mitigation.
'htpss//civitai.cco hhtps://iiitaacooommoe/////22
22

---

Prompts: two peoples,a manager, secrefary.compnn..ees moalittymasterpiies. http:ieeesttadinngeeeeee realistic,
reaaistic,<lorammoon <lorammore details:0.675
details:0.66 ControlNet parameters: ControlNet Model: control vllp sd15 openpose; control weight: 1.0; strating control step: 0.1. ending control step: 0.7; control model: balanced; resize model: just resize.
Prompts:
two peoples,a manager, secrelary.companybbes
Figure 13: The stereotype mitigation effectiveness MAS when simultaneously using LoRA and ControlNet for image control in this scenario. On the left are the original 121-generated images with stereotypes, and on the right are the images after stereotype mitigation.
23

---

H
Images
Prompt: "A photo of a white/black man and his house." model after sterastppe mitigation.
Original model without stenosepp minigstion..
Original steroctype mitigation.
SD 1.5 model after sterccoype mitigation.
Stable Cascade model without sterestype miligatioon.
Stable Cascade model sterzotype mitigation.
B FI
Original SD XL Lightning model without stereotype mitigation.
Lightning after sterectype
18
Original SD XI. Turbo model without stareotype miligation.
SD XI. Turbo model after stereotype mitigation.
Figure 14: Prompt: "A photo of white/black man and his house."

---

Prompt: "A photo of poor people and a rich people." SD XL model stasentype matagation.
Original SD XL model without mitigarion.
SI.-15 model without stevootype mitggtion.
SD-1.5 sterecypp miligation.
Original Stable Cascade model without sizreotype minggaton.
Stabie Caaccade model after stereatype mitigatioon
Original Lightning sterontype miligation.
Lighting sarcceype mitigation.
Original withont stenostype mitigation.
stereotype mitigation.
Figure 15: Prompt: "A photo of poor people and rich people."
25

---

Prompt: "A photo of a doctor on the left and a nurse on the right."
model without stemootype mitigation.
steceatype mitigation.
Original SD-1.5 model witheut starootype mitigation.
SD-1.5 model after stereotype mitigation
22
Original Stable Ciscade without stenottpp miligation.
model stescntype miligationn
Original Lightning
sterootype mit igptionn.
Lightning steroocype mitigation.
Original SD XL. Turbo model without sterewtype mitigattion.
SD XL Turbo model after sterestype mitigation.
Figure 16: Prompt:" A photo of doctor the left and nurse the right."
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction accurately reflect the contributions and scope of the paper. In Sections 3 and 4 we provide detailed theoretical explanations and comprehen- sive experimental validation of the contributions proposed in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the paper in section S
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
27

---

Answer: [Yes]
Justification: We provide detailed and sufficient reference sources and proof processes for the mathematical models constructed in the paper in Appendix and B
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversell, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed information for all experiments in the paper support the reproducibility of the results. In Appendix C D and G we report comprehensive experimental parameters, datasets, computational resources, prompts, and other relevant information.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
28

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the core code of the experiments and access links to the relevant datasets in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://niis.cc.
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
7 //nips.cc/putt wwtpc//pppppg.udeeeeeeeepeeepeee httpa///ggiieee/ooeeuuubiiieees irnooeSubmissionPol111.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed information for all experiments in the paper to support the reproducibility of the results. In Appendix C D and G we report comprehensive experimental parameters, datasets, computational resources, prompts, and other relevant information.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In the paper, we provide the error bounds for all experimental results and report the calculation methods for these results in Appendix [ In Tables 7 "XXXXXX" represents the actual calculated mean, and " X-/+XX/+XXX denotes the error bounds.
Guidelines:
The answer NA means that the paper does not include experiments.
29

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In Appendix C we report the computational resources used for the experiments. The model training experiment is conducted on four NVIDIA A100-80GB GPUs, and all evaluation experiments are conducted on a single NVIDIA A100-80GB GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't it make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics https: https:://tunrtrtr http::://///uiip/// http:////c..////ccccciiieeeeeeee
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification:
All research in this paper complies with the NeurIPS Code of Ethics in every
aspect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
30

---

Justification: In this paper, we discuss the potential societal impacts of our work. Our research effectively mitigates uusociation-engendered stereotypes in T2I, thereby reducing the generation of biased content by generative AI. This provides a strong assurance for the safe development of generative Al.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: To reduce the potential risks of T21 diffusion models generating unsafe images, we report the models, parameter settings, prompts, and other relevant configurations used in our experiments in Appendix C D and G These configurations help ensure the generation of safe images and reduce the risk producing unsafe content.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All data, models, and baselines used in this paper are open source. We provide citations or footnotes for all sources throughout the paper.
31

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the open-source code the supplemental material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper did not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
32

---

Justification:
The paper did not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
33


Reviews:
[{"review_id": "2GxiR9R0S0", "reviewer": "Reviewer_QwVE", "summary": "The paper proposes a framework to detect and mitigate stereotype association in Text-to-Image models. They conduct extensive experiments to demonstrate the usability of the framework.", "questions": "- Since the default representation of the images can also be biased, did the authors study how much of the bias stems from the isolated representations of identity groups and objects vs the two combined? \n- Can authors provide more details on the dataset used for training and evaluation and the grounding used for evaluating the presence of stereotypes before and after mitigation?", "limitations": "Missing limitations and discussion section.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ The authors aim to assess \"association-engendered\" stereotypes in T2I models. They model the stereotype mitigation problem as a probability distribution alignment problem and propose MAS framework to mitigate the association-engendered stereotypes.\n+ The authors conduct extensive experiments to demonstrate the effectiveness of the framework.", "weaknesses": "- I strongly recommend renaming the acronym for the Text-Image-Text CLIP model. The current acronym 'TIT-CLIP' is inappropriate, particularly given the project's focus on detecting stereotypes in Text-to-Image models. A more suitable name that reflects the project's serious and sensitive nature would be advisable. \n\n- The claim that the default representation of different identity groups (e.g., 'black' and 'white' people) is not stereotypical is false (Lines 8-10, Figure 1). Research has demonstrated that the default representations of identities are indeed stereotypical, and default representations of objects can also exhibit stereotypical characteristics [1, 2, 3, 4].\n\n[1] Easily Accessible Text-to Image Generation Amplifies Demographic Stereotypes at Large Scale. Bianchi et al. 2023 \\\n[2] ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generation, Jha et al., 2024 \\\n[3] Stable bias: Evaluating Societal Representations in Diffusion Models, Luccioni et al. 2023 \\\n[4] AI’s Regimes of Representation: A Community-Centered Study of Text-to-Image Models in South Asia. Qadri et al., 2023 \\", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses**\n\n- **W1(the acronym issue):** We sincerely appreciate your constructive suggestions. The original acronym, Text-Image-Text CLIP, was chosen from an encoding perspective, focusing on the encoding of Text (prompt, stereotype description) and Image. We agree that this name may not be precise enough and does not fully reflect the project's theme. We will rename it to **Prompt-Image-Stereotype CLIP**. This new name better describes the objects that CLIP encodes and highlights the key aspects of the prompt and the stereotypes manifested in the generated images, making it more aligned with the project's focus than TIT-CLIP.\n- **W2 (about the default representation stereotype): We did not deny that default representations of identity are stereotypes**; therefore, we use the words \"may not\" in lines 8-10. Our focus in lines 8-10 and Figure 1 is on the emergence of new and subtle stereotypes when the sensitive attribute of race is associated with the object of a house, rather than on the stereotypes inherent in the representations themselves.\n\n**Questions**\n\n- **Q1 (the default stereotype issue):**  As stated in W2, the primary purpose of our paper is to demonstrate that the stereotype can be engendered by the association of two objects when these two objects individually are not stereotypical. **This association-engendered stereotype is not a simple inheritance or addition of individual default representational stereotypes but is based on the interaction between the two, engendering a new stereotype.** \n- **Q2 (the dataset issue):** We have provided a detailed description of the experimental datasets in Table 8 of Appendix C.1 of the paper. Most of the data comes from publicly available datasets, which can be accessed via the references provided in the paper. \n\n**Missing Limitations** \n\n- In lines 328-334 of the paper, we already discussed the limitations of our method and the potential social impacts.  We would extend that in the revised paper if that were not enough.\n\nOverall, we will carefully revise our paper based on these valuable comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vIx1JbQcBW", "reviewer": "Reviewer_ioQ9", "summary": "This paper presented the first step to mitigate association-engendered stereotypes in Text-to-Image (T2I) diffusion models. A probability distribution alignment problem was first formulated, and then a probability distribution model was constructed for non-association-engendered and association-engendered stereotypes. This paper further presented a MAS framework, which consists of the Text-Image-Text CLIP (TIT CLIP) and Sensitive Transformer. Comprehensive experiments demonstrated that the MAS framework is an effective mitigation approach for association-engendered stereotypes in T2I.", "questions": "Q1: In line 167, for the Algorithm 1, what is the output?  Please clarify. \n\nQ2: More output examples of the proposed framework MAS and baseline methods should be shown to illustrate the mitigation effects.\n\nQ3: In Appendix B2 Figure 5(a), what does \"extreme\" mean in this context?", "limitations": "- Please clarify the \"stereotype-free distribution\", which is not very clear in this paper. \n\n- While the paper provides an Appendix for detailed experimental settings, some implementation details may be needed in Section 4.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "+  A novel and important research problem that tackles the stereotypes engendered by the association of multiple objects, which was ignored by previous work. This research adds critical insights into the stereotype mitigation area.  \n\n+ A neat framework MAS was proposed to mitigate such association-engendered stereotypes. MAS innovatively models the stereotype problem as a probability distribution alignment problem, which means aligning the stereotype probability distribution of the generated image with the stereotype-free distribution.  \n\n+ Two effective components, TIT CLIP and Sensitive Transformer, were proposed to enable MAS.  The framework MAS learns the mapping of prompts, images, and stereotypes via the TIT CLIP and constructs sensitive constraints via the Sensitive Transformer to guide the T2I diffusion model in generating stereotype-free images by embedding these sensitive constraints into the T2I diffusion process.\n\n+ A novel metric, Stereotype-Distribution-Total-Variation (SDTV), was introduced to evaluate association-engendered stereotypes accurately due to the insufficiency of existing metrics. \n\n+ Extensive experiments were conducted, supported by the 13 Appendix pages.", "weaknesses": "- In line 167, for the Algorithm 1, what is the output?  Please clarify. \n\n- More output examples of the proposed framework MAS and baseline methods should be shown to illustrate the mitigation effects.\n\n- While the paper provides an Appendix for detailed experimental settings, some implementation details may be needed in Section 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses**\n\n- **W1 (the algorithm issue):** The output of Algorithm 1 is the embedding of prompt, image, and stereotype.\n- **W2 (about the suggestions of output examples):** We sincerely appreciate your constructive suggestions. We will add examples of outputs from MAS and other baselines to the paper to demonstrate the mitigation effects alongside the results reported in Table 3.\n- **W3 (the experimental settings):** We describe the main pipeline and key evaluation methods in Section 3. Due to the paper's page limit, we have to place the detailed experimental settings in the appendix.\n\n**Questions**\n\n- **Q1:** Please see **W1**.\n- **Q2:** Please see **W2**.\n\n- **Q3 (the definition of extreme): Extreme** refers to a scenario in a sensitive attribute dimension where the occurrence probability of a specific sensitive attribute value significantly exceeds the combined occurrence probability of other attribute values. We refer to this specific sensitive attribute value as an **extreme attribute value**. For instance, when using \"*an image of a beautiful woman*\" as a prompt and generating some images, considering the woman's race, 90% of the images may depict the white race, while other races only account for the remaining 10%.\n\n**Clarifications**\n\n- As mentioned in lines 132-136 of the paper, stereotype-free means that a T2I model should generate images with equal probability across different sensitive attribute values, avoiding significant disparities in probability distribution caused by the dependence of sensitive attributes on the object being generated. **Stereotype-free distribution** means that the T2I model generates images with equal probability for each sensitive attribute value, resulting in a **probability distribution of sensitive attributes** close to **a uniform distribution**.\n\n- Please see **W3**.\n\nOverall, we will carefully revise our paper based on these valuable comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qTIbTHChH9", "reviewer": "Reviewer_E5YY", "summary": "This paper presents a novel framework (MAS) to address biases in text-to-image (T2I) models. Traditional methods focus on individual object stereotypes but fail to tackle stereotypes arising from object associations. MAS models the stereotype issue as a probability distribution alignment problem, utilizing a Text-Image-Text CLIP (TIT CLIP) and a Sensitive Transformer to align generated image distributions with stereotype-free distributions. The paper also introduces the Stereotype-Distribution-Total-Variation (SDTV) metric for better stereotype evaluation. Experiments show MAS effectively mitigates both single and association-engendered stereotypes.", "questions": "Please see the weaknesses.", "limitations": "Please see the weaknesses.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper addresses a relatively unexplored area in stereotype mitigation for T2I models, focusing on the association of multiple objects rather than individual objects.\n2. The paper is well-structured, with a clear presentation of the problem, methodology, and results. The introduction provides a solid context for the need to address association-engendered stereotypes, and the explanation of the MAS framework is easy to follow.\n3. The paper extends to improving the societal and ethical aspects of AI-generated content, making it a valuable contribution to the field.", "weaknesses": "1. The proposed framework adds considerable complexity to the existing T2I pipeline, which might pose challenges for practical implementation and integration into existing systems, especially for developers with limited resources or technical expertise.\n2.  It is advisable to provide more details on the computational overhead introduced by the Sensitive Transformer within the MAS framework, especially in terms of real-world application performance and scalability for large-scale T2I generation tasks.\n3. I wonder if the MAS framework can be adapted to other generative models beyond text-to-image, such as text-to-video or text-to-audio models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses**\n\n- **W1 (the complexity issue):** As shown in Section 4.2 of the paper, we conducted experiments to evaluate the impact of MAS on the computational load of the T2I diffusion model. Table 6 demonstrates that MAS effectively mitigates stereotypes while maintaining image generation efficiency and quality.\n\n- **W2 (the computational overhead issue):** The time complexity of the Sensitive Transformer is consistent with that of the Transformer. For an input batch size of  $b$, sequence length $N$, and an $l$ -layer Transformer model, the computational complexity is $O(l(bNd^2 + N^2d))$, where $d$  represents the dimension of the word embeddings.\n\n  We conducted MAS tests on our local server. As reported in Table 6 of the paper, the performance is similar to the original T2I model, with a difference of about $20$ seconds over a generation task of $100$ batches with a batch size of $10$. This results in an average efficiency decrease of $0.02$ seconds per image, which is negligible for practical generation tasks. Regarding the scalability of MAS, we applied it to mainstream T2I models for stereotype mitigation. Table 2 demonstrates the effectiveness of our MAS across different stable diffusion pipelines. Additionally, our MAS implements modular integration across various pipelines, allowing stereotype mitigation by simply embedding our MAS module into the original T2I workflow.\n\n- **W3 (the adaptability issue):** Currently, our approach has only been tested for effectiveness in Text-to-Image generation. Though the concept of our approach looks feasible for Text-to-Video and Text-to-Audio generation, honestly speaking, we can not make a concrete answer about its adaptability without further deep exploration.\n\nOverall, we will carefully revise our paper based on these valuable comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x4o7upqODG", "reviewer": "Reviewer_L6vF", "summary": "The authors observe that some stereotypes only appear when an association of objects is required in T2I and propose to address association-engendered stereotypes for the first time. They use a pre-trained Text-Image-Text CLIP to map a prompt to its potential stereotypes and generate sensitive constraints using a Sensitive Transformer. The sensitive constraints are then used with the input prompt to affect T2I generation to be fairer. Besides, a new metric to evaluate association-engendered stereotypes in T2I results is proposed.", "questions": "See Weaknesses for major questions.\n\nOther minor issues include:\n- L41: I think the explanation in brackets is unrelated to its previous statement. Stereotype mitigation on a single object is different from stereotype mitigation on a single sensitive attribute.\n- Why there are [SA] indications in some prompt templates? If the SA is given in the prompt, the evaluation of stereotype mitigation across different SA classes is rather trivial.\n- How are the sensitive constraints incorporated into the prompt input? Maybe the authors could consider redrawing Figure 2 (3) for a clearer illustration.", "limitations": "The authors fail to provide a discussion on limitations and potential societal impact.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors make direct observations of association-engendered stereotypes in T2I results. From these observations, the motivation to address association-engendered stereotypes is validated.\n- The idea of learning mappings from the prompts to their corresponding stereotypes is interesting and the overall framework is a novel design.\n- The novelly-proposed evaluation metric for association-engendered stereotypes with extensive proof and discussion is valuable.\n- Extensive experiments are conducted with structural evaluations, proving the effectiveness of the method.", "weaknesses": "- I am wondering why the TIT CLIP can learn the mapping from prompts to stereotypes by maximizing the cosine similarity of three pairs. It is intuitively understandable that the original 2-dimensional CLIP uses contrastive learning to associate visual appearance and textual description of a single image. However, in this case, the stereotypes are something unrevealed in a single image but spotted by a statistical distribution over a set of images. The validity of pulling <prompt, stereotype> pair closer should be further explained. Besides, it would help the reader to understand the concept better if the authors could provide some examples of the stereotype descriptions used for training the TIT CLIP.\n- Technical details of training the Sensitive Transformer is missing. Is it trained together with the diffusion model with the distribution alignment guidance or what is the training objective of it?\n- How does the Sensitive Transformer handle engendered stereotypes across multiple attributes? Is it achieved implicitly or explicitly? Please provide further explanation.\n- How to get the probability distribution for evaluation is unclear. Does it involve using pre-trained sensitive attribute classifiers like in previous works? If so, I am wondering how accurate the classifiers are when there are multiple objects in a given image.\n- It would be beneficial to provide ablation studies to showcase the effectiveness of each proposed component. For example, I am curious about how a simple MLP would work in mapping the prompts to their stereotypes, compared to the TIT CLIP.\n- It is visually hard to tell that the race bias is reduced in Figure 3(b), leading to concerns that the method cannot effectively mitigate biases that are strongly bonded to certain races. \n- Examples after stereotype mitigation in G.2 show violation to prompt requirement \"female\", leading to concerns that the method might overlook the prompt input. \n- One can hardly tell which person is \"poor\" from some examples in Figure 12. It raises concerns about how to distinguish harmful stereotypes (like relating poverty to certain races) from desirable visual concepts (like relating poverty to affordable clothes) during generations. \n- The authors do not provide a discussion on the limitation of the method and its societal impact although claim to do so in the checklist. \n- The stereotype mitigation target is even distribution. How can the model adapt to certain circumstances where not all attribute classes are correct? For example, the model should not generate Asian figures when asked for German soldiers in WW2.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses (the first part)**\n\n- **W1 (TIT CLIP mapping issue):** In fact, the association-engendered stereotype in T2I can only manifest in the generated images. Therefore, we need to use images as a medium to connect the relationship between prompts and stereotypes. In our task, we consider each image's stereotype to focus on the description of the object and its sensitive attributes. Thus, each image includes two descriptions: the original 2D CLIP description of the image's meaning and the stereotype-focused description that concentrates on the object and its sensitive attributes. **As shown in Figure 1 of `supplementary_images.pdf`**, although the stereotypes in the paper are derived from probabilistic statistics, we annotate the stereotypes of the images through probabilistic descriptions rather than using the probability model for prompt-image-stereotype association mapping. Finally, in Figure 1 of **`supplementary_images.pdf`**, we provide examples of stereotype descriptions.\n\n  > Training data type:    <prompt, image, stereotype description>\n\n- **W2 (technical details):** Our Sensitive Transformer is based on Transformer[1]. As shown in Equation (5) of the paper, $\\text{Sensitive Matrix}(V) = \\text{softmax}\\left(\\frac{QK^T}{\\sqrt{d_k}}\\right)V$, we changed the Transformer's task from translation to generating sensitive constraints. \n\n  **The Sensitive Transformer is trained together with the diffusion model under distribution alignment guidance**.  The training objective of the Sensitive Transformer is to generate sensitive constraints that align with the given prompts. Additionally, to generate the optimal sensitive constraints for the prompt, we optimized the generated sensitive constraints. As shown in Equation (6), $\\sigma^* = \\mathop{\\arg\\min}\\limits_{\\sigma \\subseteq S_{\\mathcal{Y}}} \\sup |\\sigma(p_x^{v(s)}) - p_x^{u(s)}|$, the optimization objective is to minimize the total variation distance between $\\sigma(p_x^{v(s)})$ and $p_x^{u(s)}$.\n\n  > [1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 2017, 30.\n\n- **W3 (the issue of handling multiple attributes):** **As shown in Figure 2 of `supplementary_images.pdf`**, the Sensitive Transformer generates sensitive constraints for each object in the original prompt using the prompt-image-stereotype embedding obtained from TIT CLIP. These sensitive attribute values are generated in the form of a dictionary. If the original prompt already specifies certain sensitive attributes, they will not appear in the sensitive constraints dictionary. **By simultaneously constraining multiple sensitive attributes, we suppress the association-engendered stereotypes**, achieving this in an **implicit** manner.\n\n- **W4 (the evaluation issue):** As mentioned in lines 219-221 of the paper, we provide a brief description of the evaluation of SDTV values presented in Table 2 and Table 3. The detailed calculation method for SDTV values, including single and multiple objects and sensitive attributes, is provided in Appendix B. Given the potential inaccuracy of classifiers, we evaluate the SDTV values of these images using mathematical statistical methods rather than relying on classifiers.\n\n- **W5 (the MLP suggestion):** Initially, we had the same idea as the reviewer: to use MLP to map prompts to their stereotypes. However, **as shown in Figure 1 of `supplementary_images.pdf`**, given our training data, if we skip the images and only use prompts and stereotype descriptions, we encounter the problem where the same prompt corresponds to multiple different stereotype descriptions, as illustrated in the table below. \n\n  | prompts                          | stereotype description                                       |\n  | -------------------------------- | ------------------------------------------------------------ |\n  | a photo of a doctor and a nurse. | doctor, nurse, male, female,...\\[doctor always male, ...\\][M-O & M-SA] |\n  | a photo of a doctor and a nurse. | doctor, nurse, female, female,...\\[doctor always male, ...\\][M-O & M-SA] |\n  | a photo of a doctor and a nurse. | doctor, nurse, male, male,...\\[doctor always male, ...\\][M-O & M-SA] |\n  | ...                              | ...                                                          |\n\n  This problem causes the MLP's loss to remain constant, preventing it from learning effective features. By encoding the image into the embedding, we can address this issue while preserving the semantic relationship between the prompt and the image, which is the main advantage of TIT CLIP over MLP. \n\n- **W6 (about Figure 3):** In Figure 3(b), the left image of the terrorist predominantly displays Middle Eastern facial features in terms of the sensitive dimension of race. After mitigation, the Middle Eastern features are covered by a mask, and the generated image then focuses on the terrorist's inherent features rather than displaying obvious racial characteristics.\n\n---\n\n**`Note:`** Due to the word limit imposed on each rebuttal by NeurIPS, we have included our responses to **W7-W10** and the replies to the \"**Questions**\" in the **Author Rebuttal** cell. We **kindly request the Reviewer L6vF to switch to the Author Rebuttal cell to view the second part of our detailed responses**. We sincerely appreciate your review.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**Dear Chairs and Reviewers,**

​	We kindly thank all the reviewers for their time and for providing valuable feedback on our work. 

​	In response to the reviewers, we have added the **`supplementary_images.pdf`** file. This file contains annotations and explanations for the data used in our experiments and provides a more detailed description of the Sensitive Transformer. 

Kind regards,

The authors

------
------

**Response to reviewer L6vF's questions (the second part):**

**Weaknesses**
- **W7 (about the Appendix G.2):** This issue arises because the "female" position in the prompts is towards the end. According to the stable diffusion encoding rules, the weight of "female" is reduced, leading to a loss of the word's features. We retested this issue, and as shown in Figure 4 of supplementary_images.pdf, adjusting the prompt weights resolves the problem. Additionally, regarding the reviewer's concerns about semantic preservation problems, we conducted semantic preservation experiments in Section 4.2 of the paper. Table 4 indicates that our MAS maintains a similar level of semantic preservation to the original T2I model.

- **W8 (about Figure 12):** In Figure 12, due to the training data of the diffusion model, the disparity between rich and poor individuals is not always very pronounced in some examples. However, an apparent phenomenon is that images of both poor and wealthy individuals predominantly depict Black and White men. This phenomenon may engender harmful stereotypes, as it suggests that being poor or wealthy is always associated with either the Black or White races. The stereotype-free T2I diffusion model should distribute the probabilities equally across all races for both poor and wealthy individuals. A wealthy person could be White, Black, Asian, etc.; similarly, a poor person should also be represented equally across all races, not just limited to White and Black men.
- **W9  (missing limitations):** In lines 328-334 of the paper, we already discussed the limitations of our method and the potential social impacts.  We would extend that in the revised paper if that were not enough.
- **W10  (the mitigation target issue):** When the prompt explicitly includes specific sensitive attributes, MAS prioritizes those specified in the prompt. As we explained in **W3**, if the original prompts already specify certain sensitive attributes, they will not appear in the sensitive constraints dictionary. For example, when generating an image of a World War II German soldier, the weight of the "German" attribute in the prompts will be significantly higher than the sensitive constraints. This ensures the model does not produce unrealistic results, such as a German soldier with Asian features.

**Questions**

- **Q1  (the Line 41 issue):** The "single object" refers to Non-Association-Engendered stereotypes. As stated in Appendix A.2, Non-Association-Engendered stereotypes include single object and single/multiple attribute cases, and the parenthetical statement aligns with Non-Association-Engendered stereotypes. We apologize for any confusion our previous wording may have caused. We will revise the sentence as follows:

  > Previous works on stereotype mitigation have been limited to a single object, referred to as *Non-Association-Engendered stereotypes* (e.g., only mitigating the stereotype problem in the occupation or gender dimension), and cannot effectively address stereotypes involving the association of multiple objects, referred to as *Association-Engendered Stereotypes*. 

- **Q2 ([SA] issues in prompt word templates):** In some scenarios, the prompt already includes some sensitive attributes (SA). In such cases, we must ensure that the SA specified in the prompt remains semantically aligned while mitigating stereotypes for the unspecified SA. For example, in the prompt "a female nurse," the SA of "female" is already included. We need to maintain the "female" SA unchanged while mitigating stereotypes related to other sensitive attributes, such as the race and age of the female nurse. Therefore, when setting up the prompt template, it is crucial to consider cases where specific SA is already known.

- **Q3 (the sensitive constraints incorporated issue):** In the **`supplementary_images.pdf`**, we supplement the embedding process of the sensitive constraint in T2I, as shown in Figure 3. Using the `np.concatenate` function, we concatenate the original prompt embedding with the generated sensitive constraints embedding to form the input for the diffusion process.

**Missing Limitations** 

- In lines 328-334 of the paper, we already discussed the limitations of our method and the potential social impacts. We would extend that in the revised paper if that were not enough.

Overall, we will carefully revise our paper based on these valuable comments.

Meta-review: The paper studies the problem of association biases in text to image generation that occurs when the text prompt includes both people and objects. All of the reviewers found the paper proposed a novel method for an important problem and the evaluation was solid.
Decision: Accept (spotlight)