FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation
Authors: Christopher T.H Teo
Milad Abdollahzadeh
Xinda Ma
Ngai-man Cheung
Venue: NeurIPS 2024 poster (2024)
Keywords: fairness
generative modelling
text-to-image models
bias mitigation
prompt learning
Primary area: fairness

Abstract: Recently, prompt learning has emerged as the state-of-the-art (SOTA) for fair text-to-image (T2I) generation. Specifically, this approach leverages readily available reference images to learn inclusive prompts for each target Sensitive Attribute (tSA), allowing for fair image generation. In this work, we first reveal that this prompt learning-based approach results in degraded sample quality. Our analysis shows that the approach's training objective--which aims to align the embedding differences of learned prompts and reference images-- could be sub-optimal, resulting in distortion of the learned prompts and degraded generated images. To further substantiate this claim, **as our major contribution**, we deep dive into the denoising subnetwork of the T2I model to track down the effect of these learned prompts by analyzing the cross-attention maps. In our analysis, we propose a novel prompt switching analysis: I2H and H2I. Furthermore, we propose new quantitative characterization of cross-attention maps. Our analysis reveals abnormalities in the early denoising steps, perpetuating improper global structure that results in degradation in the generated samples. Building on insights from our analysis, we propose two ideas: (i) *Prompt Queuing* and (ii) *Attention Amplification* to address the quality issue. Extensive experimental results on a wide range of tSAs show that our proposed method outperforms SOTA approach's image generation quality, while achieving competitive fairness. More resources at FairQueue Project site: https://sutd-visual-computing-group.github.io/FairQueue

FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation
Christopher T. Teo"
Milad Abdollahzadeh mi bilad_ebbbllahhadddddddd...
prrristopher_teommmmalll sutd. edu. sg
Xinda Ma
Ngai-Man Cheung! ngaiman__ n cheung@sutd, edu.sg
xinda_ma@sutd, edu.
Singapore University of Technology and Design (SUTD)
Abstract
Recently, prompt learning has emerged as the state-of-the-art (SOTA) for fair text- to-image (T2I) generation. Specifically, this approach leverages readily available reference images to learn inclusive prompts for each target Sensitive Attribute (tSA), allowing for fair image generation. In this work, first reveal that this prompt learning- baasing-based approach results in degraded sample quality. Our analysis shows that the approach's training objective-whit which aims to align the embedding differences of learned prompts and reference images could be sub-optimal, resulting in distortion of the learned prompts and degraded generated images.
To further substantiate this claim, as our major contribution, we deep dive into the denoising subnetwork of the T2I model to track down the effect of these learned prompts by analyzing the cross-attention maps. In our analysis, we propose novel prompt switching analysis: I2H and H2I. Furthermore, we propose characterization cross-attention maps. Our analysis reveals abnormalities the early denoising steps, perpetuating improper global structure that results in degradation in the generated samples. Building on insights from our analysis, we propose ideas: (i) Prompt Queaing and (ii) Attention Amplification to address the quality issue. Extensive experimental results on a wide range tSAs show that our proposed method outperforms SOTA approach's 's image generation quality, while achieving competitive fairness. More resources at Project Page.
quantitative
1 Introduction
There has been significant progress in the quality of text-to-image (T2I) generation [1-3] resulting in increasing adoption in different applications [4-10]. With this comes concerns regarding the fairness of these T2I models and their societal impacts [11-15].
Fair T2I Generation. T2I models may inherit biases present in their training data, Several approaches have been proposed to mitigate these biases [16-19] (See related work Supp). Particularly, Inclusive T2I Generation (ITI-GEN) [16]-the existing SOTA-suggests that fair T2I approaches based on hard prompts (HP) (e.g., "A headshot a person with fair skin tone") are limited by linguistic ambiguity. For example, Skin Tone is often challenging to define and interpret based on HP, resulting sub-optimal performance. To overcome this linguistic ambiguity, ITI-GEN adopts the notion that "a picture is worth thousand words" and leverages readily available reference
"Equal Contribution
i Corresponding Author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

ITI-GEN Degraded Samples
Reference Dataset
Directional Alignment
Smiting)
E
AP
ISA Smiling
AP
Observation
CLIP
Py
Space
A1
Egd5
Maal
0.05
Young
0.01
0.47
Smiling
1, Voe
Cheezboose
Qualitative Analysis (Attention Maps)
Quantitative Analysis (Attention Maps)
Prompt Switching Analysis
HP
12H:
Supp DMZZ, R...s.r (0a-1)
(b) Analysis
DIZ.)
nor
"beamiingg
<0utgut>
0:40
144
& 5.10
6 0.75
ITI-GEN
"s"
1121:
JOWZ, Ef Po..P.....
622
8.4
Amplinate
"o."
<0etputo
ISA Smillng
(c) Proposed Solution
Prompt Queuing
Attention Amplification
Results
IMIZ, T (Qun
201.1
36-3
Figure 1: Our work re-visits SOTA fair T2I generation, ITI-GEN. We question ITIIGEN's central idea of prompt learning via alignment between the directions of prompt embeddings and reference image embeddings. (a) We observe degradation in images generated through ITI-GEN's learned prompts. We note that the direction reference image embeddings could include unrelated concepts beyond tSA differences (e.g., variations in accessories) resulting in learning of distorted prompts using ITI-GEN. Furthermore, we observe misalignment between the direction credible hard prompts and that of reference images/learned prompts. (b) As main contribution and to further understand how these distorted prompts affect the image generation process, we deep dive into the denoising network and analyze the coss-attention maps, revealing their abnormalities e.g., higher activity for maps associated with non-tSA tokens ('of"", "a'"). We examine the degraded global structures resulting from these distorted prompts in the early denoising steps. Moreover, propose I2H and H2I (Bq.2) analysis to understand impact of these degraded global structures and abnormalities in later denoising steps. In addition, propose metries (Eq.3) cross-attention maps quantify these abnormalities. (c) Building on insights from our analysis, we propose a solution to address distorted prompts while maintaining competitive fairnsss. Our solution FairQueue includes attention amplification. Frr and F, are CL.IP text and image encoder resp. [20]. T, F, are the base prompt, hard prompt with minimal linguistic ambiguity, and ITI-GEN prompt, resp.
ideas: prompt queuing and
images learn an inclusive prompt for each tSA category. This approach translates visual attribute differences present in the reference images into prompt differences, enabling the learned prompts to be used to generate images of all tSA categories, regardless their linguistic ambiguity. Fairness is achieved by uniformly sampling the learned prompts to condition the T2I generation. Central to this approach is the enforcement of directional alignment between learned prompt embeddings and reference image embeddings corresponding
pair of iSA categories.
In this work, we question the central idea of prompt learning via alignment between the direction of prompt embeddings and the direction of reference image embeddings in the context of fair T2I generative models. Our work starts with examining the generated images and observes that moderate amount of degraded images are generated based on ITI-GEN. We argue that using the direction of reference image embeddings as guidance could be sub-optimal, as the difference between reference images could include additional unrelated concepts other than the tSA difference

---

(Fig 1). For example, reference images of "A headshot of a person smiling" and "A headshot of a person not smiling" contain differences in poses, accessories, hairstyling, in addition to the difference in smiling. Therefore, the direction of reference image embeddings could be noisy and include additional unrelated concepts other than the iSA difference. We perform an analysis on the direction of embeddings to further understand the issue. We hypothesize that using the direction of reference image embeddings as guidance could lead to distortion in the learned prompts, resulting in artifacts and quality degradation in the images generated by T21 models.
To further substantiate this claim, as our major contribution, we deep dive into the denoising subnetwork of the T2I model to amalyze ITI-GEN prompts in the generation pipeline. Our analysis include examination of the cross-attention maps of the learned prompts at individual time steps of the denoising process, We propose novel prompt switching analysis: ITI-GEN to HP (I2H), and HP to ITI-GEN (H2T). We further propose new quantitative metries for cross-attention map characterization. Our analysis reveals cross-attention maps of the learned prompts have abnormalities in the initial time steps of the denoising process. This results in synthesizing improper global structures. Interestingly, we find that the learned prompts have a minimum abnormality in the later steps-the learned prompts perform adequately in generating the desired iSA category provided that proper global image structures could be synthesized in the initial denoising steps. To justify our analysis of cross cross-attention attention, we remark that cross-attentin contextualizes prompt embeddings with the latent representation of images and has been shown to play key role in T2I models [21, 22].
Building on the insights of our analysis, we propose a solution address degraded generated images without compromising faimess and diversity. Particularly, we propose Prompt Queuing to apply base prompts (without iSA tokens) in the initial time steps and ITI-GEN learned prompts the later time steps of the denoising process. We further propose Attention Amplification to balance the quality and fairness of the T2I generation. Overall, our solution can effectively address the degraded quality issue in ITI-GEN while maintaining competitive fairness. Our contributions are:
We examine the generated images from the prompt learning- based fair T21 generation approach and reveal a moderate amount of generated images with degradation (Sec 3.
We argue that the direction of reference image embeddings could be noisy and include unrelated concepts in addition to iSA difference, and prompt learning based on alignment with the direction of reference image embeddings could be sub-optimal (Sec 3.1).
We deep dive into the denoising subnetwork of the T2I model and analyze cross-attention maps with our proposed prompt switching analysis 12H and H21, and our proposed quantitative metrics for cross-attention maps. Our analysis reveals and characterizes abnormalitics in cross-attentions of ITI- GEN prompts in the denoising process (Sec 3.2).
We propose FairQueue, a solution based on prompt queuing and attention amplification to improve generation quality while maintaining competitive fairness (Sec 4).
2 Preliminaries
T2I Generation. SOTA T21 generation is based diffusion model (DM) [1-3]. In the forward diffusion process, Gaussian noise is incrementally added to the training data to train the DM. Then, during reverse diffusion, the DM generates samples by randomly sampling latent noise Zo- N NN00 I) as an input. For more control, text-conditioning [1, 23-25] was introduced, where we denote the reverse diffusion (denoising) of single step / by Ze+1 +- (M,,,, R, 1, Here, Z, is the latent of the noisy image, R the input prompt, E [0, the denoising step. and random seed. Central to text conditioning is the cross-attention mechanism which contextualizes prompt embeddings with the image latent [21, 26]. Specifically the cross-attention map M E grxmxn xn whhree r is the number of tokens in the prompt, and m n shows map size for each token-is computed by:
M = SoftMar OKT
(1) where, Q = (o())))) is the linear projection of the latent spatial features d(z)), and K
Va
(E is the linear projection of the textual embedding Er ()) (usually CLIP text encoder [20]). For ease of notation, we refer to the token-specific attention maps as M egg. M('''') E mmx refers to the cross-attention map for the token "of"" in R. As our work focuses on the reverse
diffusion process, we utilize Zo as the noisy latent input and Zt as the final latent output. This Z₁ is then finally passed into the DM decoder to output generated image, D(Z)).
3

---

Fairness in Generative Models. In generative models, fairness is defined as equal representation [27, 28], where for a tSA with K categories, a fair generator will generate an equal number of samples for each category. As an example, for a T21 model G with text prompt "A headshot of a person" as input, we consider G as fair model w.r.t iSA = Young-with two categories {Young, 01d} [29, 27, 28] if generates an equal number of samples for each categories of this tSA [30, 31].
Hard Prompts for Fair T2I Generation. A baseline for achieving fairess in T2I models is to append the tSA-related prompt to the base prompt [17, 19]. Considering the same (SA=Young, and the base prompt "A headshot of a person", adding a tSA-related prompt for each category results in the HPs: "A headshot of person young/ ood'. For fair generation, we query T2I with each of these HPs uniformly. Note that HP although very effective with certain tSA, in most cases, is ineffective due to the tSAs having linguistic ambiguity [32]-having misleading or deceptive language.
Prompt Learning for Fair Lext-to-Image Generation. To resolve the issue of ambiguous tSAs, inspired by the recent success of prompt learning 34), ITI-GEN [16] aims to achieve fairness in a pre-trained T2I model by learning inclusive tokens for each category of the ISA. Assuming the tokenized base prompt as T E Rexd, where p is the number of tokens and d is the dimension of the embedding space, for each category k E {1, K. of ISA, learns q additional tokens. S = [S S ........ 1 Roxd In [16], q is set to 3. Then, ITI-GEN prompt is constructed by appending these learned tokens to the original tokens: Pa = [T; S*1 E ( These tokens are learee using set of labeled reference images (w.F. tSA) Dref { yi ) E {1, K} to provide stronger signals for describing tSA. More specifically, for a pair of categories (i. of tSA, directional loss [35] is used to match the direction of learned prompts and images for this pair in
CLIP embedding [20] space i.e., ming. Ldir = (A (PPs.))
(+,ي0)) where (1())) (AP()))
denotes the direction between images (text prompts) of two categories and in CLIP's embedding space, and directional loss Lair is minimized to learn tSA tokens S 5 for these categories. Finally, using Ps input prompt, fairness is achieved by uniformly sampling the K categories of the tSA. We will omit the category index &o when it is clear from context, and denote learned prompt and tokens by P and : S resp.
3 A Closer Look at Prompt Learning for Fair Text-to-Image Generation
In this section, we take a closer look at ITI-GEN [16]. First, in Sec. 3.1, we analyze ITI-GEN performance where we find quality degradation in moderate number of generated samples. We attribute this to the sub-optimal learning objective in ITI-GEN, which captures unrelated concepts that distort the learned tokens in P. Then, in See. 3.2, we analyze ITI-GEN prompts during sample generation by inspecting the cross-attention mechanism. Our analysis reveals that ITI-GEN prompts give rise to abnormality particularly damaging to the early steps of the denoising process.
To conduct the following analysis on ITI-GEN prompts' behavior we require a strong base- line pseudo-gold standard compare against. To address this, we found that when considering certain tSA with minimal linguistic ambiguity (MLA) [32]-a few iSA that can be described without misleading or deceptive language-HPs can serve as this strong baseline. Therefore, in this section, we focus on tSAs with minimum linguistic ambiguity. Later, in experiment section, we will include all ISAs, with or without ambiguity.
Remark.
3.1 Limitations of Prompt Learning for Fair T2I Generation
Although ITI-GEN [16] improves fairness in T21 generation, a closer examination of its outputs reveals potential trade-off: compromised image quality. In this section, first, we perform a systematic experiment to showcase these quality issues and then explore the potential root causes behind them.
Experimental Setup. To evaluate our generated samples, we utilize the metrics: i) Fairness Dis- crepancy (FD) [27, 31, 11, 36] measure fairness, ii) Text-Alignment (TA) [37, 22] and FID [38] to measure quality, and iii) DreamSim (DS) [39] to measure semantic preservation. Next, we determine a set of tSA with MLA to compare ITI-GEN with HP (as a pseudo- gold standard). Specifically, we follow [16] and use pre-trained Stable Diffusion (SD) [1] as T2I model. Then as mentioned in Sec. 2, for HP, we append the tSA-related prompts to the base prompt. We empirically found that iSAs Samiling, High Cheekbones], J. are unambiguous by classifying 500 generated sample per HP utilizing CLIPP classifier [20], where on average they both achieve a 98% accuracy (Experiment details in Supp). Then, for ITI-GEN [16], we strictly follow [16] and use publicly available fair

---

BP:
BAADOT o Person"
FD(U)
TA(1)
FID(U)
DS(U)
Eutulling
1 Eeadabot of Penaon desting"
*A Nendetttt of Peesco nct Beutting"
8.4e
0.672
79.82
0.323
ITI-GEN:
124e
Ours
Boatshik
69c
0.674
80.02
0.284
X Thautetue of's Pienzn with mag Chowictccss"
Ilexsietne Per Chaeeta Chawiithonse"
3.68c
0.672
80.32
0,332
ITI-GEN:
STR-EDD LEOTTD
318e
0.595
86.42
0.538
Ours:
4.92
0.685
79.73
0.330
Figure 2: T21 generation performance of HP, ITI-GEN [16], and our proposed FairQueue for target Sensitive Attributes (tSAs) with minimal linguistic ambiguities. Samples generated by HP demonstrate outstanding performance with good fairness (FD), high quality (FID and FD), and good semantic preservation (DS). Meanwhile, ITI-GEN moderately degrades sample quality, impacting fairness and semantic preservation. FairQueue demonstrates comparable performance to HP, even surpassing HP in both quality and semantic preservation in many cases. Note that HP only performs well for unambiguous tSAs, and can not be used for general fair T2I generation purposes, as it can not be defined well for ambiguous iSAs (See Supp for detailed discussion).
image sattasetssampld from CelebA [29]-as reference images to learn inclusive tokens, S. Finally, generate and evaluate ITI-GEN samples based on the same latent noise input HP. See Supp for experiment and metric details.
Fig. 2 shows some generated samples together with quantitative results. A moderate number of gen- erated images with ITI-GEN have quality degradation often with unrelated content (e.g., generating dog, multiple degraded faces, vague cartoons, etc.). Quantitative results show that for both tSAs, HP performs better in fairness (lower FD), quality (higher TA, lower FID), and semantic preservation (lower DS). We postulate degraded samples stem from ITI-GEN's sub-optimal training objective.
Issue of Directional Loss for Fair T2I Generation. We hypothesize that directional loss is sub- optimal in learning tS -SA-related tokens S. Particularly. the differences in reference images AI can include unrelated concepts in addition to variation in tSA categories. For example, considering ISA=Smiling in Fig. la (col 2), the reference images used for learning these two categories contain differences in pose, accessories, etc., in addition to the difference in Smi Smiling. We further explore this potential of encoding unrelated concepts in A by taking a closer look into the CLIP embedding space, where ITI-GEN's learning process happens. Recall, that as we utilize tSA with MLA and the HPs only differ in the tSA categories, considering (SA=Smiling Semmmiiiigg the related tokenized prompts of HP CLIP space can be computed as follows: F, Er ('A headshot of person smiling'), and F, Er ('A headshot of person not ami smiling'). with Ey denoting CLIP's text encoder. Then AF = F, shows the direction of the tokenized prompts in the CLIP embedding space.
utilize them references in analysis. For example,
Our results in Fig. (col3) shows directional loss between AI and AF iee, Lav (I,F), for different (SAs using the reference images for each tSA. Note that Lair O means perfect alignment. Our comparison reveal considerable misalignment between AI and F implying that unrelated concepts are potentially encoded in AI. Mcanwhile, AI and AP near perfect alignment implies that

---

"pererra
"amtinn
"beaming
grinning
porsaat
Step
Input
Output Input
96
Output
2
50
Hard prompt (HP)
ITI-GEN prompt
Figure Comparison of cross-attention maps during the denoising process with HP (left) and ITI-GEN (right). Here, we use iSA tSA=Smiling and plot the denoising process for one sample generation. Each denoising process consists of 50 steps initiated with the same noisy input. Each cell depicts the attention map for the respective token (column) at the respective step (row) overlaid on the input. We highlight 3 key observations: ① ITI-GEN tokens S: have abnormal activities compared to the corresponding -SA-rel tokens in HP by attending to unrelated regions (backgrounds) or scattered attention. 2 non-tSA tokens like "of" and "'a" are abnormally more active in the presence of ITI-GEN tokens. 3 As compared to the HP counterpart, issues created by ITI-GEN tokens (①) & 2) degrade the global structure in the early denoising steps (e.g.. Step 15), for example, human face in HP some unrelated structure in ITI-GEN. The same behavior is observed for some other samples and tSAs (see Supp for more samples, and other tSAs with more denoising steps).
these unrelated concepts are potentially transferred to P via ITI-GEN's learning objective, resulting in distorted learned token S.
3.2 Analyzing the Effect of ITI-GEN Prompts in T2I Generation
In the previous section, we observed degraded sample quality in ITI-GEN which we attribute to the sub-optimal training objective that results learning distorted tokens. In this section, we take a step further to answer the question: "Given pre-trained T2I model and some distorted learned prompts as input, how do these distorted prompts affect the image generation process of the T2I model?"
To answer this, we deep dive into the latent denoising network [1] and analyze the cross-attention mechanism [40] the bridge for text and image modules in T2I models [1, 23-25]. In this analysis, we visualize the tross-attention maps to investigate potential anomalies caused by distorted tokens in the denoising process. Specifically, we compare cross-attention maps of ITI-GEN prompt against HP with minimal linguistic ambiguity (as reference). To allow fair token-to-toker comparison, in this experiment, we lengthen HP by including additional tokens containing synonyms of the tSA. Note that this did not augment HP's behavior, and similar results are seen in the original HP. See Supp for more details.
Visualizing Cross-attention Maps. We follow DAAM [41] for visualizing cross-attention maps by tracing attention scores in the cross-altention module demonstrate how an input token within prompt influences parts of the generated image. Specifically, to visualize the cross-attention map of a token, DAAM interpolates and accumulates the attention scores over all scales (layer of the U-Net [42] as the denoising network [1]), and all denoising steps. However, we tailor DAAM to the requirements of our fine-grained analysis by introducing further controls. First, we isolate the attention maps for each denoising step to allow for both step-wise and multi-step analysis. Second, we introduce a prompt-switching mechanism, allowing for the interchangeable tracing of different prompts at any particular denoising step.

---

ISA Smiling
* quaddise
parsor
millon
fgersno
Japur
Output
Tapat
Cutpar
Stage
Chiipar
Stage
Cheekbones
Input
Curput
Impet
B
Cuippul
Stagy
Iopat
20
Quigat
Ingun
K TORR "d
Durpptt
Stage
SA tokind
0,045
0.050
0.655
0,500
Corna
HP (F)
ITI-GEN (1)
Figure 4: Analyzing the accumulated cross-attention maps for the denoising process in our proposed prompt switching analysis I2H and H21. Here, we use two tSAs: Smil Smiling, High Cheekbones. For each tSA, we show the accumulated gross-attention maps for H21 and I2H, with some quantitative results. In the H2I (12H) experiment, the first row shows the accumulated cross- attention maps during the early denoising steps with HP (ITI-GEN) as the input prompt, and the second row shows the maps during later steps, after switching to ITI-GER (HP). Observation 1: Learned tokens in ITI-GER affect early denoising steps, degrading global structure synthesis; such degraded global structure disrupts the final output. This is observed in I2H. Observation 2: Learned tokens in ITI-GER works decently in the later stage of the denoising process if the global structure is synthesized properly. This is observed in H2I. As we show in Supp. similar observations can be made for other samples and other tSAs. Bottom: Histograms of our proposed metrics cross- coosenattntionn maps demonstrate the abnormalities in many samples.
Abnormalities in the Presence of Distorted Tokens. To investigate potential anomalies arising from distorted tokens learned by ITI-GEN, we comprehensively analyze cross-attention maps of all denoising time steps on 500 generated samples per category of each ISA, for both ITI-GEN and HP. Fig. 3 shows one of these cross- cross-attention maps comparing ITI-GEN and HP for iSA=Smi 1 ing (more examples in Supp). Our empirical investigation reveals four points: i) global structure is synthesized in the early steps of the denoising process aligning with previous works [21] that the denoising process progressively synthesizes the image. ii) Learned ITI-GEN tokens have abnormal attention compared to the tSA-related tokens in HP ("Smiling" in col. 7 of HP), e.g., [[S]] contain unrelated or scatter activation (Issue 1). iii) In the presence of the ITI-GEN tokens, other non-tSA tokens (like M ['aa") and M ('ooo)")) are abnormally more active (Issue 2). We remark that tokens interact with each other in the denoising steps. iv) Considering Issues 1 & 2, we observe that degraded global structure is synthesized in the early steps of denoising, and eventually a degraded sample is generated at the end of the denoising. Note that similar issues occur with many other samples and tSAs (details in Supp).
To further understand issues, we isolate effect of distorted tokens by proposing two analyses focusing on different denoising steps. These two analyses dissect the influence of distorted tokens in key denoising steps (Recall denoising of a single step t is denoted by Z1+1 + DM (Z,, R. t, s). Sec. 2):
1] DM F,1,8) LE [m,
DM, P,t,)) E [0, n 1]
.H2I= [DM F,t,s) E [0, n 1] (D, P,4,,)
12H-
(2)
LE [n,
To do this, as seen in Fig. lb (col 2), we first propose Analysis 1: switching prompt from ITI-GEN to HP (12H) during the denoising process. This allows for a better understanding of how the global structure's degradation in early denoising steps may affect the final generated output. Specifically, as in Eq. 2, 12H first utilizes ITI-GEN prompt P in early denoising steps which potentially leads to degraded global structure. This then followed by utilizing hard prompt F for the remaining denoising steps. Next, to investigate if ITI-GEN tokens will create the same issues in the later steps of the denoising process, we propose Analysis 2: HP to ITI-GEN (H2I). Converse to the previous experiment, we utilize F in early steps of denoising, and then switch to using P as input prompt. For each experiment, we plot and analyze the cumulative ross-attenttion maps for early steps (0 to 1) and later steps (n to !) separately. Fig. 4 shows an example of the cross-atted attention maps for
7

---

these two experiments with iSA=(Smiling High Cheekbones]. J. See Supp for more samples and details. Considering results in Fig. the following observations can be made:
Observation 1: Learned tokens in ITI-GEN affect the early steps of the denoising process leading
to degradation in synthesizing global structure. More specifically comparing the first row of the cross-attention map between 12H and H2I in Fig. 4, we can have the following observations: i) ITI- GEN tokens have more scattered attention or attending to unrelated regions compared (SA-related tokens in HP; ii) non-tSA tokens like "a", and "off" are more active in the presence of the ITI-GEN tokens. These two abnormalities result in degraded global structure in the early steps. In addition. considering the second row of the 12H in Fig. 4, the degraded global structure in the early steps leads to disrupted final output even though the (non-distorted) HP prompt used in later steps.
Learned tokens in ITI-GEN works decently in the later steps of the denoising process if the global structure is synthesized properly. More specifically, considering H21 in Fig. 4, when HP prompts synthesize proper global structure in early steps, the ITI-GEN tokens attend to proper regions and contribute to adding the finer details related to ISA, as shown in the second row of H2I.
Observation 2:
Quantitative Metries for Cross-attention Maps. In addition to the visual demonstration, we propose two metrics to support further our observed abnormalities of ITI-GEN tokens in the early steps of denoising for large number of generated samples. Specifically, for each generated sample: i) To quantify abnormally active attention associated with non-tSA tokens, we compute the expectation of attention amplitude: E(s,y) { [J],, where J is a non-tSA token such as "of"". ii) We analyze the scatter in attention by measuring the second central moment [43] for each tSA token K:
(KK)=
E((( - x)² + (y- - ?) (2.,))
2.91
Here, M = (M[K]/E 2.9 M[K),, and (z. g) is the centroid. The two metries are computed on the accumulated cross-altention maps from stage of 12H (for ITI-GEN) and H21 (for HP). The histograms of these two metrics for 500 generated samples in Fig. lb (col 3) and Fig. 4 demonstrate Issues 1 & 2 in many generated samples.
Remark. Our thorough analysis in this section shows that distorted tokens learned by ITI-GEN only have destructive performance in the early steps of denoising, and they generally have decent performance in later steps when the global structure is formed properly (H21). We remark that even though H21 has decent performance in fair and high-quality T21 generation, is only applicable to ISA with minimal linguistic ambiguity. In the next section, we will discuss our proposed method to address fair and high-quality T21 generation encompassing both ambiguous and unambiguous iSA.
Proposed Method
In this section, we present our proposed method, FairQueue new generation framework consisting of two additions: Prompt Queuing and Attention Amplification to improve the sample quality when implementing fair T2I generation. In addition to quality improvements, FairQueue also allows for better semantic preservation of the original sample generated from the base prompt T.
Prompt Queuing. Recall that when utilizing ITI prompt P-which is tuned to generate samples containing the 1SA-degraded global structure occurs in early denoising steps for moderate number of samples. Conversely, utilizing HP with minimal linguistic ambiguity enables high-quality and fair T21 generation. However, as such HPs are not available for all tSAs [16], we naturally consider the next best available option-the base prompt T (a natural language prompt without the distorted trainable lokens)-and propose prompt queuing. Specifically, as seen in Fig. I(c). prompt queuing first utilizes T in the early n denoising steps, thereby allowing for the global structures to form properly. Next, we transit to ITI-GEN prompt P for the remaining (I - n) steps. This allows the more fine-grained iSA semantics to be developed on top of the already well-defined global structures.
Attention Amplification. By implementing prompt queuing, the output samples may experience a reduction tSA expression due the reduced exposure to the ITI-GEN prompt P. To address this, we propose Attention Amplification, an intuitive solution that emphasizes the expression of the ISA by scaling the ITI-GEN token's 's cross- cross-attention maps, i.e., c ((.)) where e >
5 Experiments
In this section, we evaluate our proposed (FairQueue) against the existing SOTA ITI-GEN [16] over various (SA. Then, we conduct an ablation study by first evaluating the contribution brought by each
8

---

Table 1: Evaluating Proposed FairQueue against ITI-GEN, We utilize FD: Fairesss Discrepancy (4), TA: Text-Alignment (t), FID (4), and DS: DreamSim (4) to determine the faimess, quality, and semantic preservation, respectively. For FD a combination of CLIP [20], off-the-shelf classifier [44, 45] and human evaluator were utilized as tSA classifier. For TA utilize CLIP [20] as the feature extractor. Overall, our proposed method demonstrates the best ability to balance between sample quality and fairness, while preserving the semanties of the original base-prompt T.
Single ISA (CelebA)
ISA
(D)
(f)
FID (4)
DS (U)
ITI-GEN
6.41e 4.2e-3
0.855
1.2g
78.9 + 1.3
0.337 1.
Gender
Ours
6.41e 3 3.8e
0.676 1: 5.
78.3 1: 1.5
0.308 1.2c
Young
ITI-GEN
13.1e-3 + 8.1e"
0.653 9.4e-)
82.9 1.4
0.552 3.2e
Ours
15.5e 3.8e
0.678 8. le
75.3 1: 2.1
0.370 2.7e
Smiling
ITI-GEN
12tc 9.26-
0.605 1.2c
88.6 = 0.9
0.557 2.20-
Ours
69.0c + 4.2e
0.674 + 1. 1.7e"
80.0 + 1.3
0.284 + 1.0e
Bigh Checktones
ITI-GEN
318c 3 12.0e
0.595 1:2g -3
86, 2.1
0.538 Le
Ours
4.92c 3.60
0.685 = 7
79.7 + 2.4
0.330 + 2 20
Skin
ITI-GEN
1.41c 8 1.2c
0.646 1.8c 2
101.3
0.525
Ours
1.41e
1.20
0.666 =: 1.9.
97.0 1:00=:0 3.2
0.408 = 3 3.0e
Ryaglassea
GEN
Ours
14.1e
25.4e 1 1.9e
2.6c
0.670 1: 6.1e
0.654 3.3e
79.4 :1 2.3
83.5 1.4
0.391 +: 1.6c
0.486 1.4e
Mustache
ITI-GEN
26.2e-1
1.8e-
0.670 4.2c
85.0 3.3
0.452 1.9e
Ours
22.6e
1.2e
0.680 5.3e
80.2 1. 3.0
0.345 3.
Chubby
ITI-GEN
Ours
112: -3 8c
119e 7.2c -3
0.675 2.3e
0.617 2.26
78.3 + 1.4
79.2 1.5
0.387 3.0e
0.551 Ge
Gray Bair
ITI-GEN
280e
6.8e a
0.610 + 4.3e 0.669 3. 3.
87.3 2.1
0.533 = 2 9e
Ours
266c
7.1c
82.2 2.3
0.417 # 3.1e
Multi 15A (CelebA)
ITI-GEN
39.1c 1 1.2e
0.668 1 7.1e
72.6 1: 3.1
0.458 :: 7.8e
Gender x Young
Ours
12.40 + 2.30
a
0.686 5.7e
T1.7 2.5
0.373 1.10
Gender Young Eyeglasees
ITI-GEN
Ours
208 4 7.3e
257e
8.7e
0.854 1 3.3e
4.1e
61.51 2.7
65.2 1: 1.6
0.360 6.3c
0.475
1.le
ITI-GEN
100e
1.7e-2
0.643 + 7.
65.5 2.7
0.475 + 9. 9.
Gender x Young Eyeglasses Smiling
Ours
168e
1.0e
0.661
1.1
0.379
9.7e
Multi ISA (Fairface & Fair Benchmark)
Gender x Age
ITI TEE
142e
4.2e a
0.659 7.2c 8
0.445
Ours
108e
a 4.3e
0.672 = 1.1e
58.81 + 3.3
0.359 : 3.5e
Gender 8kin Tone
160e 3.7e-i
0.670
2.26
59.56 3.6
0.463
7.7e-3
Ours
116e #1 4.de
0.686 1 2.3e
54.66 1 2.7
0.390 1 1.8c
component for FairQueue i.e., Prompt queuing, and Attention Scaling. Then we revisit the task initially proposed by ITI-GEN: Training Once-for-All Token. Overall, we show that FairQueue achieves new SOTA performance.
Experimental Setup. Following [16], we utilize the publicly available reference dataset from CelebA [29]. FairFace [45] and FAIR benchmark [44]. For CelebA, we perform both single tSA and multi- ISA experiments. Note that in this dataset each iSA has two categories. In FairFace and FAIR datasets, (SAs have more categories, e.g. Age and Skin tones contain 9 and 6 classes, respectively, Therefore, fair generation is more challenging in these datasets. For these experiments we use T=E, ("A headshot of person") as base prompt, and for fair comparison, we utilize exactly the same learned from ITI-GEN's original code [46] for both ITI-GEN and proposed FairQueue. In addition, we randomly sample set of 500 latent codes and use the same latent codes for both approaches. As earlier discussed Sec. 3, we utilize fairness discrepancy (FD) to evaluate fairness, Text-Alignment (TA) and FID for quality, and DreamSim (DS) to measure semantic preservation. See Supp for more details. We repeat this process times and report the mean and standard deviation.
Our results in Tab. 1 demonstrate that FairQueue is able to match ITI-GEN fairness performance closely, and in some cases even improve upon For example, for (SA=Smi -0ngmmnnnn. FairQueue indicates a significantly lower bias (FD=6.9e -3 than ITI-GEN (FD=124e 3). In addition, considering sample quality, FairQueue achieves an overall better performance than ITI-GEN for all datasets. For example, in CelebA, FairQueue's 's TA> 0.666 while ITI-GEN's 's TAs 0.655, with the worst performance with High Cheekbones (TA=0.595). These results are similarly reflected in FID. We remark that this quality degradation largely contributes to ITI-GEN fairness degradation. Finally, when considering semantic preservation (DS 4) of the original sample generated with T, FairQueue achieves unparalleled performance by ITI-GEN.

---

Ablation study: evaluating prompt queuing and attention scaling, To evaluate the contribution brought by FairQueue, we consider the same setup as Sec. 5 in the main manuscript focusing on the iSA Smiling. Here, we compare the performance when utilizzng different attention amplification scaling factors, e, and different prompt queuing transition points i.e, switching from T P. Specifically, we consider gradual increments in [0, 12] and shifting of the transition points, step {0, 0.11, 0.21, 0.3/} when 50.
Our results in Fig.5 illustrate that generally when e increases, fairness improves. However, saturation point (c 10) exists where quality and semantic preservation beyond this point degrades. Then when considering different transition points, we find that at step 0.2/ FairQueus achieves the best quality and semantic preservation performance while still achieving good fairness measurements. However, increasing beyond this point results in significant fairness degradation.
0.15
0.69
0.8
0:13
0.113
0.45
0.078
0.075
0.965
02
0.088
0138
0,883
0.15
0.84
0.59
2 4 10
0.11 0:21
0.51
8 10
0.11
Attentio Amolication
Attention Amplication
Attention Amprication scaling
Prompi
scaling
Transiiion Point
factor,
Transiton Point
Transtion Point
Figure 5: Ablation Study: Comparing FairQueue performance when varying i) attention amplifica- tion factor, or ii) Prompt Queuing transition point from T - P for tSA Smil Smiling.
Ablation study: revisiting training once-for-all token. Utilizing FairQueue we follow [16] and re-visit adapting pre-trained ITI-GEN tokens, S; to new Base Prompt T E, ("A headshot of doctor") by pre-pending. Then we generate samples utilizing both FairQueue and ITI-GEN with the same noise input. As seen in Fig. 6 FairQueue demonstrates better performance than ITI-GEN achieving both better quality and semantic preservation the sample generated by T" while still having good ISA representation- more illustration in Supp.
ITI-Gen:
Presa
Ytark
Sualinal
High
With
Willer
headshot
ductor"
Ours:
Figure Illustration of samples generated by ITI-GEN and FairQueue with new Base Prompt T' E headshot of doctor") via pre-pending. FairQueue improves sample quality and ability to preserve the original sample's semantics while mainly adapting only the tSA.
6 Conclusion
In this paper, we reveal quality degradation in ITI-GEN tthe existing SOTA fair T21 prompt learning approach. Our analysis reveals that this quality degradation is due to the distorted learned tokens in ITI-GEN prompt impacting cross-attention in the early steps the denoising (reverse diffusion) process. To address this, we propose FairQueue simple but effective solution consisting of: Prompt Queuing and Attention Amplification. Overall, our extensive experimentation demonstrates FairQueue achieves SOTA performance in balancing quality, faimess, and semantic preservation. Limitation, related work and additional experiments can be found in the Supp.
Acknowledgements
This research is supported by the National Research Foundation, Singapore under its AI Singapore Programmes (AISG Award No.: AISG2-TC-22 USS2--CC-0022-077, The Agency for Science, Technology and Research (A STTAR) under its MTC Programmatic Funds (Grant No. M23L760021). This research supported by the National Research Foundation, Singapore and Infocomm Media Development Authority under its Trust Tech Funding Initiative. Any opinions, findings and conclusions rec- ommendations expressed this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore and Infocomm Media Development Authority.
10

---

References
[1] Rohin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pultern recognition, pages 10684-10695, 2022.
[2] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-Shot Text-to-lmage Generation, February 2021.
[3] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang. Vijay Vasudevan, Alexander Ku, Yinfei Yang. Bureu Karagol Ayan, al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arxv:22006, 10789, 2(3):5, 2022.
[4] Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas, Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 5907-5915 2017.
[5] Tao Xu, Pengehuan Zhang, Qiuyuan Huang, Han Zhang, 7he Gan, Xiaolei Huang, and Xiaodong He. Atingan: Fine-grained text to image generation with attentional generative adversarial networks. In Proceedingy of the IEEE conference on computer vision and pattern recognition, pages 1316-1324, 2018.
[6] Bowen Li, Xiaojuan Qi, Thomas Lukasicwicz, and Philip Torr. Controllable text-to-image generation. Advances in neural information processing systems, 32, 2019.
[7] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pages
Pmlr, 2021.
[8] Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering io-agee- generation via transformers. Advances in Neural Information Processing Systems, 34:19822 19835, 2021.
[9] Chenfei Wu, Jian I Liang, Lei Ji, Fan Yang. Yuejian Fang. Daxin Jiang, and Nan Duan. Nüwa: Visual synthesis pre-training for neural visual world creation. European conference on computer vision, pages 720-736. Springer, 2022.
[10] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew. Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference Machine Learning, pages 1688441604. PMLR, 2022.
[11] Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generation models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 3043-3054, 2023.
[12] Charlotte Bird, Eddie Ungless, and Atoosa Kasirzadeh. Typology of risks of generative text-to-imase to-image models. Proceedings of the 2023 AAAI/ACM Conference on Ethics, and Society, pages 396 996-410, 2023.
[13] Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni. and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on faireess. arXiv preprint 23022 10893, 2023.
[14] Aditya Chinchure, Pushkar Shukla, Gaurav Bhatt, Kiri Salij. Kartik Hosanagar, Leonid Sigal, and Matthew Turk. Tibet: Identifying and evaluating biases in text-to-image generative models. arXiv preprint arXiv:2312.01261, 2023.
[15] Noa Garcia, Yusuke Hirota, Yankun Wu, and Yuta Nakashima. Uncurated image-text datasets: Shedding light on demographic bias. In Proceedings of the IEFF/CVF Conference Computer Vision and Pottern Recognition, pages 6957-6966, 2023.
[16] Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre lti: Inclusive eoxt-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3969-3980, 2023.
[17] Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. How well can text-to- image generative models understand ethical natural language interventions? arXiv preprint arXiv:2210.15230 2022.
11

---

[18] Ching- Yao Chuang, Varun Jampani, Yuanzhen Li. Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts. arXiv preprini arXiv:2302.000770 2023.
[19] Ming Ding, Zhuoyi Yang. Wenyi Hong. Wendi Zheng. Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-lo-image generation via transformers. Advances in Neural Information Processing Systems, 34: 14419822-19835, 2021.
[20] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763, PMLR, 2021.
[21] Amir Hertz, Ron Mokady, Jay Tenenhaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to- prompt image editing with Representations, 2022.
control. In The Eleventh International Conference on Learning
[22] Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Yan Zhu, Multi-concept customization of text-to-image diffusiio. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1931-1941, 2023.
[23] Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song. Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXivv2211.01333 2022.
[24] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, al. Photorealistic text-to- image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479 -34994, 2022.
[25] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 20Xi:2204.006155. 1(2):3 2022.
[26] Yunji Kim, Jiyoung Lee, Jin-IWa Kim, Jung-Wor Woo Ha, and Jun- Yan Zhu. Dense text-to-image generation with attention modulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7701-7711, 2023.
[27] Christopher Teo, Milad Abdollahzadeh, and Ngai-Man Man Cheung. On measuring fairness in generative models. Advances in Neural Information Processing Systems, 36, 2023.
[28] Kristy Choi, Aditya Grover, Trisha Singh, Rui Shu, and Stefano Ermon. Fair generative modeling via weak supervision. In International Conference Machine Learning, pages 1887-1898. PMLR, 2020.
[29] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEFE international conference on computer vision, pages 3730-3738, 2015.
[30] Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Chcung. Fair generative models via transfer learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 2429-2437, 2023.
[31] Christopher TH Teo, Milad Abdollahzadeh, and Ngai-Man Cheung. Fairtl: A transfer learning approach for bias mitigation in deep generative models. IEFE Journal of Selected Topics in Signal Processing, 2024.
[32] Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West. Alexander Koller, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. We'r re afraid language models aren't modeling ambiguity. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https:/ //openeevie...... met/forummiddwaae w 3bb3.
[33] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 13009):233--2344 2022.
[34] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16815, 2022.
[35] Rinon Gal, Or Patashnik, Haggai Maron, Amit H Bermano, Gal Chechik, and Daniel Cohen Or. Stylegan- nada: Clip-guided domain adaptation of image generators. ACM Transactions on Graphics (TOG), 41(4): 1-13, 2022.
12

---

[36] Ching- Yao Chuang, Varun Jampani, Yuanzhen Li. Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts. arXiv preprini urXiv:2302.00070, 2023.
[37] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: reference-Iree evaluation metric for image captioning. In Proceedings the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2514-7528, 2021.
[38] Martin Heusel. Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by time-scale update rule converge a local nash equilibrium. Advances neural information processing systems, 30, 2017.
[39] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data, arXiv preprint vrXiv:2306.09344 2023.
[40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[41] Raphael Tang, Linging Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang. Karun Kumar, Pontus Stenetorp. Jimmy Lin, and Ferhan Ture. What the DAAM: Interpreting stable diffusion using cross attention. In Proceedings of the 61st Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, URL ht https: acl anthology. org/ ww/g//22...o acl long. 310.
[42] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. Medical image computing and computer assisted intervention MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[43] Rafacl C Gonzales, Richard E Woods, and Steven L Eddins. Digital image processing using MATLAB. Pearson Prentice Hall, 2004.
[44] Haiwen Feng, Timo Bolkart, Joachim Tesch, Michael J. Black, and Victoria Abrevaya. Towards racially unbiased skin tone estimation via scene disambiguation. In European Conference on Computer Vision, 2022.
[45] Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender. and age bias measurement mitigation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1588-158, 2021.
[46] Itigen. https: /ttp//tttttttubtt www./tuaannenninngaaa/iiIin
[47] Ninarch Mchrabi, Palash Goyal, Apury Verma, Jwala Dhamala, Varun Kumar, Qian Hu, Kai-Wei Chang, Richard Zemel, Aram Galstyan, and Rahul Gupta. Resolving ambiguities text-to-image generative models. In Proceedings of the 61st Anmual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 14367-14388, 2023.
[48] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 141266980, 2014.
[49] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEESCVF conference on computer vision and pattern recognition, pages 4401-4410, 2019.
[50] Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 11410-1 11400, 2022.
[51] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 220.::2204406155,, 1(2):3, 2022.
[52] Midjoumny, http: /www www.miiddoore..oomm com// Accessed: 2024-01-11.
[53] Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniy Taigman. Make-a-scene: Scene-based text-to rext-to-image generation with human priors. In European Conference on Computer Vision, pages 106. Springer, 2022.
13

---

[54] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang. Vijay Vasudevan, Alexander Ku, Yinfei Yang. Burcu Karagol Ayan, Ben Hutchinson. Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, and Yonghui Wu. Scaling autoregressive models content-rich text- to-image generation. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https: https://openrevievv met/forummidd AFd.d--FDDYKKDDD Featured Certification.
[55] Eva Cetinic and James She. Understanding and creating art with ai: Review and outlook. ACM Transactions on Multimedia Compuring, Communications, and Applications (TOMM), 18(2):1-22, 2022.
[56] Pierre Chambon, Christian Bluethgen, Curtis P Langlotz, and Akshay Chaudhari. Adapting pretrained Aision-langual foundational models to medical imaging domains, arXiv preprint
2022.
[57] Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, et al. Holistic evaluation of text-to-image models. Advances in Neural Information Processing Systems, 36, 2023.
[58] Ahmed Imtiaz Humayun. Randall Balestriero, and Richard Baraniuk. Magnet: Uniform sampling from deep generative network manifolds without retraining. In International Conference on Learning Representations, 2021.
[59] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pages 709- 127-727. Springer, 2022.
[60] Milad Abdollahzadch, Touba Malekzadeh, Christopher TH Teo, Keshigeyan Chandrasegaran, Guimeng Liu, and Ngai-Man Cheung. survey on generative modeling with limited data, few shots, and zero shot. arXiv preprint arXiv:2307. 14397, 2023.
[61] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman, Dream- booth: Fine tuming text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 22500-22510, 2023.
[62] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh International Conference on Learning Representations, 2022.
[63] Kihyuk Sohn, Huiwen Chang, José Lezama, Luisa Polania, Han Zhang, Yuan Hao, Irfan Essa, and Lu Jiang. Visual prompt tuning for generative transfer learning. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 19840-19851, 2023.
14

---

Supplementary
This supplementary provides additional experiments as well as details that are required to reproduce results. These were not included in the main paper due to space limitations. The supplementary arranged as follows:
Section A: More Experimental Results
A.1 Identifying FairQueue as the optimal combination
A.2 Cross-Attention Analysis
A.3 More on Ablation Studies
* A.3.1 Analyzing the Effects of Attention Amplification
A.3.2 More Illustrations for Training Once-fo-All Tokens
*
A.3.3 Human-recognized Assessment Comparing ITT-Gen and FairQueue Quality
A.4 More Illustration
A.5 Evaluating Minimal Linguistic Ambiguity for tSA
Section B: Experimental Details
B.1 Details of Calculating Directional Loss for Prompt Tuning
B.2 Details of the Ambiguities in Text Prompts
B.3 Details of Model Hyper Parameters
B.4 Computation Resources
B.5 Details of Evaluation Metrics
B.6 Visualizing the Learned Embedding vs Base Prompt
Section C: Limitations and Broader Impacts
Section D: Related Works
15

---

A More Experimental Results
A.1 Identifying FairQueue the optimal combination
Table 2: Analysis of all possible different combinations for Prompt Queuing (PQ) and iSA Attention Amplification (AA). We summarize our findings from main paper for the tSA "Smilling". Note that (S)) notates AA for tSA tokens, ITI-GEN prompt P=[T;S], and results in bold and italics are the best and second best. Notice that 26:FairQueur (PQ+AA) provides the best combination: it achieves both outstanding sample quality (C6: TA=0.674 FID=80.02 similar to C1: TA=0.681 & FID=76.9 with the best quality but poor fairness) and faimesss (C6: FD=0.069 similar to C4: FD=0.05 with the best fairness but poor quality).
Prompt Queuing (PQ)
Attention Amplification (AA)
Stuge (Prompt)
Stuge (Prompt)
FDX
TA(1)
FID(J)
DS(
remarks
CI: no PQ. AA for Base Prompe
No
No
T
T
0.211
0.681
76
C2: no PQ, no AA ITl-Gen
No
No
(T:S)
(T:8)
0.124
0.005
88.63
0.537
Unimplementable
C3: AA only for Base Prompt
combination due
No
Yes
T
T
N.A
N.A
N.A
N.A
the absence
1SA tokens for AA
C4: AA Only for ITT--Gen Gen
No
Yes
[7:8]
[7so(S)]
0.05
0.610
89.41
0.550
C5: PQ Only
Yes
No
T
[7:8]
0.145
0.674
80.15
0.240
C6: PQ + AA
Both PQ and AA are present iee,, FairQuene
proposed: FairQuene)
Yes T
[Ta((S))
0.069
0.674
80.02
0.284
In this section, we discuss in more detail how we identified FairQueue - with its two mechanisms: Attention Amplification and Prompt Queuing as the pessfoerforming solution. Specifically, we summarize our findings, discussed throughout the paper, when exhaustively considering all possible combinations. Our results are as follows:
C1: Base Prompt T Only (no AA no PQ): It lacks tSA-specific knowledge and results in poor fairness. Additionally, without tSA tokens S, AA is not applicable for C3.
C2: ITI-Gen prompt P Only, in Tab. 1 (no AA no PQ). Our analysis in Sec 3.2 shows it has poor quality due to distortion in global structure during sample generation. Without PQ, the issue of distorted global structure persists for some tSAs
C4: Attention Amplification (AA) Only, in Fig. 5 when PQ transition point=0. results in poor quality since only ITI-Gen is used. We remark that utilizing only AA for ITI-Gen may deceptively improve fairness. but the generated samples have poor quality e.g., Smiling cartoons. The reason is (similar to C2): without PQ, the issue of distorted global structure persists for some tSAs.
C5: Prompt Queuing (PQ) Only, in Fig. 5 when c 0. By replacing the distorted ITI-Gen prompt with the Base prompt in Stage 1, PQ leads to improved quality, but without AA, the fairness remains poor given reduced exposure tSA tokens in the denoising process.
C6: FairQueue (PQ+AA), in Tab Our proposed solution with optimal quality and fairness. Specifically, it combines the effects of Prompt Queuing- enabling the global structure to be properly formed resulting in good quality samples, and Attention Amplification-enhanne the ISA-specific expression for better fairness.
Overall, our results in Tab. 2 reveal that FairQueue (C6) is the superior combination balancing between faireess and quality. Specifically, Prompt Queuing is necessary whereby utilizing either only ITI-GEN (C2) or only Base Prompt (C1) results quality and fairness degradation, respectively. Furthermore, our results show that both PQ and AA are necessary to obtain high-quality samples with good fairness performance, as without PQ (C4) sample quality is poor, and without AA (C5) fairness performance is degraded.
16

---

A.2 Cross-attention analysis
Sec. 3.2 analyzes the effect of inclusive tokens S by comparing the accumulated aross-attention maps of individual tokens between HP and ITI-GEN is observed that distorted tokens learned by ITI-GEN negatively affect the development of global structure in the early steps of denoising. The destructive effect arises with abnormally high activity of non-tSA tokens (e.g., "'of",, and the tSA- tokens attend to unrelated regions with scattered attention. A quantitative analysis is performed over 500 sample generations for different tSAs to affirm the observations. The below details how token- specific accumulated cross-attention maps are obtained from the SD pipeline, discusses interaction among tokens, and presents additional representative results for tSAs Smiling, High Cheekbones, Gray Hair, and Chubby.
Details of visualizing token-specific accumulated cross-attention map. Cross-attention is often used to contextualize prompt embeddings with latent representations per sample generation step. Following DAAM [41], coordinate-avee aware attention scores [[ are extracted from the latent diffusion network (i.e., U-Net) for the token S; at the layers where cross-attention take place. These token-specific attention scores, each with the same spatial dimensions as the latent representation, are upscaled bicubically to the image size (512 x 512 in this case) to reveal where attention is paid per token and accumulated within the assigned step(s). The resulting 2D matrix is visualized in Fig. and Fig. 4 and referred to an "accumulated cross-attentis map".
Interaction among tokens. We remark that the cross-attention map of given token is dependent on the others in the prompt. There are two channels where the effect of tokens may interact: 1) via latent representation, as is a function of input tokens and serves as the query in the cross-attention (see 2)c.2); 2) softmax operation, as a component in the attention pipeline, softmax is taken across all tokens when processing attention scores. These two effects become increasingly apparent as we move through different cross-attention layers of the U-Net and perform more denoising steps.
HP vs. ITI-GEN: : qualitative analysis. To investigate potential abnormality of ITI-GEN embed- dings, images of different iSA are generated conditioning on HP (F) and ITI-GEN (P) respectively. The cross-attention map employed as tool to explore the cause of degraded generations. of fair
pursuit comparison, for some tSAs the original HPs ("HPI", see Tab. 6) are extended to align with P in the number of tSA tokens ('HP2"). Nonetheless, as one can find in the samples in Fig.7 to 14, the extension does not change the behavior of HPs significantly.
7 to 17 give overview of cross-attention maps during the denoising process. One may find that the tSA tokens in the HP((s) tend to concentrate on the region(s) semantically associated with the ISA, e.g, mouth for iSA Smi ing, cheek for ISA High Cheekbones, and hair for iSA Gray Hair, On the other hand, ITI-GEN iSA tokens' activity tends to be less focused and attends broadly. With more steps than Fig. 3, it is clearer that the global structure of the images is synthesized in the early steps, which motivates the prompt switching experiments.
Prompt switching experiments and quantitative analysis. To further investigate HP and ITI-GEN prompts' behaviors in the early steps, the prompt switching experiments (i.e., I2H and H2I) are proposed in Sec. 3.2. Fig. 18 to 21 present representative outcomes of the experiments. One can find that the destructive effect caused by ITI-GEN prompts only occurs at the early steps, i.e., Stage in the figures.
In addition, the activation patterns are more clear the accumulated cross-attention maps. The non-tSA tokens in ITI-GEN prompts are in general more active, and the tSA tokens tend to attend more broadly, which may explain the drastic semantic deviations from HPs in Fig.7 to 17. The latter observation is particularly evident for iSA Smiling. highly localized facial expression, which is supported by the histogram of central moments in Fig.I. The other tSAs, though may not be directly associated with specific facial feature, share the same trend, manifested statistically by the histograms in Fig.22 and 23g.23.
Utilizing Base Prompt (T) in FairQueue. In Prompt Queuing the use of T, in place of the HP, is similarly grounded on the 12H/H2I analysis, as both T and HP are natural language prompts free of learned tokens. This can be seen in the embedding analysis in Supp B.6 where the HP and 7 are seen to be close to one another. As a result, the sample generated by is expected to be of similar quality as the HP.
17

---

"sesstatto""
Tran Bingn
Step
Osipat Inper
Ozipa
Figure 7: Cross-attention maps during the denoising process with HP1 (left), HP2 (middle, equal #tSA tokens to ITI-GEN ), and ITI-GEN (right) prompts. ISA-Smiling.
In Fig. 24, we provide further visualizations of T's effectiveness in generating the global structure in early denoising steps. Specifically, we compare the cross-attention maps of FairQueue with ITT--Ge during sample generation, together with quantitative analysis. Results in col illustrate 7's effectiveness in synthesizing the global structure in stage 1, and non-abnormal attention (in Fig. 25), resulting in effective global synthesis than ITI-Gen and better sample quality.
18

---

Phoedthoo
Ettniin
ppersoo
"amallag"
"annoon
Figure 8: Cross-attention maps during the denoising process with HP1 (left), HP2 (middle, equal
#tSA tokens to ITI-GEN ), and ITI-GEN (right) prompts. tSA=Smiling.
19

---

"perBoo""
"heastsbos
"houdiboo
or
80
81
89
Figure 9: Cross-attention maps during the denoising process with HP1 (left), HP2 (middle, equal
#tSA tokens to ITI-GEN] ), and ITI-GEN (right) prompts. ISA=Smiling.
20

---

Figure 10: Cross-attention maps during the denoising process with HP1 (left), HP2 (middle,
equal #tSA tokens ITI-GEN) ), and ITI-GEN (right) prompts. tSA=High Cheekbones.
21

---

"persoo"
or
Figure 11: Cross-attention maps during the denoising process with HP1 (left), HP2 (middle,
equal #tSA tokens ITI-GEN) ), and ITI-GEN (right) prompts. tSA=High Cheekbones.
22

---

"persoo"
Figure 12: Cross-attention maps during the denoising process with HP1 (left), HP2 (middle, equal #tSA tokens ITI-GEN) ), and ITI-GEN (right) prompts. (SA=High Cheekbones.
23

---

"pensonn"
ahibbyy
"gersoon"
Seeding
Chiipn Engut
Inpar
Figure 13: Cross-attention maps during the denoising process with HPI (left), HP2 (middle, equal #tSA tokens to ITI-GEN) ), and ITI-GEN (right) prompts. iSA=Chubby.
24

---

Step
ehinboo
Beeliyy
Curper Inpui
St
68
Figure 14: Cruss-attention maps during the denoising process with HP1 (left), HP2 (middle,
equal #tSA tokens to ITI-GEN ), and ITI-GEN (right) prompts. (SA=Chubby.
25

---

"headshot"
"headshot"
Step
"person"
"with"
"gray"
"hair"
Input
A
"of"
"a"
Output Input
"A"
""
80
S1
82
Output
19
20
Figure 15: Cross-attention maps during the denoising process with HP (left) and ITI-GEN
(right) prompts. ISA=Gray Hair.
26

---

"headshot"
"headshot"
Step
"person""
"person"
"with""
"gray"
"hair"
Input
$
"of"
"
Output Input
"A"
"a"
80
S1
62
Output
16
17
20
45
Figure 16: Cross-attention maps during the denoising process with HP (left) and ITI-GEN
(right) prompts. ISA=Gray Hair.
27

---

"headshot"
"headshot"
Step
"person"
"person"
"with"
"gray"
"hair"
Input
"of"
"a"
Output Input
AA"
80
S1
Outpur
14
15
16
19
20
25
30
Figure 17: Cross-attention maps during the denoising process with HP (left) and ITI-GEN
(right) prompts. ISA=Gray Hair.
28

---

"gersSio"
"eeudatiol
"garsolr"
Stage
90
Output
Ovipat
meadssel!
H&r
"parsos"
Deaniin
K
off.
K
50
Stage
Inpel
*
30
Oerout
Iapal
Chrppet
K
off
Stape
Inpurt
80
Curgut
lapal
Output
BER
Theamate
12H
"perssnn"
Stape
Ingur
Ourgun
Cuippt
handdahai
"porsin""
Stage
* padiias
Ousqun
Iuput
Output
Griening
IB
"perosnn
Stage
Styn
Chuiput
"panson"
"Deumiin"" "milla
Chitpat
Stage
Stage
"partonn
comaling
Thand
"parson""
Inpei
Chippat
Lapot
Outpul
Stage
Figure 18: Accumulated cross-attention maps for the denoising process in our proposed prompt switching experiments I2H and H2I: ISA Smi ling.
29

---

"porsso""
"oigh
"gorsol
Suage
20
63
Ouiput
Cuipat
meatlabel!
Har
"parsce""
Amiddchas
K
ro
K
50
Stage
Inper
*
Ourper
Impal
COU -O QUaR
Outpul
Stage
.K' rpendebil
HEDE
"patwon"
"2igo
"sliesk"
K
96
Stape
Ingur
20
68
Ourpue
Iapal
Outsul
HERL
"partar"
12H
"persson"
Stape
Teger
Ourgen
Topat
nagh
egoaok
Chiiput
Stage
"weadstiol"
BERL
"partion"
"cheskk
-K
Stage
"goreoo
Output
"pariier"
Stage
meadshal!
HERE
"porecn"
"eigh
"aheak
Chrtout
Chirtert
Stage
mundehok
Stage
$5..0
Isger
Trendohicl
"parson
"higo
"chesk
Ouiput
Iapat
K
haadahhi
"parsion"
Chitpet
Output
Stage
Figure 19: Accumulated cross-attention maps for the denoising process in our proposed prompt switching experiments I2H and H2I: ISA High Cheekbones.
30

---

BRI
"perstio
"Deeto"
"gersol
Stage
Ouipput
Ouiput
K
meadisbl!
of
"parsoo
"chibbay
"ahesso
K
50
Stage
Insut
*
30
62
Inpat
Output
Stage
KK'
Beasity
K
90
Stape
Inpert
30
Charpes
Iapal
aanso" Output
12H
K
Beasing
"persann
Stape
Toger
Ourgen
Ouipput
BERC
"persioi""
Stage
20
Output
Impat
Output
"pansint
Stape
"ggat
meadetol!
HERE
"pirsto"
"chuubly
"aheeso"
Chrtpar
Chitrett
Stage
Stage
"partonn
Thand
"parsion"
Inpui
Ouiput
lapat
90
Outpul
Stage
Figure 20: Accumulated cross-attention maps for the denoising process in our proposed prompt switching experiments I2H and H2I: ISA Chubby.
31

---

Traidebao
HERE
Tueddenn
Twittt
Stape
Inper
"
83
Taper
ogrial
Culpat
38.E
Euput
K
Ortiput
2A
Iaper
Outvaa
Iupal
Output
Stage
"noudation
geraoaa
hotelahoi
*
"a""
K
"o""
S1
Chatput
Ispat
Outuan
Iupal
Outpul
Stage
R2ZT
"gareon"
K moaddauur
12H
20
Stage
S1
68
Ourpun
Inpol
K
Cuipptt
Stage
12
"parson""
"K''
Stage
Impst
Outpen
Imput
2955
Ouiiput
EIZE
roaddanc
"parsoe
Stage
Cutiput
"pordcm"
Stage
muadehan
*parson"
"parsion
Eaput
Ouiput
Iapat
90
51
Chitpet
Stage
Figure 21: Accumulated cross-attention maps for the denoising process in our proposed prompt switching experiments I2H and H2I: ISA Gray Hair.
32

---

ISA Chubby
016
011
0.2
0.15
6.1
6:5
DIES
Amplitude
Central moment
Figure 22: Histograms for cross-attention analysis in prompt switching experiments 12H and H2I: tSA Chubby.
ISA Gray Hair
13A sokena
00
D420
0005
0.500
0.14
0.66
Amplitode
Central
HP (1)
ITI-GEN
Figure 23: Histograms for cross-attention analysis in prompt switching experiments I2H and H2I: tSA = Gray Hair.
Dugo proup
29
procaps
Oagut
Accumulated cross-attention maps for Base prompt, FairQueuing and ITI-Gen, ISA=Smiling, High Cheekbones. Same setup as Sec 3.2 of the main manuscript (e.g.. Fig.4). The mid column presents FairQueue, with Base prompt (T) Stage and ITI-Gen (I) in Stage the left column is only based on Base prompt and the right is only based on ITI-Gen. Note Base prompt behaves similarly the HP in forming good global structures frames); in the second stage, the iSA token of ITI- TTI-Gen can attend to ISA-related regions (c.g., eyes and mouth for Smiling, or the lower half of the face and cheeks for High Cheekbones, see magenta frames) and enhance associated facial features.
Figure 24:
the first stage (annotated by red
33

---

"of"
"a.
Smiling
100
100
0-
0.18
0.02
0.03
0,04
0.05
0.06
0.07
High Cheekdonee
100
100
0.08
0.10
0.12
0.16
0.18
0.02
0.03
0.04
0.05
0.06
0.07
Amplitude
Base Saammpp//airQuuuSSStuuge 1] (T)
ITI-GEN (I)
Figure 25: Histograms for non-tSA tokens in the first stages, ISA=Smiling and High Cheekbones for Fig. 24.
34

---

A.3 More on Ablation Studies
A.3.1
Analyzing the Effects of Attention Amplification
In this section, we provide illustrations for the ablation study. Our results in Fig. 26, illustrate the effect of attention application with different scaling factors (c). Notice that at c = 0 the tSA expression may still be lacking but increasing results emphasized tSA expression.
Smiling
CHO
Ca8
C=8
C:10
C:12
Figure 26: Illustration of FairQueue samples utilizing different attention amplification scaling factors (c), ISA=Smiling. In each row, we utilize the same seed and prompt queuing transition point.
A.3.2 More Illustrations for Training Once-fo-All Tokens
In Fig. 27, we provide more illustrations for the analysis on Revisiting Training Once-for-Al Token.
ITI-Gen:
dicter
(a) Random Sample
ITI-Gen:
Risson
decter"
(b) Random Sample
TAD
dactar"
Ouis:
(c) Random Sample 4
Figure 27: Illustration of samples generated by ITI-GEN and FairQueue with a new Base Prompt T" = Ex("A headshot of a doctor"} via pre-pending, Notice that FairQueue im- proved sample quality and ability to preserve the original sample's semantics while mainly adapting only the tSA.
35

---

A.3.3 Human-recognized Assessment Comparing ITI-Gen and FairQueue Quality
In this section, we carried out human user study to evaluate the sample quality and faimess of the generated sample from FairQueue against ITI-GEN. Specifically, we utilize the same seed to generate 100 sample pairs with ITT-Gen and FairQueue for ( Smilligg High Cheekbones, Gender, and Young J. Then, utilizing Amazon Mechanical Turk we conduct 2 tasks:
Quality comparison by A/B testing: Human labelers select the better quality sample between ITI-Gen and FairQueue (from the same seed). Each sample was given labelers.
Fairness comparison by human-recognized tSA: labelers identified the tSA class for each sample. The final label was based the majority of 3 labelers. labelers were also given "unidentifiable" option the class could not be determined. Finally, the labels were used to measure FD.
Our results in Tab. reveal that FairQueue generates better quality samples than ITI-Gen (>62.0% preference) and Tab. 4 shows that FairQueue achieves competitive fairness with ITI-Gen. Overall, this aligns with our quantitative results in Tab. 1.
Table 3: A/B testing: Human assessment comparing quality between ITT-Gen FairQueue for 200 samples per tSA. Col and indicate the percentage of labelers that prefer the method's sample quality. A larger value is better.
ITI-GEN
FairQueue
Smiling
1.3%
98.7% 08.7%
High Cheekbones
2.7%
97.3%
Gender
33.0%
67.0%
Young 38.0%
62.0%
Table 4: Fairness comparison by human-recognized (SA: ITI-Gen vs FairQueue for 200 samples per tSA.
Human assessment to compare FD for
VEN-GEN FD(L)
FairQueue FD(L)
Smiling
0.106
0.014
High Cheekbones
0.144
0,021
Gender
0.014
0,014
Young
0.014
0.028
36

---

A.4 More Illustration
In this section, we provide more samples generated by FairQueue based on the setup in Sec. 5. Recall that here we utilize the base prompt T Er"A headshot of person" and consider the tSAe (Male, Young, Smiling. Low Cheekbones, Pale Skin, Eyeglasses, Mustache)) Each sample is then generated based on the same 10 fixed noise inputs.
Figure 28: Base- Base-Prompt images T with fixed latent noise input
Figure 29: FairQueue with (SA=Female with fixed latent noise input
Figure 30: FairQueue with tSA=Male with fixed latent noise input
Figure 31: FairQueue with tSA=01d with fixed latent noise input
Figure 32: FairQueue with (SA=Young with fixed latent noise input
Figure 33: FairQueue with iSA=not Smiling with fixed latent noise input
Figure 34: FairQueue with tSA=Smi ingg with fixed latent noise input
37

---

Figure 35: FairQueue with (SA=Low Cheekbones with fixed latent noise input
Figure 36: FairQueue with LSA=High Cheekbones with fixed latent noise input
Figure 37: FairQueue with iSA=not Pale Skin with fixed latent noise input
Figure 38: FairQueue with tSA=Pale Skin with fixed latent noise input
Figure 39: FairQueue with (SA=no Eyeglasses with fixed latent noise input
Figure 40: FairQueue with tSA=with Eyeglasses with fixed latent noise input
Figure 41: FairQueue with iSA=no Mustache with fixed latent noise input
Figure 42: FairQueue with iSA=with Mustache with fixed latent noise input
A.5 Evaluating Minimal Linguistic Ambiguity for tSA
In this section, we provide an experiment to search for tSA with minimal linguistic ambiguity: To this, we utilize the and their respective HPs in Tab. Then, with these HPs, we generate 500 samples per category of the individual tSA. Finally, we classify the respective samples and evaluate the accuracy of the T2I model generating samples with the respective categories of the (SA. Our results in Tab. 5 report the accuracy of the T2I generative model in accurately
38

---

interpreting the HPs to generate the respective tSAs. Based on our results, we can determine that the ISAE (Male, Young, Smiling, High Cheekbones] are with minimal linguistic ambiguity, as their HPs can be easily interpreted by the T21 generator as seen by the high accuracy.
Table 5: Accuracy of the T2I model in generating the tSA with the HP. See Tab. for list on all the HPs.
ISA
Positive Prompts
Negative Prompts
Nale
99.8%
99.2%
Young
97.8%
98 98.4%
Smiling
98. 78.2%
97. 97.8%
High Cheekbones
96.6%
98.8%
Pale Skin
91.3%
7.8%
Eyeglasses
97.4%
2.4%
Nustache
90.4%
13.6%
Gray Hair
97.2%
22.4%
Chubby
98.2%
30.4
B Experimental Details
B.1 Details of Calculating Directional Loss for Prompt Tuning
In this section, provide more details the directional loss, Cair utilized by ITI-GEN. Recall that in ITI-GEN the direction loss takes in two components: 1) direction of Image embeddings, A I, and 2) direction of ITI-GEN token embedding, AP. For ease of discussion, utilize binary ISA. but the same concept can be simply scaled to multi-class ISA.
Specifically, we first measure AI where ITI-GEN first evaluates the mean image embedding for each category of the tSA i.e., a = E& [Ey (x*)), where E₁ is CLIP Image encoder [20] and x² are the reference image for & samples in given mini-batch. Then, we calculate the directional Image embeddings: AI, = ag - a, where and j are the different categories of the selected tSA. Then when considering direction of ITI-GEN token embedding we simply calculate AP E (P.) Fr (P)), where ET is the CLIP text encoder.
B.2 Details of the Ambiguities in Text Prompts
Ambiguities arise because of the potentially multiple interpretations of the same utterance. Ambiguity well-known concept in Large Language Models (LLMs), and recently there have been some attempts to understand the impact of these ambiguities in the intersection of the LLM and image generation models, a.k.a T21 models. Among different types of ambiguity, three major types can affect the quality of the T21 generation [47]:
Syntax: where there could be different interpretations of the same text. As an example, in input prompt "the mouse looks at the cat standing on rug", it is not clear whether the mouse standing
the rug or the cat.
Semantics: where the words within text have multiple meanings. As an example, "a photo of bat", it is not clear whether it refers to a nocturnal flying mammal or the flat wooden club used in sports like baseball or cricket (with 'cricket' itself having two different meanings ;)). underspecification where the used text prompt can not completely describe the required attributes in the image. For example, "a woman with th light hair" can not specifically determine the category of the hair color.
Within the context of this paper, ambiguity is generally used with the Hard Prompts (HP) where we append the description of specific (SA to the base prompt. For example, considering the base prompt "a headshot of person", and considering the 1SA=Smiling one can create the HP as "a headshot of person smiling", and "a headshot of a person not smiling". In our paper, we empirically realize that for some tSA, creating HP like this will result in strong baseline for fair T2I generation, observed by our quality and fairness metrics. We call these, tSAs empirically found that tSAs like Young,
with minimum linguistic ambiguity, and as an example, Smiling, Gender fall within this category. Due to their superior performance, HP with these tSAs is used as our pscudo-gole standard in the analysis of the quality degradation issue in ITI-GEN is
39

---

Sec. 3. However, other tSAs suffer ambiguity in either finding proper terms to define them or inferior performance in terms of the quality or fairness of the generation. This enforces using more advanced techniques like prompt learning to overcome these ambiguities.
B.3 Details of Model Hyper Parameters
Models and training hyper-parametert In our experiments, we follow the same setup as ITI-GEN [16] and utilize Stable Diffusion v1.4 [1] as our T2I generator. Then when implementing ITI-GEN we utilize an S; with a token length of per tSA which is optimized based on learning rate of Ir = 0.01. For reference datasets, we utilize [16] readily available datasets with contain 200 reference images per category of each tSA. An Adam [48] optimizer is utilized during prompt learning. For sample generation, we follow the recommended diffusion steps of = 50 and utilize an Attention scale of c 10 and an Attention Queuing transitioning step =10.
Hard Prompt. Tab, illustrates the list of HPs utilized in our experiments. We remark that not all ISAs have a clear HP. For example, with the tSA=Skin Colour it is difficult to find reasonable HP to describe an individual with a specific skin tone.
Table 6: Hard Prompts utilized in T2I generation.
ISA
Positive Prompts
Negative Prompts
Xale
"I. headshot person Young"
person Nale"
"A headshot persen Fenale"
Young
"A. headshot of a person with Smiling"
"g person with no/ei thout Smilingg
person 0ld"
Smil ing
headshot
Righ Cheakbonea
"A headshot of person with high cheelthoras'
"A headahot: of a peraos with los sheakbonea"
Pale Skin
"A headsbot of
with headshat peraon
ekin" person eyegl gasnans mustache"
"A headshot of parson with no/without pale skin
Byaglassea
headshot,
"A headshot. "A headshat of person without mustache"
na/vithout eyaglasses"
Muatache
Gray Hair
"A headshot of person with gray bais'
"A headebot of pereon without gray hair'!
Chubby
"AA headahot of person chubay""
"A bandshot of
no chubby"
Stable Diffusion Version. We verified that the problems of poor performance with Hard Prompts as indicated by Zhang et [16] would still persist even in more recent versions of Stable Diffusion e.g., SD 3.0. This issue can be observed by simply inputting the prompt "A headshot of a person without glasses" to SI) 3.0 where the generated samples still frequently have the wrong tSA class ("with glasses") indicating this ambiguity still exists. For example, when utilizing the same setup as Tab with HP (from Tab. 6) and SD3.0, our results show poor fairness performance for Eyeglasses (FD=670e-") 3) and Pale Skin (FD=580e=") )).
Increasing the size of the reference dataset (2k reference images per category per tSA). We verified that increasing the size of the reference dataset does not resolve ITI-GEN's quality degradation. Specifically, we repeated the experiment in Tab. with iSA Smiling with 2k reference samples per class. Our results measured an FD= 27e -3 TA=0.591, FID==9.2,, and DS=0.532 which is similar to the results in Tab. (based on 200 reference images). This indicates that the core problem may not be the data size, and may need specific data curation (including sample pairs with only semantic differences in (SA, and similar semantics elsewhere), which poses scalability and applicability challenges.
B.4 Computation Resources
Tab. illustrates the amount of the compute including the GPU Hours for different steps of our research and the estimated carbon emission for our experiments.
Table 7: Estimated Computation time. The carbon emission values computed using https:
/ //mlco2, amlco2.github github. io/ impact.
Experiment
Hardware
GPU Hours
Carbon emitted (kg)
T21 Sample Generation
RTX3090
25.0
4.87
Directional Alignment analysis
RTX3090
0.25
0.048
Cross Attenton Analysis
RTX3090
1.0
0.195
Prompt Leaming
RTX3090
1.0
0.195
Total:
27.25
5.308
40

---

B.5 Details of the Evaluation Metrics
In this section, we provide more detail on the evaluation metries used in our work.
Fairness. We use the fairness discrepancy (FD) metric which compares tSA distribution in generated images with ideal uniform distribution [27]. In this metric FD=0 would indicate perfect fairness. Following [16, 11, 36] we use a combination of CLIP [20] models, human evaluators, and off-the- shelf models for predicting the distribution of the tSA in target images. We remark that to evaluate FD we utilize classifiers that achieve reasonably high accuracy when evaluated on CelebA [29] reference dataset, as seen in Tab. 8.
Table 8: Accuracy of the iSA classifier when evaluated with CLIP/off-shelf classifier.
ISA
Accuracy
Male
99.6%
Young
98.8%
Smiling
98.6%
High Cheekbones
97.2%
Pale Skin
98.4%
Byeglasses
96.7%
Mustache
87.4%
Gray Hair
88.3%
Chubby 90.4%
Quality. Text alignment (TA) [37, 22] and Fréchet Inception Distance (FID) [38] are utilized quality metrics. Specifically, Text-alignment is based the notation that sample generated based on ITI-GEN prompt (P) or HP (F) for a given tSA, should retain the semantics of the original base prompt T. unless the sample has degraded in quality. For example, samples generated based on F = Er("A headshot of a person young") should still retain the semantic of T = ET("AA headshot of person""). Therefore, images (based on the P or F) with base prompt T using text-image cosine similarity in CLIP's feature space [37, 22]. FID [38] then compares the feature statistics of the generated samples against a reference fair FFHQ [49] dataset from the CleanFID [50] library.
evaluate quality, we compare the generated
Semantic Preservation: When faimess approach enforces fairesss w.r. tSA, changes related to that iSA are more favorable. Considering our setup, for the same latent code Z, the generated images with base prompt and fairness scheme are favorable to be different only in features related to tSA, and similar in other features. To measure this behavior, for each input latent code Z, the generated image by querying the T21 model with base prompt T considered as the reference image. Then, for the same latent code Z, we measure the similarity of the generated image by querying the T2I with HP and ITI-GEN prompts with that reference image using DreamSim [39] features. DreamSim improves upon existing semantic measurement by considering both low-level features, mid-level, and high-level features for more holistic semantic measurement.
tue.mpe. de/; http:://ggttttb..o///c////////// http://// caa///uniieearrr//pppreeee.ooooo
'nttps://trust.te tue
41

---

B.6 Visualizing the Learned Embedding vs Base Prompt
In addition to the direction loss values provided in the main paper show that AP not aligned well with AT as pseudo-gol standard direction-wiie -hhch increases the possibility of encoding unrelated knowledge and leading distorted learned tokens. Here, we provide an additional visualization of the embedding in Fig. 43 to show the difference between the learned embedding in ITI-GEN and the embeddings of the base prompt and hard prompt. one can see, even though the embeddings the base prompt and three different variants of the hard prompt are clustered well, the embeddings of the learned tokens ITI- TT-GGEN are quite far from this cluster increasing the possibility unrelated concepts and degrading the generation quality, One may argue that using regularizer can push these embeddings towards this cluster. However, our experimental results suggested that this will result in a compensated performance in terms of the fair T2I generation.
the encoded
EiGen BSA seient
Sonee Convenent
M Corverent
tedthnng: 154 poer
(n) Gender (Male)
(b) Young
(c) Smiling
Figure 43: PCA Analysis on CLIP-text embedding for ITI-GEN, well-defined HP and Base Prompt. Utilizing a pre-trained CLIP text encoder we attain the text-embedding for the i) Base Prompt T="A headshot of a person", ii) ITI-GEN tokens, and iii) selected well-defined Hard Prompts for [Gnder (Male), Young). Then we apply Principle Component Analysis (PCA) for dimensional reduction. We remark that these same text embeddings are later used in the SDM for sample generation.
C Limitations and Broader Impact
In this section, we discuss some limitations regarding our work as well as some potential societal impacts that it may have.
Limitations. Firstly, FairQueue work follows ITI-GEN which utilizes a reference dataset to first optimize ITI-GEN prompt P. This setup requires dozens of reference images for each category of the tSA which may not always be readily available. Second, although FairQueue provides better quality and semantic preservation of the original base prompt sample, its sample generation is still prone to experiencing entanglement in certain tSA. Specifically, entanglement in tSA could result in some unwanted augmentation of non-tSA during sample generation.
Broader impact. Our work FairQueue takes a significant step towards enhancing fairness in text- to-image generation. By improving the quality of samples generated through fair text-to-image algorithm, we facilitate greater adoption of these techniques by the general public. This increased adoption can help prevent the perpetuation of unwanted biases in everyday applications, promoting more equitable and inclusive use of technology in society.
42

---

D Related Work
Text-to-Image Generation. There was surge in text-to-image (T2I) models in the last years. exemplified by models like DALL-E [7, 511, Stable Diffusion [1], Midjourney [52] (with over 16 million active users in July 2023), and many others [53, 24, 54]. These models have demonstrated their capability to generate high-quality samples across different domains and new applications are defined around these models in different areas such as art [55], design, and even medical imaging [56]. In contrast to the earlier generative models which primarily served research purposes within research and scientific settings, current T2I models offer much broader accessibility [57]. However, this increased accessibility also amplifies the potential consequences of bias within these generative models [27]. Our work aims to address this issue by mitigating bias and prompting fair T2I generation.
Fairness in Generative Models. In generative modeling, faireess is usually defined as equal representation where all categories of tSA are supposed to be represented with similar probability. Different approaches are proposed to improve the fairness of the conventional generative models including weak supervision to achieve fairness with the importance weighting of a fair dataset [28], transfer learning from large biased dataset to a small fair dataset [30, 31], enforce uniform sampling from the latent feature space [58]. In the context of fair T2I generation, using the pretrained T21 models and adapting prompts for a fair generation has recently attracted a lot of attention. Bansal al. [17] proposes the use of "ethical interventio" prompts for text-to-image generators to encourage the concept of independence w.r.t. the the sensitive attributes. Specifically, these ethical intervention prompts can be appended to the original prompt e.g., "a photo of bride from diverse cultures" and "a photo of person wearing a hat irrespective of their [SA]" Furthermore, the work substantiates this approach by mentioning that these neutral language prompts exist in the training data. Overall, these additional prompts have empirically been shown to produce diverse samples (a different definition of diversity w.e.t SA) and high-quality samples (human-voted) Chuang et al. [36] proposes to project of the biased direction of the text embeddings as form for bias mitigation. Specifically, given known sensitive attribute, we are able to minimize the equalization loss and find a text embedding that is of equal distance from the biased prompt's embedding. Then utilizing this optimized (debiased) text-embedding. we are able to generate samples with a fairer SA distribution. In addition, recently prompt learning has been proposed to learn the tSA knowledge from a set of reference images in ITI-GEN [16]. In this work, we address the issues that arise due to unrelated concepts being encoded in these prompts, leading to defects in the cross-attention maps and degrading the image generation quality.
Prompt Learning. Prompt learning has recently been shown be very effective and efficient way of adapting pretrained large vision-language models to different downstream tasks like zero-shot classification [34, 33, 59], image editing [21], personalization of diffusion models [22, 60-62), and few-shot image generation [63]. In this work, we analyze the potential issue of prompt learning in the context of fair T21 generation by analyzing the cross-attention module in the presence of the learned prompts and propose simple yet efficient approach to integrate prompt learning more efficiently for fair T2I generation.
43

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
[Yes] The main contribution of this paper is the analysis of the shortcomings in prompt learning for fair T2I generation, and then proposing new approach to address these shortcomings.
Answer:
Justification: [Yes] The claims are justified with both provided extensive analysis in cross- attention map (in the main paper and Supp), together with experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? [Yes] There are some limitations inherited from the setup used in the literature regarding the required dutaset to learn the prompts and also the possible entanglement between different target Sensitive Attriuttes (1SAs)
Answer:
Justification: [Yes] Please see Sec. for the detailed discussion. Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create a separate "LLimitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the
implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recogmition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimeess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
44

---

Answer:
[NA]
We don't 't have theoretical assumptions and proofs.
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes] In addition to the basic experimental details provided in the main paper, the detailed setup for enabling reproducibility is discussed in Sec. B. In addition, to facilitate reproducibility we have also provided the anonymous link for the code used in this paper.
Justification: [Yes] Please check Supp for the code, and Sec. B for the details of the experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (c.g., a large language model), then there should either be way access this model for reproducing the results way reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g.. to registered users), but should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
45

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes] The anonymous link to the code is provided in the Supp. In addition, all datasets used in this paper are publicly available. The link for datasets is also added in the readme file the uploaded code.
Justification: Yes]] Please refer to Supp. Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.. cc/ public/guiden http///dddee/oeeuubiiiinen Cat///ooeeeeeeeeeeeeeeeeoeaa ss wwwiisssssosssoiiiee comPolicy) for more details.
While we encourage the release of code and data, we understand that this might not possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
//nips. wttppbllc/gguiiiiccccceeeooooeee htt///uddd////// Seriides/ooeeuuue /CodeSubmiiiicciccccee ssi siiooliccy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reprodueible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes] All the details necessary understand the results are included in Sec. B. Justification: [Yes] Please see Supp Sec. B. Guidelines:
The answer NA means that the paper does not include experiments. The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes] For the experimental results provided for the comparison of our proposed
FairQueue with SOTA approach ITI-GEN as mentioned in Sec. 5 the experiments run 5 times and the error bars are included in the Tab.
Justification: [Yes] Please refer to Tab. 1.
Guidelines:
The answer NA means that the paper does not include experiments. The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing some parameter, or overall run with given experimental conditions).
46

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g,, Normally distributed errors). It should be clear whether the enror bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma enror bar than state that they have 96% CI. if the hypothesis
Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
The detailed amount of compute resources together with carbon emission estimations are provided in Supp Sec. B.4.
Answer: [Yes]
Justification: [Yes] Please check Supp Sec. B.4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttp///eerppp////eee/eee/eeeeeee cc cc/ccc/pp//c ptcppppic//////eeeccceeccclllllt Ceteiii//iiieeeee cewuidelineeneeeoc inee??
Answer: [Yes] We carefully read the NeurlPS's 's code of ethic to make sure we follow it to
the best of our ability.
Justification: [NA]
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., there is a special consider- ation due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes] The broader impact of our work is discussed in Sec. C which enables safer adoption of T2I models by addressing the fairness concerns.
Justification: [Yes] Please see Supp Sec. C
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations
47

---

(e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point out. For example, it legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA] We use the publicly available stable diffusion as our T21 model in our research, and considering that this model already has the safe-checker to prevent generating NSFW content, we do not add any additional safeguards in our research. Justification: [NA] Please read our answer above.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
All dataset, codes and other assets used in this paper are cited properly throughout the whole paper.
Answer: [Yes]
Justification: [Yes] Please check the citations.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, paperswi thcode. com / datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
48

---

For existing datasets that are re- ee-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged reach the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA] We are not introducing new assets in our reserach.
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA] We do not have human subjects in studies.
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA] We do not have human subjects our sludies.
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
49


Reviews:
[{"review_id": "WqwnhUigpH", "reviewer": "Reviewer_89En", "summary": "In the field of fair text-to-image generation, the ITI-GEN paper demonstrated good performance by learning and embedding tSA. However, this paper argues that the embeddings learned in ITI-GEN can include unrelated attributes, resulting in noisy embeddings and significantly degrading the quality of the generated samples. To address this, the paper proposes using prompt queuing to apply tSA after a certain timestep and employs attention amplification to prevent the weakening of tSA.", "questions": "- Is there a difference between the original ITI-GEN and your re-implementation as I mentioned or did I just misunderstand? (I am not an expert of ITI-GEN)\n- Wouldn't this problem be naturally resolved if a large number of reference images were used to learn the embeddings?", "limitations": "The authors describe the limitations and potential societal impact in their supplemental material.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "The paper provides a highly detailed analysis of how the learned tSA embeddings are actually reflected in the images using attention maps. It also demonstrates that the learned embeddings do not properly reflect differences in the images.", "weaknesses": "First, the degradation seems severe compared to the ITI-GEN paper. If I understand correctly, ITI-GEN appears to aggregate all learned tSA embeddings by summing them, while this paper seems to concatenate a large number of tokens, as indicated by (p+q)*d. This difference in implementation may be causing a decline in baseline performance and requires an explanation.\nAdditionally, the proposed method to address this issue seems naive, and there is a lack of ablation experiments. Also, even with the proposed method, there appears to be a degradation in sample performance compared to HP. For example, in Figure 2, an originally female image appears to have been changed to a male image using the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> R1Q1: “ITI-GEN appears to aggregate all learned tSA embeddings…This difference in implementation may be causing a decline in baseline performance”\n\nR1A1: We thank the Reviewer for their comment, and apologize if it was unclear. We clarify that our code of  ITI-Gen as a baseline is **100% identical** to the original ITI-Gen paper as we utilize the official GitHub source code released by ITI-Gen authors (line 282). Therefore, **we are certain that the quality problem is sourced from ITI-Gen**. \n\nFor multiple tSAs, following ITI-Gen exactly, we used the aggregation mechanism denoted in ITI-Gen’s paper Eqn 3. We will include these details in our paper for further clarification.\n\nWe reassure the Reviewer on the above matters by directing them to the submitted source code to observe the existence of this aggregation function: **[Annon. Link line 489] \\> iti_gen \\> model.py (line 225-230)**.\n\nWe hope the provided details address the Reviewer's concerns.\n\n$ $\n\n>R1Q2: the proposed method to address this issue seems naive.\n\nR1A2: We respectfully clarify that proposing a solution for the quality degradation issue is **non-trivial**. Our proposed scheme stems from uncovering this issue in the context of prompt learning for fair T2I generation. **Only through detailed analysis of the issue (highlighted as a core contribution) and examination of attention maps of individual denoising steps, were we able to propose a simple yet effective solution.**\n\nSpecifically, first, we uncover the issue of quality degradation in ITI-Gen (based on their code). Then, we attribute this issue to the sub-optimal directional loss used for prompt learning in ITI-Gen leading to encoding unrelated concepts in learned tokens (Fig 1a-col3). We further trace back the effect of these learned tokens in sample generation by analyzing the cross-attention mechanism (Fig. 3), and propose H2I and I2H analysis to analyze the effect of the learned tokens on different stages of the denoising process.\n\n**Only upon this analysis** do we recognize that the distorted ITI-Gen tokens affect the global structure in the early denoising steps, but work well in enforcing tSA expressions if the global structure is formed properly–by natural language prompts like HP and Base Prompt. This inspires us to propose prompt queuing to bypass degrading global structure in the early steps of denoising.\n\nAlthough Prompt queuing prevents degrading global structure, as there are no tSA-related tokens in the early stage, it results in reduced tSA expression. However, our H2I analysis suggests that with proper global structure, tSA tokens attend to tSA-related regions to enforce fair generation. Inspired by this analysis, we propose attention amplification to enlarge the effect of tSA tokens and intensify tSA expression to achieve fairness without degrading image quality.\n\n$ $\n\n>R1Q3: there is a lack of ablation experiments\n\nR1A3: We respectfully correct the review that ablation studies are included in Supp. A.2 (mentioned in line 297) due to space limitations. In these ablation studies, we experimented with different intervals for prompt queueing and scales for attention amplification. These ablation studies verify the effectiveness of individual components. \nWe additionally also compared FairQueue against ITI-Gen in the application of  Training-Once-For-All. Our results again demonstrate FairQueue improved quality over ITI-Gen. \n\n$ $\n\n\n>R1Q4: with the proposed method, there appears to be a degradation in sample performance compared to HP… e.g. in Figure 2, female image appears to have been changed to a male.\n\nR1A4: We clarify that based on the DS values (measured on 1k samples per tSA), our proposed approach on average has better performance in semantic preservation compared to HP, e.g, 0.284 and 0.330 for ‘Smiling’ and ‘High Cheekbones’ compared to 0.323, and 0.332 for HP (DS($\\downarrow$) is a metric for measuring semantic preservation, see our paper). We remark that comparing degradation of two approaches based on a single sample might not be that easy, as in Fig. 2, HP also shows some discrepancies similar to what indicated by Reviewer, e.g., in col. 4, row 5, HP changes a male to female.\n\nImportantly, we remark that HP is only applicable to a few tSAs which have minimal linguistic ambiguity (MLA). For most tSA which are non-MLA (see Supp A.4), HP has a poor performance in enforcing fairness (as also discussed in the ITI-Gen paper). In contrast, as shown in Tab. 1, FairQueue delivers a good performance in terms of both quality and fairness across various tSAs.\n\n$ $\n\n> R1Q5: Is there a difference between the original ITI-GEN and your re-implementation\n\nR1A5: As answered in R1A1: Our deployment of ITI-Gen as a baseline is 100% identical to the original work. \n\n$ $\n\n\n>R1Q6: Wouldn't this problem be naturally resolved if a large number of reference images were used to learn the embeddings?\n\nR1A6: We believe that simply increasing the reference data size may not resolve the problem of unrelated knowledge. Specifically,  the existing ITI-Gen reference dataset is already relatively sizable. For example, in CelebA, 200 samples are used for each class of tSA, and yet this problem persists. \n\nTo verify this, we conduct the same experiment with tSA Smiling with 2k reference samples per class. Our results measured an FD=$127e^{-3}$, TA=$0.591$, FID=$89.2$, and DS=$0.532$ which is similar to Tab.1, with no improvement by using a large number of reference images. This indicates that the core problem may not be the data size, and may need specific data curation (including **sample pairs** with only semantic differences in tSA, and similar semantics elsewhere), which poses scalability and applicability challenges.\n\n$ $\n\n**We appreciate the positive feedback and valuable comments of the reviewer. we sincerely hope that reviewers could consider increasing the ratings if our responses have addressed all their questions**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FIRmSrKnaW", "reviewer": "Reviewer_8Wzu", "summary": "This paper introduces an approach to improving fairness and quality in text-to-image generation models by rethinking prompt learning. The current Inclusive Text-to-Image Generation (ITI-GEN) methods often degrade image quality due to suboptimal training objectives and embedding misalignment between learned prompts and reference images.\nKey innovations include Prompt Queuing, ensuring regular and fair prompt usage, and Attention Amplification, enhancing the impact of prompts on image generation. Experiments show that FairQueue improves both image quality and fairness compared to state-of-the-art methods.", "questions": "- I'm curious why the experiments were conducted on only one dataset. \n- Some of the metrics used in the paper are unfamiliar. Are these metrics validated, and are they commonly used in other research papers as well?\n- Additionally, I would like to know how you plan to address the limitations mentioned in the limitations section.", "limitations": "They mentioned limitations and broader impact in the paper.", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "- Through detailed analysis, they revealed the limitations of the previous approaches, specifically how suboptimal training objectives and embedding misalignments degrade image quality.", "weaknesses": "- The readability of the paper is poor.\n- The evaluation of the model is insufficient, as there are only evaluations on the datasets of human faces, which lacks in evaluating the generalizability of the model. More experiments on other datasets are needed. \n- The paper emphasizes quantitative metrics but does not give enough attention to qualitative assessments. Including user studies or expert reviews could provide valuable insights into the perceived quality and fairness of the generated images, complementing the quantitative data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">R2Q1:The readability of the paper is poor.\n\nR2A1: We remark that Reviewer 89En scored our presentation excellent, and Reviewer iQtH described it as “well written and logically structured.” We will review the paper once again.\n\n$ $\n\n> R2Q2: I'm curious why the experiments were conducted on only one dataset.\n\nR2A2: We respectfully clarify that following ITI-Gen paper [16], our work conducts experiments on **3 datasets** including CelebA [29], FairFace [45] and Fair Benchmark [44] (line 277). \n\n$ $\n\n>R2Q3: The evaluation of the model is insufficient, as there are only evaluations on the datasets of human faces, which lacks in evaluating the generalizability of the model.\n\nR2A3: We remark that for the type of datasets, we followed literature [16, 28, 30] on fairness. Specifically, as the literature is mostly focused on the human-centric tSA (e.g., “Gender”, “SkinTone”, …) expressed by human faces, existing fairness datasets are developed and curated based on this concept.\n\nWe remark that our approach is **agnostic to the dataset type** with no restriction regarding the type of content. However, to the best of our knowledge, the fairness community has not developed a curated dataset for this purpose. As an example, the FairFace dataset is curated for fairness study to include 6 skin tone classes and considers the Individual Typology Angle [44] to ensure that race is uncorrelated with skin tone. However, a similar effort has not been carried out for non-human concepts in fairness literature to enable studying fairness in these concepts.\n\n$ $\n\n> R2Q4: The paper emphasizes quantitative metrics but does not give enough attention to qualitative assessments.\n\nR2A4: We thank the Reviewer for the comment. First, we clarify that we have tried to provide extensive qualitative assessment between ITI-Gen and FairQueue samples in [Annon. Supp link line 489]  \\> more resources \\> Tab_1 (due to size limitation) containing 1.8k samples per approach. Nevertheless, we follow Reviewer’s suggestion and include a user study experiment to compare ITI-Gen and FairQueue. Specifically, we utilize the same seed to generate 100 sample pairs with ITI-Gen and FairQueue for ‘Smiling’, ‘High C.’, ‘Gender’, and ‘Young’. Then, utilizing Amazon Mechanical Turk we conduct 2 tasks:\n\n-  **Quality comparison by A/B testing**:  Human labelers select the better quality sample between ITI-Gen and FairQueue (from the same seed). Each sample was given to 3 labelers.\n\n- **Fairness comparison by human-recognized tSA**: labelers identified the tSA class for each sample. The final label was based on the majority of 3 labelers. labelers were also given an “unidentifiable” option if the class could not be determined. Finally, the labels were used to measure FD.  \n\nOur results in Tab. A reveals that FairQueue generates better quality samples than ITI-Gen (>62.0% preference) and Tab.B shows that FairQueue achieves competitive fairness with ITI-Gen. Overall, this aligns with our quantitative results in Tab.1.\nWe will include these results in the final paper. \n\n$ $\n\nTab. A: **A/B testing:** Human assessment comparing quality between ITI-Gen vs FairQueue for 200 samples per tSA. Col 2 and 3 indicate the percentage of labelers that prefer the method’s sample quality. A larger value is better.\n\n|  |  ITI-Gen | FairQueue |\n|---|---|---|\n| Smiling | 1.3 % | 98.7 % |\n| High.C | 2.7 % | 97.3 % |\n| Gender | 33.0% | 67.0% |\n| Young | 38.0% | 62.0% |\n\n$ $\n\nTab.B: **Fairness comparison by human-recognized tSA:** Human assessment to compare FD $\\downarrow$ for ITI-Gen vs FairQueue for 200 samples per tSA. \n\n|  |  ITI-Gen FD | FairQueue FD |\n|---|---|---|\n| Smiling | 0.106 | 0.014 |\n| High.C | 0.144 | 0.021 |\n| Gender | 0.014 | 0.014 |\n| Young | 0.014 | 0.028 |\n\n\n\n$ $\n\n>R2Q5: Some of the metrics used in the paper are unfamiliar. Are these metrics validated, and are they commonly used in other research papers as well?\n\nR2A5: We assure the Reviewer that as discussed in line 140, with more details in Supp B.5, these are indeed popular performance metrics. Specifically,\n\n- FD is a common fairness metric utilized in many popular fair generative modeling works [16,27,28,30,31] \n- FID [16,38,28,30,31,35,60] is a very popular quality metric used in generative modeling.\n- Text Alignment [16,37,11,60] is a popular quality metric used in T2I models.\n- DreamSIM [39,B,C] extends on LPIPS [35,60], a popular metric to evaluate semantic similarity of images.\n\n$ $\n\n>R2Q6: Additionally, I would like to know how you plan to address the limitations mentioned in the limitations section.\n\nR2A6: We appreciate the Reviewer's interest in our work. We remark that the limitations section is included with the aim of full transparency. In this paper, as we focus on improving quality of fair generative models, these limitations, which require substantial research effort, are beyond the current scope. Considering these points, in what follows we provide some discussion for potential future works.\n\nWe first discuss attribute entanglement, which we believe is largely due to the current setups in fair generative models which lack an ideal dataset that fully disentangles the tSAs. For example, the attribute ‘Bald’ is often associated with ‘Male’ rather than ‘Female’. As a result, the entanglement issue is shared among all fairness approaches and a solution is non-trivial. However, it is indeed a very interesting research direction, which we hope to explore.\n\nNext, we address the need to have a carefully-constructed reference dataset to guide the training of ITI-Gen tokens. In future works, we may take inspiration from the few-shot generation literature like StyleGAN-Nada [35] which utilizes an expert auxiliary model to guide the training in place of reference data. \n\nWe will add these details to future work.\n\n$ $\n\n**We appreciate the valuable comments and sincerely hope that reviewers could consider increasing the ratings if our responses have addressed all their questions**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ybYVkB4E5W", "reviewer": "Reviewer_iQtH", "summary": "The authors propose FairQueue, a simple and effective solution to solve the quality degradation problem in ITI_GEN through prompt queuing and attention amplification.", "questions": "1.\tIn Chapter 4, the author proposed a method for attention amplification. Is the amplified area Si related to tSA? If so, please explain the specific impact relationship.\n2.\tAre there specific metrics to measure the relationship between generation quality and fairness?", "limitations": "The author can further improve the description of the method part of the paper and describe the innovation more clearly.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper is well-written and logically structured.\n2.\tThe proposed FairQueue effectively solves the quality degradation problem in ITI_GEN.\n3.\tThe paper is very rich in experiments", "weaknesses": "Some of the contribution points of the article can be optimized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">R3Q1: Some of the contribution points of the article can be optimized.\n\nR3A1: We appreciate the Reviewer’s comment and would like to assure the Reviewer that we have already optimized our contributions with further discussion in the supplementary (due to space limitations). Specifically,\n\n- In Supp. A.1., to optimize the delivery of our findings in cross-attention analysis, we provide substantial qualitative samples on different tSAs to demonstrate the characteristics of natural language and ITI-Gen prompts. Specifically, ITI-Gen prompts affect the global structure in the early denoising steps, while effectively enforcing the tSA once the global structure has been well formed by natural language prompts.\n\n- In Supp. A.2, we present an ablation study to optimize the effect of both mechanisms in our proposed method: Prompt Queuing and Attention Amplification. Here, we observe that each component contributes to improving either fairness or quality of the generated sample.\n\n- In Supp B.6, to advance our understanding of the distorted ITI-Gen prompts, we provide further discussion on the issues with ITI-Gen prompts. Here, we provide additional analysis to show that ITI-Gen’s prompt embeddings deviate significantly from the Base prompt and Hard prompt embedding. These findings further support our understanding of the differences between ITI-Gen prompts and natural language prompts (HP and Base prompt).\n\nWe hope that these additional resources address the reviewer’s comment. We will include additional pointers to them.\n\n\n$ $\n\n\n>R3Q2: Is the amplified area Si related to tSA? If so, please explain the specific impact relationship.\n\nR3A2: We remark that the Reviewer’s understanding of the amplified area is correct. Specifically, the $S_i$ tokens in the ITI-Gen learn to encode the tSA by minimizing the directional loss (as discussed in lines 108-118). Therefore, the attention map correlating to the $S_i$ tokens is responsible for expressing the tSA at related regions, so is the amplified $S_i$.\n\nParticularly, recall that prompt-queuing allows for the global structure to be first generated with base prompt $T$ followed by tSA adaptation with learned prompt $P$. This reduces the number of denoising steps that generated samples are exposed to the $S_i$ tokens, i.e., only in stage 2 and hence affecting tSA expression. To compensate for this, Attention Amplification (discussed in lines 267-270) was introduced, which scales the attention maps correlating to these $S_i$ (the tSA-specific tokens). This results in the intensification of the tSA-specific regions (e.g., attending with more intensity to the mouth region in tSA=’Smiling’), and therefore enhances the tSA expression in the generated samples.\n\nTo further understand the impact of attention amplification, we direct the Reviewer to our ablation study in Supp. A.2, where we augment the attention amplification factor $c$. Here, we observe that when $c=0$, the FD is still relatively large due to the lack of tSA expression. Then, by increasing this to $c=10$, we observe the increment in the tSA expression lowers the FD. To further support this point, in this rebuttal, we provide some additional qualitative results seen in Fig. C of the attached pdf. Here, we observe the same effect on ‘Smiling’ where the tSA expression is poor at $c=0$. Then when amplifying the attention intensity ($c>0$) to the related regions (e.g., mouth, as seen in rebbutal Fig. A col 2–FairQueue) we observe that the tSA expression also intensifies i.e., samples begin to smile.  \n\n\n$ $\n\n\n>R3Q3: Are there specific metrics to measure the relationship between generation quality and fairness?\n\nR3A3: We thank the Reviewer for the insights and agree with the Reviewer’s intuition that there should exist a metric to evaluate the relationship between fairness and quality since this trade-off can exist. For example, some existing fair generative models in the literature [27, 28, 30]  require balanced datasets for the tSA, which are commonly small due to their difficulty to collect, and therefore affect sample quality. Unfortunately, to the best of our knowledge, the fair generative modeling community has not studied such a metric yet. Therefore, in our manuscript, following related work [16,28,30,31] we consider existing well-known individual metrics in the literature to assess fairness and quality, separately. Specifically as mentioned in line 140 and discussed in more detail in Supp. B.4, we have followed the existing works in the literature [16, 28, 30, 31], and used the following metrics for evaluating the fairness and quality of the generated images:\n\n- Fairness discrepancy (FD) is a common metric utilized in many popular fair generative modeling works [16,27,28,30,31] to assess fairness in the context of sample generation.\n\n- FID [16,38,28,30,31,35,60] is a very popular quality metric used in generative modeling to compare the feature embedding distribution on the generated sample against an ideal reference dataset.\n\n- Text Alignment [16,37,11,60] is a popular quality metric used in evaluating text-to-image models.\n\n- DreamSIM [39,B,C] extends on the well-known LPIPS [35,60] metric in the literature, to determine semantic similarities but additionally considers the high-level, mid-level, and low-level semantics concurrently.\n\n\n$ $\n\n\n>R3Q4: The author can further improve the description of the method part of the paper and describe the innovation more clearly.\n\nR3A4: We appreciate the reviewer's feedback and hope that our responses have clarified the details of the method in our paper. In the final manuscript, we will include this content from the Supp. We remark that all our code and necessary instructions are included in the Supp for full reproducibility of our work. \n\n$ $\n\n**We appreciate the positive feedback and valuable comments of the reviewer. we sincerely hope that reviewers could consider increasing the ratings if our responses have addressed all their questions**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NmW8YA4fIc", "reviewer": "Reviewer_9bUQ", "summary": "This paper examines the prompt learning method used in fair text-to-image (T2I) generation, highlighting its impact on image quality. The authors identify that aligning prompt embeddings with reference image embeddings introduces noise due to unrelated concepts, leading to degraded image quality. They conduct an in-depth analysis of the T2I model's denoising subnetwork, introducing novel prompt switching analyses (I2H and H2I) and new quantitative metrics for cross-attention map characterization. Their findings reveal abnormalities in the early denoising steps, which affect global structure synthesis.\n\nTo address these issues, the paper proposes two solutions: Prompt Queuing, which applies base prompts initially and ITI-GEN prompts later in the denoising process, and Attention Amplification, which balances quality and fairness. Extensive experiments demonstrate that these methods improve image quality while maintaining competitive fairness and diversity. The paper's contributions include identifying issues in the current prompt learning approach, providing a detailed analysis of the denoising process, and proposing effective solutions to enhance T2I generation quality.", "questions": "1. The authors provide a more detailed explanation or evidence on why the base prompt T is more effective in forming global structures during the early denoising steps compared to ITI-GEN prompts. Including theoretical justifications or experimental comparisons that illustrate how the base prompt helps establish the foundational elements of the image more effectively would be beneficial. Specific examples or visualizations from your experiments could clarify this point.\n2. How does switching to ITI-GEN prompts in the later stages of denoising improve the detailed expression of tSAs without compromising the overall image quality?\n3. The author should elaborate on the mechanism by which attention amplification enhances tSA expression. How is the scaling factor chosen and validated?\n4. The paper would be strengthened by more detailed experiments and results analysis, especially comparisons between using only the base prompt, only ITI-GEN prompts, and the FairQueue method. A thorough analysis of these results would help demonstrate how FairQueue effectively addresses the issues identified with the original method. By incorporating these detailed explanations and analyses, the paper would more convincingly show how FairQueue resolves the problems it aims to address, enhancing its credibility and scientific rigor.", "limitations": "1. Although the authors have identified and analyzed the issues with the current methods, the explanation of how their proposed method, FairQueue, effectively addresses these issues lacks detail. Regarding the effectiveness of Prompt Queuing, the authors should provide a deeper explanation of why the base prompt is more effective in the early denoising steps. This could be supported by theoretical analysis and specific experimental results, such as demonstrating how the base prompt gradually establishes the fundamental elements of the image during the denoising process. Further explanation of how using the base prompt in the early steps helps form a robust global structure, allowing the application of ITI-GEN prompts in later steps to better add details without compromising the overall structure, would be beneficial.\n2. For the mechanism of Attention Amplification, the authors need to elaborate on why scaling the cross-attention maps effectively enhances tSA expression. This section could include mathematical derivations or experimental data to show how different scaling factors impact image quality and tSA expression. Providing specific examples of how attention amplification compensates for the reduced tSA expression due to prompt queuing in various scenarios would strengthen the argument.\n3. The authors should include more experimental results, particularly comparisons between using only the base prompt, only ITI-GEN prompts, and the FairQueue method. Detailed analysis of these results would help demonstrate how FairQueue addresses the issues identified with the original method, such as image quality degradation and insufficient tSA expression. Providing specific quantitative metrics and visual comparisons can more intuitively show the advantages of the proposed method.\n4. The paper lacks a thorough analysis of the computational complexity and the necessary resources for implementing FairQueue. Understanding the computational demands is essential for assessing how feasible and scalable the method is in real-world applications. Without this information, it is difficult to determine whether FairQueue can be efficiently deployed in various environments, particularly those with limited computational resources.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper embarks on an in-depth examination of the denoising subnetwork within the T2I model. The goal is to understand how learned prompts influence image generation throughout the denoising steps. This involves dissecting the entire process from noisy input to the final image, which allows for pinpointing where and how the learned prompts might be causing degradation in image quality. This deep dive includes analyzing how noise is progressively reduced and how the model’s intermediate representations evolve at each step. By doing so, the authors can track the influence of different prompt tokens on the generation process, providing a granular view of the internal dynamics of the model.\n\n2. The cross-attention maps are visual tools that show how attention is distributed across different parts of the image in response to various tokens in the prompt. By mapping out these attention distributions, the paper identifies specific patterns and irregularities. The analysis reveals that certain tokens, which are not directly related to the target sensitive attributes (tSAs), exhibit abnormally high levels of activity. For instance, non-tSA tokens like \"of\" and \"a\" show higher attention scores, indicating that the model's focus is scattered and not properly aligned with the relevant attributes. This misalignment disrupts the formation of coherent global structures in the early stages of image generation.\n\n3. The paper highlights that abnormalities in cross-attention maps are particularly pronounced during the initial steps of the denoising process. At these stages, the model is supposed to establish the foundational global structure of the image. However, the distorted prompts from ITI-GEN interfere with this process, leading to poorly structured outputs. For example, the analysis shows that when using ITI-GEN prompts, the model often fails to correctly focus on key image regions, resulting in artifacts and unintended features. This is contrasted with the behavior of hard prompts, which show more targeted and consistent attention patterns that help maintain image quality.\n\n4. This paper includes extensive experimental results demonstrating the superiority of FairQueue over ITI-GEN in terms of fairness, image quality, and semantic preservation. These  experiments are conducted on multiple datasets and with various tSAs.", "weaknesses": "1. The proposed innovations, Prompt Queuing and Attention Amplification, are relatively incremental and might be seen as a combination of existing techniques rather than a breakthrough innovation. The combination of these two mechanisms does improve performance but lacks a strong theoretical underpinning to justify why this specific combination is superior.\n2. The proposed method, FairQueue, while innovative, could benefit from a more detailed explanation of how it addresses the issues identified in the paper. In the current form, the explanation of Prompt Queuing lacks depth regarding why the base prompt T is more effective in the early denoising steps. The authors should delve deeper into the reasons behind this, possibly by providing theoretical analysis and specific experimental results that demonstrate how the base prompt gradually establishes the fundamental elements of the image during the denoising process. \n3. Additionally, the transition to ITI-GEN prompts in the later stages should be better explained, particularly in terms of how it enhances the detailed expression of tSAs. Comparative illustrations of image generation results at different stages using different prompts would effectively show the benefits of this transition.\n4. The mechanism of Attention Amplification needs a clearer explanation. The authors should detail why scaling the cross-attention maps enhances tSA expression, possibly through mathematical derivations or experimental data showing the impact of different scaling factors on image quality and tSA expression. \n5. Although the experimental results are comprehensive, the paper lacks detailed descriptions of the experimental settings and specific parameters used, which could hinder reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer for detailed feedback. Due to lack of space we consolidate the comments from Weakness(W), Questions(Q), and Limitations (L) into 7 responses.\n\n$ $\n\n >R4Q1: “proposed innovations, Prompt Queuing, and Attention Amplifcation are relatively incremental” (W1)\n\nR4A1: We respectfully remark **our proposed solution for quality degradation in sophisticated fair generative model is non-trivial**, requiring our deep analysis of attention maps in individual denoising steps via our novel H2I/I2H analysis: \n\n- I2H reveals **learned tokens (via ITI-GEN) affect early denoising steps, degrading global structure**\n- H2I reveals **learned tokens work well in enhancing tSA expression in the later denoising steps, if global structure synthesized properly** \n\n*only from these observations*, can we propose FariQueue which is both simple and effective in mitigating quality degradations.\n\n$ $\n\n>R4Q2: “ why the base prompt $T$ is more effective in the early denoising steps” (W1,W2,Q1,Q4,L1,L3)\n\nR4A2: We apologize if emphasized insufficiently that the effectiveness of base prompt $T$ is grounded on our I2H and H2I analysis. Recall that I2H reveals quality degradation is related to using learned tokens (via ITI-GEN) in early denoising stage, and H2I reveals that using HP (no learned tokens) instead can prevent it.  **Using $T$ is in fact grounded on this analysis: both $T$ and HP are free of learned tokens.**  This is further seen in Supp B.6 where $T$ and HP are close in the embedding space while the learned prompts are far. (Recall that HP can only be used for a few tSA with minimal linguistic ambiguity, thus using $T$ is needed as a solution for many tSA).\n\nIn addition, per Reviewer’s comment, we provide visualizations of $T$’s effectiveness in generating the global structure in early denoising steps. Specifically, we compare the cross-attention maps of FairQueue with ITI-Gen during sample generation, together with quantitative analysis. Results in col 2 vs 3 of Fig. A (rebuttal pdf) illustrate $T$’s effectiveness in synthesizing the global structure in stage 1, and non-abnormal attention (in Fig B), resulting in effective global synthesis than ITI-Gen and better sample quality.\n\n$ $\n\n>R4Q3: “How does switching to ITI-GEN prompts in the later stages of denoising improve the detailed expression of tSAs without compromising the overall image quality” (W1, W3, Q2, Q4, L1, L3)\n\n\nR4A3: As our I2H/H2I analysis suggests, given good global structures synthesized in stage 1, ITI-Gen tokens enforce the tSA in stage 2 by having good *localized* attention that attends well to *tSA-specific regions* resulting in gradual addition of detail that enhances tSA expression.  \n\nAlso, per Reviewer’s comment, we provide visualization for  FairQueue vs $T$, where they are the same in stage 1 but are different in stage 2 (only FairQueue contains learned tSA tokens).\n\nResults in col 1 vs 2 of Fig. A (rebuttal pdf) show that with FairQueue, the learned tokens $S_i$ emphasize attending to tSA-related regions (e.g., eyes and mouth for Smiling, or lower half of the face and cheeks for High Cheekbones),  to output samples with the tSA features. Furthermore, as the attention is generally localized to the tSA-specific regions, the sample’s quality is well preserved. \n\n$ $\n\n>R4Q4: “Elaborate on the mechanism by which attention amplification enhances tSA expression. How is the scaling factor chosen and validated” (W1, W4, Q3, L2)\n\nR4A4: As discussed in R4A3, learned ITI-Gen tokens enforce tSA expressions by attending to tSA-specific regions. Attention Amplification then increases this tSA expression by amplifying the attendance at tSA-specific regions.\n\nThen, an ablation study in Supp A.2 determined the optimal scaling factor $c$. Our results suggest that $c=10$ delivers Pareto optimal performance for fairness and quality. \n\nAdditionally, per Reviewer’s comment, we provide some qualitative results (Fig C. rebuttal pdf) to support this analysis. Our results show that at $c=0$, even with prompt queuing, samples may lack tSA expression (Smiling). Increasing  $c$ helps to enhance tSA expression. \n\n$ $\n\n>R4Q5: Justify why this specific combination (PromptQueing and Attention Amplifcatio n) is superior. (W1)\n\nR4A5: To verify the contribution of both PromptQueuing and Attention Amplification, our ablation study in Supp A.2. varied the amplification scaling factor, $c$, and Prompt Queuing transition point.\n\nOur analysis reveals that both components contribute to FairQueue's superior performance and **removing any one results in worsened performance**, i.e., when $c=0$, FD increases (less fair), and when the transition point=0, TA decreases (poorer quality). \n\n$ $\n\n >R4Q6: “The paper lacks detailed descriptions of the experimental settings and specific parameters used …” (W5)\n\nR4A6: We respectfully clarify that all experimental reproducibility details are already available in Supp:\n- **Model + hyperparameter** in Supp B.3 \n- **Code + reproduction files** in the anon. link (line 488)\n- **Evaluation metrics**  in Supp B.5\n\nFurthermore, all reproducibility resources are in the Supp.\n\n$ $\n\n >R4Q7: “The paper lacks a thorough analysis of computational complexity.” (L4)\n\nR4A7: We clarify that we had conducted a thorough analysis to verify that FairQueue contributes **negligible complexity**. \nFor inference: the inference time of the generation is ~45s regardless of base prompt, ITI-Gen, or FairQueue. It is because PromptQueing adds negligible computation to replace $T$ with $P$, and Attention Amplification only introduces a multiplication operation. General computation resources info is provided in Supp B.4, and we will add more details to clarify this. \nFor prompt learning: prompt learning (ITI-Gen/FairQueue) is generally light-weight due to: \n\n- T2I generator is frozen and only three tSA tokens are trained.\n- T2I generator is not in the training loop\n\nOn average, learning tokens for a tSA takes ~5min with one RTX 3090.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Global Response (GR): We thank all the reviewers for their valuable time and effort in reviewing our work. We appreciate the Reviewers' kind comments, such as:

- **Presentation**: Reviewer 89En giving our paper excellent score (4) for presentation; Reviewer iQth for praising our paper as “well-written and logically structured”

- **Analysis**: Reviewer 89En, 8wzu, and 9buQ recognizing our  “highly detailed analysis”, “detailed analysis” and “in-depth analysis”, respectively.

- **Experiments**: Reviewer iQth: recognizing our efforts in making our paper “very rich in experiments” 

- **Solution**: Reviewer iQth acknowledging “FairQueue effectively solves the quality degradation problem in ITI_GEN” and Reviewer  9buQ: Acknowledging the “superiority of FaiQueue over ITI-Gen”

$ $

We would also like to express our appreciation to all the Reviewers for allowing us to clarify our work, as well as the constructive comments.  We have considered all comments seriously. 

$ $

To briefly recap our work:

1. Our detailed study first uncovers that ITI-Gen –Prompt learning– degrades the quality of the generated samples, an observation missed by recent works [A] –based on high-level analysis. 

2. Upon deeper analysis, it is revealed that the quality degradation sources from ITI-Gen’s sub-optimal directional loss used during training, where unrelated knowledge distorts the tokens in ITI-Gen prompt –responsible for encoding the tSA expression. 


3. Based on these findings, we deep dive into analyzing the impact of these distorted ITI-Gen tokens during sample generation. Specifically, we inspect the attention maps with our novel H2I/I2H analysis. Our findings reveal to us that:

	- The quality degradation sources from the distorted ITI-Gen tokens affecting the global structure in the early denoising steps

	- Interestingly, given that the global structure is well-synthesized, ITI-Gen tokens work well (in the later stage) to enforce the tSA expression. 

4. **Only based on this analysis,** are we then able to propose our Prompt Queuing mechanism (an evolution of H2I), which instead utilizes the Base Prompt in place of HP in the early denoising steps (to first synthesize good global features), followed by ITI-Gen prompt (to enforce tSA expression). Note that HP and Base Prompt share the similarity of being natural language prompts (demonstrated in Supp B.6) -undistorted by the directional loss in contrast to ITI-Gen– and hence produce good global structure.

5. Finally, recognizing that Prompt Queuing would impact the tSA expression due to the reduced denoising steps with ITI-Gen prompt, we propose Attention Amplification which scales up the attention maps for the tSA-specific tokens ($S_i$) resulting in enhanced tSA expression. We coin this combination of Prompt Queuing and Attention Amplification as FairQueue.

6. We then perform comprehensive experiments comparing FairQueue with the existing SOTA ITI-Gen to demonstrate our improved quality and semantic preservation while preserving the fairness performance. Our experiments include:
	-  **3 datasets (fair generative modeling gold-standard benchmarks)**: CelebA [29], FairFace [45], and Fair Benchmark [44] 

	- **12 different tSA combinations**

	- **4 popular performance metrics**:  Fairness discrepancy (FD) [16,27,28,30,31], FID [16,38,28,30,31,35,60], Text Alignment [16,37,11,60], DreamSIM [39,B,C].

$ $

Additionally, due to space limitations, we divert some key resources to the supplementary:

1. **Ablation studies** (Supp. A.2): demonstrates the individual contribution of Prompt Queuing and Attention Amplification.    

2. **Exploration of ITI-Gen, HP, and Base prompt relationship in their embeddings space** (Supp.B.6): Verified that HP and Base Prompt are similar  (close in their embeddings) and ITI-Gen prompt is dissimilar (distant embeddings).

3. **Hyper Parameters** (Supp. B.3): details for reproducibility.

4. **Code and Qualitative Illustrations** ([Annonymous Link in Supp line 489]). 
	- Code and ReadMe
	- Qualitative comparison of ITI-Gen vs FairQueue illustration in \> [more resources] \> [Tab_1_More_illustration]



$ $

In this rebuttal, we have also included a few additional experiments and visualizations, as requested by the reviewers. All of the newly added results help to support the effectiveness of FairQueue:

1.  Human-recognized assessment (Amazon-Mechanical Turk) of the quality and fairness of generated samples by ITI-Gen vs FairQueue

2. Additional Qualitative and quantitative analysis on the attention maps between BasePrompt vs FairQueue vs ITI-Gen

3. Qualitative assessment of the effects of Attention Amplification

4. Evaluating ITI-Gen’s performance with a larger reference dataset (2k)

$ $

In what follows, we provide comprehensive responses to all questions. We could provide more details if there are further questions. We hope that our responses can address the concerns. 

$ $

[A] Fernández, Daniel Gallo, et al. "Reproducibility Study of" ITI-GEN: Inclusive Text-to-Image Generation"." Transaction on machine learning research (2024).

[B] Ghazanfari, Sara, et al. "LipSim: A Provably Robust Perceptual Similarity Metric." International Conference on Learning Representations (2024).

[C] Liu, Yinqiu, et al. "Cross-Modal Generative Semantic Communications for Mobile AIGC: Joint Semantic Encoding and Prompt Engineering." arXiv preprint arXiv:2404.13898 (2024).

Meta-review: This work focuses on improving fairness and quality in T2I generation via prompt learning and proposed the FairQueue that is demonstrated to be superior over the current ITI-Gen method. Reviewers are generally positive (one accept and three borderline accept) by acknowledging the contribution and its effectiveness, without obvious objections. Though it mainly targets on improving one specific existing work, extensive experiments verify the advantage of proposed methods. It points out a promising direction of how prompt learning affect the generation and the analysis on attention maps help better understand the black box of big diffusion models, which is of great value to the community. Therefore, after checking all comments, the area chair made a decision of acceptance. Some concerns including adding ablation study and necessary explanations presented in the rebuttal should be addressed in the revised version.
Decision: Accept (poster)