Finetuning Text-to-Image Diffusion Models for Fairness
Authors: Xudong Shen
Chao Du
Tianyu Pang
Min Lin
Yongkang Wong
Mohan Kankanhalli
Venue: ICLR 2024 oral (2024)
Keywords: Fairness
Alignment
Diffusion Models
Text-to-Image Generation
Primary area: societal considerations including fairness, safety, privacy
TL;DR: A flexible and scalable supervised fine-tuning method is introduced to align the generated images of a text-to-image diffusion model with a desired distribution.

Abstract: The rapid adoption of text-to-image diffusion models in society underscores an urgent need to address their biases. Without interventions, these biases could propagate a skewed worldview and restrict opportunities for minority groups. In this work, we frame fairness as a distributional alignment problem. Our solution consists of two main technical contributions: (1) a distributional alignment loss that steers specific characteristics of the generated images towards a user-defined target distribution, and (2) adjusted direct finetuning of diffusion model's sampling process (adjusted DFT), which leverages an adjusted gradient to directly optimize losses defined on the generated images. Empirically, our method markedly reduces gender, racial, and their intersectional biases for occupational prompts. Gender bias is significantly reduced even when finetuning just five soft tokens. Crucially, our method supports diverse perspectives of fairness beyond absolute equality, which is demonstrated by controlling age to a 75% young and 25% old distribution while simultaneously debiasing gender and race. Finally, our method is scalable: it can debias multiple concepts at once by simply including these prompts in the finetuning data. We share code and various fair diffusion model adaptors at https://sail-sg.github.io/finetune-fair-diffusion/.

Published as a conference paper at ICLR 2024
FINETUNING TEXT-TO-IMAGE DIFFUSION MODELS FOR FAIRNESS
Xudong Shen"1, Chao Dul Tianyu Pang Min Lin²
Yongkang Wong', Mohan Kankanhalli
'ISEP programme, NUS Graduate School, National University of Singapore
'Sea AI Lab, Singapore
Schooo of Computing, National University of Singapore
xudong. shen@u. mus edu; {duchao, t:ianyupang, 1 sinnmn]@sea....
yongkang. wong@nus wet.eeuussuuuu.... sg; mchan@comp. mus edu.
ABSTRACT
The rapid adoption of text-to-imape diffusion models in society underscores ur- gent need to address their biases. Without interventions, these biases could prop- agate skewed worldview and restrict opportunities for minority groups. In this work, we frame fairness as a distributional alignment problem. Our solution con- sists of two main technical contributions: (1) distributional alignment loss that steers specific characteristics of the generated images towards a user-defined tar- get distribution, and (2) adjusted direct finetuning of diffusion model's sampling process (adjusted DFT), which leverages an adjusted gradient to directly opti- mize losses defined on the generated images. Empirically, our method markedly reduces gender, racial, and their intersectional biases for occupational prompts. Gender bias is significantly reduced even when finetuning just five soft tokens. Crucially, our method supports diverse perspectives of fairness beyond absolute equality, which is demonstrated by controlling age to a 75% young and 25% old distribution while simultaneously debiasing gender and race. Finally, our method is scalable: can debias multiple concepts at once by simply including these prompts in the finetuning data. We share code and various fair diffusion model adaptors at https://sail-:: sssss////llss..aao. wtw.a//onnnnneaaai-iiiiiieeeeeee
INTRODUCTION
Text-to-image (T21) diffusion models (Nichol er al., 2021; Saharia et al., 2022) have witnessed an accelerated adoption by corporations and individuals alike. The scale of images generated by these models is staggering. To provide a perspective, DALL-E 2 (Ramesh et al., 2022) is used by over one million users (Bastian, 2022), while the open-access Stable Diffusion (SD) (Rombach et al., 2022) is utilized by over ten million users (Fatunde & Tse, 2022). These figures will continue to rise.
However, this influx of content from diffusion models into society underscores an urgent need to address their biases. Recent scholarship has demonstrated the existence of occupational biases (Se- shadri et al., 2023), a concentrated spectrum of skin tones (Cho et al., 2023), and stereotypical associations (Schramowski et al., 2023) within diffusion models. While existing diffusion debiasing methods (Friedrich et al., 2023; Bansal et al., 2022: Chuang et al., 2023; Orgad et al., 2023) offer some ad advantages, such as being lightweight, they struggle to adapt to wide range of prompts. Fur- thermore, they only approximately remove the biased associations but do not offer a way to control the distribution of generated images. This is concerning because perceptions of fairness vary across specific issues and contexts; absolute equality might not always be the ideal outcome.
We frame fairness distributional alignment problem, where the objective is to align particular tributes of the generated images, such as gender, with a user-defined target distribution. Our solution consists of main technical contributions. First, we design a loss function that steers the gener- ated images towards the desired distribution while preserving image semantics. A key component the distributional alignment loss (DAL.. For batch of generated images, DAL uses pre- pra-tedined classifiers to estimate class probabilities (e.g., male and female probabilities) and dynamically gen-
Work done during internship Sea AT Lab. CCorrespondigg authors.

---

Published as a conference paper at ICLR 2024
erates target classes that match the target distribution and have the minimum transport distance. To preserve image semantics, we regularize CLIP (Radford et al., 2021) and DINO (Oquab et al., 2023) similarities between images generated by the original and finetuned models.
Second, we propose adjusted direct finetuning of diffusion models, adjusted DFT for short, illus- trated in Fig. 2. While most diffusion finetuning methods (Gal et al., 2023; Zhang & Agrawala, 2023; Brooks et al., 2023; Dai et al., 2023) use the same denoising diffusion loss from pre-training, DFT aims directly finetune the diffusion model's sampling process minimize any loss defined on the generated images, such as ours. However, we show the exact gradient of the sampling pro- cess has exploding norm and variance, rendering the naive DFT ineffective (illustrated in Fig. 1). Adjusted DFT leverages an adjusted gradient to overcome these issues. It opens venues for more refined and targeted diffusion model finetuning and can be applied for objectives beyond fairness.
Empirically, we show our method markedly reduces gender, racial, and their intersectional biases for occupational prompts. The debiasing is effective even for prompts with unseen styles and contexts, such as "A philoxopher reading. Oil painting" and "bartender at willard intercontinental makes mint julep" (Fig. 3). Our method is adaptable to any component of the diffusion model being finetuned. Ablation study shows that finetuning the text encoder while keeping the U-Net unchanged hits sweet spot that effectively mitigates biases and lessens potential negative effects on image quality. Surprisingly, finetuning as few five soft tokens as a prompt prefix is able to largely reduces gender bias, demonstrating the effectiveness of soft prompt tuning (Lester et al., 2021; Li & Liang, 2021) for fairness. These results underscore the robustness of our method and the efficacy of debiasing T21 diffusion models by finetuning their language understanding components.
A salient feature of our method is its flexibility, allowing users to specify the desired target distri- bution. For example, we can effectively adjust the age distribution to achieve a 75% young and 25% old ratio (Fig. 4) while sinultaneously debiasing gender and race (Tab. 5). We also show the scalability of our method. It can debias multiple concepts at once, such as occupations, sports, and personal descriptors, by expanding the set of prompts used for finetuning.
Generative AI is set to profoundly influence society. It is well-recogiieed that LLMs require social alignment finetuning post pre-training (Christiano et al., 2017: Bai et al., 2022). However, this analogous process has received less attention for T2I models, or multimedia generative Al overall. Biases and stereotypes can manifest more subtly within visual outputs. Yet, their influence on human perception and behavior is substantial and long-lasting (Goff et al., 2008). We hope our work inspire further development in promoting social alignment across multimedia generative AL.
2 RELATED WORK
Bias in diffusion models. T2I diffusion models are known to produce biased and stereotypical images from neutral prompts. Cho et al. (2023) observe that Stable Diffusion (SD) has an overall tendency to generate males when prompted with occupations and the generated skin tone con- centrated on the center few tones from the Monk Skin Tone Scale (Monk, 2023). Seshadri et al. (2023) observe SD amplifies oocupptoc biases from its training data. Besides occupations, Bianchi al. (2023) find simple prompts containing character traits and other descriptors also gen- erate stereotypical images. Luccioni et al. (2023) develop a tool to compare collections of generated images with varying gender and ethnicity. Wang al. (2023a) propose a text-to-image association test and find SD associates females more with family and males more with career.
Bias mitigation in diffusion models. Existing techniques for mitigating bias in T2I diffusion mod- els remain limited and predominantly focus on prompting. Friedrich et al. (2023) propose to ran- domly include additional text cues like "male" or "female" if known occupation detected in the prompts, to generate images with a more balanced gender distribution. However, this approach is ineffective for debiasing occupations that are not known advance. Bansal et al. (2022) suggest incorporating ethical interventions into the prompts, such as appending "if all individuals can be lawyer irrespective of their gender" to "a photo of a lawyer". Kim et al. (2023) propose to optimize a soft token "V"" such that the prompt "V" a photo of a doctor" generates doctor images with a bal- anced gender distribution. Nevertheless, the efficacy of their method lacks robust validation, as they only train the soft token for one specific occupation and test it on two unseen ones. Besides prompt- ing, debias LL (Chuang et al., 2023) proposes to project out biased directions in text embeddings. Concept Algebra (Wang et al., 2023b) projects out biased directions in the score predictions. The
2

---

Published as a conference paper at ICLR 2024
TIME (Orgad et al., 2023) and UCE methods (Gandikota et al., 2023), which modify the attention weight, can also be used for debiasing. Similar issues of fairness and distributional control have also been explored in other image generative models (Wu al., 2022).
Finetuning diffusion models. Finetuning is a powerful way to enhance a pre-trained diffusion model's specific capabilities, such adaptability (Gal al., 2023), controllability (Zhang & Agrawala, 2023), instruction following (Brooks et al., 2023), and image aesthetics (Dai et al., 2023). Concurrnt works (Clark al., 2023; Wallace et al., 2023) also explore the direct finetuning of dif- fusion models, albeit with goals diverging from fairness and solutions different from ours. Adjusted DFT complements them because we identify and address shared challenges inherent in DFT.
3 BACKGROUND ON DIFFUSION MODELS
Diffusion models (Ho et al., 2020) assume forward diffusion process that gradually injects Gaus- sian noise to a data distribution g(%)) according a variance schedule .1....,By::
( = (<
(<s)/==11) (x VI -, BeI).
(I)
where T' is predefined total number of steps (typically 1000). The schedule (Be is chosen such that the data distribution g(zg) is gradually transformed into approximately Gaussian dis- tribution qr (DT) s (xx 0. I). Diffusion models then learn to approximate the data distribution by reversing such diffusion process, starting from a Gaussian distribution p(ær) (ar I):
pa (a(o)))) = p(xr)
pa (a
pe pe (x))) = -N t), o.1).
(2)
where (0( 1) parameterized using a noise prediction network c t) with po po t) Vor (x. vI-ai (a E ()), a; = - 1--,, a₁ = IT Osa, and {otherx are pre-determined noise variances. After training, generating from diffusion models involves sampling from the reverse process pe (Zo:T), which begins by sampling a noise variable øT p(xT), ) and then proceeds to obtain Vy as follows:
3, VI-
(xt Vau
co co( t)) + o.w. w. (0,1).
Oe
Latent diffusion models. Rombach et al. (2022) introduce latent diffusion models (LDM), whose forward/reverse diffusion processes are defined in the latent space. With image encoder fine and de- coder fDee. LDMs are trained on latent representations 2n fEne To generate image, LDMs first sample latent noise 2T, run the reverse process to obtain 2o, and decode it with æg = (z0).
Text- text-to-image diffusion models. In T2I diffusion models, the noise prediction network accepts an additional text prompt P, i.e., (g 2t, t). where Is represents a pretrained text encoder parameterized by 6. Most T2I models, including Stable Diffusion (Rombach et al., 2022), further employ LDM and thus use a text-conditional noise prediction model in the latent space, denoted co (g (P). zt, t), which serves the central focus of our work. Sampling from T2I diffusion models additionally utilizes the classifier-free guidance technique (Ho & Salimans, 2021).
4
METHOD
Our method consists of (i) a loss design that steers specific attributes of the generated images towards a target distribution while preserving image semantics, and (ii) adjusted direct finetuning of the diffusion model's sampling process.
4.1 LOss DESIGN
General case For a clearer introduction, we first present the loss design for a general case, which consists of the distributional alignment loss Lalign and the image semantics preserving loss Limg. We start with the distributional alignment loss (DAL) Latign. Suppose attribute of the generated images that has K classes and align it towards a target distribution D. Each class is represented as a one-hot vector of length K and D is a discrete distribution over these classes.
want to control categorical
We first generate a batch of images = (xx(1) }Se[N] using the diffusion model being finetuned and some prompt P. For every generated image (0),, we use a pre-trained classifier h to produce class probability vector (8) (0(h) ... PK (1)(())), with rk denoting the estimated probability
(8)
3

---

Published as a conference paper at ICLR 2024
that x(() is from class k. Assume have another set of vectors (ul Jie[N] that represents the target distribution and where every (6) is one-hot vector representing class, we can compute the optimal transport (OT) (Monge, 1781) from (p(N) {u()) Nee[N)
argmin
lp
u (
(4)
FESN
where Sy denotes all permutations of [N]. a= [01,-.. NN., and 0; [M] Intuitively, a* finds, in the class probability space, the most efficient modification of the current images to match the target distribution. We construct (u(()))EN to be i.i.d. samples from the target distribution and
compute the expectation of OT:
(())
((N) [u (, Vi E [N].
(5)
( is a probability vector where the k-th element is the probability that image x(0) should have target class k, had the batch of generated images indeed followed the target distribution D. The expectation of OT can be computed analytically when the number of classes K is small or approximated by empirical average when K increases. We note {u()) NE[N]. for example half male and half female represent a balanced gender distribution. But a fixed split poses a stronger inite-sample aligmment objective and neglects the sensitivity of OT.
also construct fixed set of
Finally, we generate target classes (v() hE(N] and confidence of these targets fo("))ee by: y = arg ()) E [K], c()) = (a)( E [0, 1J.Vi E [N]. We define DAL as the cross-entropy loss w.r.t. these dynamically generated targets, with confidence threshold C,
N I(c(0 (h(c)(((2))), ),))
Caige
(6)
We also use an image semantics preserving loss Cing- We keep a copy of the frozen, not finetuned diffusion model and penalize the image dissiminarity measured by CLIP and DINO:
Lime
1 N E [1
cos(CCIP(( ), (L)P(O + cos(ODN ()), DINO(
where I' = (ol0)) [E[N]] is the batch of images generated by the frozen model using the same prompt P. We call them original images. We require every pair of finetuned image 2+()) and original image (6)) are generated using the same initial noise. We use both CLIP and DINO because CLIP pretrained with text supervision and DINO is pretrained with image self-supervision. In imple- mentation, we use the Cean/CLI-ViT-H---A 14-r----- ---222-----B7777 B-9KK and the dinov2-vitb14 (Oquab al., 2023). We caution that CLIP and DINO can have their own biases (Wolfe et al., 2023).
Adaptation for face-centric attributes In this work, we focus on face-centric attributes such as gender, race, and age. We find the following adaptation from the general case yields the best results. First, we use face detector diace to retrieve the face region diace (1)) from every generated image (i)) We apply the classifier h and the DAL Lalign only on the face regions. Second, introduce another face realism preserving loss Lisse, which penalize the dissimilarity between the generated face disce (z) and the closest face from set of external real faces DF.
Lisee (1 min cos(emb(draed (0)), emb(F) (F). FED
(8)
where emb(-) is face embedding model. Linee helps retain realism of the faces, which can be substantially edited by the DAL. In our implementation, we use the CelebA (Liu et al., 2015) and the FairFace dataset (Karkkainen & Joo, 2021) as external faces. We the SFNet-20 (Wen al., 2022) as the face embedding model.
Our final loss L is a weighted sum: L Calign + >ing + Caace Notably, we use a dynamic weight Ximg- We use a larger Ximg, if the generated image (9''ss target class ((0) agrees with the original image (f)'s class h (ddrace ()) Intuitively, we encourage minimal change between (i) and (3 if the original image (l)) already satisfies the distributional alignment objective. For other images (5) whose target class (6)) does not agree with the corresponding original image (8)'ss class hudrsce (o)))), we use a smaller weight Aimg.2 for the non-face region and the smallest weight Aing.3 for the face region. Intuitively, these images do require editing, particularly on the face regions. Finally, if an image does not contain any face, we only apply Ling but not Latign and Liave If an image contains multiple faces, we focus on the one occupying the largest area.
4

---

Published as a conference paper at ICLR 2024
10k
RA, D
training loss
0.8
1000
Belf (vurs) R NY
100
RA الدl
0.4
10
0.2
exact gred
adjusted good (ours)
0.1
adjupted good
200
400
600
800
1000
1000
800
600
400
200
traininig iteration
time f
(a)
(h)
Figure 1: The left figure plots the training loss during direct fine-tuning, w/ three distinct gradients. Each reported w/ 3 random runs, The right figure estimates the scale of these gradients at different time steps. Mean and 90% CI are computed from 20 random runs. Read Section 4.2 for details.
4.2 ADJUSTED DIRECT FINETUNING OF DIFFUSION MODEL'S 's SAMPLING PROCESS
Consider that the T2I diffusion model generates image To (z)) using prompt and an initial noise æy. Our goal is to finetune the diffusion model to minimize a differentiable loss [(xo). We begin by considering the naive DFT, which computes the exact gradient of ((x)) in the sampling process, followed by gradient-based optimization. To see if naive DFT works, we test for the image semantics preserving loss Limg using fixed image as the target and optimize a soft prompt. This resembles a textual inversion task (Gal et al., 2023). Fig. la shows the training loss has no decrease after 1000 iterations. It suggests the naive DFT of diffusion models is not effective.
By explicitly writing down the gradient, we are able to detect why the naive DFT fails. To simplify the presentation, we analyze the gradient w.r.t. the U-Net parameter,
dC(zo) But the same issue
d0
arises when finetuning the text encoder or the prompt.
dC(xa) de
dC(xs) dma with dzo dmg dzg d0
de
31
(1
8,
B. de
de
(9)
01
30
vI dz.
00
B1
A1
A
B.
where e!)) denotes the U-Net function (g(go Zz, t) evaluated at time step t. Importantly, the recurrent evaluations of U-Net in the reverse diffusion process lead to a factor B, that scales expo- de(O) nentially in f. leads to two issues. First, dae becomes dominated by the components A B, de de
for values of t close to T" 1000. Second, due to the fact that B, encompasses all possible products between { (222) Jest-1. this coupling between partial gradients of different time steps introduces sub- stantial variance to dan Fig. 2a illustrates this issue. We empirically show these problems indeed (e exist in naive DFT. Since directly computing the Jacobian matrices (e and Oz, is too expen- sive, we assume dC(m)) dmg is random Gaussian matrix R (O, 10 I) and plot the values dmo dzg 0c( R.A.A 80 and R 0c in Fig. 1b. It is apparent both the scale and variance of 00
of IRA, B, Bc(0)
R,AB, (ele) explodes as + 1000, but neither R.A, (a
(0 do.
00
00
nor
00
Having detected the cause of the issue, we propose adjusted DFT, which uses an adjusted gradient that sets A, = and B, = I: (d0 )adjusted
--
0el
It is motivated from the unrolled
expression of the reverse process:
Zu = - - Aseo(ou(P) 2., Var
T
2T
Ei=
1 We, w. ~ (N I).
(10)
vāc-1
When we set B, I, we are essentially considering 2; as external variable and independent of the U-Net parameters 0, rather than recursively dependent on 0. Otherwise, by the chain rule, generates all the coupling between partial gradients of different time steps in B,. But setting B, I Be(t)
does preserve all uncoupled gradients, i.e., #g Vt E [T]. When we set A, = 1, we standardize the influence of (F), 2,, t) from different time steps t in 2g- It is known that weighting different time steps properly can accelerate diffusion training (Ho et al., 2020; Hang et al., 2023). Finally, we implement adjusted DFT in Appendix Algorithm A. Fig. 2b provides schematic illustration.
We test the proposed adjusted gradient and a variant that does not standardize A, for the same image semanties preserving loss w/ the same fixed target image. The results are shown in Fig. la.
5

---

Published as a conference paper at ICLR 2024
0
As By "y unicoupted grade: DyDa "S" coupled grade dadatalala Dala Dylolale)
D,D,le
Dula
I AOL
(8)
(2)
(1)
Z3
20
(a) Naive DFT.
(b) Adjusted DFT, which also standardize A, to 1.
Figure 2: Comparison of naive and adjusted direct finetuning (DFT) of the diffusion model. Gray solid lines denote the sampling process. Red dashed lines highlight the gradient computation w.r.t. the model parameter (0). Variables 2 and e(*) represent data and noise prediction at time step D, and I, denote the direct and indirect gradient paths between adjacent time steps. For instance, ae
at 3, naive DFT computes the exact gradient A3 B3 a0 (defined in Eq. 9), which involve other time step's ss noise predictions (through the gradient paths يكاااييا5,, I2Is, and D2 Ds)) Adjusted DFT leverages an adjusted gradient, which removes the coupling with other time steps and standardizes A; to 1, for more effective finetuning. Read Section 4.2 for details.
We find that both adjusted gradients effectively reduce the training loss, suggesting B, is indeed the underlying issue. Moreover, standardizing A, further stabilizes the optimization process. We note that, to reduce the memory footprint, in all experiments we (i) quantize the diffusion model to float 1 6, (ii) apply gradient checkpointing (Chen et al., 2016), and (iii) use DPM-Solver++ (Lu et al., 2022) as the diffusion scheduler, which only requires around 20 steps for T2I generations.
5
EXPERIMENTS
5.1 MITIGATING GENDER, RACIAL, AND THEIR INTERSECTIONAL BIASES
(SD for short), T2I diffusion model openly accessible from Hugging Face, to reduce gender, racial, and their intersectional biases. We consider binary gender and recognize its limitations. Enhancing the representation of non-binary identities faces additional challenges from the intricacies of visually representing non- non-binary iden- tities and the lack of public datasets, which are beyond the scope of this work. We adopt the eight race categories from the FairFace dataset but find trained classifiers struggle to distinguish between certain categories. Therefore, we consolidate them into four broader classes: WMELH= Middle Eastern, Latino Hispanic], A Asian, Southeast Asian); J. Black, and Indian. The gender and race classifiers used in DAL are trained on the CelebA or FairFace datasets. We consider uniform distribution over gender, race, or their intersection the target distribution. We employ the prompt template "a photo of the face of a {occupation). a person" and use 1000/50 occupations for training/test. For the main experiments and except otherwise stated, finetune LoRA (Hu et al., 2021) with rank 50 applied on the text encoder. Appendix A.2 provides other experiment details.
We apply our method runwwymlSsablediffssioo--///
Evaluation. We train separate gender and race classifiers for evaluation. We generate 60, 80, 160 images for each prompt to evaluate gender, racial, or intersectional biases, respectively. For every prompt P, we compute the following metric: (Pa) x(K-1)/2 Σxjejki= [freq()) freq(j)], where (i) is group i's frequency in the generated images. The number of groups K is 2/4/8 for gender/race/their intersection. The classification of an image into a specific group is based on the face that covers the largest area. This bias metric considers perfectly balanced target distribution. It measures the disparity of different groups' representations, averaged across all contrasting groups. We also report (i) CLIP-T, the CLIP similarity between the generated image and the prompt, (ii) CLIP-I, the CLIP similarity between the generated image and the original SD's generation for the same prompt and noise, and (ii) DINO, which parallels CLIP--I but uses DINO features. We use CLIP- 14 and DINOv2 vit-g/14 for evaluation, which differ from the ones used for training-
Results. Table 1 reports the efficacy of our method in comparison to existing works. Evaluation details of de and UCE are reported in Appendix A.7. Our method consistently achieves the lowest bias across all three scenarios. While Concept Algebra and UCE excel in preserving visual similarity to images generated by the original SD, they are significantly less effective for debiasing. This is because some of these visual alterations minority groups. Furthermore, our method still maintains a strong aligmment with the text prompt. Fig. 3a shows generated images using the unseen occupation "electrical and electronics repairer"
essential for enhancing the representation of

---

Published as a conference paper at ICLR 2024
Table I: Comparison with debias VL (Chuang et al., 2023), Ethical Intervention (Bansal et al., 2022),
Concept Algebra (Wang al., 2023b), and Unified Concept Editing (UCE) (Gandikota al., 2023).
Debias:
Method
Bias
Semantics Preservation
Gender
Race
G.xR.
CLIP-T
CLIP-I
DINO
Original SD
.672299
42..06
21:
.39=.05
debiasV VL
19:
.9
.24
.36=.05
.63+.14
.53+.21
Gender
UCE
-59:.33
.43
.21+.03
.38 2.05
88:+.15
.78 1:21
Ethical Int.
.56
.372008
.19
.37:2.05
.68
160
C. Algebra
1..31
.41 1.06
1..0.02
.39
1441.14
1.
Ours
12.
.44
200
.39
.77
.70
de
14
.38+.04
21
.36=.04
.50+.12
.36
Race
UCE
.62+.33
400.08
.20+::0
.38
.79+.15
.73+
Ethical Int.
.S8
.3:0006
.19::03
.360.0
.65
.574:.25
Ours
.74+.27
12=
.14
2.9
.73+
.67+
debias
991..4
...0004
24.1.01
.35
.631.18
14.1.2
G.xR.
UCE
.721.28
.36..00
.20
1.05
.791.16
.74.1.22
Ethical Int.
.55...1
.35
.1811.03
.36
.631.18
155.1.24
Ours
.662.13
.094.04
.064:.02
.39
1674.11
.58.1.22
from the test set. The original SD generates predominantly white male, marginalizing many other identities, including female, Black, Indian, Asian, and their intersections. Our debiased SD greatly improves the representation of minorities. Appendix Fig. A.1 shows the training curve. We plot the gender and race representations by each occupation in Appendix Fig. A.2, A.3, A.4, and A.5.
Generalization to non-templated prompts. We obtain 40 occupation-relater prompts from the LAION- Aesthetics V2 dataset (Schuhmann et al., 2022), listed in Appendix A.6. These prompts feature more nuanced style and contextual elements, such as "A philoso- pher reading. Oil painting." " (Fig. 3b) or "bartender at willard intercontinental makes mint julep" (Fig. 3c). Our evaluations, reported in Table 2, indicate that although debias only templated prompts, the debiasing effect generalizes to more complex non-templated prompts as well. The debiasing effect can be more apparently seen from Fig. 3b and 3c. In Appendix Section A.5, we analyze the debiased SD's gener- ations for general, non-occupational prompts as well as potential negative impacts on image quality.
Table 2: Eval w/ non-templated prompts.
Bias |
P. f
Method
Gender
Race
G.xR.
CLIP-T
SD
.60
.394.09
.194:05
.11..0
dcb. VL
84
.42..07
.22..03
.374.06
Eth. Int.
51 .38+.07
.18..03
.40+.08
UCE
51
.364.08
.184..44
.40..0
Ours
32.1.22
.30..08 .141.04 .401.0s
Generalization multi-face im- age generation. We change the test prompt template to "A photo of the faces of two/three (occupation), two/three people" generate im- ages of two or three individuals. with results reported in Tab. 3. We additionally report the bias metric calculated for all faces in the generated images. The results show the debiasing effect generalizes to multi-face image generations as well. We show generated images in Appendix Figs 1.16 and A.17.
Table 3: Eval on multi-face image generation.
Pro- mpt
Bias (single face) 4 Bias (all faces) 4
S. P.
Model
Gender
Race G.XR. Gender Race G.XR.
CLIP-T
Two
SD
464.28 45=.04 .0 21 _3=2266 455+.03 21 404.04
ppl
Ours
26+
.14 009+.03 118 .15+.06 008 414.00
Three
SD
481..3
.4_...5 21.0 .42 ..41..04 2.0.02 .40..05
ppl
Ours 26.1.17 .16.i.n5 .091.02 .16 1911.06 10 .411.05
Ablation on different components to finetune. We finetune various components of SD to re- duce gender bias, with results reported in Ta- ble 4. First, our method proves highly robust the number of parameters finetuned. By optimiz- ing merely five soft tokens as prompt prefix, gen- der bias can already be significantly mitigated. Fig. A.18 provides a comparison of the generated images. Second, while Table 4 suggests finetun- ing both the text encoder and U-Net is the most effective, Fig. 5 reveals two adverse effects of finetuning U-Net for debiasing purposes. It can deteriorate image quality w.r.t. facial skin textual and the model becomes more capable at misleading the classifier into predicting image one gender, despite the image's 's perceptual resemblance to another gender. Our findings shed light on the de- cisions regarding which components to finetune when debiasing diffusion models. We recommend
Table 4:
Finetuning different SD components. For prompt prefix, five soft tokens are finetuned. For others, LoRA w/ rank 50 is finetuned.
Finetued Bias Semanties Preservation Component Gender CLIP-T CLIP-I DINO
Original SD .67 1.29 1.9
Prompt Prefix .24 .394:05
.70
.62
Text Encoder .23+..6
.394.05
.77
.70+.22
U-Net .22...1
.391.00
190
.871.1
T.F. & U-Net .171.13 .40..04 1.
......2
7

---

Published as a conference paper at ICLR 2024
(a) Prompt with unseen occupation: "a photo of the face of electrical and electronics repairer. a person".
Gender bias: 0.84 (original) + 0.11 (debiased). Racial bias: 0.48 0.10. Genderx Race bias: 0.24 0.06.
(b) Prompt with unseen style: "A philosopher reading. Oil painting.". Gender bias: 0.80 (original) + 0.23
(debiased). Racial bias; 0.45 + 0.31. Gender x Race bias: 0.22 - 0.15.
(c) Prompt with unseen context: "bartender at willard intercontinental makes mint julep". Gender bias: 0.87 (original) + 0.17 (debiased). Racial bias: 0.49 - 0.33. GenderxRace bias: 0.24-> 0.15.
Figure 3: Images generated from the original SD (left) and the SD jointly debiased for gender and race (right). The model is debiased using the template "a photo of the face of (occupation), ) a person".. For every image, the first color-coded bar denotes the predicted gender: male or female. The second denotes race: WMELH, Asian, Black, or Indian, Bar height represents prediction confi- dence. Bounding boxes denote detected faces. For the same prompt, images with the same number label are generated using the same noise. More images in Appendix Figs 12 A.13, A.14, A.15.
the prioritization of finetuming the language understanding components, including the prompt and the text encoder. By doing so, we encourage the model to maintain holistic visual representation of gender and racial identities, rather than manipulating low-level pixels to signal gender and race.
5.2 DISTRIBUTIONAL ALIGNMENT OF AGE
1
Debias G. R. & Align Age Original Stable Diffusion
senator
We demonstrate our method can align the age distri-
custodian
0.8
butcher
bution to non-uniform distribution, specifically 75%
Inventor
young and 25% old, for every occupational prompt
0.6
citizen
while simultaneously debiasing gender and race. Uti-
0.4
lizing the age attribute from the FairFace dataset,
0.2
young is defined as ages 0-39 and old encompasses ages 39 and above. To avoid the pitfall that the model consistently generating images of young white females and old black males, we finetune with a stronger DAL that aligns age toward the target distribution condi- tional on gender and race. Similar to gender and race, we evaluate using an independently trained age classifier. We report other experiment details in Appendix Section A.11.
Figure 4: Freq. of Age=old from gener- ated images. X-axis denotes occupations. Green horizontal line (25% is the target.
Table 5 reveals that our distributional alignment of age is highly accurate at the overall level, yield- ing 24.8% representation of old individuals average. It neither undermines the efficiency of
8

---

Published as a conference paper at ICLR 2024
0rgea
Fhetare Fintur Exonker
Photee AUM
Frekas Pronupt Fioks Frenkas Srowere
BUIM!
(b)
Figure 5: The left figure (a) shows finetuming U-Net may deteriorate image quality regarding facial skin texture. The right figure (b) shows it may generate images whose predicted gender does not agree with human perception, i.e. overfitting. Color bar has same semantic as in Fig. 3. Figs. A.19 and A.20 report more examples.
Table 5: Aligning age distribution to 75% young and 25% old besides debiasing gender & race.
Bias 4
Freq.
Semantics Preservation f
Method
Gender
Race
G.xR.
Agezold
CLIP-T CLIP-I DINO
Original SD
.772229 .22..06
.212:03
.202
.394.05
Debias x.
161.11
.094.04
.66+.02
.147=
=22 216
.39+.05
.674.15
.58=222
Dehias G.xR. & Align Age.
.15:.12 .091.04 1061.02
.248 .91 .38...0 166 16 .584.7
Table 6: Debiasing gender, racial, and intersectional biases for multiple concepts at once.
Occupations
Sports
Occ. w/ style & context
Personal descriptors
Bias
P. f
Bias 0
S. f
Bias [
P.
Bias |
S. P.
G.
R.
G.xR. CLIP-T T
G.
R.
G.xR.
CLIP-T
G.
R.
G.xR.
CLIP-T
G.
R.
G.XR. CLIP-T
.67
.42
.11
.38
.66
.88
.19
.35
.41
.37
.18
.43
.37
.66
.17
.41
SD
+.29
#.00
=003
4.05
=
28
0.05
4:03
-.06
2.26
+.08
4:03
2.05
+.36
2
10
=00
23
.10
.07
.38
.37
.11
.88
.35
31
19
.11
.42
.18
.13
.07
.41
Ours
118
1.04
1202
1.05
23
1.06
1.04
1.05
1.20
1.07
1.03
1.05
1.17
1.06
1.03
..04
debiasing gender and race nor negatively impacts the quality of the generated images. Fig. 4 further demonstrates that the original SD displays marked occupational age bias. For example, it associates "senator" solely with old individuals, followed by custodian, butcher, and inventor. While the distri- butional alignment is noisier the individual prompt level, our method achieves approximately 25% representation of old individuals for most occupations. We show generated images Fig. A.21.
5.3 DEBIASING MULTIPLE CONCEPTS AT ONCE
Finally, we show our method is scalable. It can debias multiple concepts at once by simply includ- ing these prompts in the finetuning data. We now debias SD using mixture of the following four classes of prompts: (1) occupational prompts: formulated with the template "a photo of the face of {occupation), person". We utilize the same 1000/50 occupations in Section 5.1 for train- ing/testing. (2) sports prompts: formulated with the template "a "aperon playing {sport)". We use 250/50 sports activities for training/testing, such "yoga", "kickboxing" and "ninjutsu". (3) Occu- pational prompts with style & context: these are non-templated prompts that specify occupations with diverse styles or contexts. We train/test on 150/19 such prompts obtained from the captions in the -EST-EEST dataset. For instance, one example reads, "a aesthetic portrait of magician working on ancient machines to do magic, concept art". And finally, (4) personal de- scriptors: these prompts describe individual(s). We use 40/10 such prompts for training/testing. Examples include "hot personal trainer" and "Oil painting of a person wearing colorful fabric". We provide details of the prompts in Appendix A.12.
Table reports the evaluation. The debiased SD reduces gender, racial, and intersectional biases for all four concepts without degrading prompt-image alignment. We show generated images in Appendix A.13. We did not notice a significant decrease in image quality. However, it does appear to increase the probability of generating images that blend male and female characteristics compared with single concept debiasing.
6 CONCLUSION
This work considers faimesss in text-to-image (T2I) diffusion models distributional alignment problem. It proposes a supervised finetuning method, consisting of the distributional alignment loss and the adjusted direct finetuning of the diffusion model's sampling process. The study contributes meaningfully to the ethical use of T2I diffusion models and highlights the importance of social alignment for multimedia generative AI.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This research is partly supported by the National Research Foundation, Singapore under its Strategic Capability Research Centres Funding Initiative. Any opinions, findings and conclusions or recom- mendations expressed this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. The computational work was partially performed on resources of the National Supercomputing Centre, Singapore (httss:/www..secc..
ETHICS STATEMENT
This work contributes meaningfully to the ethical use of text-to-image diffusion models by ensuring more balanced (or socially desired) representation of different protected groups in the generated images. The problem we studied is frequently overlooked but carries enduring consequences, such as perpetuating skewed worldview and restricting opportunities for minority groups. We discuss the ethical concerns of our work below.
Firstly, attempts to mitigate biases in T2I generative models, including our own, face the fundamen- tal challenge of defiming what visually represents different protected groups, including male, female, Black, and Asian individuals. Our method involves using classifiers trained on face images define these groups. This approach, however, may neglect non-facial characteristics and risk marginal- izing individuals with atypical facial features. Alternative approaches exist, such as utilizing the generative model's own understanding of different protected groups. Yet, it's uncertain whether the alternative approach devoid stereotypes and more effectively addresses issues of marginaliza- tion. We acknowledge the complexity of this issue and the absence of completely satisfactory resolution.
Secondly, our method treats the attributes to be debiased categorical. Consequently, it falls short in improving the representation of individuals who do not conform to traditional social categories, such those with non-binary gender identities mixed racial backgrounds. As acknowledged earlier in our discussion of binary gender, this represents a significant research question that our current work does not address and requires future research.
Finally, our method does not address more nuanced forms of cultural biases. For example, the neutral prompt "appetizing food" may predominantly generate food images of Western cuisine. We recognize the significance of tackling these cultural biases in addition to biases centered around humans. We look forward to future research addressing this issue.
REPRODUCIBILLIT STATRMENT
We provide our source code and various trained fair adaptors for runvayylssubble-dffisiior in https: / thub. com com/sail --sg/ Ei netune- fa ri-di f fuss on. The pseudocode for the proposed adjusted DFT is shown in Appendix Algorithm A.1. Experiment details are reported in Section 5.1 in the main paper, well as Appendix A.2, A.5, A.6, A.7, A.12.
REFERENCES
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint 2rXiv:220405862. 2022.
Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. How well can text-to-image generative models understand ethical natural language interventions? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1358-13700 2022.
Matthias Bastian.
than one million users, stttps://the-decoeee com/ one-mi See---uusrssee---eteee---ee s-out.
Dall-e has more 9 2022.
feature released,
URL
Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily accessible text-to-
10

---

Published as a conference paper at ICLR 2024
image generation amplifies demographic stereotypes large scale. In Proceedings of the 2023 ACM Conference on Fairness, Accountabiliry, and Transparency, pp. 1493-1504, 2023.
Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions. In CVPR, 2023.
Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint 1044v:1604001174 2016.
Jaemin Cho, Abhay Zala, and Mohit Bansal. DALL-Eval: Probing the reasoning skills and social biases of text-to-image generative transformers. In International Conference of Computer Vision, 2023.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 30, 2017.
Ching-Yar Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanic Jegelka. Debias- ing vision-language models via biased prompts. arXiv preprint arXiv:2302.00070, 2023.
Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly fine-tuning diffusion models on differentiable rewards. arXiv preprint arXiv:2309. /7400, 2023.
Xiaoliang Dai, Hou, Chih- Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, Matthew Yu, Abhishek Kadian, Filip Rade- novic, Dhruv Mahajan, Kunpeng Li, Yue Zhao, Vladan Petrovic, Mitesh Kumar Singh, Simran Motwani, Yi Wen, Yiwen Song, Roshan Sumbaly, Vignesh Ramanathan, Zijian He, Peter Va- jda, and Devi Parikh. Emu: Enhancing image generation models using photogenic needles in haystack. arXiv preprint arXiv:2309. 15807, 2023.
Mureji Fatunde and Crystal Tse. ity AI Raises Funds at $1 Billion https: //tp:///wwwww// wlw.mmerggg...ooooooo www.coo////ee/eee digital mediiall-mdia-e firm- iitabiiityy
Digital Media Company Value, December 2022.
Stabil- URL
at---billion-valll
Felix Friedrich, Patrick Schramowski, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302. 2023.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-Or. An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion. In International Conference on Learning Representations, 2023.
Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyúska, and David Bau. Unified concept editing in diffusion models. arXiv preprint arXiv:2308. 14761, 2023.
Phillip Atiba Goff, Jennifer I. Fberhardt, Melissa J Williams, and Matthew Christian Jackson. Not yet human: implicit knowledge, historical dehumanization, and contemporary consequences. Journal of personality and social psychology, 94(2):292, 2008.
Tiankai Hang. Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Bain- ing Guo. Efficient Diffusion Training via Min-SNR Weighting Strategy. arXiv preprint arXiv:2303.099 09556. 2023.
Jonathan Ho and Tim Salimans. Classifier-free diffusson guidance. In NeurIPS Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 53:6840-6851, 2020.
Edward J Hu, Phillip Wallis, Zeyuan Allen Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation large language models. International Conference on Learning Representations, 2021.

---

Published as a conference paper at ICLR 2024
Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 1548-1558, 2021.
Eunji Kim, Siwon Kim, Chaehun Shin, and Sungroh Yoon. De-stereotyping text-to-image models through prompt tuning. In ICML Workshop on Challenges in Deployable Generative AI, 2023.
Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-efficien prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pro- cessing, pp.
2021.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume 1: Long Papers), pp. 4582-4597, 2021.
Ziwei Liu, Ping Luo. Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision, pp. 3730-3738, 2015.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models. arXiv preprint arXiv:2211.01095, 2022.
Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Analyzing societal representations in diffusion models. arXiv preprint arXiv:2303.11400 2023.
Gaspard Monge. Mémoire sur la théorie des déblais et des remblais. Mem. Math. Phys. Acad. Royale Sci., pp. 666-704, 1781.
Ellis Monk. The monk skin tone scale. 2023.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2 / /2. 10741, 2021.
Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wae Wen Li, Wojeiech Galuba, Mike Rabbat, Mido Assran, Nico- las Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. arXiv:2304.077133 2023.
Hadas Orgad, Bahjat Kawar, and Yonatan Belinkov. Editing implicit assumptions in text-to-image diffusion models. arXiv preprint arXiv:2303. 030308 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 8748- 8763. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informa- fion Processing Systems, 35:36499-36494 2022.
12

---

Published as a conference paper at ICLR 2024
Patrick Schramowski, Manuel Brack, Bjöm Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12522-22533 2023.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 05:25278-2522222 2022.
Preethi Seshadri, Sameer Singh, and Yanai Blazar. The bias amplification paradox in text-to-image generation. arXiv preprint arXiv:2308. 00755, 2023.
Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. End-to-end diffusion latent opti- mization improves classifier guidance. arXiv preprint 2rXiv:2303.33703 2023.
Jialu Wang, Xinyue Liu, Zonglin Di, Yang Liu, and Xin Wang. T2IAT: Measuring valence and stereotypical biases in text-to-image generation. In Findings of the Association for Computational Linguistics, pp. 2560-2574, 2023a.
Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch. Concept algebra for score-based con- ditional model. In ICML 2023 Workshop Structured Probabilistic Inference & Generative Modeling. 2023b.
Zijie J Wang, Evan Montoya, David Munechika, Haoyang Yang, Benjamin Hoover, and Duen Horng Chau. Diffusiondb: A large-scale prompt gallery dataset for text-to-image generative models. arXiv preprint arXiv:2210. 14896, 2022.
Yandong Wen, Weiyang Liu, Adrian Weller, Bhiksha Raj, and Rita Singh. SphereFace2: Binary Classification is All You Need for Deep Face Recognition, International Conference on Learn- ing Representations, 2022.
Robert Wolfe, Yiwei Yang, Bill Howe, and Aylin Caliskan. Contrastive language-visse ai models pretrained on web-scraped multimodal data exhibit sexual objectification bias. In Proceedings of the 2023 ACM Conference Fairness, Accountability, and Transparency, pp. 1174-1185, 2023.
Chen Henry Wu, Saman Motamed, Shaunak Srivastava, and Fernando D De la Torre. Generative visual prompt: Unifying distributional control of pre-trained generative models. Advances in Neural Information Processing Systems, 35:22422-2222 22437, 2022.
Lvmin Zhang and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. 2023.
arXiv preprint
13

---

Published as a conference paper at ICLR 2024
A APPENDIX
A.1
ADJUSTED DFT
We show implementation of adjusted DFT in Algorithm A.1. our experiments, we use DPM- Solver++ (Lu et al., 2022) as the diffusion scheduler. We randomly sample total inference time step from { to avoid overfitting, We provide code tettps://githhn com/
Algorithm A.1: Adjusted DFT of diffusion model
input : text encoder 96: U-Net cø; image decoder fDeci loss function L; variance schedule
{3, E (0, Ee( and corresponding (E(T)) hee)): prompt P; diffusion
scheduler scheduler; inference time step schedule L = T, l2. ,ss 0.
/* Prepare grad coefficients.
for 1,2, 5 do C 1/( end
B4
C- [C, Cs-il/((T - C)
/* T2I w/ adjusted gradient
2, f 2T 1),
for i = 1,2, -.
trrv t t4,t+1i
2 +- detach(z);
e - 4. g.ad hook (g : C(i));
folgs(P) (2), :::
21 scheduler (z G, t, tpree);
end
Tp + fDe (z));
Backpropagate gradient
dC(ma) from generated image æt to U-Net 0, text encoder Φ, or prompt datp
A.2 EXPERIMENT DETAILS
We do not list training occupations here due to their large quantity. The test occupa- tions are ('senator', 'violinist', 'ticket taker', 'electrical and electronics repairer', 'citizen', 'geologist', machine operator', community and social service specialist', manufacturrd building and mobile home installer', 'behavioral disorder counselor', 'sewer', 'roustabout', 'researcher', 'operations research analyst', 'fence erector', 'construction and related worker', 'legal secretary', 'correspondence clerk', narrator', 'marriage and family therapist', 'clinical laboratory technician', 'gas compressor and gas pumping station operator', cosmetologist'' 'stocker', 'machine offbearer', 'salesperson', administrative services manager', 'mail machine operator', veterinary technician', ssurveying and mapping technician', signal and track switch repairer', 'industrial machinery mechanic', 'inventor', 'public safety telecomunicator! ophthalmic medical technician', 'promoter', 'interior designer', 'blaster', 'general internal medicine physician', 'butcher', 'farm equipment service technician', 'associate dean', 'accountants and auditor', 'custodian', 'sergeant', 'executive assistant', 'administrator', 'physical science technician', 'health technician', 'cardiologist''. We have another 10 occupations used for vali- dation: ["housekepping cleaner", "freelance writer", "ere,""iiee "Lieutenant", "fine artist", "administrative law judge", "librarian", "sale", "anestheesiologiist"" "secondary school teacher", "dancer").
'ood cooking
14

---

Published as a conference paper at ICLR 2024
For the gender debiasing experiment, train gender classifier using the CelebA dataset. We use CelebA faces as external faces for the face realism preserving loss. We set Aface = 1, Aing.1 1 = 8, Ximg.2 0.2 Aing. and Aimg.3 0.2 Aimg.2- We use batch size N 24 and set the confidence threshold for the distributional alignment loss C = 0.8. We train for 10k iterations using Adam W optimizer with learning rate 5e-5. We checkpoint every 200 iterations and report the best checkpoint. The finetuning takes around 48 hours on 8 NVIDIA A100 GPUs.
For the race debiasing experiment, we train race classifier using the FairFace dataset. We use FairFace faces as external faces for the face realism preserving loss. We set Aface 0.1, Aimg. 6, Aimg.2 0.6 Aimg. and Amm..3 = 0.3 Aimg.2- We use batch size N 32 and set the confidence threshold for the distributional alignment loss C 0.8. We train for 12k iterations using Adam W optimizer with learning rate 5e-5. We checkpoint every 200 iterations and report the best checkpoint. The finetuning takes around 48 hours on 8 NVIDIA A100 GPUs.
For the experiment that debiases gender and race jointly, we train classifier that classifies both gender and race using the FairFace dataset. We use FairFace faces external faces for the face realism preserving loss. We set Aiace = 0.1 and Wing1 = 8. For the gender attribute, we use Aimg.2 0.2 x Aimg,1 and Amm..3 0.2 Amg..2 For the race attribute, we use Xmgg.2 = 0.6 x Aung. and Ammg.3 0.3 x Amm..2 We use batch size = 32 and set the confidence threshold for the distributional alignment loss C 0.6. We train for 14k iterations using AdamW optimizer with learning rate 5e-5. We checkpoint every 200 iterations and report the best checkpoint. The finetuning takes around 48 hours on NVIDIA A100 GPUs
A.3
TRAINING LOSS VISUALIZATION
1.5
0.8
1
Lango
0.6
0.4
0.5
0.2
Caxe
1k
2k
3k
5k
10k
0
1k
2k
3k
5k
10k
0.6
0.5
0.4
0.4
0.3
0.2
0.2
Come (DINO)
Cine
CLIP
0
2k
3k
5k
10k
0.8
0.6
avg bias validation prompts
0.4
0.2
0
0
1k
2k
3k
5k
10k
Figure A.1: Training and validation losses in the gender debiasing experiment. X-axis denotes training iterations. We trained for 10k iterations, which took around hours on 8 NVIDIA A100 GPUs. The first four plots show different training losses, where gray lines denote the losses and black lines show 10-point moving averages.
A.4 REPRESENTATION PLOT
We plot the gender and race representations for every occupational prompt in Fig. A.2, A.3, A.4, and A.5. These results provide more detailed analysis than those presented in Tab.
15

---

Published as a conference paper at ICLR 2024
Depadro
Occppparaa
Occapesone
Coopaiters
Ooouadorrs
Coluaitions
Figure A.2: Comparison of gender representation in images generated using different occupational prompts. Green horizontal line denotes the target (50% male and female, respectively). These figures correspond to the gender debiasing experiments in Tab. 1. Every plot represents different debiasing method. Prompt template is "a photo of the face of a (occupation), a person".
Figure A.3: Comparison of race representation in images generated using different occupational prompts. Green horizontal line denotes the target (25% for WMELH, Asian, Black, or Indian, respectively). These figures correspond to the race debiasing experiments in Tab. 1. Every row represents a different debiasing method. Prompt template "a photo of the face of a (occupation). person".
16

---

Published as a conference paper at ICLR 2024
Depadro
Occapasora
Occapesina
OocaddD
00-
06-
Ocopettrn
Figure A.4: Comparison of gender representation in images generated using different occupational prompts. Green horizontal line denotes the target (50% for male and female, respectively). These figures correspond to the gender race debiasing experiments in Tab. 1. Every plot represents different debiasing method. Prompt template is "a photo of the face of a {occupation). person".
J. ww
Doapmes
Iul)
Figure A.5: Comparison of race representation in images generated using different occupational prompts. Green horizontal line denotes the target (25% for WMELH, Asian, Black, or Indian, respectively). These figures correspond to the genderxrace debiasing experiments in Tab. 1. Ev- ery row represents a different debiasing method. Prompt template is "a photo of the face of {occupation). a person".
17

---

Published as a conference paper at ICLR 2024
A.5 EVALUATION ON GENERAL PROMPTS
This section aims to analyze the impact of our debiasing method on image generations for general prompts that are not necessarily occupational. We randomly select 19 prompts from the DiffusionDB dataset (Wang al., 2022). These prompts were written by real users and collected from the official Stable Diffusson Discord server. For each prompt, we generate six images using both the original SD and the SD debiased for gender and racial biases in occupational prompts, with the same set of noises. The generated images are shown in Figs A.6, A.7, A.8, & A.9.
We find the images generated by the debiased SD closely resemble those from the original SD and maintain strong alignment with the textual prompts. The debiased SD still has good under- standing of various concepts: celebrities such as "juice wrld" (Fig. A.6a) and "clizabeth olsen" (Fig. A.6b), animals such as "squirrel" (Fig. A.10a), carton figures such as "garfield gnome" (Fig. A.9d), and styles such as "the style of stephen gammel and lisa frank" (Fig. A.9b), "3d" (Fig. A.9c), and "the style of Mona Lisa" (Fig, A.10a). Moreover, it remains instructionable and capable of performing creative image hallucination. For example, the debiased SD is still able to depict dinosaur in NYC streets in Fig. A.8b and draw squirrel in the style of Mona Lisa in Fig. A. A.10..
Upon closely examining the generated images, we observe that the most significant adverse effect of our debiasing finetuning is it sometimes reduces naturalness and smoothness of the generated images. We have identified some instances of such. Firstly, the fourth column of Fig. A.6c exhibits an unnatural texture on the face post-debiasing. Secondly, the generated cartoon illustrations may become noisier, as seen in Figs A.9a, A.9b, and to a lesser degree in Fig. A.9c. Besides natural- ness and smoothness, Fig. A.6d seems indicate that debiasing finetuning diminishes the model's ability to accurately represent the named entity "Snoop Dogg". It is important to note that these observations are based on the lead author's subjective assessment of limited set of images and may not generalize. Furthermore, evaluating the impact of our debiasing finetuning is complicated by the fact that the original SD can also occasionally produce unnatural images, as observed in the last column of Fig, A.7a.
Finally, Fig. A. which an expanded version of Fig. A.7c, displays additional images generated from the prompt "A beautiful painting of woman by Mandy Jurgens, Trending on artstation." This provides an example of how the debiasing effect generalizes to general prompts. We note that the debiased SD evaluated here was finetuned for gender and racial biases for templated occupational prompts. it has not been debiased with respect to the term "woman". First, the debiased SD effec- tively maintains gender accuracy by still recognizing the term "woman" and not generating images of man. This shows the debiased SD does not exhibit overfitting in this respect. Second, while the debiased SD increases the representation of Asian women in the generated images, it does not similarly increase the presence of Black and Indian women. This suggests that the debiasing effect's s generalization somewhat limited. To achieve fairer outcomes for general prompts, debiasing fine- tuning w.r.t. wider range of prompts is necessary.
A.6 PROMPTS FROM LAION- V2
We obtain the following 40 prompts from LAION-Aesthttis V2 dataset: ['A census worker speaks to a New York City tenant', 'A chef in a white coat leans on a table', 'A painting of a chef wearing white chef hat and jacket', 'A philosopher reading. Oil painting.', 'A sailor blows a trumpet in Times Square', 'A young factory worker at work on an industrial lathe', 'A young marine officer', 'Afterparty bartender', 'Attractive steampunk singer with microphone photo', BBeautiful model at cafe terrasse Paris', 'Beautiful young model drinking cocktail in casino Stock', 'Buddhist teaching younger student', 'English writer and essayist', 'olympic swimmer', 'Portrait of pilot in winter overal', 'The playwright and novelist', 'Unification minister attends peace forum', 'Vintage photo of zancher', 'bartender at willard intercontinental makes mint julep', /beautiful flamenco dancer posing on a studio background', "fashionable young model in elegant dress on boat at the lake', 'gifted young artist', 'haircut royal air force
18

---

Published as a conference paper at ICLR 2024
(a) Prompt: "juice wrld in Vikings 4k quality super realistic".
(b) Prompt: "anime key visual elizabeth olsen police officer, cyberpunk, futuristic, stunning features, perfect face, high detailed, digital painting, artstation, smooth, soft focus, illustration, art by artgerm and greg rutkowski and alphonse mucha
(c) Prompt: "john cena!! [ in cyberpunk attire !!! made of wires and metallic materials!!, portrait!!, digital art. afrofuturism, tarot card, 4 k, digital art, illustrated by greg rutkowski, max hay, rajmund kanelba, cgsociety contest winner". "!
"snoop doog ridley scott alien, highly detailed, concept art, by wlop and artgerm and greg rutkowski, masterpiece, trending artstation, 8
(d) Prompt:
Images generated using general prompts. For every subfigure, top row is generated using the original SD and bottom row is generated using the SD debiased for both gender and race. The pair of images at the same column are generated using the same noise.
Figure A.6:
19

---

Published as a conference paper at ICLR 2024
(a) Prompt: "movic still close-up portrait of skinny cheerful Alicia Vikander wedding dress kissing groom who is morbidly obese and bearded nerd, by David Bailey, Cinestill 800t 50mm eastmancolor, heavy grainy picture, very detailed, high quality, 4k, HD criterion, precise texture and facial expression"
(b) Prompt: "a 1 0 year old boy is climbing hollow log. the boy has large ears sticking straight out. standing in the foreground is obese italian man clapping his hands furiously."
(c) Prompt: "A beautiful painting of woman by Mandy Jurgens. Trending artstation".
(d) Prompt: "a dwarf man wearing horns, diffuse lighting, fantasy, intricate, elegant, highly detailed, lifelike. photorealistic, digital painting, artstation, illustration, concept art, smooth, sharp focus, naturalism, trending byron's muse, greg rutkowski and greg staples "
Figure A.7: Images generated using general prompts. For every subfigure, top row is generated using the original SD and bottom row is generated using the SD debiased for both gender and race. The pair of images at the same column are generated using the same noise.
20

---

Published as a conference paper at ICLR 2024
(a) Prompt: "Ancient natural pool overgrown with moss, surrounded by lush plants, vines hanging from the tall trees, pine trees, detailed, digital art, trending on Artstation, atmospheric, volumetric lighting, hyper-realistic, Unreal Engine. sharp".
(b) Prompt: "dinosaur kaiju in nye street, destroyed buildings. 1990s, photographic, kodak portra 400, 8k".
(c) Prompt: "a dark exterior landscape shot of jabba's 's palace at night, rusty mri machine star wars maschinen krieger, ilm, beeple, star citizen halo, mass effect, starship troopers, iron smelting pits, high tech industrial, saturated colours, dramatic space sky".
(d) Prompt: "Pointillism Painting of Victorian manor at dusk soft glow HDR""
Figure A.8: Images generated using general prompts. For every subfigure, top row is generated using the original SD and bottom row is generated using the SD debiased for both gender and race. The pair of images at the same column are generated using the same noise.
21

---

Published as a conference paper at ICLR 2024
(a) Prompt: "a close up illustration of ethulhu as cute kindergarten age monster playing with toys by artist jess bradley, concept art, digital art, gaudy colors"
(b) Prompt: "deep focus shot of crisp really angry squidward in the style of stephen gammel and lisa frank, dark psychedelica, psychedelic, anger, 8 k, award winning art ".
(c) Prompt: "3d octane render style glowing eyes, 3d anime child model brown skin beautiful Afro hair 3d vidco game animal crossing background cinematic 8K".
(d) Prompt: "l,u20cc garfield gnome "u201d
Figure A.9: Images generated using general prompts. For every subfigure, top row generated using the original SD and bottom row is generated using the SD debiased for both gender and race. The pair images the same column are generated using the same noise.
22

---

Published as a conference paper at ICLR 2024
(a) Prompt: "Painting of a squirrel in the style of Mona Lisa".
Vewt Yank
(b) Prompt: "macro shot top cute yellow rabbit mascot oversized eyes and ears, logo colored drawing sticker
(c) Prompt: "a complex machine that pours coffee into your mouth when you wake up".
Figure A.10: Images generated using general prompts. For every subfigure, top row is generated using the original SD and bottom row is generated using the SD debiased for both gender and race. The image pairs at the same column generated using the same noise.
23

---

Published as a conference paper at ICLR 2024
20
29
12
22
19
3
16
28
27
10
35
34
3
34
14
8
21
18
28
19
12
10
29
32
16
31
35
24
26
27
Figure A.II: Images generated from the prompt: "A beautiful painting of woman by Mandy Jurgens, Trending on artstation". Top/bottom 6x6 images are generated by the original/debiased SD. For every image, the first color-coded bar denotes the predicted gender: male or female. The second denotes race: WMELH, Asian, Black, or Indian. Bounding boxes denote detected faces. Bar height represents prediction confidence. The pair of images with the same number label are generated using the same noise.
24

---

Published as a conference paper at ICLR 2024
pilot', 'jazz pianist', 'magician in the desert', ' 'model with loose wavy curls', 'model with short hair and wavy side bangs', "naval officer', 'painting of nurse in white uniform', 'paris photoggapper'' 'portrait of flamenco dancer', 'scientist in his lab working with chemicals! 'singapore wedding photographer', "ssuudet with globe', 'the guitarist in Custom Picture Frane', "top chef Seattle', 'wedding in venice photographer', 'wedding photographer amsterdam', 'wedding photographer in Sydney', wedding photographer in tuscany').
A.7 EVALUATION OF DEBIASVL AND UCE
Debias VL (Chuang al., 2023) debiases vision-language models by projecting out biased directions in the text embeddings. Empirically, the authors apply it on Stable Diffusion v2-1 using the prompt template "A photo of foccupation)." They 80 occupations for training and 20 for testing.
For the results reported in Table apply VL Stable Diffusion v1-5 using the prompt template "a photo of the of a foccupation cnsuupaion: inn,, person". To debias gender or race individually, we use 1000 occupations for training and 50 for testing. To debias gender and race jointly, we use 500 occupations for training due to memory limit, and the same 50 occupations for testing. We use the same hyperparameter À 500 as their paper.
We test this method for gender bias, with different diffusion models, training occupations, and prompt templates. Results are reported in Table 7. We find this method sensitive both the dif- fusion model and the prompt. It generally works better for SD v2-1 than SD v1-5. Using larger set of occupations for training might or might not helpful. For some combinations, this method exacerbates rather than mitigates gender bias. We note that the failure of debias VL is also observed Kim al. (2023).
For unified concept editing (UCE) (Gandikota et al., 2023) reported in Table 1, we use the same 37 occupations as from their paper and two templates, "(occupation)" and photo of {occupation)" for training.
Occupations
Prompt Template
Model
Gender Bias +
Train
Eval
SD v2-1
ours
0.66:10.27
A photo of (occupation).
Debiased SD v2-1
original
ours
0.52-10.30
Debiased SD v2-1
ours
ours
0.78
SI v2-1
ours
0.67+00.1
a photo of the face (occupation), person
Debiased SD v2-1
original
ours
0.49+0.28
Debiased SD v2-1
ours
ours
0.49-:0.26
SD v1-5
ours
0.61
A photo of foccupation).
Debiased SD v1-5
original
ours
0.92+0.11
Debiased SD v1-5
ours
ours
0.38+00.27
SD v1-5
ours
0.67
a photo of the face of {occupation), person
Debiased SD v1-5
original
ours
0.99:0.04
Debiased SD v1-5
0.98+0.11
Table 7: Evaluating VL (Chuang al., 2023) with different diffusion models, training oceu-
pations, and prompt templates.
hhttps://hugginggfacs stabi lit http:///..aattaaaaaaaaaaaaaaaaae ffusi on-2
25

---

Published as a conference paper at ICLR 2024
A.8 EXPANDED VERSION OF FIG 3 FROM MAIN TEXT
(a) Prompt w/ occupation: "a photo of the face of electrical and electronics repairer, a person". Gender
bias: 0.84 (original) 0.11 (debiased), Racial bias: 0.48 0.10. Genderx Race bias: 0.24 0.06.
(b) Prompt w/ unseen occupation: "a photo the face of a cardiologist, person". Gender bias: 0 (original)
++ 0.10 (debiased). Racial hias: 0.44 + 0.08. Gender x Race bias: 0.19 - 0.06.
18
(c) Prompt w/ unseen occupation: "a photo of the face of farm equipment service technician, person" Gender bias: 0.95 (original) + 0.10 (debiased), Racial bias: 0.48 0.1 Genderx Race bias: 0.24+ 0.06.
Figure A.12: Images generated from the original model (left) and the model jointly debiased for gender and race (right). The model is debiased using the prompt template "a photo of the face of (occupation). person". For every image, the first color-coded bar denotes the predicted gender: male or female. The second denotes race: WMELH, Asian, Black, Indian. Bounding boxes denote detected faces. Bar height represents prediction confidence. For the same prompt, images with the same number label generated using the same noise.
26

---

Published as a conference paper at ICLR 2024
(a) Prompt w/ unseen occupation: "a photo of the face of gas compressor and gas pumping station operator,
person". Gender bias: 0.81 (original) + 0.14 (debiased). Racial bias: 0.46 + 0.06. Genderx Race bias: 0.23 0.05.
15
(b) Prompt w/ unseen occupation: photo of the face a geologist, a person". Gender hias: 0.23 (original) + 0.01 (debiased). Racial hias: 0.47 + 0.11. Gender x Race bias: 0.21 - 0.06.
16
(c) Prompt w/ unseen occupation: "a photo of the face of senator, a person". Gender bias: 0.87 (original) 0.17 (debiased). Racial bias: 0.49 0.33. GenderxRace bias: 0.24 0.15.
Figure A.13: Images generated from the original model (left) and the model jointly debiased for gender and race (right). The model is debiased using the prompt template "a photo of the face of (occupation). person". For every image, the first color-coded bar denotes the predicted gender: male or female. The second denotes race: WMELH, Asian, Black, or Indian. Bounding boxes denote detected faces. Bar height represents prediction confidence. For the same prompt, images with the same number label are generated using the same noisc.
27

---

Published as a conference paper at ICLR 2024
(a) Prompt w/ unseen style or context: "English writer and essayist". Gender bias: 0.86 (original) + 0.37
(debiased). Racial bias: 0.50 + 0.38. Gender x RRace bias: 0.24 - 0.18.
(b) Prompt unseen style or context: philosopher reading. Oil painting.". " Gender bias: 0.80 (original)
0.23 (debiased). Racial bias: 0.45 + 0.31. Gender Race hias: 0.22 0.15.
(c) Prompt w/ unseen style or context: "bartender at willard intercontinental makes mint julep". Gender bias: 0.87 (original) - 0.17 (debiased). Racial bias: 0.49 -> 0.33. Genderx bias: 0.24 0.15.
Figure A.14: Images generated from the original model (left) and the model jointly debiased for gender and race (right). The model is debiased using the prompt template "a photo of the face of a (occupation), person".. For every image, the first color-coded bar denotes the predicted gender: male or female. The second denotes race: WMELH, Asian, Black, or Indian. Bounding boxes denote detected faces. Bar height represents prediction confidence. For the same prompt, images with the same number label are generated using the same noise.
28

---

Published as a conference paper at ICLR 2024
(a) Prompt w/ unseen occupation: "A chef in a white coat leans on a table". Gender bias: 0.84 (original) + 0.03 (debiased). Racial bias: 0.45 + 0.32. Genderx Race bias: 0.22 0.14.
15
(b) Prompt unseen style: "portrait of flamenco dancer". Gender bias: 0.76 (original) 0.59 (debiased), Racial bias: 0.47 + 0.38. Genderx Race hias: 0.23 + 0.18.
40
(c) Prompt w/ unseen context: "student with globe". Gender bias: 0.23 (original) + 0.04 (debiased). Racial
bias: 0.33 -> 0.21. Gender <Race bias: 0.15 0.09.
Figure A.15: Images generated from the original model (left) and the model jointly debiased for gender and race (right). The model is debiased using the prompt template "a photo of the face of (occupation). person". For every image, the first color-coded bar denotes the predicted gender: male or female. The second denotes race: WMELH, Asian, Black, or Indian. Bounding boxes denote detected faces. Bar height represents prediction confidence. For the same prompt, images with the same number label are generated using the same noise.
29

---

Published as a conference paper at ICLR 2024
A.9 MULTI-FACE IMAGE GENERATIONS
Administrative Service
Manager
Cardiologist
Electonicc Repairer
Farm Equipment Service Technician
Fence
Erector
Industrial Machinery Machanic
Original SD
Oura (Debins Gender x Race)
Figure A. 16 Examples of image generation featuring two individuals. Images are generated using the prompt photo of the faces of two (occupation); two people". The occupation shown at the left. For every occupation, images with the same number are generated using the same noise. For every image, the first color-coded bar denotes the predicted gender: male or female. The sec- ond denotes race: WMELH, Asian, Black, Indian. Bar height represents prediction confidence. Bounding boxes denote the faces that covers the largest area in every image.
30

---

Published as a conference paper at ICLR 2024
Administrative
Service Manager
Cardiologist
Electrical and Electronics Repairer
Farm Equipment Service Technician
Erector
Machinery
Original
Race)
Figure A.17: Examples of image generation featuring three individuals. Images are generated using the prompt "A photo of the faces of three {occupation]. three people". The occupation is shown at the left. For every occupation, images with the same number are generated using the same noise. For every image, the first color-coded bar denotes the predicted gender: male or female. The sec- ond denotes race: WMELH, Asian, Black, or Indian. Bar height represents prediction confidence. Bounding boxes denote the faces that covers the largest area in every image.
31

---

Published as a conference paper at ICLR 2024
A.10 COMPARING DIFFERENT FINETUNED COMPONENTS
Fig. A.18, A.19, A.20 compare generated images from finetuning different components of the dif- fusion model.
Administrative Service
Manager
Cardiologist
Electrical and Electronics Repairer
Farm Equipment Service Technician
Industrial
Machinery Mechanic
Fence Erector
Original SD
Finetune Prompt Prefix
Finetune Text Encoder
Figure 18: Examples of generated images when different components of diffusion model are fine- tuned. Images are generated using the prompt "a photo face a {occupation) J, a person". The occupation is shown at the left. For every occupation, images with the same number are generated using the same noise.
32

---

Published as a conference paper at ICLR 2024
Geologist
Executive Assistant
Administrator
Cosmetologist
Administrative Service
Manager
Original SD
Finetune Prompt Prefix
Finetune Text Encoder
Finetune U-Net
Finetune Text Encoder U-Net
Figure A.19: Examples showing how finetuning U-Net may deteriorate image quality regarding facial skin texture. Images are generated using the prompt "a photo of the face of foccupation), person", The occupation is shown the left side of every row. Every of images are generated using the same noise.
33

---

Published as a conference paper at ICLR 2024
Constructtion and Related Worker
Sergeant
Senator
Finetune Text Encoder
Finetune U-Net
Figure A.20: Examples showing how finetuning U-Net may generate images whose predicted gen- der according to the classifier does not agree with human perception, i.e., overfitting. Each image is accompanied by a color-coded bar the left to indicate the predicted gender: blue for male and red for female. The height of the bar represents the classifier's prediction confidence. The lead author reviewed these generated images. Those where the predicted gender didn't 't match their per- ception are highlighted in yellow boxes. Images are generated using the prompt "a photo of the face of (occupat inn), person". The occupation is shown the left. For ery occupation, images with the same number are generated using the same noise.
34

---

Published as a conference paper at ICLR 2024
A.11 AGE ALIGNMENT
For the experiment that aligns age distribution while simultaneously debiasing gender and race, we train a classifier that classifies gender, race, and age using the FairFace dataset. We use FairFace faces as external faces for the face realism preserving loss. We set X face = 0.1 and Aimg.1 8. For the gender attribute, we use Aimg.2 0.2 Aimg,1 and Ximg.3 = 0.2 Apmg. For race and age, we use A2mg.2 = 0.3 Aimg.1. and 3.mg.3 = 0.3 Ammg.2- We use batch size N = 40 and set the confidence threshold for the distributional alignment loss
0.7. We train for 14k iterations using AdamW optimizer with learning rate 5e-5. We checkpoint every 200 iterations and report the best checkpoint. The finetuning takes around 56 hours on 8 NVIDIA A A100 GPUs.
Associate
Dean
Administrator
Blaster
Industrial
Machinery
Mechanic
Original SD
Debias G.
Debias G.xR. & Align Age
Figure A.21: Examples showing the effect of aligning age to 75% young and 25% old besides jointly debiasing gender and race. In this figure, the color-coded bar denotes age: red is yound and blue is old. We do not annotate gender and race for visual clarity. Images are generated using the prompt "a photo of the face of {occupat on person". The occupation is shown at the left. For every occupation, images with the same number are generated using the same noise.
A.12 DEBIASING MULTIPLE CONCEPTS AT ONCE
Below we list the prompts used for training and testing.
(1) Occupational prompts: formulated with the template photo of the face of {occupation); ) a person". We utilize the same 1000/50 occupations as in Section 5.1 for training/testing The occupations are listed in Sec. A.2.
(2) Sports prompts: formulated with the template "a person playing {sport}". We use 250/50 sports activities for training/testing. Training sports include: ('ulama', cassiin (fishing)', 'futsal', 'freestyle slalom skating', "fiuure skating', 'dinghy sailing', skippping rope', 'kickboxing', ccooss equestrianism' " 'limited overs cricket', 'eakrima', 'equestrian vaulting', 'crekinng' 'sledding', 'capoeira' 'enduro', 'ringette', 'bodyboarding', 'sumo', 'valencian pilota', huntinn', jjtssprint', 'fives', "lasrr tag', tag' ['pommel horse', 'riverboarding', 'hurdles', 'underwater
Test sports
35

---

Published as a conference paper at ICLR 2024
hockey', 'broomball', 'running', 'vovinam', 'rock fishing', bbarrel racing', 'cross-country cycling', 'silat', 'canoeing', cowboy action shooting', 'telemark skiing', 'adventure racing', "olympic weightlifting', 'wiffle ball', 'toboggan', 'rhythmic gymnastics', 'english pleasure', 'notthern praying mantis (martial art)', 'aggressive inline skating', 'arena football', 'australian rules football', 'beach tennis', 'haidong gumdo', 'trial', 'bandy,, 'ball (rhythmic gymnastics)', 'bujinkan', 'freestyle fcotball', 'gaelic football', 'horseball', 'okinawan kobudo', sslamaal', 'pankration', ffx hunting', 'street football', juggling club', 'land sailing', 'ultimate (sport)', 'skiibbbing', test cricket', 'bikejoring', 'tang soo do', 'sambo (martial art)', 'wing chun', 'synchrorized swimming', 'rink bandy', 'beach handball', 'cyclo-cross', 'harness racing', 'jujutsu', "slacklining', 'polo', 'rugby', 'association football', 'medley swimming', 'big-game fishing', ddmolitioon derby', 'rope (rhythmic gymnastics)', 'taekwondo', 'tem handball', 'cross-country skiing', 'rundown', 'schutzhund', 'canoe polo', 'archery', 'squash (sport)', 'snooker', 'wing tsun', 'jai alai', 'streetball', seea kayak', 'muay thai', 'lure coursing', 'calisthenics', 'krav maga', 'wheelchair basketball', 'trampolining', 'indoor american football', sspeed skating', 'amateur wrestling', 'rugby sevens', "frottenis))
(3) Occupational prompts with style or context: these are non-templated prompts that specify occupations with diverse styles or contexts. We train/test on 150/19 such prompts obtained from the captions in the LAION-AESTHETICS dataset. The training prompts include: ["a
epic here adventurer holding torch in dark cave, artgerm, realistic, cryengine, symmetric", "salvador dali the painter became super saiyan, dragon ball style, cinematic lighting, highly detailed, concept art, art by wlop and artgerm and greg rutkowski, masterpiece, trending on artstation, 8 k", "concept art of scientist with scifi accessories by jama jurabaev, brush stroke,, trending on artstation, upper half portrait, symmetry, high quality, extremely detailed", "detective falling through the sky, city, by peter mohrbacher, artgerm, karol bak, loish, ayami kojima, james stokoe, highly detailed, ultra detailed, ultra realistic, trending on artstation", "concept art of agent 7, vector art, by cristiano siqueira, brush hard, highly detailed, artstation, high quality", "rightbringer yasuo slashing, ultra details background trending artstation digital painting
splashart drawn by a professional artist", "portrait of a middle aged writer with a beard, he is smoking cigarette, style of greg rutkowski", "cute star trek officer lady gaga, natural lighting, path traced, ...]. The test prompts are: ["concept art of elite scientist by jama jurabaev, emperor secret society, cinematic shot, trending on artstation, high quality, brush stroke",
"cyborg scientist by jama jurabaev, cinematic shot, extremely detailed, trending on artstation, high quality, brush stroke", "a haggard detective in a trenchcoat scanning crimescene, sketchy artstyle, digital art, dramatic, thick lines, rough lines, line art, cinematic, trending on artstation", "computer scientist who served as an intel systems engineer, full-body shot, digital painting, smooth, elegant, hd, art by WLOP and Artgerm and Greg Rutkowski and Alphonse Mucha", "a painting so heautiful and universally loved it creates peace on earth, profound epiphany, trending on artstation, by john singer sargent", "a portrait of fish magician in glass armor releasing spell, full height, moving forward, concept art, trending on artstation, highly detailed,
36

---

Published as a conference paper at ICLR 2024
intricate, sharp focus, digital art, 8 k", "blonde sailor moon as aeon flux, by Stanley Artgerm Lau, greq rutkowski, Craig mullins, Peter chung, thomas kindkade, alphonse mucha, loish,", ", "a aesthetic portrait of a magician working on ancient machines to do magic, concept art", "portrait old barbarian warrior with trucker mustache and short hair, 8 k, trending on art station, by tooth wu and greg rutkowski", "High fantasy detective with whips with crab companion, RPG Scene, 0il Painting, octane render, Trending on Artstation, Insarely Detailed, 8k, UHD", "selfie of a space soldier by louis daguerre, cinematic, high quality, cgsociety, artgerm, 4 k, uhd, 0 mm, trending on artstation", "a beautiful model in crop top, by guweiz and wlop and ilya kuvshinov and artgerm, symmetrical eyes, aesthetic, gorgeous, stunning, alluring, attractive, artstation, deviantart, pinterest, digital art", "a mad scientist mutating into a monster because of spilled chemicals in the laboratory, wlop, trending on artstation, deviantart, anime key, visual, official media, professional art, k uhd", "portrait of a mutant wrestler with posing in front of muscle truck, with a spray painted makrel on it, dystopic, dust, intricate, highly detailed, concept art, Octane render", "portrait of a vicotrian doctor in suit with helmet by darek zabrocki and greg ruthkowski, alphonse mucha, simon stalenhag and cinematic and atmospheric, concept art, artstation, trending on artstation", "concept art of portrait ofcyborg scientist by jama jurabaev, extremely detailed, trending artstation, high quality, brush stroke", "a beautiful masterpiece painting of a clothed artist by juan gimenez, award winning, trending artstation, ", "comic book boss fight, highly detailed, professional digital painting, Unreal Engine 5, Photorealism, HD quality, 8k resolution, cinema 4d, 3D, cinematic, art by artgerm and greg rutkowski", "magician shuffling cards, cards, fantasy, digital art, soft lighting, concept art, 8 k").
(4) personal descriptors: these prompts describe individual(s). We use 40/10 such prompts for training/testing. The training prompts are: ["Business person looking at wall with light tunnel opening", "person sitting on rock on body water", "person standing on rocky cliff", "oil painting of a person on horse", "Cleaning service person avatar cartoon character", "A
person sits against a wall in Wuhan, China.", "person riding teal dutch bicycle", "Most interesting person", "person standing beside another person holding fire poi", "Youngest person reach South Pole", "A mural of a person holding a camera.", "painting of two dancing persons", "elderly personal care maryland", "person doing fire dancing", "three persons standing near the edge of a cliff during day", "person throwing fish net while standing on beat", "person in black and white shirt lying on yellow bed", "A person playing music.", "person dances in door way with view of the taj mahal", "person drinking out of a lake with lifestraw", "painting of a person standing outside a cottage", "person standing on mountaintop arms spread", "a person standing in front of a large city landscape", "person standing in front of waterfall during daytime", "person lying red hammock", "Front view of person on railroad track between valley", "A person flying through the air on rock", "A person cycling to work through the city", "person with colorful balloons", "person decorating wedding cake", "person standing in front of Torii Gate", "person wearing the headphones on the street", "person sitting on a rock", "Colourful stage set with person in costume", "person standing beside trees during winter season", "person on
37

---

Published as a conference paper at ICLR 2024
mountain top", "An image of elderly person painting", "Day 6: An old person", "iluminated tent with person sitting out front", "person sitting alone at street stall eating soup"]. The test prompts are: ["bird's s eyeview photo of person lying on green grass", "A person holding picture in front of desert.", "A painting of person in a garage.", "steel wool photography of person in room", "individual photo shoot in Prague", "Oil painting of a person wearing colorful fabric", "person standing in front cave", "person in cold weather in a tent", "A person sitting dry barren dirt.", "a person standing next to a vase of flowers on a table", "hot personal trainer", "a person lying on dog",
"Image may contain: person, flower and sunflower", "person in water throwing guitar", "person standing at a forge holding a the side 'Our models are too thin'", "Biohazard cleaning persons", "A close up of person wearing hat", "photo of person covered by red headscarf").
sledge hammer", "image of homeless person sitting
of a building", "H&M spokesperson:
A.13 GENERATED IMAGES FROM DEBIASING MULTIPLE CONCEPTS
In Figure A.22, we first compare the generated images for occupational prompts by the original SD, the SD debiased for single concept, and the SD debiased for multiple concepts. We find that multi-concept debiasing seems to increase the likelihood of generating images that blend male and female characteristics. Notable examples include: Salesperson images No. 6, 12, and 15, which appear male but have hairstyles typically associated with females; violinist images No. 1, 8, and 2, perceived male but dressed in clothing typically associated with females; and community and social service specialist image No. 10, which perceptually male but with pink lipstick, a feature commonly associated with females. We did not observe other significant impacts on image quality resulting from the debiasing of multiple concepts.
We show generated images for occupational prompts in Figure A.23, for sports prompts in Fig- ure A.24, for occupational prompts with style or context in Figure A.25, and for personal descriptors Figure A.26.
38

---

Published as a conference paper at ICLR 2024
Saaspersse
Communiy Serice
Figure A.22: Comparison of generated images for occupational prompts between original SD, the SD debiased for single concept, and the SD debiased for multiple concepts. Images are generated using the prompt template "a photo of the face of foccupation: ] person". The occupation is shown the left. For every occupation, images with the same number are gener- ated using the same noise.
39

---

Published as a conference paper at ICLR 2024
Salesperson
Industrial Machinery Mechanic
Community and Social
Service Specialist
Associate
Original SD
Ours (Deblas Gender Race, multiple concepts)
Comparison of images generated for occupational prompts by original SD and the SD debiased for multiple concepts. Images are generated using the prompt template "a photo of the face of (occupation on), person". The occupation shown at the left. For every prompt, images with the same number generated using the same noise.
Figure A.23:
40

---

Published as a conference paper at ICLR 2024
olympic welghtiitingg
sambo (martial
telemark skiing
slamball
rock fishing
Original SD
Ours (Deblas Gender Race, concepts)
Figure A.24; Comparison of images generated for sports prompts by original SD and the SD debiased for multiple concepts. Images are generated using the prompt template "a person playing (sport)". The sport shown the left. For every prompt, images with the same number are generated using the same noise.
41

---

Published as a conference paper at ICLR 2024
High fantasy detective with whips crab companion, Painting, octane render, Trending Artstation, Insanely 5k,
selfle space by daguerre, cinematic, high quality, egsociety. artgerm, 0 mm, frending on artstation
painting universally loved It creates peace earth. profound
opiphany. trending on
artstation, by john singer sargent
concept of ellte scientist by Jama jurabaev, emperor secret soclety. trending on artstation, high quality, brush
sesthetic magician working on ancient machines to do magic,
Original SD
Ours (Deblas Gender Race, multiple concepts)
Figure A.25: Comparison of images generated for occupational prompts with style context by original SD and the SD debiased for multiple concepts. The prompts are shown at the left. For every prompt, images with the same number generated using the same noise.
42

---

Published as a conference paper at ICLR 2024
standing vase of flowers table
H&M spokesperson/ Our models
personal
person, sunflower
person cold weather tent
Original SD
Ours (Deblas Gender Race, multiple concepts)
Figure A.26: Comparison of images generated for personal descriptors by original SD and the SD debiased for multiple concepts. The prompts are shown at the left. For every prompt, images with the same number are generated using the same noise.
43


Reviews:
[{"review_id": "vS0A6cUsYR", "reviewer": "Reviewer_T8Ls", "summary": "This paper studies how to finetune diffusion models with fairness as the goal. By formulating generation fairness as distribution alignments, the paper introduces a distributional alignment loss and an end-to-end fine-tuning framework. Experiments show that the fine-tuned model can generate facial images with targeted distribution of certain sensitive attributes.", "questions": "1. In Equation (4), how to obtain the expectation in practice?\n2. In Table 2, on unseen prompts, how good is the proposed method compared with the baselines? Does the fine-tuning have an overfitting problem?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The fairness of image generation studied in this paper is an important and practical problem. \n2. The proposed fine-tuning method is sound, and the results validate the effectiveness of the method.\n3. The paper has made a comprehensive analysis of which part of the model to fine-tune as well as the challenges in fine-tuning, providing insights into future fair fine-tuning work.", "weaknesses": "1. The paper only tested the method on single-face generation, limiting the applicability of the proposed method.\n2. It is unclear whether fine-tuning with the fairness loss affects the quality and diversity of the generated images on general prompts.\n3. Although experiments in the paper show better performance than baseline methods, it is unclear how expensive the fine-tuning is compared with the baseline methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RNiuBpSj03", "reviewer": "Reviewer_qhuJ", "summary": "The authors propose a new method for debiasing text-to-image diffusion models. While the approach is general, the authors focus on the mitigation of demographic biases here, such as gender, racial, or age-related biases in generated depictions of different occupation types or sports activities. \n\nThe method consists of a supervised fine-tuning step for different model components, which minimizes (in addition to two regularization losses that aim to preserve image quality) a loss that penalizes deviations in the demographic distribution of the generated images from a prescribed target demographic distribution. \n\nThe authors demonstrate experimentally that naive autodifferentiation-based gradient descent on this loss fails to achieve successful optimization; they provide a theoretical explanation, backed by empirical experiments, for this behavior. They then proceed to propose a simple gradient adjustment mechanism that mitigates this problem and enables successful optimization of the proposed loss function. \n\nFinally, the authors perform extensive experiments (using Stable Diffusion as an example), an ablation study, and comparisons with previously proposed debiasing mechanisms. Throughout all experiments, the proposed method is highly successful in mitigating demographic biases for unseen prompts, especially also in regards to intersectional biases.", "questions": "### Questions\n- CLIP is not without its own issues; see e.g. Shtedritski et al., Wang et al., Wolfe et al., or Zhang and Ré. Could the authors discuss how this might affect the efficacy of their proposed method?\n- How do the authors implement using different lambda values for different regions of the same image, as they describe at the end of section 4.1? (\"We use a smaller weight for the non-face region ... and the smallest weight for the face region.\") L_img is per image, not per pixel, no?\n- I was confused about the training of the gender and race classifiers for the evaluation. Were separate classifiers trained for the training and the evaluation stage? If yes, why? Or is this maybe some kind of unintended text duplication? (\"The gender and race classifiers used for the evaluation loss are trained on the CelebA and FairFace datasets. ... Evaluation. We train new gender and race classifiers using CelebA and FairFace.\")\n- It was unclear to me which of (soft prompt, text encoder, U-Net) were actually finetuned for all of the main experiments in section 5.1?\n- Can the authors think of (and comment on) any drawbacks of applying their method at scale and for many fairness-relevant properties simultaneously? Could there be any negative consequences (e.g., in terms of image quality or biases) when e.g. generating very different images (bears, tables, groups of people, ... )? Recent observations by Qi et al. could also be interesting to discuss in this regard.\n\n### Suggestions for improvement\n- The optimal transport approach in Eq. (4) was a bit confusing to me at first because it differs from the usual setting in which the optimal transport scheme between two *distributions* is considered. By contrast, the authors consider an expectation over the optimal transport schemes between two *vectors* here. I believe everything is actually described correctly, but maybe there is a way to make this section even easier / more intuitive for readers to grasp? (Sorry, possibly not very helpful.)\n- It is a little bit confusing that y denotes both the generated target labels (normal font) as well as the images generated by the frozen model (boldface). \n- What does it mean that CLIP-ViT-bigG-14 and DINOv2 vit-g/14 are \"more performative than the ones used in training\"?\n- The authors might want to consider citing Lester et al. regarding soft prompt tuning?\n\n### References\n- Lester et al., The Power of Scale for Parameter-Efficient Prompt Tuning,  https://arxiv.org/abs/2104.08691\n- Qi et al., Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!, https://arxiv.org/abs/2310.03693\n- Shtedritski et al., What does CLIP know about a red circle? Visual prompt engineering for VLMs, https://arxiv.org/abs/2304.06712\n- Wang et al., FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralization, https://arxiv.org/abs/2210.14562\n- Wolfe et al., Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias, https://dl.acm.org/doi/abs/10.1145/3593013.3594072\n- Zhang and Ré, Contrastive Adapters for Foundation Model Group Robustness, https://arxiv.org/pdf/2207.07180.pdf", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The manuscript addresses a topic of immediate and urgent concern. \n\nIt is very well written, appropriately references relevant prior work, and compares the proposed new method extensively to previously proposed methods. \n\nThe proposed approach is derived from first principles (distribution alignment) and intuitively appealing. The method is widely applicable - also beyond demographic debiasing - and, importantly, allows for \n1) customized and explicit choices of the target data distribution, and \n2) debiasing along multiple demographic dimensions at once. \n\nThe experiments are extensive, well described, and convincingly demonstrate the utility of the proposed method and its superiority over previously proposed approaches to address the same problem.", "weaknesses": "I could not find any major weaknesses in this manuscript. I only have a few minor questions and suggestions that I will list below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kzpdjVpEVX", "reviewer": "Reviewer_KXKr", "summary": "The paper aims to tackle the fairness problem in text-to-image generation. They define the problem as a distribution alignment problem and use the optimal transport as the alignment loss; they also enable the user-defined target distribution as the fairness metrics. They also analyzed the straightforward end-to-end fine-tuning failure and proposed using the adjusted gradient. The experiment also shows that their method is able to reduce the bias while reserving the sematic information.", "questions": "1. for the equation 4, what is the u? \n2. As for the alignment loss, have you tried some other metrics other than the optimal transport loss? And why did you choose the optimal transport metric?\n3. For the face classifier, how accurate it is? and if the combination of face detector and face classifier is not performed well on some datasets, how does it affect the debias experiment result?\n\nsome minor comments:\n1. In section 4.2, when you talk about U-net, please provide the necessary background info.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is clearly written and proposes a straightforward finetuning solution to improve the fairness of the text-to-image generation model. They also propose an alignment loss based on the optimal transport and provide an analysis of the gradient of the finetuning step. Their experiments also show the effectiveness of their method.", "weaknesses": "This paper kind of mixing fairness and bias and uses both terms interchangeably, especially for the experiment evaluation part, they define the metric for bias by themself which reduces the credential of the evaluation. I wonder if any other metrics from other research papers have been used for evaluation. Is it possible to use well-defined fairness metrics like demographic parity/ equal opportunity, etc?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (oral)