AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generation

## Abstract

Abstract Despite the high-quality results of text-to-image generation, stereotypical biases have been spotted in their generated contents, compromising the fairness of generative models.
In this work, we propose to learn adaptive inclusive tokens to shift the attribute distribution of the final generative outputs.
Unlike existing de-biasing approaches, our method requires neither explicit attribute specification nor prior knowledge of the bias distribution.
Specifically, the core of our method is a lightweight adaptive mapping network , which can customize the inclusive tokens for the concepts to be de-biased, making the tokens generalizable to unseen concepts regardless of their original bias distributions.
This is achieved by tuning the adaptive mapping network with a handful of balanced and inclusive samples using an anchor loss.
Experimental results demonstrate that our method outperforms previous bias mitigation methods without attribute specification while preserving the alignment between generative results and text descriptions. Moreover, our method achieves comparable performance to models that require specific attributes or editing directions for generation. Extensive experiments showcase the effectiveness of our adaptive inclusive tokens in mitigating stereotypical bias in text-to-image generation. The code will be available at https://github.com/itsmag11/AITTI .

###### Keywords:

## 1 Introduction

Text-to-image (T2I) generation has gained widespread usage thanks to its ability to produce visual content from user-specified text descriptions. However, alongside the technical achievements, concerns have arisen regarding the presence of stereotypical biases in the generated outputs, as analyzed by various studies [, , , , , ].
The unfairness in the T2I generation is usually reflected by an unequal representation of different social groups. It is observed that in circumstances where no specific attributes are specified in a human-related prompt, the T2I models tend to generate human figures resembling certain genders and races, reinforcing harmful social discrimination [, ]. In particular, some occupations are strongly associated with specific genders following stereotypes, *e.g*., male for doctor and female for nurse. Furthermore, negative concepts like poverty or unattractiveness lead to the generation of people from colored races [, , ]. Exposure to such an unfair environment will strengthen the stereotypes and biases, justify hate in our society, and undermine the rights of minority groups to be treated fairly.
While unfair training datasets are often recognized as the origins of biases in large models, the pervasive nature of stereotypes in our society makes the complete elimination of biases challenging. This triggers the necessity to design de-biasing algorithms to mitigate biases in the pre-trained models.

The main goal of this paper is to devise a bias mitigation method for biased concepts(^1^11For clarity, we define the key terms as: sensitive attributes $\mathcal{A}$ refer to the attributes of interest for achieving fairness (*e.g*., gender), and attribute classes $\mathcal{A}_{c}$ denotes possible classes of the attribute (*e.g*., female and male). Biased concepts $\mathcal{C}$ describe the concepts that contain potential stereotypes.).
An ideal inclusive T2I model yields results with evenly distributed sensitive attributes across all attribute classes, *e.g*., 50% male and 50% female in gender, when no attribute-related instructions are provided. A crucial aspect of a fair model is its ability to generate inclusive outcomes without direct instruction regarding the target attribute class. Besides, users’ unawareness of potential biases related to a target concept should be respected. Therefore, we argue that a good de-biasing algorithm should: (1) achieve fairer results without explicit specification of the target attribute class during generation. (2) require no prior knowledge of the original bias distribution associated with the concept (*e.g*., the doctor concept is stereotypically biased towards males).

Achieving the aforementioned inclusive properties is non-trivial in the absence of direct attribute specification and prior knowledge about the bias distribution. Our study shows that the simple approach of learning a fixed inclusive token via textual inversion [] fails in mitigating biases in concepts originally biased toward different classes (see Sec. 3.2 for detailed discussion).
In addition, it also risks modifying the semantic concepts specified in the original prompt. In this work, we follow the mainstream prompt tuning method but aim at learning adaptive inclusive tokens that do not specify any target class, and yet can shift the bias attribute in generation outcomes towards a more equitable distribution, regardless of which class it was originally biased to.

We hypothesize that the token embeddings of a biased concept, *e.g*., the doctor concept, encode information regarding its bias distribution. Hence, to enable the adaptability of the learned inclusive tokens across various biased concepts, a lightweight adaptive mapping network can be used to find the optimal inclusive token, taking into account the target concept.
In addition, an anchor loss is proposed to guide the desired properties of the inclusive token, minimizing the discrepancy between de-noising UNet predictions of prompt with inclusive token and prompt with the ground truth attribute class. The anchor loss ensures the inclusive token has an aligned impact with attribute class tokens and alternates among all possible classes.

Our proposed method is validated on the widely adopted Stable Diffusion (SD) framework []. Utilizing small-scale balanced datasets generated by the SD model itself to train our adaptive inclusive tokens, the fairness of the final model outputs can be significantly improved. Notably, the learned adaptive inclusive tokens demonstrate the capacity to generalize to unseen concepts and prompts, and be concatenated to mitigate multiple biases along various attributes.

The key contributions of our work are summarised as follows: We introduce a simple yet effective prompt-tuning approach to achieve inclusive text-to-image generation without attribute specification or prior knowledge of biased concepts. In particular, we propose the adaptive mapping network together with the anchor loss to address the issue of generalizability across different attribute class domination. Extensive experiments demonstrate the effectiveness of our method both quantitatively and qualitatively.

## 2 Related Work

#### 2.0.1 Bias in Text-to-Image Generation.

Comprehensive analyses have been conducted to study the bias and unfairness observed in T2I generation [, , , , , ]. Wang *et al*. [] first introduce the Implicit Association Test (IAT) [] in social psychology to measure biases in the task of T2I. IAT is designed to reveal implicit biases that an individual may hold unconsciously towards certain concepts. By experimenting with the valence and stereotype IATs on T2I output images, it is found that valence and stereotypical biases exist in state-of-the-art T2I models at various scales, *e.g*., the pleasant attitude is significantly biased towards straight sexual orientation than homosexual ones. Bianchi *et al*. [] make a comprehensive study on the wide range of stereotypes related to gender, race, nationality, and other identities associated with traits, occupation, and even object descriptions. It shows that attempts to either specify counter-stereotype prompts by users or add system “guardrails” cannot prevent stereotypes from spreading in the T2I results. Ghosh *et al*. [] spot the over-representation of Caucasian males in general terms like “person” and the over-sexualization of women of color without specification. TIBET [] propose to identify and measure biases in any T2I models using counterfactual reasoning, which breaks the limitation of pre-defined bias axes in previous studies. Wang *et al*. [] propose BiasPainter which provides an automatic and systematic study of gender, race, and age biases. The BiasPainter augments a seed image with a clear identity with queries of professions, activities, types of objects, and personality traits and compares the attributes of the identity between augmented and seed images to identify biases in queries. Jha *et al*. [] focus on exploring geo-cultural stereotypes in T2I models with a large scale of 135 nationality-based identity groups. It is revealed that the severity of bias varies for different identity groups and the “default” (without specification on bias attributes) representations of identity groups contain stereotypical appearances.

#### 2.0.2 Bias Mitigation in Text-to-Image Generation.

Various approaches have been developed to alleviate stereotypical biases in T2I generation, including model fine-tuning [, , ], prompt tuning [, , , ], concept editing [, ], and inference guidance []. A straightforward strategy is to fine-tune the entire T2I model on a large-scale dataset that is carefully synthesized to cover various classes of bias attributes []. Besides, techniques of fine-tuning the sampling process of the diffusion model have been proposed so that distributional constraints on the generative outputs, which is the direct interpretation of fairness, can be applied [, ]. However, such fine-tuning methods require a heavy load of computations. Prompt tuning-based methods aim to modify or add textual tokens to affect the T2I generation outputs. Gal *et al*. [] learn a fairer word for a biased concept from a small dataset. Bansal *et al*. [] explore the impact of ethical interventions added to the original prompts on the fairness of generative results. Li *et al*. [] introduce a fair mapping network that projects the text embeddings of a neutral prompt to the middle of prompts with all possible classes. The above prompt tuning methods do not modify the prompt to provide explicit guidance during generation. On the contrary, Zhang *et al*. [] propose to overfit one class token for each attribute class by image prompts and apply ad-hoc post-processing to loop over all combinations of target attributes to achieve inclusive generation. Orgad *et al*. [] and Gandikota *et al*. [] apply model editing to enforce the generation of non-stereotypical classes by optimizing the cross-attention weights of the diffusion models. However, determining the appropriate editing strengths is challenging considering variations in bias strengths across different concepts and classes. Friedrich *et al*. [] employ a training-free approach to incorporate fair guidance at inference, steering a biased concept in pre-determined directions to increase fairness. Though this approach requires no training or fine-tuning, prior knowledge of the biased concept is essential, and each generation is guided by a specific semantic direction.
Our approach adopts the prompt tuning approach without necessitating heavy model fine-tuning or prior knowledge about the original bias distribution, making it computationally efficient and highly generalizable.

#### 2.0.3 Prompt Tuning.

Prompt tuning is a technique that adapts a large language model (LLM) to new concepts by optimizing some prompt parameters with the model weights fixed. Prompt tuning has been applied to various downstream tasks such as image classification [, ], customized generation [, ], and bias mitigation [, ]. Zhou *et al*. propose to optimize trainable context tokens in class prompts to boost the performance of zero-shot classification using CLIP [] classifier [] and further tailor the tokens to be input-conditioned for better generalization to unseen classes []. Textual Inversion [] is proposed to invert a visual concept from a small set of images to a new pseudo word and customize generation on the visual concept. Similarly, Ruiz *et al*. [] and Kumari *et al*. [] update a trainable token together with some parts of the model to represent a particular visual content. The ability of prompt tuning to mitigate biases has been demonstrated by Gal *et al*. [] as mentioned in the previous subsection. Kim *et al*. [] adopt a similar approach as prompt tuning to learn new words in prompt for a fair generation. However, they optimize the tokens on the sampling stage of the diffusion model, leading to an extensive amount of computation and memory required.
Our method tackles the limitation of fixed inclusive tokens on the transferability to different domination classes, as observed in both previous methods, by learning adaptive inclusive tokens.

## 3 Methodology

### 3.1 Preliminary

#### 3.1.1 Diffusion Model.

Stable Diffusion [] is a commonly used latent diffusion model for image generation. With classifier-free guidance of textual conditions, SD demonstrates excellent performance in the T2I generation task. During each training step, a training image $x_{0}$ is first encoded into latent space as $z_{0}$ by a pre-trained image encoder $\mathcal{E}(\cdot)$. Then, a noise latent and a de-noising timestep $t$ are sampled to compute the ground truth noise $\epsilon$, which will be added to the encoded image to obtain the noisy latent $z_{t}$ in the current timestep. On the other side, a textual prompt $T$ that describes the image is tokenized to token embeddings $v_{T}$ and encoded to textual embeddings $e_{T}$ by the pre-trained CLIP text encoder []. Subsequently, a de-noising UNet $\epsilon_{\theta}$ [] takes in the noisy image latent, the de-noising timestep, and the textual condition to predict the noise added to the image latent. The learning objective of the de-noising stage is formulated as follows:

$$ $\mathcal{L}_{\textit{denoise}}=\mathbb{E}_{z_{0},\epsilon\sim\mathcal{N}(0,1), t,e_{\textit{T}}}\left[\left\|\epsilon-\epsilon_{\theta}\left(z_{t},t,e_{T} \right)\right\|_{2}^{2}\right]\ .\vspace{4pt}$ (1) $$

During inference, the de-noising UNet predicts the noise to be removed at each timestep from a Gaussian noise latent, conditioned on textual input.

### 3.2 Is Textual Inversion Effective for De-biasing?

Our method is inspired by the textual inversion framework introduced by Gal *et al*. []. Their paper briefly discussed the application of the textual inversion technique on bias reduction and provided a simple demonstration. Specifically, they curate a small and diverse dataset for a particular concept and learn a fairer word to substitute the original concept. The effectiveness of textual inversion in bias mitigation demonstrates that the fairness of T2I results can be significantly affected by the text condition, and it is feasible to invert a more balanced distribution to pseudo words from a small set of images. To enable the generalizability to unseen concepts, we revise their approach by disentangling the concept information from the learned tokens, making them solely represent the inclusiveness of a biased attribute. To elaborate, while the original textual inversion paper proposes learning a pseudo word <inclusive-doctor> to replace the original word “doctor”, the revised variant learns a pseudo word <gender-inclusive> applicable to any human-related concepts for a fairer generation in gender. The experimental results of this naïve revised version of the textual inversion approach (labeled as TI), are reported in Tab.1.

Figure: (a) Quantitative comparison of the distribution discrepancy $\mathcal{D}_{KL}$ across occupations stereotypically dominated by different genders. Each domination group consists of 5 unseen occupations (detailed lists are provided in the supplementary materials). MD: male-dominated occupations. FD: female-dominated occupations.
Refer to caption: extracted/5677187/figures/chief.jpg

Although the approach effectively reduces bias in gender and race attributes numerically, it exhibits two notable limitations in its generated outputs. Firstly, a single learnable inclusive token is inadequate in mitigating biases in concepts originally biased toward different classes. Secondly, the semantic meanings of certain concepts have been altered by the learned inclusive token.
Upon testing with unseen occupations that are biased to different genders (*e.g*., “software developer” stereotypically dominated by male figures, and “flight attendant” stereotypically dominated by female figures), results in Fig.1(a)
show that the revised textual inversion method manages to significantly improve fairness in stereotypically female-dominated occupations but exhibits less effectiveness in male-dominated ones with a single inclusive token. This observation aligns with findings by Kim *et al*. [], who conclude that the transferability of their de-stereotyping prompt is limited to unseen classes sharing the same dominant attribute class. Such sub-optimal results suggest that the token learns to simply generate more male figures instead of understanding the fair distribution.
In addition, as illustrated in Fig.1(a), the term “chief” appears to deviate from its original semantic in the SD model under the influence of the inclusive token learned via the revised textual inversion method. Though the drifted representations of “chief” figures are not erroneous, given the term’s broad definition, our objective is to ensure the learned inclusive tokens exclusively affect the biased attributes of the generated human figures without altering the semantics of the biased concepts.

Figure: Figure 2: Framework of our proposed adaptive inclusive token for text-to-image generation. The blue color indicates frozen weights, and the green color indicates trainable weights. Left: single training stage. Right: details of text model with adaptive mapping network. The adaptive inclusive token is concept-specific. $TokenIDs$ are for illustration only.
Refer to caption: x1.png

### 3.3 Adaptive Inclusive Token

To address the aforementioned limitations, instead of employing a fixed inclusive token for all concepts, we introduce a lightweight adaptive mapping network to predict the inclusive token tailored for each specific concept. To prevent the adaptive inclusive token from capturing irrelevant information from the training set that potentially causes semantic drift on bias concepts and degrades fairness in the generated results, an anchor loss is proposed to constrain the effect of the token. The framework of our proposed method is illustrated in Fig.2. Details of our proposed modules are described below.

#### 3.3.1 Adaptive Mapping Network.

The adaptive mapping network $\mathcal{F}_{am}$ is introduced after the token embedding lookup and before the text transformer in our framework. The inclusive prompt $T_{i}$ contains an inclusive token denoted by a placeholder <$i$> $\in\{$<$i_{g}$>,<$i_{r}$>,<$i_{a}$>$\}$, representing <gender-inclusive>,<race-inclusive>,<age-inclusive>, respectively. The placeholder is processed by the text tokenizer and the token embedding lookup table along with other words, resulting in an initial inclusive token embedding $v_{i}$. Then, the inclusive token embedding $v_{i}$ and the biased concept embeddings $v_{c}$ are jointly fed into the adaptive mapping network to predict a concept-adaptive inclusive token $v_{i}^{c}$. The fundamental hypothesis is that the token embeddings of a concept inherently contain necessary information about its bias distribution. Therefore, customizing the inclusive tokens for each concept facilitates the selection of a more suitable token to effectively shift the final distribution towards fairness. Finally, the adaptive inclusive token $v_{i}^{c}$ substitutes $v_{i}$ in the original token embeddings $v_{T_{i}}$ to form $v_{T_{i}}^{c}$, which is then forwarded to the text transformer model. The overall process of the text model with adaptive mapping is outlined in Algorithm 1.

Figure: Algorithm 1 Text Model with Adaptive Mapping Network

#### 3.3.2 Anchor Loss.

Given the observed issue of concept drifting in the revised textual inversion results, we hypothesize that the issue may stem from ambiguous instructions on what information to be captured by the inclusive token during training. To guide the learnable token on the desired property for it to learn and promote fairness, we add constraints to the UNet noise prediction with inclusive text condition, as it represents the influence of text condition on the final generative outputs. Previously, Li *et al*. [] interpret fairness constraints as requiring the text embeddings of a de-biased prompt to be of equal distances from those of all class-specific prompts for one bias attribute. However, we argue that the notion of fairness and inclusiveness should not entail an average distance across all possible classes, but involve an equal-possibility shifting within those classes. Therefore, we proposed an anchor loss which is formulated as follows:

$$ $\mathcal{L}_{\textit{anchor}}=\mathbb{E}_{z_{0},\epsilon\sim\mathcal{N}(0,1),t ,e_{T_{i}}^{c},e_{T_{a}}}\left[\left\|\epsilon_{\theta}\left(z_{t},t,e_{T_{i}} ^{c}\right)-\epsilon_{\theta}\left(z_{t},t,e_{T_{a}}\right)\right\|_{2}^{2} \right]\,$ (2) $$

The common de-noising process notations follow Eqn.1. Here, $e_{T_{i}}^{c}$ indicates the text embeddings of inclusive prompt $T_{i}$, and $e_{T_{a}}$ represents the text embeddings of anchor prompt $T_{a}=[T(c);a]$ where the inclusive token is replaced by the ground truth attribute class $a$ of the training sample. For example, for a training sample of a female firefighter with gender as the target bias attribute, if $T_{i}$ is “a photo of a <$i_{g}$> firefighter” then $T_{a}$ will be “a photo of a female firefighter”. Rewarding the similarity of UNet noise prediction under $e_{T_{i}}^{c}$ and $e_{T_{a}}$ conditions has two potential benefits. Firstly, it provides indications that the learned inclusive token should specifically affect certain attributes of the output, *e.g*., gender, as the anchor words. Secondly, it facilitates inclusive generations by allowing the effect of the inclusive token to shift among all possible classes.

The overall training objective for our adaptive inclusive token is defined as:

$$ $\mathcal{L}=\mathcal{L}_{denoise}+\lambda\mathcal{L}_{anchor},$ (3) $$

where $\mathcal{L}_{denoise}$ is calculated based on the inclusive prompt $T_{i}$, and $\lambda$ is the weighting parameter.

## 4 Experiments

### 4.1 Experimental Setup

#### 4.1.1 Scope.

We validate the idea of the adaptive inclusive token in mitigating three commonly studied bias attributes - gender, race, and age: $\mathcal{A}\in\{gender,race,age\}$. These attributes reflect harmful sexism, racism, and ageism observed in real life.
Regarding gender bias, we consider binary classes of male and female, acknowledging its limitation in representing non-binary genders. However, we argue that identifying certain appearances as non-binary genders may reinforce stereotypes within these less-represented social groups, and therefore should be avoided until more carefully collected public data on non-binary genders is available. For racial bias,
we refer to the seven racial groups from the FairFace dataset []. To simplify the analysis, we group East Asian and Southeast Asian into a single category Asian, resulting in six distinct groups: White, Black, Asian, Middle Eastern, Indian, and Latino Hispanic.
As for age bias, binary groups of young and old people are examined. In this study, we focus on bias concepts related to human figures, particularly in occupations: $\mathcal{C}\in\{occupation\}$.

#### 4.1.2 Training Data.

We use 24 occupations that span diverse bias attributes as our training occupations. For each occupation and attribute class, 20 images are generated using SD1.5(^2^22runwayml/stable-diffusion-v1-5) [] with 50 de-noising steps. The RetinaFace detector [] is applied to filter generation without valid faces. In total, there are $(24\times 2\times 20)$ images for gender attribute, $(24\times 6\times 20)$ for race, and $(24\times 2\times 20)$ for age. Details on preparing training data can be found in the supplementary materials.

#### 4.1.3 Implementation Details.

Our main experiments are conducted on the SD1.5 network, yet the generalizability to other models is demonstrated. By default, the inclusive token $v_{i}$ is initialized to the embedding of the token “individual”, which we find to be a natural inclusive token (detailed discussion on natural inclusive tokens can be found in the supplementary materials).
The length of our adaptive inclusive token is set to 1, resulting $v_{i}^{c}\in\mathbb{R}^{1\times 768}$, as $768$ is the token embedding dimension of the SD1.5 text model. The models are trained for 3000 steps with batch size $=1$ (gradient accumulation steps $=4$), taking approximately 1 hour on one NVIDIA A100 GPU. The base learning rate is set to $5.0e-04$ and we employ the $AdamW$ optimizer. As for training prompts, we follow Textual Inversion [] to use the ImageNet templates describing objects, but add adjectives to describe the attribute of interest. The template training prompts can be found in the supplementary materials. For the adaptive mapping network, we use transformer architecture with six attention heads and four transformer blocks.

#### 4.1.4 Evaluation Protocols.

To evaluate the performance of mitigating stereotypical biases in T2I generations, we construct a test set of unseen 24 occupations exclusive from the ones in the training set.
For each biased concept, we analyze 100 images with valid faces to measure the distributions of different attributes.
All images for evaluation are generated using the test prompt “a photo of a <$i$> {occupation}” with 25 de-noising steps. The racial inclusive token <$i_{r}$> is introduced after 10 sampling steps on base prompt $T(c)$ to better balance six classes.
Attribute classifiers are employed to detect and gather statistics on the distributions among various attribute classes in the generated images.
Following previous works [, , ], we utilize the CLIP [] zero-shot classifier to classify sensitive attributes, prompting with “a photo of a [attribute] person”.

Figure: (a) Visual evaluation on stereotypically male-dominated occupations.
Refer to caption: extracted/5677187/figures/qualitative_male.jpg

Figure: (a) Visual evaluation on race bias mitigation.
Refer to caption: extracted/5677187/figures/qualitative_race.jpg

#### 4.1.5 Evaluation Metrics.

We use three metrics to evaluate the effectiveness of our method. First, the fairness of de-biased attribute distribution is assessed by the KL divergence ($\mathcal{D_{KL}}$)$\downarrow$ against an even distribution. Second, Fréchet Inception Distance (FID)$\downarrow$ is used to quantify the image quality. Lastly, to ensure our adaptive inclusive token does not cause concept drifting, we report the alignment between generated images and prompts using the CLIP-Score (CLIP)$\uparrow$.

#### 4.1.6 Baselines.

Baseline methods implementations are briefly explained as follows: (1) Stable Diffusion (SD1.5) []: we use it as the original pipeline with base prompt $T(c)=$ “A photo of a {occupation}”. (2) ITI-GEN []: we train ITI-GEN on different attributes separately using the additional datasets provided. During generation, it loops over all classes of certain attribute. To ensure optimal results as even distribution, we generate an equal number of images for each class, allowing a slightly larger amount of images to be evaluated when the number of classes is not divisible by 100. Since race attribute is not studied in their work, we do race classification on the inclusive skin tone generation and recognize its limitations. (3) TIME []: the model is trained on each test occupation individually since it cannot generalize to unseen concepts. The editing strength parameter $\lambda$ is tuned in choices of $[10,1,0.1]$ to select the best one for each occupation. (4) Fair Diffusion []: we follow the default editing parameters and edit the model to the non-stereotypical class for gender and age attributes with binary classes. For the race attribute consisting of six classes, we loop over all six classes as the editing directions to give a comprehensive result. (5) Ethical Intervention []: we add best-performing intervention “…if all individual can be a {occupation} irrespective of their {attribute}” as reported in their paper to the base prompt. (6) Fair Mapping []: The model is trained on three attributes separately following the implementation details provided until the training loss converges. (7) Revised Textual Inversion [] method is implemented as discussed in Sec.3.1.

**Table 1: Comparisons with baseline methods on fairness, quality, and text alignment of generative results across three bias attributes. Abbreviations are used for simplicity: SD1.5: Stable Diffusion v1.5. TIME: time diffusion. FD: fair diffusion. EI: ethical intervention. FM: fair mapping. TI: revised textual inversion. ^∗ indicates editing-based methods that require careful tuning of editing strengths to achieve reasonable results. The best and second-best results are marked by bold and underline. Unlike FID and CLIP, the best results of $\mathcal{D_{KL}}$ in respective categories are indicated by bold to better distinguish these two paradigms.**
|  | Gender | Race | Age |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ | $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ | $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ |
| SD1.5[] | 0.3584 | 281.12 | 0.2823 | 0.5973 | 281.12 | 0.2823 | 0.2319 | 281.12 | 0.2823 |
| With attribute specification or prior knowledge on the bias distribution |  |  |  |  |  |  |  |  |  |
| ITI-GEN[] | 0.0078 | 278.21 | 0.2753 | 0.3699 | 247.05 | 0.2679 | 0.1560 | 243.09 | 0.2648 |
| TIME^∗[] | 0.2908 | 277.79 | 0.2733 | 0.5463 | 270.03 | 0.2663 | 0.2285 | 271.09 | 0.2738 |
| FD^∗[] | 0.2420 | 278.10 | 0.2718 | 0.4987 | 277.64 | 0.2738 | 0.2246 | 280.33 | 0.2740 |
| Without attribute specification |  |  |  |  |  |  |  |  |  |
| EI[] | 0.1666 | 283.52 | 0.2758 | 0.6033 | 281.11 | 0.2745 | 0.2258 | 289.82 | 0.2773 |
| FM[] | 0.1174 | 222.82 | 0.2341 | 0.3722 | 220.37 | 0.2391 | 0.3823 | 255.72 | 0.2402 |
| TI[] | 0.2590 | 283.38 | 0.2777 | 0.8065 | 275.32 | 0.2799 | 0.3113 | 286.22 | 0.2823 |
| Ours | 0.1298 | 272.35 | 0.2789 | 0.3625 | 277.15 | 0.2808 | 0.2168 | 268.53 | 0.2798 |

### 4.2 Experimental Results

#### 4.2.1 Single Bias.

The quantitative comparisons of our proposed method against baselines in mitigating single bias are shown in Tab.1. The methods are categorized into two groups: those with attribute specification or editing direction guidance and those without. As shown in the table, the method with attribute specification (ITI-GEN) generally yields the best inclusiveness as expected. However, as mentioned in Sec.1, we believe achieving fairer results without specifying attribute classes or editing direction poses a more meaningful and challenging task. Without any additional information, our method achieves superior performances in mitigating race and age biases and remains comparable to the Fair Mapping (FM) approach [] that performs the best in gender bias. While FM achieves less distribution discrepancy, its CLIP-Score experiences a notable decline, suggesting insufficient visual semantics regarding the target occupation in the generated images. Qualitative comparisons for the approaches without additional information are provided in Fig.3 and Fig.4. The misalignment of FM generative outputs with input prompts can be observed, *e.g*., the “flight attendant” case in Fig.4.
Additionally, an interesting observation is that,
generated images of the Ethical Interventions (EI) method tend to contain multiple individuals in line with the inclusive requirement in the ethical interventions.

**Table 2: Ablation studies on proposed components for single-bias mitigation.**
| $L_{anchor}$ | $\mathcal{F}_{am}$ | Gender $\mathcal{D_{KL}\downarrow}$ | Race $\mathcal{D_{KL}\downarrow}$ | Age $\mathcal{D_{KL}\downarrow}$ |
| --- | --- | --- | --- | --- |
|  |  | 0.2590 | 0.8065 | 0.3113 |
| ✓ |  | 0.1822 | 0.3540 | 0.3999 |
|  | ✓ | 0.3981 | 0.8090 | 0.2581 |
| ✓ | ✓ | 0.1298 | 0.3625 | 0.2168 |

**Table 3: Variant experimental settings for single-bias mitigation.**
| No. of Inclusive Tokens | Gender $\mathcal{D_{KL}\downarrow}$ | Race $\mathcal{D_{KL}\downarrow}$ | Age $\mathcal{D_{KL}\downarrow}$ |
| --- | --- | --- | --- |
| 1 (Ours) | 0.1298 | 0.3625 | 0.2168 |
| 2 | 0.1560 | 0.3893 | 0.2337 |
| 3 | 0.1575 | 0.3646 | 0.2131 |
| Diffusion Model | Gender $\mathcal{D_{KL}\downarrow}$ | Race $\mathcal{D_{KL}\downarrow}$ | Age $\mathcal{D_{KL}\downarrow}$ |
| SD2.1 | 0.4166 | 0.5819 | 0.2062 |
| SD2.1 with ours | 0.3074 | 0.5722 | 0.1934 |

**Table 4: Performance of combining adaptive inclusive tokens to mitigate multi-biases.**
| <$i_{g}$> | <$i_{r}$> | <$i_{a}$> | Gender $\mathcal{D_{KL}\downarrow}$ | Race $\mathcal{D_{KL}\downarrow}$ | Age $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  | 0.3584 | 0.5973 | 0.2319 | 281.12 | 0.2823 |
| ✓ | ✓ |  | 0.1518 | 0.3141 | - | 270.19 | 0.2780 |
|  | ✓ | ✓ | - | 0.5691 | 0.2079 | 267.61 | 0.2782 |
| ✓ |  | ✓ | 0.1568 | - | 0.2848 | 263.37 | 0.2793 |
| ✓ | ✓ | ✓ | 0.1417 | 0.5932 | 0.2272 | 262.95 | 0.2769 |

#### 4.2.2 Ablation Studies.

Ablation studies are conducted to evaluate the effectiveness of our proposed components. The quantitative results are shown in Tab.2. As demonstrated, combining two proposed components yields the best results in general. However, the second row shows that the adaptive mapping network is not as effective in mitigating race bias as in the other two. This could be caused by the fact that most occupations follow the same racial bias distribution to favor White individuals, reducing the need for the adaptive mapping network to adjust to different bias classes in race bias mitigation. Additionally, alternative experimental settings are tested. The top three rows in Tab.3 demonstrate the effectiveness of a single-length adaptive token in achieving a more inclusive distribution in the output. The last two rows showcase the generalizability of our method to different T2I models beyond SD1.5. Details and more ablation studies can be found in the supplementary materials.

**Table 5: Performance of adaptive inclusive tokens in complex scenes. Reported metrics are on gender bias. Additional prompts are added to “A photo of a <$i_{g}$> {occupation}".**
| Metrics | $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ | $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ | $\mathcal{D_{KL}\downarrow}$ | FID$\downarrow$ | CLIP$\uparrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Prompts | + “drinking coffee.” | + “reading a book.” | + “listening to music.” |  |  |  |  |  |  |
| SD1.5 | 0.3567 | 266.75 | 0.3116 | 0.3992 | 326.80 | 0.3045 | 0.4246 | 274.67 | 0.3033 |
| Ours | 0.2404 | 264.88 | 0.3082 | 0.2669 | 312.87 | 0.3020 | 0.2195 | 275.71 | 0.3009 |

#### 4.2.3 Multiple Biases.

To validate the ability of our adaptive inclusive tokens to mitigate multiple stereotypical biases simultaneously, we concatenate previously single-attribute-trained inclusive tokens during inference with prompt: “A photo of a <$i_{g}$> <$i_{r}$> <$i_{a}$> {occupation}”. As the results in Tab.4 demonstrate, the adaptive inclusive tokens
can be combined to achieve inclusive generation across various attributes in most cases.
Furthermore, we observe that combining these individually trained inclusive tokens during inference may even enhance the de-biasing performance,
posing an intriguing insight into the intersections of bias attributes. While addressing these complex correlations lies beyond the scope of this work, we recognize it as a potential future work.

#### 4.2.4 Complex Scene.

We use three additional prompts describing people engaged in various activities to assess the generalizability of our tokens to unseen and more complex prompts. The evaluation is done on the <$i_{g}$> token only for a preliminary demonstration. The results are reported in Tab.5. It appears that our adaptive inclusive token can be effectively generalized to achieve inclusive generation in more complex scenarios with comparable text-image alignment and even better image quality.

## 5 Conclusion

In this work, we study the challenges faced by previous prompt tuning-based bias mitigation approaches. To address the limitation of generalization across different dominant attribute classes, we propose to learn a concept-specific inclusive token through an adaptive mapping network, instead of a fixed one. For the concept drifting issue, we introduce an anchor loss that constrains the impact of the adaptive inclusive token on the final outputs. As a result, our proposed method manages to significantly enhance inclusiveness in T2I generation, with the ability to generalize to unseen concepts and scenarios and mitigate multiple biases simultaneously. In addition, our approach is not model-specific and can be applied to various T2I models to boost fairness in their outputs. We hope our work can bring insights to all who care and work in this field. We believe it should be our effort to make the world a more inclusive place, where every individual regardless of background or identity, can feel valued and respected.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: See above weaknesses.
soundness: 3
strengths: 1. The paper focuses on addressing the bias problem in current T2I models, particularly gender, race, and age biases, which has important social benefits.

2. The proposed method is simple yet model-agnostic, as stated by the authors, and can be easily adopted in existing approaches to help reduce bias issues.

3. Extensive experiments were conducted to evaluate the effectiveness of the proposed method in mitigating biases.
summary: The paper focuses on addressing the bias problem in current T2I models. To tackle this, the authors introduce a prompt-tuning approach based on an adaptive mapping network and anchor loss. Experimental results demonstrate its effectiveness in mitigating gender, race, and age biases.
weaknesses: 1. The biases present in current T2I models are not primarily due to the models themselves but stem from other non-technical issues, such as dataset limitations or unclear prompts. Additionally, for evaluation, the authors use a CLIP zero-shot classifier to classify sensitive attributes. However, since CLIP is likely trained on biased datasets, the classifier may more accurately identify attributes that are highly frequent in its training data, which could, in turn, affect the accuracy of evaluation results.

2. From the quantitative comparison shown in Table 1 and the qualitative results, the proposed method does not show significant improvement over other methods, particularly FM, in addressing different biases. As discussed in the paper, the proposed method is model-agnostic—can it enhance the performance of other methods?

3. What are the experimental settings for Figure 1(a)? More details are needed to illustrate the bias problems present in current T2I models.

4. I am confused about the ablation studies shown in Table 2. What is the setting for without adaptive mapping network? Without AM, how is the anchor loss in Eq. (2) calculated?

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: Provide a deeper exploration of the theoretical foundation and motivation behind the adaptive mapping network and anchor loss. Discuss how these components compare to existing bias mitigation techniques.

Include a more detailed description of the structure and training process of the adaptive mapping network in the paper. This will help other researchers better understand and replicate the method.

Enhance the clarity and readability of the figures and diagrams in the paper to effectively convey the authors' intentions.

Increase the number of experiments in complex scenarios. Discuss the impact of fairness on accuracy and how to mitigate such issues.

Provide some practical application cases to demonstrate the effectiveness and potential challenges of the method in real-world scenarios. This will help readers better understand the practical application value of the method.
soundness: 3
strengths: The paper introduces a novel approach by using adaptive inclusive tokens to mitigate bias in text-to-image generation models without the need for explicit attribute specification or prior knowledge of bias distribution, sounds interesting.

Experimental results demonstrate that the proposed method outperforms previous bias mitigation techniques in scenarios without attribute specification. 

The method seems generalizes well and this paper is easy to read.
summary: This pape addresses the critical issue of stereotype bias in text-to-image generation models. The authors propose an innovative method that adjusts the attribute distribution of the final generated output by learning adaptive inclusive tokens. This approach does not require explicit specification of attributes or prior knowledge of bias distribution. Instead, it customizes concept-specific inclusive tokens through a lightweight adaptive mapping network. The method introduces an anchoring loss to constrain the influence of adaptive inclusive tokens on the final output, thereby enhancing the fairness and inclusivity of the generated results without compromising the consistency between the text and the generated images.
weaknesses: Potential Introduction of Factual Errors: The proposed method may introduce factual inaccuracies when dealing with non-neutral concepts. For example, generating an image of a female U.S. president, which does not align with historical facts. The authors do not discuss or evaluate the conflict between accuracy and fairness in their approach.

Lack of Motivation and Theoretical Foundation: The adaptive mapping network proposed by the authors lacks a clear motivation and theoretical discussion. This makes it difficult to understand the underlying principles and rationale behind the approach.

Implementation Details: While the paper outlines the overall framework of the method, the description of the implementation is vague, particularly regarding the specific structure and training process of the adaptive mapping network. The diagrams provided are unclear and fail to effectively convey the authors' intentions.

Quality of Generated Models: There is a lack of effective discussion and comparison regarding the quality of the generated models. The impact of the prompts, models, and data volume on the final model is not adequately addressed.

Limited Experimental Setup: The experimental setup is relatively limited, especially in terms of evaluating performance in complex scenarios and across different datasets. This restricts the understanding of the method's robustness and applicability.

Practical Application Value: The paper lacks a discussion on the practical application value of the proposed method. It is unclear how the generated images can be ensured to be both fair and factually accurate.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. The main metric $D_{kl}$ for attribute distribution demonstrates that the original SD1.5 outperforms SD2.1 and SDXL (SD1.5>SD2.1>SDXL). This is not consistent with the qualitative results. Does it mean the metric there is a problem with the metric itself?
 2. Why does a single token perform better than multiple tokens (with higher complexity)?
soundness: 3
strengths: 1. The paper introduces a novel method for reducing biases in text-to-image generation, which is critical to AI ethics and fairness. The concept of learning adaptive inclusive tokens that can shift attribute distributions in generative outputs is a creative solution.
2. The method's ability to generalize to unseen concepts and handle multiple attributes is a major advantage, showcasing its potential for broad application across various domains.
summary: The paper presents a novel approach to mitigate stereotypical biases in text-to-image (T2I) generation models. The authors introduce a method for learning adaptive inclusive tokens that can adjust the attribute distribution of the generated images to be more equitable, without the need for explicit attribute specification or prior knowledge of the bias distribution. The core of their approach is a lightweight adaptive mapping network that customizes inclusive tokens for concepts that are targeted for de-biasing, which enhances the generalizability of the tokens to new, unseen concepts. The method is evaluated on the Stable Diffusion framework and demonstrates improved performance over existing bias mitigation techniques, showing its effectiveness in reducing stereotypical biases in T2I generation.
weaknesses: 1. While the authors claim to have developed a lightweight adaptive mapping network to address bias mitigation, the training time of nearly one hour suggests that it may not be as lightweight as initially proposed. This process raises concerns about the practicality and efficiency of the solution, especially in contexts where rapid deployment or real-time adjustments are required.
2. While the authors demonstrate quantitative improvements over the baseline in comparative analysis, the qualitative examples presented in the supplementary material show that the method is not entirely effective, with many prompts still generating images with a predominantly single gender. This inconsistency between quantitative and qualitative results casts doubt on the overall effectiveness of the method and its ability to achieve the desired inclusiveness in text-to-image generation.
3. Overall, I think the authors' approach of training to shift certain biased concepts towards less ambiguous representations using a better mapping network, akin to a new token, is quite straightforward. However, as mentioned earlier, I am concerned about whether such a simple method can be truly effective.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: Please see Weaknesses.
soundness: 2
strengths: 1. The paper is well-written and reads clearly.
2. Learning an adaptive mapping network to generate inclusive tokens specific to different occupation is reasonable and justified.
3. Experimental results effectively demonstrate the method’s effectiveness.
summary: This paper proposes a bias mitigation approach for T2I generation using adaptive inclusive tokens, enhancing fairness without needing attribute specifications. By employing an adaptive mapping network and anchor loss, the approach effectively mitigates biases while maintaining quality and text alignment. The method is versatile but has limitations when handling non-neutral concepts and potential semantic drift.
weaknesses: 1. Calculating anchor loss requires predefined information on non-dominant bias attributes for each occupation in the training data, which necessitates human pre-determination.
2. The paper primarily focuses on biases introduced by the text encoder and does not examine potential biases inherent to the diffusion model itself. However, text embedding may be attacked[1]. The proposed method may become ineffective after being attacked.
3. The reported FID metric is in the range of 200–300, whereas this metric usually falls within the magnitude of 10^1. This discrepancy may result from insufficient statistical sampling, casting doubt on whether FID accurately reflects image quality here. ImageReward [2] or PickScore [3] might be fairer metrics in this context.

[1] Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models

[2] ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

[3] Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation

Meta-review: This paper introduces AITTI, a method using adaptive inclusive tokens to mitigate biases in text-to-image generation. The approach involves tuning an adaptive mapping network with inclusive samples using an anchor loss to mitigate biases without requiring explicit attribute specification or prior knowledge of the bias distribution.

While the targeted problem is a critical issue in AI ethics and fairness, multiple concerns raised by the reviewers question the motivation behind the proposed approach, its robustness, and its limited advancement beyond existing methods. Considering these factors, I think this paper is not yet ready for publication at ICLR, and thus I recommend rejecting this paper.

Decision: Reject