Paper: Fair Text-to-Image Diffusion via Fair Mapping (ICLR-2024)

Reviewer comment (Theory):
This work assumes that the bias can be corrected with an appropriate text embedding, which in turn assumes that the diffusion model is able to generate the specified human characteristics with diverse demographic properties. However, it is possible that the bias in the association is so strong (due to insufficient or biased training data) that the model could not generate the correct image, despite being asked for a specific property. For example, if there are no female plumbers in the training set, the model may not be able to generate the specified image even if "female plumbers" are explicitly asked for in the prompt. Since the method only trains a linear projection head, it is unlikely to be able to generate unbiased images.

Author response:
We admit that if there is a lack of some specific data, then our method may become less efficient. **However, we do not think this is a reason that our paper should be rejected as every post-processing fairness method (our method belongs to this category) will become less efficient even for classical classification tasks if there is a lack of training data.** Thus, all current fairness methods need to assume that the training data is imbalanced or long-tailed and cannot hold the case like "if there are no female plumbers in the training set".

Reviewer comment (Theory):
The method relies on a set of predefined sensitive keywords to enable transformation. However, it is hard to make this set of keywords exhaustive, and since this is a matching-based method, the checks may be skipped if some typos are introduced.

Author response:
Yes, you are correct, in our problem we need to pre-define a set of keywords that is used for our training prompts. So our method is tailored for the case where the keyword set is finite. Current research endeavors predominantly revolve around designing limited diverse well-defined benchmarks for evaluation purposes rather than open-world environment.

Regarding typos, there are two cases: (1) if the typo is unintentional, our detector network can somehow correct these typos. In our detector algorithm, we utilize semantic similarity to assess the input. As a result, we are able to effectively handle the issue you mentioned, if semantic errors in the input, such as typos, would not significantly impact the output of the original diffusion model. (2) if the typo is intentional, or it is generated by adversarial attacks, then it is out of the scope of this paper as it is a common issue for all input-to-image diffusion models. It is notable that altering the text embedding is widely used for attackers to manipulate the system in text-to-image diffusion models, which is far beyond introducing mere unfairness in the output. Such attacks pose substantial threats to the integrity and reliability of text-to-image models.

Reviewer comment (Evaluation):
The fair mapping method proposed by the work may lead to loss of details in the prompt, when compared with its baselines, as shown in Sec 4.4 that some facial expressions are not generated according to the text prompts.

Author response:
Although the fair mapping method may prioritize reducing biases, it can inadvertently sacrifice some fine-grained details or specific prompt-related features, which is also known as the price of fairness. In our work, for image quality, we show that the alignment between generated images and human-related content is very close to that of the vanilla diffusion models. Future work can explore alternative methods or modifications to the fair mapping approach to mitigate the loss of details while still achieving fairness and unbiased image generation.

Reviewer comment (Evaluation):
In Table 1, the delta in improvement of Fair Mapping over the baselines is relatively small for race. It is difficult to understand how much a 0.01 improvement actually looks like in terms of qualitative performance.

Author response:
Compared to the baseline model, our method shows a minimum improvement of 10\% and a maximum improvement of 54\%. Due to the broad definition of race and the high efficiency of diffusion models in generating faces based on occupations, the model tends to exhibit results that are more fair in terms of race compared to gender. Therefore, achieving a 10\% improvement is actually a significant amount.

Reviewer comment (Evaluation):
The authors mention that the value of the loss weight hyperparameter can affect the visual quality. Including image quality metrics like FID would be helpful to quantify how much degradation is introduced because of the debiasing network.

Author response:
The reason why we do not apply FID is that we lack real images to compare and the Inception network is not suitable for our human-related tasks. 1)FID is is a measure that calculates the distance between the feature vectors of a real image and the generated image, widely used for reconstruction images after training all parameters in generative images, which requires real images in training data. However, in our method, we just finetune a few parameters with prompts rather than thousands of real images. 2) FID is a measure of the distance directly pressed by the multivariate normal distribution, but the extracted generative image features, like our human-related tasks, do not necessarily conform to the multivariate normal distribution.  
    
Therefore, we use a traditional metric, CLIP, to evaluate our alignment between text and generated images. To address the problem that CLIP is not suitable for human-related description work, we have designed the Human-CLIP method, which addresses the alignment issues that CLIP is not suitable for in face generation tasks. Although our method has a decrease in CLIP Score, we have demonstrated that our method performs better in face generation tasks and has overcome the limitations of existing evaluation metrics in Human-CLIP.

Reviewer comment (Presentation):
Some details that are necessary for clarity and context in the main paper have been moved to the appendix. For example, Human-CLIP is a newly introduced metric but the details are not discussed at all in the main paper. Additionally, no context for the human preference evaluations is provided (e.g., at the minimum, clarify whether higher or lower is better in Table 4).

Author response:
Due to the limited space, some details about metrics about image qualities are discussed in the Appendix. Human-CLIP is specifically designed to evaluate the quality of generated images based on human perception. It leverages the knowledge and capabilities of CLIP. The scale for the score of Human Preference is displayed to provide a reference for interpreting the scores assigned by human evaluators during preference ranking tasks. The specific details of the scale, including its range and corresponding interpretations, are provided in the Appendix section of the research document. We will also give an easy-to-understand statement of these problems in the late revised version paper. **Please see Page 18 about D.2 and Page 22 about E.4 in Appendix. We will move them to the main text in the revised version if additional pages are allowed.**

Reviewer comment (Experiments):
It is unclear why no evaluations are provided in the second row of Table 2 (L_{fair} only).

Author response:
The reason is stated in the section 4.3 about the ablation study and you can refer to it kindly. The overall efficacy of our optimization is compromised in the absence of $L_{text}$, as the total loss cannot adequately capture the semantic content embedded in textual representations. The diffusion models, devoid of $L_{text}$, tend to produce images characterized by the absence of crucial semantic details, resulting in blurriness and distortion. Consequently, assessing metrics for these experiments becomes superfluous under such conditions.

Reviewer comment (Reproducibility):
How is the mapping network initialized? In row 3 of Table 2, it is unclear why L_{text} alone would lead to more "fair" results since it is just trying to minimize the projected embedding to the original. Shouldn't the performance be the same as row 1 if the loss is optimized?

Author response:
In this paper, We employ the widely used initialization method of Random Initialization. We observe that when optimizing with $L_{text}$ alone, the generated images exhibit a stronger ability to generate face-related images. Therefore, we believe that by minimizing the difference between the projected embeddings and the original embeddings, there is a greater emphasis on maintaining the consistency of inherent attributes and ignoring sensitive attributes, leading to a reduction in biases among generative results. By minimizing the discrepancy between the projected and original embeddings, the $L_{text}$ loss function encourages the model to prioritize the preservation of non-sensitive features from the original images during image generation. This helps to mitigate over-reliance on sensitive attributes and subsequently reduces the occurrence of related biases.

Reviewer comment (Experiments):
Have the authors experimented with just optimizing d(v, v_j) rather than (d(v, v_j) - \bar{d(v, -)}) in Equation 2?

Author response:
We have indeed experimented with optimizing only $d(v, v_j)$ in Equation 2, without subtracting $\bar{d(v, -)}$. However, the observed outcomes of these experiments were significantly adverse, leading to distorted generated images and a notable loss of semantic coherence. This suggests that incorporating $\bar{d(v, -)}$ in Equation 2 plays a crucial role in achieving more favorable results, such as maintaining image quality and preserving semantic consistency.

Reviewer comment (Experiments):
Although the paper covers a good amount of relevant previous studies, the paper lacks baseline experiments. For example, despite [1] focus on fair-guidance while this work focus on pluggable mapping module, the authors can calculate FairScore and compare w.r.t. training time, overhead memory, etc.

Author response:
We would like to express our sincere gratitude to you for your valuable feedback and insightful comments on our work.

> Reply to W1

We acknowledge the importance of including baseline experiments and performance comparisons. It is important to note that during the time of our study, which was conducted several months ago, there was a scarcity of publicly available open-source code in the field of fair diffusion model generation. This limitation made it challenging for us to conduct direct performance comparisons with other approaches. However, we are pleased to find that this work has been made openly accessible, including the accompanying code recently. In the revised version of our paper, we provided a comprehensive comparison with the mentioned work and provided detailed explanations of the comparison methodology. **We elucidated its limitations in the degree of human intervention for systems and shows priority for our superiority in generation speed as demonstrated.**

| Models                  | Time (seconds) |
|-------------------------|-----------------|
| Stable diffusion        | 424             |
| Fair Diffusion          | 1463            |
| Fair Mapping (our method)| 434             |

**Meanwhile, we also evaluate the alignment and diversity of Fair Diffusion. The experimental results are copied from Table 22 in Appendix E.2.** 


|         Models        |            | Occupation |           |            |   Emotion  |           |
|:---------------------:|:----------:|:----------:|:---------:|:----------:|:----------:|:---------:|
|                       | CLIP-Score | Human-CLIP | Diversity | CLIP-Score | Human-CLIP | Diversity |
| Fair Diffusion-Gender |   0.2274   |   0.1348   |   13.87   |   0.1894   |   0.1298   |    1.43   |
|  Fair Diffusion-Race  |   0.2239   |   0.1292   |   13.79   |   0.1882   |   0.1266   |    1.39   |

Please see Page 21 - 22 for more experiments details.

Reviewer comment (Novelty):
While the unfairness is largely resolved through the proposed mapping module, such a result may not come at a surprise since FairScore and the employed fairness loss term are quite similar.

Author response:
> Reply to W2

**We kindly cannot agree that FairScore and the fair loss are similar.** Note that for any keyword $c_k$, its fairscore is defined as 
    
$$\text{FairScore}(c_k)= \sqrt{\frac{1}{\mid A \mid} \sum_{a_i\in A } \left( DBias_{a_i}(c_k) \right)^2},$$ where $DBias_{a_i}(c_k)=P\left(s=a_i \mid c=c_k\right)- \frac{1}{\mid A \mid} \sum_{a_j\in A} P\left( s=a_j \mid c=c_k\right).$
And the fair loss is defined by 

$$L_{fair} = \sqrt{\frac{1}{|A|} \sum_{a_j \in A} \left( d(v,v_j) - \overline{d(v,\cdot)} \right)^2},$$

where  $d(v,v_i)$ represents the Euclidean distance between the native embedding $v$ and the specific sensitive attribute embedding $v_i$. $\overline{d(v,\cdot)}$ refers to the average distance between the native embedding $v$ and all the sensitive attribute embeddings $v_j$. 

**We can see that FairScore is defined by the conditional probability while fair loss is defined by the Euclidean distance of the embedded vectors after being transformed by our linear network.** There are no evidence or theories to support the equivalence or similarity of these two totally different metrics (conditional probability of generated images v.s. Euclidean distance of the embedded vectors).

Reviewer comment (Reproducibility):
The authors note that a detector network is employed to identify predefined sensitive keywords in the input prompts. There is no additional detailed explanation about the detector network.

Author response:
> Reply to W3

**We have added detailed related content about the detector in the revised version of our paper. Please refer to the detailed exposition and algorithm demonstration on Page 16 in Section C of Appendix. In this section, we provide a detailed exposition of the operational principles of our detector.** We introduce an additional detector that aims to adapt the user's input prompt by finding the training prompt that is semantically consistent with it. The detector calculates the similarity distance between the input prompt and predefined prompt using a pre-trained encoder and selects the training prompt with the smallest distance. Meanwhile, the detector checks whether the transformed training prompt contains any sensitive attributes. If it doesn't, the prompt is processed through the fair mapping linear network for debiasing.

Reviewer comment (Experiments):
This method explicitly needs a labeled dataset to mitigate the demographic bias in diffusion models. However, in real-world scenarios, it may be challenging to identify and address all potential types of bias comprehensively. Further, there are remaining questions regarding whether it is feasible to (i) simultaneously eliminate multiple types of bias or (ii) sequentially address multiple biases without negatively impacting performance. If such challenges cannot be properly addressed, it would incur a significant amount of training time to erase all types of biases, and heavy memory cost to save all mapping modules corresponding to each bias type.

Author response:
>Reply to W4

We do not think this point is a weakness of our paper. First, we agree on that "in real-world scenarios, it may be challenging to identify and address all potential types of bias comprehensively." Addressing all potential types of bias now is impossible for the classical classification problem even though there are many studies. However, not addressing all bias issues does not mean these papers have weaknesses and should be rejected.

As there is only one[1] work of fairness of diffusion models, we believe it is valuable to study such a critical problem and in this paper, we provide the first post-processing method.  Due to a lack of studies on this problem, it is obviously we cannot address all biases in a single paper, that is too harsh for us. 

About your two questions: (i) "simultaneously eliminate multiple types of bias." As we mentioned, even for classification tasks, there is still no work that can achieve this. However, we also need to mention that, since our method is a post-processing approach, it can be directly integrated into any other (future) fair text-to-image diffusion models to further improve fairness. In the revised version, we add more discussions on the strengths of our method in Appendix C on Page 16. (ii) "sequentially address multiple biases without negatively impacting performance." This is also a harsh question, to the best of our knowledge, whether there is a trade-off between utility and fairness is even an open problem for classification tasks.

Reviewer comment (Reproducibility):
How many random seeds are used throughout the experiments?

Author response:
>Reply to Q1

In our experiments, there is only one seed for all text-to-image diffusion approaches. Following stable diffusion, We fixed the seeds as 42 in the whole process of our experiments. As a result, the seeds may contribute little influence to our results among different approaches.

Reviewer comment (Experiments):
The experiments are limited to biases related to gender and race, omitting other prevalent societal biases such as ageism, socioeconomic status and religious discrimination.

Author response:
We cannot agree with the reviewer's comments. As we mentioned, currently there is only one paper[1] on fairness diffusion models previously. There is a lack of unified benchmarks for evaluating the performance of approaches in this domain. In our research, we focus on investigating gender and race biases due to their well-documented presence and significant impact, as evidenced by numerous studies [1-2]. Additionally, these biases are readily observable in images, making them suitable as typical attributes for our research focus. It is worth noting that our method can be extended to address other sensitive attributes based on user preferences. The flexibility of our approach allows for its deployment and adaptation to various attribute domains, providing a versatile framework for bias mitigation in image analysis.

References

[1] Friedrich F, Schramowski P, Brack M, et al. Fair diffusion: Instructing text-to-image generation models on fairness[J]. arXiv preprint arXiv:2302.10893, 2023.

[2] Wang F E, Wang C Y, Sun M, et al. Mixfairface: Towards ultimate fairness via mixfair adapter in face recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(12): 14531-14538.