FairCoT: Enhancing Fairness in Diffusion Models via Chain of Thought Reasoning of Multimodal Language Models

Authors: Zahraa Al Sahili, Ioannis Patras, Matthew Purver
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: diffusion models;fairness; bias; chain of thought; text to image; multimodal LLMs

Abstract:
In the domain of text-to-image generative models, biases inherent in training datasets often propagate into generated content, posing significant ethical challenges, particularly in socially sensitive contexts. We introduce FairCoT, a novel framework that enhances fairness in diffusion models through Chain-of-Thought (CoT) reasoning within multimodal generative large language models (LLMs). FairCoT employs iterative CoT refinement and attire-based attribute prediction to systematically mitigate biases, ensuring diverse and equitable representation in generated images. By integrating iterative reasoning processes, FairCoT addresses the limitations of zero-shot CoT in sensitive scenarios, balancing creativity with ethical responsibility. Experimental evaluations across multiple models, including DALL-E and various Stable Diffusion variants, demonstrate that FairCoT significantly improves fairness and diversity metrics without compromising image quality or relevance. Our approach advances ethical AI practices in generative modeling, promoting socially responsible content generation and setting new standards for fairness in AI-generated imagery.

Decision: reject
Reviewer scores: 3, 5, 3, 5

Reviews and discussion:
<thread>
creation date:
1730619733732

thread_type:
official_review

<thread_object>
creator: reviewer_snle

<content>
summary:
This paper presents FairCoT, a framework designed to enhance fairness in text2image models by leveraging CoT. It includes iterative CoT refinement, attire-based attribute prediction, and bias assessment via entropy scores. Their experiments show that FairCoT achieves high fairness and diversity metrics without compromising image quality or relevance.

strengths:
1. It tackles significant questions in AI. 

2. They successfully adapt CoT to the field of Fairness and debiasing 

3. this is a simple, model-agnostic method that can be applied to both open-source and closed-source models without requiring retraining or parameter adjustments.

4. They achieve good debiasing without lowering the quality of the generated image

weaknesses:
1. While I generally enjoyed reading this paper, the excessive use of bullet points makes it appear less professional and somewhat like an unfinished outline.
2. The paper relies heavily on CLIP for attribute prediction, but since CLIP was not originally designed as an exhaustive demographic classifier, this reliance might lead to skewed results and miss more subtle and nuanced features.
3. I am also interested in understanding the typical length of the CoT path used in the experiments. Does it require a large number of inferences? Additionally, is CoT useful for other downstream applications in text-to-image models like image editing and classification?
4. In qualitative examples 5 and 6, I noticed that FairCoT often produces images with multiple people, while the baseline typically shows a single person. Does this mean that FairCoT sacrifices some control over the image? Additionally, in Figure 6, it appears that FairCoT introduces some green noise into the image.

questions:
I would appreciate it if the authors addressed the questions mentioned in the weaknesses section, and I will raise the score if their responses are convincing.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730663595376

thread_type:
official_review

<thread_object>
creator: reviewer_pnnp

<content>
summary:
The present paper proposes a novel method to tackle the lack of diversity in text-to-image models’ outputs displaying persons. To this end, the authors make use of pre-trained language models and CLIP and a discovery stage employing a Chain-of-thought process to create a set of prompt extensions. The approach is evaluated based on a prompt set covering gender, race, age, and religion and compared to a wide range of existing "debiasing" approaches.

strengths:
- The paper addresses a current and crucial problem in research and industry as exemplified by text-to-image model applications such as DALL-E and Gemini being restricted in their capabilities of generating humans by their creators to prevent the display of social biases and, in turn, a negative social impact.

- While simple, the introduced method is flexible and model-agnostic.

- The paper provides an extensive overview of related work.
 
- Extensive evaluation covering multiple diffusion models and other „debiasing“ approaches.

- The evaluation goes beyond the display of single persons and single concepts commonly used in previous work.

weaknesses:
- Instead of leveraging existing benchmark datasets such as [1], the evaluation is based on prompts selected by the authors.
Using existing datasets would enhance the comparability and reproducibility of results.

- More importantly, my main concern is the use of CLIP-T within both the methodology and evaluation metric, which could lead to a flawed evaluation. Therefore, I would urge the redesign of the evaluation process. 

- While the prompts used are listed within the Appendix, it would increase clarity if at least the number of prompts for each category is listed in the main text.

- Section 4.3 lacks details on the hand-labeled images. Can you provide details on who performed the labeling, the annotators’ qualifications, the evaluation criteria, and the process used? Further, will this dataset be shared as part of the contribution?

- Figure and table captions are very sparse. More descriptive captions would improve clarity.

References:

[1] Luccioni et al. (NeurIPS 2023): https://proceedings.neurips.cc/paper_files/paper/2023/file/b01153e7112b347d8ed54f317840d8af-Paper-Datasets_and_Benchmarks.pdf

questions:
- In Section 3.1, the term “training stage” may be misleading, as no training is conducted. This phase appears to be a bias discovery stage rather than training. Using “training” could lead readers to believe that model parameters are being tuned.

- The appendix indicates the use of GPT-3.5 as the language model. To improve reproducibility, please specify the exact version used. Additionally, have other open-weight language models been tested to reduce reliance on proprietary models?

**Minor comments**
- The abbreviation “MLLM” is introduced multiple times.
- There are missing spaces in several locations, such as in line 266 “(Figure1)” and line 410 “fairness.Further.”
</content>
</thread_object>

</thread>
<thread>
creation date:
1730685230688

thread_type:
official_review

<thread_object>
creator: reviewer_snaz

<content>
summary:
The paper present FairCoT: an iterative CoT refinement and attire-based attribute prediction technique mitigate biases present in text to image generative models. The paper develops a MLLM and CLIP guided method to develop a CoT pool, which is then used to guide a T2I model towards generating de-biased outputs. Experiments on DALL-E and various Stable Diffusion variants demonstrate that FairCoT significantly improves fairness and diversity metrics without compromising image quality or relevance.

strengths:
1. The paper tackles an important problem in T2I models - and aims to develop a method that makes them generate images which represent a broader sample of the society.
2. The paper studies multiple aspects such as age, religion, gender and race.

weaknesses:
1. Technically limited contribution. The work iteratively applies CoT to generate multiple images to steer them towards being fair, which in my opinion does not offer many novel insights or findings.
2. The paper used MLLMS and CLIP throughout their pipeline, as a means to alleviate bias in T2I models. However, the paper does not account for the bias present in these models, which has been extensively studied in [1,2].
3. Computationally inefficient. The pipeline involves multiple calls to a T2I model. The authors could refer to [3] for a survey of methods on this topic which perform for example, efficient, embedding-based methods.

[1] https://arxiv.org/abs/2108.02818
[2] https://arxiv.org/abs/2203.11933
[3] https://arxiv.org/pdf/2404.01030

questions:
1. In Section 3.1.2, why is "attire" chosen to be a good representation of a given religion? I am not sure about Figure 2? How does a man in a suit define the religion of that person? A person of a given religion might not always be wear a given attire. I don't think it is appropriate to generalize this.
2. I would like to understand how the pipeline works with the SD set of models, given they cannot be prompted the same way DALL-E can be since its integrated with GPT-4.
3. General suggestions - I think all the captions of the figures and tables should be improved.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730779383400

thread_type:
official_review

<thread_object>
creator: reviewer_wptm

<content>
summary:
This paper presents FairCoT, a new framework for improving fairness in text-to-image diffusion models by leveraging Chain-of-Thought (CoT) reasoning within multimodal large language models (MLLMs). The key idea is to use iterative CoT refinement and attire-based attribute prediction to systematically reduce biases in generated images. Experiments across multiple models show improved fairness and diversity without sacrificing image quality or relevance.

strengths:
- Reducing the biases and improving fairness in image generation models are very important topics for both research communities and the real-world applications of the models.
- The proposed approach shows significant improvement over gender, race, age, and religion attributes while maintaining image quality (measured via CLIP-T scores).

weaknesses:
- The technical contributions and the novelty of this work are not significant. Previous works [1] also explored using an iterative strategy to improve the fairness of Text-to-Image diffusion models.
- In addition, the authors did not compare the several more relevant baselines, such as IDA [1].
- It is not clear how the approach iterative updates the CoT and consequently improves the fairness of the images. It would be helpful if the authors could provide a more clear and concrete example of the refinement process.
- The presentation of the paper is not clear and should be further improved. For example, the diagram shown in Figure 3 is not very clear.

[1] Debiasing Text-to-Image Diffusion Models. He, et al. 2024.

questions:
- While the quantitative results of CLIP-T score only have small variations, in the qualitative comparison in Figure 6, the images generated by FairCoT look pretty weird and of poor quality. Could you have an explanation for this? Is this a common issue of FairCoT?
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper presents FairCoT, a framework designed to enhance fairness in text-to-image diffusion models through Chain-of-Thought reasoning within multimodal language models. The authors propose an iterative refinement process incorporating attire-based attribute prediction and bias assessment using entropy scores, claiming improvements in fairness across multiple demographic attributes while maintaining image quality.

All reviewers unanimously gave negative scores. The core technical contribution is modest, primarily combining existing techniques without substantial innovation. The evaluation method raises significant concerns, particularly the circular dependency of using CLIP for both generation and evaluation. The lack of rigorous comparison with established baselines, limited validation beyond automated metrics, and potential reproducibility issues further undermine the paper's scientific contribution. The paper needs a significant revision to address those concerns to reach the publication criteria.