F air C o T : Enhancing Fairness in Text-to-Image Generation via Chain of Thought Reasoning with Multimodal Large Language Models (2025)

Abstract: In the domain of text-to-image generative models, biases inherent in training datasets often propagate into generated content, posing significant ethical challenges, particularly in socially sensitive contexts. We introduce FairCoT, a novel framework that enhances fairness in text-to-image models through Chain-of-Thought (CoT) reasoning within multimodal generative large language models. FairCoT employs iterative CoT refinement to systematically mitigate biases, and dynamically adjusts textual prompts in real time, ensuring diverse and equitable representation in generated images. By integrating iterative reasoning processes, FairCoT addresses the limitations of zero-shot CoT in sensitive scenarios, balancing creativity with ethical responsibility. Experimental evaluations across popular text-to-image systems—including DALL-E and various Stable Diffusion variants—demonstrate that FairCoT significantly enhances fairness and diversity without sacrificing image quality or semantic fidelity. By combining robust reasoning, lightweight deployment, and extensibility to multiple models, FairCoT represents a promising step toward more socially responsible and transparent AI-driven content generation.

## introduction recent advances in text-to-image (t2i) generation have received significant attention in the natural language processing and computer vision communities. by leveraging large text corpora and multimodal data, modern generative models can produce high-fidelity, contextually relevant images from textual descriptions. these breakthroughs have enabled new applications in creative content generation, design prototyping, and accessibility tools for language-driven image creation. however, the data-intensive nature of these models also risks propagating societal biases inherent in large-scale textual and visual datasets, potentially resulting in outputs that are biased or discriminatory. addressing these biases is a critical challenge in textto-image generation, and leveraging the reasoning capabilities of multimodal large language models offers promising avenues for mitigating potential discriminatory outputs. efforts to mitigate bias in text-conditioned image generation typically focus on two broad categories: prompt engineering and model-based debiasing. on the one hand, prompt engineering strategies @xcite rely on human-authored templates, which are prone to subjectivity and require substantial manual labor @xcite . on the other hand, fine-tuning or textembedding modification approaches @xcite @xcite can be computationally expensive and often limited to specific open-source diffusion frameworks. moreover, such model-centric solutions can inadvertently hamper alignment goals or overlook certain categories of bias if they were not explicitly addressed during the fine-tuning phase @xcite . as the generative modeling ecosystem expands to include both open and closed-source platforms, there is a pressing need for lightweight and generalizable methods that can dynamically handle multiple forms of bias without costly retraining. in this paper, we introduce faircot, a novel framework that incorporates chain-of-thought (cot) reasoning within multimodal large language models (mllms) to refine and guide the generative process of t2i models toward more equitable outputs. unlike zero-shot cot approaches that may overlook complex social contexts, faircot adopts an iterative refinement mechanism to produce fairness-aware reasoning paths. it begins by assessing potential biases-such as those related to gender, race, age, or religion-and then adjusts textual prompts accordingly to ensure diverse and inclusive image generation. during iterative reasoning, the model updates its textual guidance based on intermediate feedback, thereby adapting to socially sensitive attributes in real time. additionally, we show how the method is flexible enough to incorporate multiple attributes simultaneously; here, proposing an attire-based attribute detection strategy that leverages clip to expand the scope of bias identification to include religious attire and similar subtle signals. this approach facilitates more accurate detection of sensitive attributes beyond commonly studied demographic categories. lastly, we enable task-adaptive cot inference -at inference time, the mllm selects the most relevant chain of reasoning for generating fairness-conscious prompts, functioning as a lightweight plug-in that does not require retraining or parameter updates to underlying diffusion models, making it generalizable to biases beyond professions,including animal breeds, devices, and modes of transport (figure 1 ). unlike traditional prompt engineering or finetuning methods, faircot operates at the reasoning level, making it compatible with both open and closed-source t2i models. our experiments-spanning popular systems like dall-e and stable diffusion-demonstrate improved fairness and diversity metrics without compromising image quality or semantic relevance. by offering a transparent and dynamic mechanism for controlling how t2i models respond to sensitive inputs, faircot represents a step toward more accountable multimodal systems, giving users on-demand options to address potential over-or underrepresen-tation of minority groups in specific contexts. our contributions are summarized as follows: ## related work this section discusses relevant work on bias in vision-language models, language-driven bias mitigation, advances in large language models, and attribute prediction. bias in vision-language models bias in visionlanguage models is a significant concern, as these models often inherit and amplify societal biases present in training data. @xcite investigated gender bias in zero-shot vision-language models, revealing performance disparities and calibration issues based on perceived gender in images, highlighting how language influences both expanding and biasing vision tasks. @xcite introduced fairrag, focusing on mitigating biases in human generation tasks by incorporating fair retrieval methods. their approach adjusts retrieval processes to include diverse and representative examples, reducing biases in generated content. on the other hand, @xcite proposed fine-tuning text-to-image diffusion models for fairness using a single textual inversion token, demonstrating that targeted finetuning can mitigate demographic biases, although it may be computationally intensive and limited to specific models. complementary to guidanceand token-based methods, @xcite propose a debiasing diffusion model (ddm) that integrates a latent representation learning indicator into stable diffusion to promote fairer generations without predefined sensitive-attribute conditions, and to reduce group disparities even when attributes are specified. language-driven approaches in bias mitigation language-driven strategies have been proposed for bias mitigation. @xcite used text embedding projection for debiasing vision-language discriminative and generative models. on the other hand, @xcite proposed dataset debiasing by enhancing datasets with synthetic, gender-balanced sets. in text-to-image models, @xcite suggested adding gender terms to prompts to balance gender representation in images. similarly, @xcite advocated adding ethical statements to prompts to directly encourage fairness. safety benchmarks for t2i models beyond mitigation methods, standardized evaluation is crucial. @xcite introduce t2isafety, a large-scale benchmark spanning three domains-fairness, toxicity, and privacy-with a hierarchy of 12 tasks and 44 categories, 70k prompts, and 68k human-annotated images. they also propose normalized kl divergence as a fairness metric and release an mllm-based evaluator (image-guard), revealing persistent racial fairness issues and model-to-model variability across safety dimensions. advances in large language models recent advances in llms have enhanced model reasoning and debiasing capabilities. @xcite introduced iterative bootstrapping in chain-of-thought prompting to improve problem-solving capabilities in models like chatgpt. @xcite proposed self-taught reasoner (star), enabling models to self-supervise and refine reasoning steps without additional labeled data. furthermore, @xcite focused on improving the faithfulness of reasoning steps in llms, addressing hallucination, and enhancing reliability in sensitive contexts. however, @xcite highlighted risks of zero-shot cot reasoning in socially sensitive domains, showing that models may generate biased or harmful content without proper guidance, underscoring the need for mechanisms to minimize biased outputs. attribute prediction attribute prediction is crucial for assessing demographic representations in generated content. clip @xcite has been widely used for zero-shot image classification and attribute prediction due to its ability to learn visual concepts from natural language supervision. @xcite found high accuracy (96%) using clip for gender classification across all race class labels, with around 93% for racial classification and 63% for age classification. in addition, @xcite adopted the eight race categories from the fairface dataset but found classifiers struggled to distinguish between certain categories. therefore, to improve race attribute prediction, they consolidated them into four broader classes: wmelh (white, middle eastern, latino hispanic), asian (east asian, southeast asian), black, and indian. @xcite presented a deep multi-task learning approach for heterogeneous face attribute estimation, emphasizing multitask learning to improve attribute prediction accuracy and robustness across diverse populations. our work builds upon these studies by integrating iterative chain-of-thought reasoning within multimodal generative llms to enhance fairness in t2i models. instead of relying on labor-intensive prompt engineering or costly model fine-tuning, our approach dynamically refines textual prompts through real-time feedback, effectively avoiding the pitfalls of oversimplified zero-shot cot in socially sensitive settings. additionally, an attirebased detection mechanism captures subtle markers-such as religious attire-to broaden the scope of bias identification beyond standard demographic attributes. notably, our framework generalizes to scenarios beyond single-person or professioncentric content (e.g., children, animals, objects), promoting more inclusive, ethically responsible t2i generation across diverse use cases. ## methods in this section, we introduce faircot, a framework that integrates mllms to enhance fairness in ai-generated imagery. we specifically leverage the reasoning capabilities of mllms to improve fairness in t2i models that otherwise lack detailed reasoning or have limited input tokens. faircot operates in two key phases: a cot generation phase and an inference phase. ## conclusion we have presented faircot, a chain-of-thoughtbased framework that harnesses iterative, mllmpowered reasoning to mitigate biases in text-toimage diffusion models without compromising semantic alignment. by transparently guiding the generative process, faircot effectively addresses demographic imbalances across attributes such as gender, race, age, and religion. our experiments demonstrate that faircot outperforms both prompt-engineering and fine-tuning approaches, underscoring its model-agnostic applicability across priority and open-source systems. given faircot's versatility, we anticipate its adoption in a broad range of domains, from creative content generation to large-scale enterprise applications. looking ahead, our research will explore more context-specific biases, investigate deeper integration with user feedback loops, and refine the iterative chain-of-thought mechanism to continuously incorporate evolving ethical considerations. we believe this work offers a practical and principled roadmap for embedding fairness-driven strategies within multimodal ai systems, fostering a new generation of responsible, inclusive, and socially aware technologies.