Title: FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Fairness

Generative Fairness

Image Generation

Stable Diffusion
Abstract: Text-to-image diffusion models, such as Stable Diffusion, have demonstrated remarkable capabilities in generating high-quality and diverse images from natural language prompts. However, recent studies reveal that these models often replicate and amplify societal biases, particularly along demographic attributes like gender and race. In this paper, we introduce FairImagen (https://github.com/fuzihaofzh/FairImagen), a post-hoc debiasing framework that operates on prompt embeddings to mitigate such biases without retraining or modifying the underlying diffusion model. Our method integrates Fair Principal Component Analysis to project CLIP-based input embeddings into a subspace that minimizes group-specific information while preserving semantic content. We further enhance debiasing effectiveness through empirical noise injection and propose a unified cross-demographic projection method that enables simultaneous debiasing across multiple demographic attributes. Extensive experiments across gender, race, and intersectional settings demonstrate that FairImagen significantly improves fairness with a moderate trade-off in image quality and prompt fidelity. Our framework outperforms existing post-hoc methods and offers a simple, scalable, and model-agnostic solution for equitable text-to-image generation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Fairimagen that reduces bias in text-to-image generation by projecting prompt embeddings onto a fairness-aware subspace that removes explicit group signals. They also introduce controlled noise along group-specific bias directions during inference to preserve semantic meaning while increasing diversity. The experimenst are performed on StableDiffusion3 model on extended Winobias professions dataset with 120 professions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths :\n\n1. The approach is simple since the approach requires modified PCA and a simple noise injection during inference.\n2. The approach is intuitive and easy to understand.\n\nWekanesses:\n\n1. I believe the approach struggles to scale effectively to more complex scenarios. For example, the authors present qualitative results for joint gender and race debiasing in Figure 3d, where many CEO images fail to accurately represent the CEO concept and exhibit visual inconsistencies. This suggests that cross-demographic debiasing becomes increasingly challenging as more categories beyond gender and race are introduced.\n\n2. This also ties back to my first point. From the ablation experiments, it seems like the approach is very sensitive to hyperparameters such as noise injection scale, hidden dimension size etc. makes the approach difficult to scale effectively.\n\n2. The approach debiases only the input embeddings at inference but does not tackle biases embedded within the pretrained diffusion model itself. Since the model’s parameters reflect biases from training data, these can still affect outputs, limiting the approach’s overall effectiveness.\n\n3. In Section 4.5.6, the authors demonstrate FairImagen’s robustness to demographically determined prompts, which is impressive. This means that when a prompt clearly and appropriately specifies a gender, FairImagen respects it without unnecessary overrides. However, if the dataset or embedding space strongly associates certain professions or roles with a particular gender (e.g., “doctor” predominantly male), the model may confuse implicit bias for explicit gender information. Doesn’t this imply that even neutral prompts like “doctor” could still generate mostly male images? I don’t see any component in the approach that explicitly prevents this behavior.\n\n4. The technical novelty is limited because the core idea mainly involves integrating FairPCA into the prompt embedding space of Stable Diffusion.\n\nQUESTIONS:\n1. SDID originally computes concept vectors in the h-space. However, I believe the authors compared their method to a version that performs this operation in the prompt embedding space. Did you also evaluate how the approach compares to the original SDID?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses that although recent text to image (T2I) model have introduces remarkable improvement in their sample quality and diversity, they also introduce biases. To address this, this work proposes FairImagen — a framework that modifies that the prompt embedding with FairPCA. Additionally, two additional contribution were made i) Noise injection — for improved fairness and pefromance trade-off — and cross demographic debasing — to better handle multiple protect attriubte.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength \n1. The paper is well-written and structured. Specifically, it the paper flow allows for clear understanding of the problem and solution proposed.\n\n2. The proposed method appears to be very effective (on coarse grained attributes) e.g., table 2 (a) on Gender, which is impressive.\n\n\n## Weakness  \n1. The paper seems not to be up to date with the latest works, e.g., it does not compare against \"NeurIPS24: FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation\", which has been demonstrated to be an improvement of ITI-Gen (which the paper compares extensively against).\n\n- Specifically, FairQueue has demonstrated that ITI-Gen performance degradation is attributed to the quality degradation induced by the technique (which is then addressed by FairQueue). Therefore, having not compared against FairQueue (and instead comparing against only ITI-Gen), it is unclear if FairImagen similarly addresses this issue by ITI-Gen (or is ITI-Gen — the existing SOTA — a better approach).\n\n2. The experiment depth is lacking — both ITI-Gen and FairQueue evaluate against multiple different sensitive attribute demonstrating it's generalizability.\n\n- The limited sensitive attributes experimented on (mostly coarse-grained), e.g., Gender, Race, does not fully demonstrate FairImagen strength and weaknesses. Other SOTA approach like FairQueue and ITI-Gen, in additional to the aforementioned coarse grain attributes also utilize fine-grained attributes e.g., Age (which arguably is more challenging, as it requires preservation of the original sample's features).\n\nQUESTIONS:\n1. Based on weakness-1, how does FairImagen perform when compared against the existing SOTA FairQueue?\n- This would be very useful to the research community and work. For example, if FairImagen performance better on fairness but worse on quality, then future research may consider a hybrid of the two approach.\n\n2. Based on weakness-2, How does FairImagen perform when considering more fine-grain protected attributes e.g., Facial expression, or more nuance protected attributes e.g., Skin-Color.\n\n3. What is the main difference between FairImagen and other Post-hoc editing method like ITI-Gen and FairQueue? This was briefly discuss in the related work, but would greatly benefit for a clear one to two liners e.g., FairImage differentiates itself from these approaches by ________ . \n\nOverall, I enjoyed reading this work and thank the author for their time and effort. I would be willing to increase my score, should my concerns be addressed (the idea is good, but the experimental depth needs to be addressed, in my humble opinion). Thank you.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tries to target the demographic biases in generation models that originate from CLIP encoder. Comprehensive experiments on fairness benchmarks show FairImagen offers a strong trade-off between fairness and image/prompt quality, outperforming existing post-hoc baselines on Stable Diffusion. The method tries to project the CLIP based prompt embeddings into a “fair” subspace using Fair Principal Component Analysis and aim to minimize the demographic group information while retaining semantic fidelity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. The paper is well written and easy to understand. \n\n2. The motivation of the paper is also quite reasonable given the biases that different components of many models induce. This can help refine many other pipelines as well since it can be used externally as a plugin.\n\n3. Their approach to solve the issue by projec is novel as they claim.\n\n\nWeaknesses: \n1. Their experiments are all restricted to CLIP based image generation model. New models like Stable Diffusion 3 (CLIP + T5) are not used, so it is unclear how generalizable FairImagen is to non-CLIP architectures. \n\n2. Protected attributes focus mainly on binary gender and a set of race/ethnicity categories. Results on broader or more nuanced demographic attributes could help show the generalizability. The work could also try to show some experiments with other complex/entangled concepts.\n\nQUESTIONS:\n1. The description of the method sounds reasonable. However, using it for something like T5 encoders  would help understand the generalizability. What would be the main challenges or steps needed to adapt FairImagen to other embedding architectures?\n\n2.Further, getting other encoders would also help understand how bias in models like SD3 which have mix of both CLIP and T5 is affected by this work. What would be the main challenges of such models with multiple encoders? \n\n3. The empirical noise injection seems helpful, but the choice of noise magnitude and direction remains empirical? Is there a principled way to set or adapt this parameter?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for de-biasing stable diffusion across gender and race. They provide distintions between methods in the literature in three main group of Prompt-based, Fine-tuning-based, and Post-hoc editing and chose the latest because of its simplicity and compatibility with a wide range of off-the-shelf diffusion models.\n\nSTRENGTHS AND WEAKNESSES:\n- minor: table 2-4 are subtable of table 5. either have three tables or one. here we have 4 tables in the text\n- Evaluation is wide but the depth of each table/section is not high (see questions)\n- Comparisons to literature: it is not clear what/why only some methods are selected for comparison. \n\n+ Authors chose a very important problem in the field that has high social impact.  \n+ Authors worked on latest stable diffusion 3 (while I see many papers worked on much earlier versions like SD1.4)\n\nQUESTIONS:\n- How did you choose the methods to compare in sec 4.4? Why not other methods like [18]?\n-Sec 4.5.1: I am not sure if I follow the claims in this section in the table 5. Which of the methods in table 5 are post-hoc? every method except FairPrompt?\n- Table 5: I see that many numbers are very close to each other., Can you provide other statistics, like std? or run some statistical test to see if the differences are statistically significant? especially for CLIP and MUSIQ\n- 4.5.4: How can we make sure results in Fig 3 are not cherry picked? can you provide ratio of major.minor attributes with all 4 scenarios?\n- Can you provide high-res (1Kx1K) comparison between FairImagen, FairPrompt and the third runner in MUSIQ? You have been using SD3 and I see this as a big advantage but still see small images when comparing quality. We want to know what exactly we lose in the trade-off.\n- What happens to gender neutral professions such as teacher/doctor?\n\nLIMITATIONS:\n- While table 1 gives a high level overview for tradeoffs between three methods for de-biasing, some of the rows and the way authors put fail (x) and pass (✓) is not clear. For example what is the measure for \"strong bias mitigation\" or \"easy deployment\"? or why we do not see the level of quality degradation for generations as a row? To me it seems that authors designed this table to have most ✓ for post-hoc editing that is the main method behind this paper. Unless otherwise please justify. \n- Evaluation is wide but the depth of each table/section is not high (see questions)\n- Authors mentioned they chose post-hoc method because of \"simplicity and compatibility with a wide range of off-the-shelf diffusion models\", but they did not provide the generalization to other diffusion models.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}