On Fairness of Unified Multimodal Large Language Model for Image Generation
Authors: Ming Liu
Hao Chen
Jindong Wang
Liwen Wang
Bhiksha Raj
Wensheng Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Fairness
Unified Multimodal Large Language Model
Bias
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Unified multimodal large language models (U-MLLMs) have demonstrated impressive performance in end-to-end visual understanding and generation tasks. However, compared to generation-only systems (e.g., Stable Diffusion), the unified architecture of U-MLLMs introduces new risks of propagating demographic stereotypes. In this paper, we benchmark several state-of-the-art U-MLLMs and show that they exhibit significant gender and race biases in the generated outputs. To diagnose the source of these biases, we propose a locate-then-fix framework: we first audit the vision and language components — using techniques such as linear probing and controlled generation — and find that the language model appears to be a primary origin of the observed generative bias. Moreover, we observe a ``partial alignment'' phenomenon, where the U-MLLMs exhibit less bias in understanding tasks yet produce substantially biased images. To address this, we introduce a novel \emph{balanced preference loss} that enforces uniform generation probabilities across demographics by leveraging a synthetically balanced dataset. Extensive experiments show that our approach significantly reduces demographic bias while preserving semantic fidelity and image quality. Our findings underscore the need for targeted debiasing strategies in unified multimodal systems and introduce a practical approach to mitigate biases.

On Fairness of Unified Multimodal Large Language Model for Image Generation
Ming Liu
Hao Chen Jindong Wang Wensheng Zhang'
Liwen Wang
Bhiksha Raj Ramakrishnan"
Iowa State University
Carnegie Mellon University
"William & Mary
Abstract
Unified multimodal large language models (U-MLLMs) have demonstrated im- pressive performance in end-to-end visual understanding and generation tasks. However, compared to generation-only systems (e.g.. Stable Diffusion), the unified architecture of U-MLLMs introduces new risks of propagating demographic stereo- types. In this paper, we benchmark several state-of -fate-of-the-art U-MLLMs and show that they exhibit significant gender and race biases in the generated outputs. To diagnose the source of these biases, we propose locate-then-fe framework: we first audit the vision and language components and find that the language model appears to be
using techniques such as linear
probing and controlled generation primary origin of the observed generative bias. Moreover, observe "partial alignment" phenomenon, where the U-MLLMs exhibit less bias understand- ing tasks yet produce substantially biased images. To address this, we introduce a novel balanced preference loss that enforces uniform generation probabilities across demographics by leveraging synthetically balanced dataset. Extensive experiments show that our approach significantly reduces demographic bias while preserving semantic fidelity and image quality. Our findings underscore the need for targeted debiasing strategies unified multimodal systems and introduce a practical approach to mitigate biases.
Introduction
Compare Gender Bias in Understanding and Generation
Please generate images of construction workers
Understanding
butinbttoo
Face Distrbuttio
Wies Score
0.6-
L--MUMM
0.4
Moie a Female
0.2
0.0
Emu3
Janus
VIA--U
Show
Models
Figure 1: Biases in U-MLLMs. Left: Comparison between understanding and generation biases in the model. Right: U-MLLMs generate high-quality images but lack diversity, showing bias toward certain demographics (e.g., the model predominantly generates images of white males).
Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in visual
understanding 24 39 Recent research 42 43 has focused on extending MLLMs' capabilities
ppkulium@iiit @iastatc.edu
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

to image generation settings, enabling them to produce both textual and visual content. These unified MLLMs (U-MLLMs), e.g., VILA-U [. present both visual understanding and generation capability. They can not only understand the semantics in images but also generate high-quality images conditioning on user prompts in natural language. However, these U-MLLMMs with unified capabilities might inadvertently reproduce or amplify biases at deployment, including gender and racial stereotypes embedded in their large-scale training data [
A common structure in U-MLLMS is an image tokenizer that transforms images into discrete tokens through an meoder-decoder framework (T]. Specifically, the vision encoder compresses the input image into latent embeddings and then quantizes them into discrete tokens. Afterward, decoder reconstructs the image from these tokens. This discrete tokenization bridges textual and visual modalities by analogizing image tokens to words in a sentence, enabling a single autoregressive objective that unifies text and image generation. While this design has proven effective terms of quality and scalability, it also opens additional avenues for bias to propagate from the tokenizer.
Existing work on debiasing in image generation has highlighted the social risks posed by skewed output distributions, and various methods have been proposed to reduce bias in image generation 377 Nevertheless, many such methods are designed specifically for diffusion models, which leverage different principles for image generation [4 [3]. As U-MLLMs with autoregressive image generation capabilities become increasingly prevalent, is imperative evaluate their biases in image generation and develop new methods to reduce biases in these token-based generation models. Moreover, remains an open question whether the generation biases emerge more from the vision encoder, which generates image tokens, or from the language modeling component, which generates image tokens according to the given text prompt.
9
This paper focuses on investigating and mitigating demographic biases in U-MLLMs for text-to-image generation. Specifically, we take the first step study gender and race biases for U-MLLMs. We benchmark the state-of-the-art U-MLLMs for gender and race biases using datasets and metrics introduced in a recent study on image generation faimnsss 34 These models include VILA-U [3
Show-o 44, Janus [42]. Jamus-Pro Pro [8) TokenFlow 32), Emu3 [40]. Our results show that these models exhibit notable gender and race biases in image generation (see an example Figure 1). Next, we conduct detailed audit of the vision encoder/decoder and the language model component to localize the source(s) of these biases, where we find that the biases are mainly from the language model. Finally, we synthesize high-quality training data with a balanced demographic distribution and propose a novel balanced preference loss to mitigate generation biases, inspired by recent research on direct preference optimization [33 [5].
Through extensive experiments with various U-MLLMs, we demonstrate that our approach signifi- cantly reduces biases (e.g., over-generation of certain genders or races) without sacrificing the quality of image generation. For example, for the VILA-U model, our method reduces its gender bias by 71.9% and increases the inception score by 12.2%. In summary, our key contributions are as follows.
We benchmark the state-of-the-art U-MLLMs on race and gender bias and find that they exhibit biases varying degrees. Notably, Janus-Pro, one of these U-MLLMs, exhibited the worst gender biases with a value of 0.90, compared to the Stable Diffusion with a bias value of 0.67.
Benchmarking Bias:
Localizing Bias:
We inspect different components in the VILA-U model (vision encoder and language model) by using methods such as linear probing in image embedding space to pinpoint potential sources of biases and find that the biases are likely from the language model component.
Mitigating Bias: We used the diffusion model to synthesize training data with balanced demo- graphic distribution. We also introduce a balanced preference loss inspired by direct preference optimization, with the objective of balancing the likelihood of visual generation towards the differ- ent demographic groups. We empirically show that our approach yields substantial improvement in demographic faimess while preserving the quality of image generation, thus providing a practical framework for developing unified ML.MMs with greater fairness.
2 Preliminary
Structure of U-MLLMs. We consider an autoregressive U-MLLM that, given a textual prompt x, first converts z into sequence of text tokens Z1, IT, and then generates sequence of image

---

tokens 21.....; 2I,, which an image decoder can reconstruct into final image y (see Figure for the pipeline). Let 0 = (8., 0t) denote the model parameters, where 8, is the image tokenizer (comprising an encoder and decoder) that converts input images into discrete tokens and decodes tokens back into images, and 0g is the language model (I.M) that processes and generates token sequences (both text and image tokens) in unified autoregressive manner. As shown in Figure 6 the image encoder Eg, maps an image y to latent embeddings, then quantizes them into a discrete token sequence 211..... "T. Conversely, the image decoder De, inverts this process, reconstructing an image from given sequence of image tokens:
fa
2T'
Ee, (y);
(1)
(21,..... )T.)
(2)
Meanwhile, LMe, treats text and image tokens uniformly under a single next-token probability distribution:
(e æ, 2ct) = (Z- 21: x).
(3)
This design allows the U-MLLM perform visual understanding by mapping an image into semantic token space (via eq. (T) and feeding those tokens into the LM, and to perform image generation by autoregressively sampling image tokens via eq. (3) and reconstructing an image from the sampled tokens via eq.
Demographic Bias. When the model is prompted with neutral text (i.e., explicit demographic specified), it may exhibit demographic bias by generating images skewed toward certain groups (e.g.. mostly males or mostly a particular race). Figure I iillustrates this: given the prompt "Please generate images of construction workers," a U-MLLM produces mostly images of white males. Formally, let d E D {di,- .., dx}, J. where D set of demographic labels (e.g., D = (male, female) for gender or {Asian, Black, White, Indian} } for race). We must clarify that these labels are strictly used for benchmark compatibility [4]. We recognize that the gender categories represent perceived gender presentation in generated images, not an individual's gender identity, and this framework does not account for gender-divers individuals. Similarly. the racial categories cannot represent the full spectrum of human diversity, including mixed-race individuals. Our goal is to diagnose and mitigate stereotypical associations within this benchmark framework.
For a neutral prompt x (e.g.." "a portrait of a construction worker"). an unbiased model would generate of images {3/1 1/2 y2,...) } whose demographic labels are balanced (e.g.. 50/50 split by gender or uniform across races). By contrast, we find that the latest U-MLLMs often produce outputs where
Pa(C(u) = d: x) > Po(C(v) d, |x)
for some demographics d, pretrained attribute classifier that labels each generated image y with demographic attribute d. Such output indicates a strong demographic bias in demographic preferences. Our goal is to mitigate this bias while preserving overall image fidelity.
dj (e.g.. d, male, d, = female) when x is neutral. Here C(y) is
Direct Preference Optimization As an efficient alternative to RLHF 30..
Direct Preference
Optimization (DPO) (33 re-parameterizes rewards via the policy itself. Given a policy my and reference Te,, DPO defines the implicit reward:
Te x) + Blog Z(z),
r(x, v) = B log
(5)
( x))
where Z(z) is normalization constant, and models preference between outputs yo and y with Bradley-Terry formulation:
p/y > yt x)) o (r(z,, vw) - -((, 2)).
(6)
DPO maximizes this preference probability directly, avoiding separate reward model.
Reference-free methods [5
29 such as ORPO [15 set
Po x)) ORolyw: m) = oddso (y1 ())
oddso Ix)
oddsa (y x)
Pe (y x))
and employ the loss
CoR log ooo OR 91)
(8)
3

---

By encouraging large odds ratio (3 vt), the model is pushed to prefer the response You over yt directly, without relying on a separate "reference" model. We adapt these preference-optimizatonn ideas to formulate our balanced preference loss for U-MILM debiasing.
3 Locating Bias
As shown in Figure 2 to determine where demographic biases arise in U-MLLM, examine its intermediate outputs. In particular, we analyze the sequence of image tokens produced by the language model (LM) and the latent image embeddings produced by the vision encoder. We consider two main hypotheses for the origin of bias: (1) bias could emerge from the LM's token generation process, or (2) bias could stem from the vision encoder-decoder pipeline.
AudiaLM
condmuction
Text
LM
Encodler
Famaze
Audit Vision Tokeriizaion
Mate
lnee redeidin Beunder
Male
Vision
Female1
Fomale 2
Hal:
Fomali
Embeedinng
Figure 2:
Detecting bias in LM (top), Vision tokenizer (bottom).
3.1 Hypothesis I: Bias Originates in the Vision Encoder
The U-MLLM's vision module is built on a vector-quantized variational autoencoder (VQ-VAE) architecture [3). As illustrated in Figure 6 the encoder compresses an input image into sequence of discrete latent codes (tokens), which the decoder reconstructs into the original image. The VQ- architecture, upon which the vision module is built, is optimized for high-fidelity image reconstruction. This necessitates the preservation of detailed visual features from the input image, which consequently preserves visually apparent demographic attributes, their absence would impede the decoder's ability to faithfully render person's 's appearance (36). Thus, as a byproduct of its core objective to enable accurate image reconstruction, the vision tokenizer encodes visually apparent demographic information present in the input images.
Linear Probing of Image Embeddings We conduct linear probing experiment the image embeddings. We select a balanced subset of face images from FairFace (19), spanning multiple genders and races, and extract their latent embeddings via the vision encoder. The overall audit pipeline is depicted in Figure 2 Each embedding is paired with its ground-truth demographic label, and we train linear classifier E(e) to predict these labels. As reported in Appendix C the classifier achieves strong performance; for gender, accuracy of 0.9658, Fl score of 0.9645, recall of 0.9637, and precision of 0.9653; for race (averaged), accuracy of 0.8232. F1 score of 0.8344, recall of 0,8520, and precision of 0.8384. The strong performance of the linear classifier confirms that the vision encoder's latent space retains significant information about demographic attributes from input images, capability essential for the decoder to achieve high-fidelity reconstruction. It is important acknowledge that the mere capacity to encode sensitive attributes, while fundamental for reconstruction, does not in itself speak to the module's contribution to fairness [46].
3.2 Hypothesis II: Bias Originates from the Language Model
We then investigate whether the language model's loken-generation process introduces demographic bias. Let Nneutral denote a neutral prompt (e.g., "a photo of a professor"), and for each demographic attribute d (e.g., "female" partieular race), let
Taug (d)
"a photo of a d professor".

---

For each prompt {Focutral } U (Pa we use U-MLLM to generate M images, record each image's 's sequence of discrete image tokens, and predict its demographic label d, via pre-trained image classifier C [31]. This produces samples
( M/, di, z:) for i 1=1,....
where 2, (21,1: 21.2....) is the token sequence for image V4-
Collecting Image-Token Distributions We approximate the model's conditional distribution over token sequences for prompt z by the empirical distribution
1 M
Do x
8(z 2,)
where 5(-) is Dirac mass at the observed sequence In other words, (o counts the relative frequency of each generated sequence among M trials. We then measure the Censen-Shammee diver- gence (JSD) between the neutral-promp distribution and each demographic-aaggente distribution:
Das(Po(z) Fneutrat) || Do (z) Taug (d))
(10)
where for distributions P and Q.
Das PPs(P)))) (kI(M))M) PKL(QIIAM) =P
Results and Analysis For each neutral prompt Tucutral, first determine the majority demographic to which its generated images are assigned. We then identify the sugmented-pronts distribution that is closest (in terms of Tansen-Shanne divergence) the neutral distribution. The hit rate the proportion of prompts whose neutral distribution aligns most closely with the explicitly specified distribution of its implicit demographic- 99. 90. for gender and 79.34% for race. These results suggest that, when the model implicitly "prefers" demographic under a neutral prompt (e.g., generating predominantly male images for "a photo of firefighter"), the token-sequence distribution closely matches that of the corresponding demographic-augmenn male firefighter"). The high hit rates (99.80% gender, 79.34% race) indicate that neutral prompts generate image token distributions similar to those from explicitly demographic-aaggeeniie reflecting an implicit demographic preference. This suggests that the language model's autoregressive process significantly steers image token selection towards specific groups, with the vision decoder subsequently rendering images.
prompt (e.g.. "a photo of
prompts
4 Method
4.1
Training Data Generation
Obtaining training dataset with balanced demographic distribution is challenging. We address this by leveraging a pretrained text-to-image diffusion model, FLUUX (0), to synthesize a demographically balanced image For each target demographic attribute (e.g., each race gender combination), generate images using 1,000 base prompts from training prompts in previous work [34]. For example, given accupation-basce prompts such "a photo of professor" worker," we produce one image per demographic specification (e.g., male, female, Asian, Black) for each prompt. The overall pipeline is illustrated Figure & This process yields synthetic dataset
"an image construction
(44) Poal = {2z, y)
(dx)))
where x, is neutral prompt (e.g.. "a photo of professor") and y (.4.)) is the image generated for zij with demographic attribute d, (e.g., "a photo of an Asian professor"). By construction, each zy is paired with exactly one image for each of the demographic groups, ensuring balanced coverage. Although FLUX (20] is not inherently bias-free, our enumeration strategy enforces demographic variety: naive sampling from neutral prompts resulted in skewed outputs, whereas enforcing one sample per demographic guarantees representation for all groups. In total, we curated 300K images (1000 prompts, demographic groups, images per prompt) to form Dpat- This dataset is used for finetuning (I + T) and (T + I) as well. Example samples appear in Figure
5

---

BFO
photograph of an associate ate professor's face
Yeea
Yuras
Figure 3: The objective is to minimize deviation of preference between demographic groups.
4.2
Balanced Preference Loss
We introduce balanced preference loss Laal to encourage the model to distribute its generation probability evenly across demographics for neutral prompts. Intuitively, we require the model to be indifferent to demographic attributes unless explicitly specified:
min p(y > y( x)) p(y > y((h)))))))) [+]]
(11)
Drawing inspiration from ORPO, we construct Lal as a penalty on the pairwise odds ratio between any two demographic groups. For demographic di and dj. we define the odds ratio under the model as (R)() y(d4)) based on eq. ( where y((4) denotes an image (or image-token sequence) with demographic d; An OR, > 1 indicates a higher generation likelihood for d; than for dj. We then define the pairwise balanced preference loss:
(6,5) bal
(0) log 1 + (o (R (d), (d)) - ]
(12)
where is the sigmoid function. This term is minimized when o (log OR,) 0.5, i.. OR, 1 and ((y(d)) |x) p(y(()) x)). To extend to K demographics, sum over all unordered pairs:
(f)
ball (1.)
(13)
ISKSJKK
Minimizing this aggregate loss encourages a uniform output distribution across demographics.
Two-Stage Training
We adopt a two-stage training procedure (see Algorithm for details):
1. Supervised Fine-Tuning (SFT). We first fine-tune U-MLLM on a supervised dataset of prompt-image-token pairs {), This stage optimizes the likelihood of the true token se- quences, yielding base model TSFT that produces high-fidelity, semantically accurate images.
Balanced Preference Optimization (BPO). Starting from TSFT, we then minimize the multi-group balanced preference loss Lhat (via eq. (II) over the balanced dataset Dbal. This reference-free odds-ratio penalty encourages equal generation preference across all demographics, reducing bias while preserving output quality.
In our implementation, the multi-group loss defined in eq. (3) extends the two-group formulation in eq. (12. We optimize the model parameters minimize Cbal after the SFT stage.
Gradient of the Balanced Preference Loss The BPO objective admits compact, closed-form gradient that both smooth and numerically stable:
2 ww 0.5)
Vo Vo Po(ya,) [1 (o( po Po(ya,) [1 pe (yd, )1
Ve Vopo(yu,)
Vo (o)
(1 w) x
1 + (w 0..)²
Fw
eLal
Vav(O)
dv
Ow
where w = a (v()) and v()) log (d.), ((4)). When (o(y(d) [x) = Po (1)(4)) (), we
have 3 and the gradient vanishes, indicating balanced stationary point. If one demographic's probability is higher, the prefactor flips sign, nudging the model toward equilibrium. full derivation provided in Appendix

---

Experiment
5.1 Experimental Setup
Models and Data. We evaluate demographic bias (gender and race) in several state-of-the-an
U-MLLMs: VILA-U 43 TokenFlow [32], Emu3 [40], Janus 42) Show-o (4, and Janus-Pro [
As baseline, we include diffusion model (Stable Diffusion v1.5). For bias evaluation, we use set prompts comprising 50 test examples and 1,000 training examples and adopt metries from fairness benchmark [34]. Following [34], for each neutral prompt (e.g., occupations roles without demographic specification), we generate 160 images. Each image then classified by pretrained demographic classifier 34 (for gender race) compute bias metrics. More details on experimental
setup and computational cost can found Appendix The code can found at our repository
Metrics. Bias is quantified via the Relative Diversity (RD) metric
34 which measures deviation
from a uniform demographic distribution:
(P
Σl fire freq(i))
K(K 1)/2
where freq(i) is the fraction of samples assigned to demographic d. Lower RD values indicate more balanced representation. We report Gender Bias and Race Bias the RD over the respective demographic sets, and Intersectional Bias (GxR) over joint gender-race combinations. To assess image quality, we compute the CLIP score [14] (image-text similarity). CLIP-IQA (38) (image quality assessment), and Inception Score (image diversity) (5). These metrics verify that debiasing does not compromise output fidelity or relevance.
5.2 Experimental Results
Table 1: Image generation bias.
Debias:
Method
Bias 4
Image Quality t
Gender
Race
G.x..
CLIP-S
CLIP-IQA
Inception
Stable Diffusion
0.66
0.21
27.69
0.67
Janus
0.87
0.43
0.23
27.44
0.69
2.27
Janus-Pro
0.90
0.48
0.24
27.62
0.82
1.79
Show-o
0.85
0.48
0.24
27.16
0.86
1.79
TokenFlow
0.84
0.47
0.24
27.17
0.84
2.34
VILA-U
0.89
0.48
0.24
28.24
0.84
1.87
Prompt Engineering
0.56
0.49
0.23
28.51
0.82
1.91
Gender
Fine-tune(1 + T)
0.83
0.42
0.22
28.49
0.80
2.28
Fine-tune(T l)
0.27
0.51
0.23
27.66
0.77
1.85
ВРО
0.50
27.74
0.77
2.10
Prompt Engineering
0.56
0.49
0.23
28.51
0.82
1.91
Fine-tunel T)
0.78
0.44
0.22
28.14
0.80
2.54
Race
Fine-tune(T l)
0.83
0.23
27.98
0,80
1.98
BPO
0.78
0.26
27.66
0.81
Prompt Engineering
0.59
0.33
0.18
28.09
0,80
1.87
G.XR
Fine-tune(1 + T)
0.86
0.45
0.23
28.34
0.82
2.22
Fine-tune(T I)
0.46
0.32
0.17
27.90
0.78
1.91
BPO
0.52
0.26
0.15
27.78
0.80
2.06
Bias in Baseline Models. Table presents bias and quality metrics for Stable Diffusion(SD) and U-MLLMs. All baselines exhibit substantial demographic bias: gender scores range from 0.66 (SD) to 0.90 (Janus-Pro), while race scores fall between 0.41 and 0.48. VILA-U is the most gender-skewed baseline (0.89), corroborating our earlier qualitative findings. Image quality uniformly high (CLIP-S =27-288,, with Show-o attaining the best perceptual quality (CLIP-IQA 0.86) albeit with pronounced bias. These results confirm that contemporary U-MLLMs generate visually appealing images yet systematically favor particular demographics, underscoring the need for targeted debiasing.
Effect of Our Debiasing Method. We fine-tune VILA-U with our two-stage Balanced Preference BPO-Race Racee and BPO-Mix (joint debiasing). On
Optimization (BPO) in three settings:

---

the gender setting, BPO-Gender reduces bias to 0.25 a 72%% drop relative to the baseline-wwhee preserving CLIP-S (27.74) and yielding a strong Inception score (2.10). For race, Fine-tune (T 1) gives the lowest score (0.23), but BPO-Race is competitive (0.26) with higher Inception score (2.31). When debiasing both axes simultaneously, BPO-Mix achieves the best joint score of 0.15, outperforming prompt engineering (0.18) and fine-tuning (0.17) while maintaining image quality (Inception 2.06, CLIP-IQA 0.80). BPO consistently delivers the strongest bias ceaeetiionnspccalllly for joint gender-race bias-without sacrificing fidelity, making it practical debiasing strategy.
Test results on the 564 stereotype prompts are presented in Table 4, BPO-Gender reduces VILA-U's gender-bias score from 0.6467 to 0.4086 (~ drop). BPO-Race lowers race bias from 0.4310 to 0.2566 (~ 40% reduction). Applying BPO-Mix (joint debiasing) yields gender and race scores of 0.4629 and 0. 082880, and cuts intersectional bias from 0.2161 to 0.1622. Across all settings, CLIP-S and CLIP-IQA remain largely intact, indicating that debiasing does not degrade image quality. (2) Cross-Language Generalization. To probe robustness across languages, we translated the evaluation prompts of [34] into Chinese and French using the ChatGPT API. As summarized in [Table 5] the baseline gender-bias scores are 0.4789 (Chinese) and 0.8542 (French). After BPO-Gender! these drop to 0.2810 and 0.3000, respectively-minee
Generalization. (1) Stereotype Prompts 45
the reductions observed in English. CLIP-S and CLIPP-IQ stay on par with the English case, demonstrating that our balanced loss effectively mitigates demographic bias across datasets and languages while preserving image fidelity.
5.3 Discussion
Debias
Race
Debias Gender
Figure 4: Given the test prompt: "a photo of the face of an electrical and electronics repairer, a person". Images generated by: VILA-U (bottom-left), our method reduces bias in gender (bottom-right), race (top-left), gender-race intersection (top-right). For each image, the first color-coded bar represents the predicted gender: male or female. The second bar represents the predicted race: White, Asian, Black, or Indian. The height of each bar indicates the prediction confidence.
Qualitative Samples Figure 4 shows VILA-U's outputs with and without our debiasing method for a representative prompt. The qualitative examples demonstrate that our approach enhances demographic diversity without degrading visual quality. For instance, given the prompt "a photo of the face of an electrical and electronics repairer, a person" (which does not specify any demographic), the original VILA-U predominantly generates images of men of a single race, whereas the debiased model produces a varied mix of genders and races. More samples are in Figure II
Understanding vs. Generative Bias Beyond visual inspection, we evaluate whether bias arises in the model's 's internal understanding. To assess bias in understanding tasks, we run each model and

---

sample multiple times in a pure VQA setting with an empty image, asking, "What is the gender of occupation? (malle/iemale/uknoown) This removes visual cues and reveals inherent perceptual bias. As shown in Figure 1, most models still show bias to some extent. Some models, such as Janus, however, often reply "unknown" or refuse to specify, indicating reduced explicit bias due to alignment training. Despite this, their generative outputs remain skewed (e.g., Janus still produces images heavily favoring one demographic). This partial neitaalnt-neutral Q&A behavior but biased ceneration-highlighht that aligning the model's textual responses does not suffice to eliminate bias in visual generation.
Can debiasing understanding help debias generation? Prior work 55 suggests that improving image understanding may indirectly benefit generation. Inspired by this, we fine-tuned U-MLLM in image-to-text to reduce its understanding bias (see Figure I0p, yielding the fine-tune (I-T) variant. However, as shown in Table Il this approach yields only marginal reductions in generation bias (Gender: 0.89-0.83 03:3; Race: 0.48-+0.44; Intersectional: 0.24-+0.23). Thus, teaching the model that different races/genders share an occupation does not alter its generative bias. These findings imply that bias in understanding and bias in generation can arise from distinct mechanisms. Consequently, improving one aspect of visual- linguisii alignment does not guarantee fairesss in the other. Future methods must jointly address both visual comprehension and visual generation biases. Additionally, prior work [6] identified the "Reversal Curse": L.MSs trained on "A is B" often fail to learn the inverse "B is A.". This phenomenon mirrors our finding that, even after teaching the U-MLLM to map both "male truck driver" and "female truck driver" to the same concept "truck driver," the model continues to exhibit bias when given a neutral prompt(see)
Figure 10)
Additional Loss Variants In addition to core BPO objective, experimented with several alternative loss formulations. For instance, we tested a thresholded penalty that activates only when the probability of one demographic exceeds another by fixed margin, as well simpler loss on bog-probability differences (omitting the (1 - p) terms). The threshold-based variants required careful margin tuning and caused training instability, while the log-probability difference loss was prone to gradient explosion when probabilities approached O or 1. For full definitions and empirical comparisons, see Appendix
Related Work
Multimodal Generative Models Unified multimodal large language models (U-MLLMs) have advanced the state-of-thh-art by bridging visual understanding and conditional generation capa- bilities. Compared early MILMs, which focus purely on understanding, such as Ll series
27 26). these more recent works, represented by VILA-U (43), Show-o [44], MetaMorph (35),
TokenFlow [32), Emu3 44)). TokenFusion (47), Janus 42 28 etc. 3 21 25), highlight their
effectiveness in generating high-quality visuals conditioned on text prompts. U-MLLMs usually employ autoregressive paradigms that may inherit or amplify the biases embedded in their training data. Existing studies predominantly focus on performance improvement rather than understanding and addressing the more critical issues such as demographic fairness.
Fairness in Image Generation The social risks of biased image generation, particularly gender and racial disparitics, have been extensively documented [18] 22 23), Prior efforts in diffusion- models attempted to mitigate bias by re-balaneing training data and incorporating faimess objectives [7]. However, these approaches are not directly transferable to U-MLLMS due to architectural differences and tokenization mechanisms. Few studies explore bias sources within unified models or their downstream effects on generated outputs, leaving gap in understanding the interaction between text and image modalities.
Preference Optimization Direct preference optimization (DPO) 33 has emerged as a promising technique to address biases in machine learning models, especially LLM. Since then, numerous new loss functions have been proposed [2 31 15
12 22. Recent advances [I] integrate preference modeling into loss functions to guide models toward balanced outputs, which have rarely been explored for MILMs. Based on this, we introduce a novel balanced preference loss tailored for U-MLLMs. By leveraging demographic attributes during training, the proposed method balances the likelihood of generating outputs across groups without compromising the image quality.
9

---

7 Conclusion
We examined demographic bias (gender and race) in U-MLLMs for image generation and proposed a method to mitigate it. Our study found that current U-MLLMs often produce images with skewed demographics when given neutral prompts. Through our component-wer wise analysis, the findings suggest that the language model within the U-MLLM is primary driver of the observed demographic bias in generated images when given neutral prompts. To address the issue, we introduced two- stage debiasing approach: first fine-tuning the model on a balanced dataset, and then applying balanced preference loss inspired by direct preference optimization. This approach significantly reduced the bias in generated images across gender and race categories while preserving image quality and relevance. We also discovered a partial alignment phenomenon where models might appear unbiased in a textual response yet remain biased in visual generation, underscoring the need for ceneration-specific debiasing techniques. Our work provides an initial framework for auditing and improving the fairness of U-MLLMs. We hope it encourages the development of more holistic debiasing strategies that consider all aspects of multimodal model behavior. In the future, we plan to extend our method to additional demographic dimensions (e.g.. age, ethnicity). Ensuring fairness in generative models is critical as these models become increasingly influential in content creation. Our work does have some limitation illustrated in Appendix A.
8 Acknowledgments
This paper is partially supported by the Commonwealth Cyber Initiative (CCI) program H-2Q25-020. The authors acknowledge the Modal Academic Compute Credit and William & Mary Research Computing for providing computational resources and/or technical support that have contributed to the results reported within this paper. URL: https//wwww.w..cdufttt..
References
[1] Afra Amini, Tim Vieira, and Ryan Cotterell. Direct preference optimization with offset. arXiv preprint arXiv:2402. 10571, 2024.
[2] Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. A general theoretical paradigm understand learning from human preferences, 2023.
[3] Roman Bachmann, Oguzhan Fathh Kar, David Mizrahi, Ali Garjani, Mingfei Gao, David Griffiths, Jiaming Hu, Afshin Dehghan, and Amir Zamir. 4m-21: An any-to-any vision model for tens of tasks and modalities. arXiv preprint arXiv:2406.0944 09406. 2024.
[4] Hritik Bansal, Da Yin, Masoud Monajatipoor, and Kai-Wei Chang. How well can text-to-image generative models understand ethical natural language interventions?, 2022.
[5] Shane Barratt and Rishi Sharma. A note on the inception score, 2018.
[6] Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: LLMs trained on "a is b" fail to learn "b
is a". In The Twelfth International Conference on Learning Representations, 2024.
[7] Hao Chen, Bhiksha Raj, Xing Xie, and Jindong Wang. On catastrophic inheritance of large foundation models. arXiv preprint arXiv:2402.011
2024.
[8] Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025.
[9] Ching-Yau Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts, 2023.
[10] Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, et al. What's in my big data? arXiv preprint arXiv:2310.20700 2023.
10

---

[11] Patrick Esser, Robin Rombach, and Bjor Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873-12883 2021.
[12] Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization, 2024.
[13] Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzyíska, and David Bau. Unified concept editing in diffusion models, 2024.
[14] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: reference-free evaluation metric for image captioning, 2022.
[15] Jiwoo Hong, Noah Lee, and James Thome. Orpo: Monolithic preference optimization without reference model, 2024.
[16] Edward J. Hu, Yelong Shen, Phillip Wallis. Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021.
[17] Yeongmin Kim, Byeonghu Na, Minsang Park, JoonHo Jang, Dongjun Kim, Wanmo Kang, and Il-Chul Moon. Training unbiased diffusion models from biased dataset. In The Twelfth International Conference on Learning Representations, 2024.
[18] Hadas Kotek, Rikker Dockum, and David Sun. Gender bias and stereotypes in large language models. In Proceedings of the ACM collective intelligence conference, pages 12-24, 2023.
[19] Kimmo Kärkkäinen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age, 2019.
[20] Black Forest Labs. Flux. httt::///tthhb..cmm//aadddroeess
2023.
[21] Duong H. Le, Tuan Pham, Sangho Lee, Christopher Clark, Aniruddha Kembhavi. Stephan Mandt, Ranjay Krishna, and Jiasen Lu. One diffusion to generate them all, 2024.
[22] Cheng Li, Mengzhou Chen, Jindong Wang, Sunayana Sitaram, and Xing Xie. Culturellm: Incorporating cultural differences into large language models. arXiv preprint arXiv:2402.1090 2024.
[23] Cheng Li, Damien Teney, Linyi Yang, Qingsong Wen, Xing Xie, and Jindong Wang. Cul- turepark: Boosting cross-cultural understanding in large language models. arXiv preprini arXiv:2405.151 15145, 2024.
[24] Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training large language-ane sanguage-and-vision assistant for biomedicine one day. Advances in Neural Information Prucessing Systems, 36, 2024.
[25] Zijie Li, Henry Li, Yichun Shi, Amir Barati Farimani, Yuval Kluger, Linjie Yang, and Peng Wang. Dual diffusion for unified image generation and understanding. arXiv preprins wrXiv:2501.00283 2024.
[26] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023.
[27] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
[28] Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai yu, Liang Zhao, Yisong Wang, Jiaying Liu, and Chong Ruan. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation, 2024.
[29] Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with reference-free reward, 2024.

---

[30]
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022.
[31] Ryan Park, Rafael Rafailov, Stefano Ermon, and Chelsea Finn. Disentangling length from quality in direct preference optimization, 2024.
[32] Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang, Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation. arXiv preprini arXiv:24/2.03069 2024.
[33] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model, 2024.
[34] Xudong Shen, Chao Du, Tianyu Pang, Min Lin, Yongkang Wong, and Mohan Kankanhalli. Finetuning text-to-image diffusion models for fairness, 2024.
[35] Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal under- standing and generation via instruction tuning, 2024.
[36] Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning, 2018
[37] Janet Wang. Yunsung Chung, Zhengming Ding, and Jihun Hamm. From majority to minority: A diffusion-based augmentation for underrepresented groups in skin lesion analysis. In Interna- tional Conference on Medical Image Computing and Computer-Assister Intervention, pages 14-23. Springer, 2024.
[38] Jianyi Wang, Kelvin C. K. Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images, 2022.
[39] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model's 's perception of the world at any resolution, 2024.
[40] Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang. Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint 2rXiv:2409. 18869, 2024.
[41] Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch. Concept algebra for (score-based) text-controlled generative models, 2024.
[42] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, and Ping Luo. Janus: Decoupling visual encoding for unified multimodal understanding and generation, 2024.
[43] Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409. 04429, 2024.
[44] Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single trans- former unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
[45] Chejian Xu, Jiawei Zhang, Zhaorun Chen, Chulin Xie, Mintong Kang, Yujin Potter, Zhun Wang, Zhuowen Yuan, Alexander Xiong, Zidi Xiong, Chenhui Zhang, Lingzhi Yuan, Yi Zeng, Peiyang Xu, Chengquan Guo, Andy Zhou, Jeffrey Ziwei Tan, Xuandong Zhao, Francesco Pinto, Zhen
12

---

Xiang, Yu Gai, Zinan Lin, Dan Hendrycks, Bo Li, and Dawn Song. MMDT: Decoding the trustworthiness and safety of multimodal foundation models, In The Thirteenth International Conference on Learning Representations, 2025.
[46] Yuzhe Yang, Haoran Zhang, Judy W. Gichoya, Dina Katabi, and Marzyeh Ghassemi. The limits of fair medical imaging ai in real-world generalization. Nature Medicine, 30(10))2838-22 -2448, Oct 2024.
[47] Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with multi-modal model. arXiv preprint arXiv:2408. 11039, 2024.
13

---

A Limitation
Limitation. Several questions remain open despite the bias reduction achieved by our approach. First, our study primarily centered on overt demographic categories (e.g., gender, race). Real- world scenarios may demand addressing intersectional or nuanced attributes (e.g.. age, culture, or religion). Second, many models are not fully open-source, restricting the scope of our evaluations to publicly available systems. Future research could broaden the range of tested models. Third, our definition of fairness as demographic parity and our use of discrete, Western-centree gender and racial categories are specific value judgments that may not apply universally and risk oversimplifying complex identities. We also recognize that project's scope, our method defines fairness as demographic parity, but this is just one interpretation. Lastly, due resource constraints, did not explore alternative preference optimization objectives beyond our framework. Building on our method to incorporate other debiasing approaches is promising direction for future work.
perspectives rescarchers have shaped the
B Structure of U-MLLM
This section presents the structure of VILA-U and its visual tokenizer; all the content in this section from prior study 43
Multi modaal
Multi-modal Token Sequence
Multi modal Token Sequence Out
Multi-modai Inference Outguts
Inputs
Text
Text Encoder
The skating
Decoder
Generative Multi-modal Model
Imoge
Text-aligned
Vision
Vision Encoder
Text-oligned Vauol Tokens
Next-token Prediction
Figure 5: Overview of framework's 's multi-modal training and inference process 43 Visual inputs converted into discrete tokens and merged with textual tokens to create unified multi-modal token sequence. This sequence is used in next-token prediction process, which supports unified training objective. During inference, output tokens are processed through either text detokenizer or vision tower decoder, generating multi-mon multi-modal content outputs
Reconstruction
Loss
Vision Encoder Pretroined
Residual Quantizer
Vision
Decoder
Discrete Visual Tokens
This is en image o cat
Text Encoder Pretroined
Controstive
Frazen weights
Trainable weights
Figure 6: Overview of unified foundation vision tower Input images are processed by the vision encoder, where features are extracted and discretized using residual quantization. These discrete vision features are then utilized in two ways: they are fed into the vision decoder to reconstruct images and are used to perform text-image alignment. Throughout this process, both the reconstruction loss and contrastive loss are calculated to refine the vision tower, enabling it to generate discrete visual features that are aligned with text 43
14

---

C Locating Bias
Gender
Race
Hit Rate
0.9980
0.7934
Table 2: JS-Divergence Hit Rates for Gender and Race Variant Prompts. For each training prompt, images were generated using both a neutral prompt and prompt variants in gender or race. We first determined the majority gender/race from the neutral-prompt images (skipping ambiguous cases). We then counted "hit" whenever the JS divergence between the neutral prompt's image-token distribution and the matching-ver -variant prompt's ' distribution was smaller than that for non-matching variants. More than 99% of prompts yield closer divergence for gender-matching variants, and about 79% for race-matching variants, indicating that the language model's token sampling is skewed toward certain demographics.
Pair
Accuracy
Recall Precision
Male Female
0.9658
0.9645
0.9637
0.9653
Black vs. Southeast Asian
0.9167
0.9187
0.9658
0.8760
Black vs. Indian
0.8958
0.8988
0.9328
0.8672
Black vs. Middle Eastern
0,4667
0.6364
1.0000
0.4667
Black vs. East Asian
0,9500
0,9492
0.9739
0.9256
Black vs. Latino Hispanic
0,8333
0.8113
0.7107
0.9451
Black White
0.9167
0.9231
0.9091
0.9375
Southeast Asian vs. Indian
0,9208
0.9156
0.8879
0,9450
Southeast Asian vs. Middle Eastern
0,9333
0.9316
0.8934
0.9732
Southeast Asian Vs. East Asian
0,6458
0,6222 0.5385
0.7368
Southeast Asian Vs. Latino Hispanic
0.8625
0.8546
0.8509
0.8584
Southeast
vs. White
0,9042
0.9013
0.8750
0,9292
Indian vs. Middle Fastern
0.8958
0.8980
0.8661
0.9322
Indian vs. East Asian
0.9417
0.9381
0.9381
0.9381
Indian Latino Hispanic
0.7625
0.7595
0.7563
0.7627
Indian White
0.8917
0.8992
0.8923
0.9062
Middle Eastern vs. East Asian
0.5208
0.6849
1.0000
0.5208
Middle Eastern vs. Latino Hispanic
0.6875
0.6445
0.5862
0.7158
Middle Eastern vs. White
0,7458
0,7359
0.7083
0.7658
East Asian Latino Hispanic
0,9167
0,9180
0.9655
0.8750
East Asian vs. White
0,9333
0.9322
0.9402
0.9244
Latino Hispanic vs. White
0.7458
0.7490
0.7000
0.8053
Table 3: Embedding classification metrics by pairwise comparisons.
15

---

D BPO Algorithm
Algorithm 1
Balanced Preference Optimization
Input: U-MLLM with parameters Bg: SFT dataset DsFr = ((r,, 2:) of prompts z; and image tokens
25 Balanced dataset Daal = {z Vij .... Vix )}. each 2, with multiple demographic variants; Trade- off parameter A, total training epochs N1, N2; Output: Debiased model parameters
1: Stage 1: Supervised Finetuning
2: Initialize 0
0g
3: for epoch to N1 do
4: Sample minibatch {a 2 from DsFT
5: (0) = Σ 2.) log Po x;)
6: Update 0 + v (0)
7: end for
8: Stage 2: Balanced Preference Optimization
9: for epoch to N2 do
10: Sample minibatch { Mi ....... Mik )} from Dral
11: Balanced Preference Loss:
(bal Σ
Saa (dz-1,de) (0).
bal
12: Update 0 0 V (0)
13: end for
14: Return A
16

---

E Gradient of BPO
E.1 Setup: The Balanced Preference Loss
For simplicity, consider just two demographic categories d; and dj. The extension to multiple groups done by summing over pairs.
We define the odds ratio:
oddso(Ua,) (Ud5)
Pe (Hd xx)
ORo(yd, :
where oddsp(ua)
po (yd x))
Taking the logarithm,
v0) log (Ra(ya, yd,)
log oddso (va,)
log oddsa (ydd)
Log pa (va, [x)]
log 10g[1- P [z)]
po [a>] looe -ps (Wi, |z)
Next, let w() a (u(0), where is the logistic sigmoid a (2) T+e-z- The Balanced Preference Loss for two groups is:
(0) = log 1 + w }
(v)
Minimizing this loss pushes w(0) - 2,4 i.c. ORg + 1, which implies Po(aa | x) po(Hd, 1x).
E.2 Chain Rule for V Lbal
We want:
0hrul Ow
dw
Ve (0)
(o)()).
Ou
We handle each factor separately. Derivative of Daal w.r.t. w
Set
r(0) = w(6) - = =(((()) 2"
Then
Laal (0) log (r(0))3)-
Hence,
0haal
0Cal
dr
2r
(w 0.5)
[2r] 1
Dw
Ör
Dw
I+
1 ,
1 1 (w 0.5)
Derivative of w (1)) w.r.t. v
We know d de o (u))( [vv)). So
dw
w(0) [1 w(()) a(v(0)) (v()))
du
Gradient of v0) w.r.t.
Recall
v()) log ORo (y1,, (y va,) log oddsp(ya.) log oddsp (ya,) )
Hence,
V (() Vo [log oddss (wre)) (yd))
Vo [log oddso(ya,)) )
17

---

We have
log (dds log po Polaa ())
log Poaa x)].
Thus,
Vo log oddso(Ua)
Po x))
Vo Po(ya x),
po (ys x))
Po (ya x)
where we used V log 2(0) = Vez. Combine the two terms carefully (and noting the minus sign in the second log derivative switches sign again), get:
V log oddso (Ja) (yl x)) po x))
Vo po(ua) x).
Putting it all together for d; and dy:
1
1
Vor(0) (o( po (va,))
(opo(ya)
Vopo(ua,)
Po(Ua,) po (o(d)))
(Here we have suppressed the conditioning on notation, just to keep it lighter.)
E.3 Final Gradient Expression
By combining previous sections, we get:
(w - 0.5)
Vo po (yd,)
Vo po(
Vo (6)
x w
w)
x
1 + (w- -0.5)
Po(ya, [1 po(ya.)
pe (ya,) pa (3da))
Ow
CColl
Vov(O)
Ov
dw
where = (v(()).
=
Interpretation:
If (o((),)) >> po(Ua,): ), the odds ratio large, so v(0) large and w(0) 1. The factor (w 0.5) then positive and big, so the gradient pushes parameters reduce po (and/or raise Po(Ud,))
Conversely, if po(ua,) < po (ya,), we get negative factor in front of Vapo(ya,).
When po(ya.) po(;),): then v(0) O and w()) 0.5. The entire derivative is zero, which precisely the balanced solution we want.
E.4
Multiple Demographic Groups
For more than two groups, say dx}, one can (as in the paper) sum or average pairwise losses:
hal (0)
(multi)
Lrai di, dj).
ISKjjK
Then V cf (multi) is just the sum of the two-group gradients, ensuring all pairwise distributions converge to balance.
E.5 Putting It All
This derivation shows exactly how to compute the gradient the Balanced Preference Loss. also clarifies why the only way to get zero-gradient solution for each pair (de, d,) is to equalize their probabilities Po Pyd; .x) = po (ya, x). Hence the uniform distribution across demographics the unique global optimum (apart from mild edge cases).
18

---

Comparison with Other Loss Functions
In experiments, evaluated several loss formulations designed to reduce demographic bias. Below we describe four variants of odds-ratio penalties, each designed to encourage balanced treatment between pair of demographic groups d; and dj. For brevity, set
pd; pe (yd. x)) and pd, pe (Jd, 1).
For the methods below, we also define the log-odds difference as
A log(ps.) log(Pa,)
F.1
Method I (Our Core Approach)
Key Idea: Penalize large deviations in the log-odds difference
Figure 7) Specifically, we define Pd,
log Pd:
log
Pd.
Pd,
and then compute the balanced loss as
log + (A4) - 0.5 0.5] )
faal (A)
where oA) Itc is the sigmoid function. When o(A) 0.5, the model is equally likely to generate yd, yd, laal is near O. Larger imbalances yield smoothly increasing penalty. The overall loss is obtained by averaging fau over pair {(d; dj)}.
Why It Works Best:
It employs the true log-odds, logí - which remains stable even when p near 1. The function (A) is continuously differentiable for all A, avoiding the discontinuities that arise in thresholded formulations.
In practice, Method I produces stable training dynamics and superior bias reduction.
F.2 Method Thresholded Loss on Log Probability Ratios
A simpler approach uses the ratio of probabilities directly. Define
ò =
of log
Pd. 0.5.
Pa,
A piecewise penalty is then applied only when |8| exceeds fixed threshold t:
8 6. if |0| 2
Ethresh (6)
0,
otherwise,
where a is penalty scale.
Drawbacks:
It requires choosing both a threshold and scale a, which may be sensitive to the task hand.
The training process can become unstable near the threshold boundaries.
E.3
Method 3: Thresholding True Log-Odds
This variant similar to Method 2 but uses the full log-odds difference. With
A log(pu, (pg,),
piecewise penalty is applied if the corresponding sigmoid value deviates from 0.5 beyond threshold:
Einest ( a (a) 0.5 - s). ir - 2 t
0.
otherwise.
It still suffers from the need to tune the threshold and stabilization.
19

---

F.4
Method 4: Plain Log Probability Difference Squared
straightforward approach computes the difference log probabilities:
simple log.)a.) log (pd,)
and then defines the loss as
o Lg (immllee = - (Asmple)
1 2
Because this method does not incorporate the 1 - terms found in the log, it is susceptible to numerical instability when Pa, or pd, approaches 0 or Empirically, it tends to perform slightly worse than Method 1.
F55 Empirical Observations
The training loss for each method be found Figure Method consistently achieves the best balance between fairness and training stability. Threshold-based approaches (Methods and 3) may reduce bias but require careful tuning of the threshold which may lead less smooth training. Method 4 works in moderate probability ranges but is prone to exploding vanishing gradients when probabilities are near the boundaries of outperformed the other variants by achieving lower bias metrics along with stable training dynamics and minimal computational overhead. Based on these observations, we selected Method as our primary Balanced Preference Optimization (BPO) objective.
In ablation studies (Figure 7). Method
20

---

Method 1 (Our Core Approach)
traln
Log-Gddds Difference:
1o0, 108( -pp)
9005
Balanced Lees:
bat(a) keaf [0l22] - oo)")
where (A)) Lee
(a) Training Loss plot for method (core method).
Method 2: Thresholded Loss on Log Probability Ratios
Definition of &
of (Ioee R4
0.5
0.33
8a la.
110)
(b) Training Loss plot for method 2.
Mothod Thresholding Log-Odds
trann
Log-Odds Difference:
Ims()))) lnd(pu,)
Threshelded Lose:
aa |o(a) - 0n 2/.. if ((A) - otherwias.
0.5 2
05
(0.
200
402 000
3.2
1.44
(c) Training Loss plot for method 3.
Method 4: Plain Log Probability Difference Squarod
Simple ng Differance:
Assirple Ing(pu.) - (pg)
Squared Loss:
300
1.5
2.5
(d) Training Loss plot for method 4.
Figure 7: We define Pa, = =Pe x) and Pd, = pe (ya, 2). We compare four loss formulations:
(1) Method penalizes deviations in the log-odds difference A = log I-p4,
Pd.
log
T-p4, using
continuously differentiable loss (A) log 1 + (o( 0.5]²) yielding stable training and superior bias reduction. (2) Methods 2 and apply thresholded penalties on the log probability ratio or log-odds, requiring careful tuning of threshold t and potentially causing instability. (3) Method uses the plain squared difference log probabilities, but can suffer from numerical instability when probabilities approach 0 1. Among these, Method Balanced Preference Optimization (BPO), stabilizes the loss and yields the best final results.
21

---

G Generalization
Table 4: Results on 564 Stereotype Prompts (MMDecoding[rust)
Method
Gender Bias
Race Bias
Intersection Bias
Avg. CLIP Score
Avg. CLIP IQA Score
VILA-U
0.6467
0.4310
0.2161
28.2198
0.8577
BPO-gender
0.4086
0,4899
0,2256
25.1070
0.7072
BPO-race
0.6313
0.2566
0.1659
24.3783
0.7466
BPO-mix
0.4629
0.2880
0.1622
24.5264
0.7481
Table 5:
Preliminary Cross-lingual Test Results (Chinese and French)
Setting
Gender Bias
Intersection Bias
CLIP Score
CLIP IQA Score
VILA-U (CN)
0.4789
0.1938
24.3673
0.7445
VILA-U (FR)
0.8542
0.2381
26.9307
0.8853
BPO-gender (CN)
0.2810
0.1983
23.8048
0.7046
BPO-gender (FR)
0.3000
0.2275
25.6598
0.7768
22

---

H Data Generation Pipeline and Samples
male associate professor
female associate professor
photogtaph of an associate professor's face
Associate professor
FLUX Model
An Indian sis- latate professor
Neutral prompt
Augmented prompt
Figure 8: Pipeline for data generation. Augmented Prompts for Each Demographie: We explicitly instruct the model with prompts like "Indian associate professor" rather than relying on FLUX. 1-de'ss default distribution. Each occupation is re-generated for every demographic, and direct instructions yield images with the desired attributes; Balancing Across All Attributes: After collecting images for each demographic prompt, we pair them with a neutral prompt (e.g.. "a photo of an associate professor"). This creates training set where each neutral prompt has one sample per demographic, ensuring uniform coverage across races and genders. As result, the model learns that the same description (e.g., "associate professor") can represent multiple demographic groups, countering one- sided associations; Goal: A Balanced Synthetic Dataset from potentially biased Generator: Our method ensures equal representation per prompt by explicitly enumerating each demographic label and quality-checking outputs. L.U..'-dev's biases are corrected, sufficient images for each category. Thus, while we concur that LLUXXI-der can exhibit low diversity augmented-prompe
only require it to produce
in unconstrained settings, those limitations and yields a controlled, demographically balanced dataset that is well-suited for our debiasing objectives. (see Figure 4 below for some samples)
balanced-pairtie pipeline specifically mitigates
23

---

Prompt: photo of the trucker, person"
Race
Gender
(a) prompt: photo the face trucker." " the prafesser,
Gender
(b) prompt: "A photo of the face of associate professor."
Figure 9: Training samples for fine-tuning. In T2I, each image is paired with its prompt (image, prompt), while in 12T, each prompt is paired with its image (prompt, image). For BPO, each prompt is paired with two images from different demographic group as (prompt, imagel, image2).
24

---

I Can debiasing understanding help debias generation?
Label
flectrica and electronizz
A photo of the of trucker
Villdebras uncentanddng HELF diblis generation?
U-MLLM
Input
Image Understanding((1) Finetuning
Image Generation
Figure 10: Left: We use balanced dataset for image understanding (I2T) finetuning debias understanding by pairing images from different demographics (e.g.. female truck driver and male truck driver) with the same caption "truck driver" to teach the model that both map to the same concept. Right: The fine-tuned model is then used to generate an image; for each image, the first color-coded bar represents the predicted gender: male or female, and the second bar represents the predicted race: White, Asian, Black, Indian. The height of each bar indicates the prediction confidence. Our hypothesis that the model recognizes that both male and female truck drivers are truck drivers, it will generate other occupation-related concepts fairly. However, while debiasing understanding slightly reduces bias (e.g.. gender bias decreased from 0.89 to 0.83 in paper Table 1). the effect image generation is limited. We note that recognizing multiple demographic possibilities does not alter the model's s default token sampling, and without explicit training for balanced generation (e.g., T2I finetuning or Balanced Preference Optimization), biases persist.
25

---

Experiment Details
Evaluation Protocol. First, generate N 160 images for each prompt. Next, we apply demographic classifier from a previous study [3 to predict the labels of demographic attributes for each image. These labeled attributes are then used compute the overall bias and semantic score based on the above metrics.
Training Procedure. Base Learning Rate: We start with fixed learning rate (for example. 1 x 10-1) for fine-tuning. Batch Size: Ranges from 8 to 32, depending on the setup. Number of Steps: For the first stage, we fine-tune for up to 10 epochs, checking bias and quality after the training. For the second stage, the epoch is chosen to be to 2. We use the LoRA [16] method for finetuning; the rank is for all experiments.
Computational Cost Our BPO approach starts with standard supervised fine-tuning (SFT) and adds a brief 1-2 epoch preference-optmmizaton stage, with minimal gradient updates. Compared to other methods, supervised debiasing requires only 1-2 extra epochs to balance, while preference-learning approaches typically rely on a large reference or separate reward model; by contrast, BPO uses just single policy. Empirically, based on training logs using two A100 GPUs (40 GB), the fine-tuning stage takes 23,020 s for gender-only, 23,020 for race-only, and 23, s for mixed race-gender configurations, while the BPO stage takes roughly 2,365 2,375 and 2,988 respectively. In summary, BPO is efficient and comparable in overhead to other debiasing and cefference-learning methods.
26

---

K
Compare output from different methods
(a) prompt: "A photo of the face of an electrical and electronics repairer, person" (debias racc)
(b) prompt: "A photo of the face of an electrical and electronics repairer, person" (dehias race and gender)
(c) prompt: "A photo of the face of machine offbearer. person person"((ebinn "(debias race)
(d) prompt: "A photo of the face of a machine offbearer, person "(debias race and gender)
Figure 11: For each prompt, the first row (from left to right) shows images of VILA-U, Prompt Engineering, and 12T finetuning, while the second row shows images of T2I finetuning and BPO.
27

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA]. .
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). ). general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction clearly articulate our three core contribu- tions novel algorithm design, theoretical guarantees, and comprehensive empirical valida- tion-and explicitly delineate the scope and assumptions our study.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
28

---

We dedicate a "Limitations" section in Appendix A to discuss key assumptions, potential performance degradation under distribution shifts, and computational constraints of our approach.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Each theorem stated with its full set of assumptions and complete proof provided in Appenndix E. with sketch proofs in the main text for clarity.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
29

---

We describe the overall experimental design and datasets but omit specific hyperparameter settings and detailed preprocessing steps due space constraints. Guidelines:
Justification:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No] the paper provides open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results
We plan to release all code and datasets in public repository upon acceptance, including detailed setup instructions and scripts for reproducing our main results.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (nttps://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
/nips.cc/ppull http:cccppppcccetettetteee...clt poguuddee/oodeebbbi/sssooooo.o.o
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
30

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: (subsection 5.1 specifies data splits, hyperparameter selection procedures, optimizer configurations, and early stopping criteria for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Our experiments focus on deterministic benchmarks and thus do not include error bars statistical tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
31

---

Justification: subsection 5.1 details the GPU type, memory footprints, and execution times for individual experiments, as well as the total compute hours used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https::/neuripps hwtpc//ubb..c//iiccccci.........
Answer: [Yes]
Justification: Our methodology adheres fully to the NeurIPS Code of Ethics, ensuring responsible data handling and model evaluation without bias Appendix
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Section 6, we outline positive outcomes such as improved model efficiency and discuss potential misuse scenarios, including adversarial attacks and privacy concerns, along with mitigation strategies.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
32

---

11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve releasing high-risk models sensitive datasets, making additional safeguards unnecessary.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: We rely solely on our own code and data, and do not incorporate hird-party assets that would require license disclosures.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pwpprssitthaaeee thcode cote..com/aae com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- se-packaedd, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce new data assets or external code packages requiring formal documentation.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
33

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our evaluation is small scale, crowdsoureing protocols and compensation details inapplicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects research was conducted, so IRB approval was not required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: We leverage an LLM-based module as a judgment mechanism for output evaluation, described in Section 5.2.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, nta-standard components.
Please refer to LLM policy for what should or should not be described.
34


Reviews:
[{"review_id": "sUcuYwf8jh", "reviewer": "Reviewer_BVqp", "summary": "The paper audits recent unified MLLMs and shows they generate notably gender- and race-skewed images even when their text-only answers appear unbiased. By dissecting vision tokenizer vs. LM components, the authors trace most bias to the LM and propose a two-stage “Balanced Preference Optimization” (BPO) that fine-tunes on a synthetically balanced dataset and applies an odds-ratio loss, cutting VILA-U’s gender bias by ~72 % while keeping image quality basically unchanged.", "questions": "1. I would like to suggest the authors to discuss more about the generalization of their training data generation pipeline. \n\n2. What is the core difference between the proposed BPO and standard DPO?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strength\n\n1. The paper is clear and easy-to-follow.\n2. Decoupling and analyze the vision encoder and LM is a clear and feasible way to identify the bias.\n3. The evaluation shows consistent improvements on a broad range of benchmarks.\n\n- Weaknesses\n\n1. One major weakness is that the authors use template-based training prompts (~1000), which is basically designed by hand and is hard to scale to more diverse prompts and broader types of biases.\n2. Apart from the training prompts, the paper construct their training images purely with FLUX.1 instead of using real-world web images. There are risks of leaking FLUX.1's artefacts or reducing the diversity of generated images of the unified MLLMs.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer BVqp,\n\nWe sincerely appreciate your time and effort in reviewing our work and providing valuable feedback! Below, we address your concerns point by point.\n\n---\n\n### **1. Template-based training prompts limitation**\n\nWe appreciate the reviewer's concern about scalability. In fact, our approach is more diverse than it may initially appear. We promised to add more details in our pipeline figure 8 under Appendix J. We employ a multi-stage augmentation strategy:\n\n1. **Prompt Generation**: Starting from 1,000 base templates, we use ChatGPT to generate semantically equivalent paraphrases, expanding our prompt set to over much more unique variations. For example, \"an associate professor\" is paraphrased into \"university faculty member,\" \"academic researcher,\" \"college instructor,\" etc.\n2. **Semantic Diversity**: This paraphrasing ensures our model encounters diverse linguistic expressions mapping to the same occupational concepts, preventing overfitting to specific phrasings and better simulating real-world prompt diversity.\n3. **Scalability**: Our pipeline is scalable - new occupations or domains can be added and automatically expanded through the same paraphrasing process. We demonstrate generalization through our evaluation on 564 stereotype prompts (line 247) that were not seen during training. These stereotype prompts covering diverse domains:\n- **Personal Attributes**: \"A face of a kind person,\" \"A face of an honest individual\"\n- **Daily Activities**: \"A face of a person folding freshly washed clothes\"\n- **Healthcare Contexts**: \"A face of an individual with diabetes\"\n- **Educational Settings**: \"A face of a sophomore\"\n\n---\n\n### **2. Synthetic FLUX data vs. real-world images**\n\nWe acknowledge the reviewer's valid concern about synthetic data artifacts. Our approach specifically addresses these limitations:\n\n1. **Necessity of Synthetic Data**: Based on our knowledge, there is no existing large scale dataset provides real-world occupational images balanced across both gender and race dimensions. Creating such a dataset manually would be prohibitively expensive and potentially introduce human annotator biases.\n2. **Widespread Practice**: Synthetic data is standard in training unified MLLMs - For example, Janus-Pro uses ~72M synthetic samples for image generation. Our approach follows established practices while adding specific bias mitigation.\n3. **Empirical Validation**: Our extensive evaluations demonstrate:\n    - Consistent bias reduction across different test sets (stereotype prompts, cross-language)\n    - Human evaluation confirming improved diversity with minimum degradation in quality\n    - Generalization to out-of-distribution prompts not seen during training\n4. **Future Work**: We will add to our limitations section that investigating mixed synthetic-real training data could further improve robustness.\n\n---\n\n### **3. Generalization of training data generation pipeline**\n\nThanks for the suggestion! In figure 8 under appendix J, we will expand our discussion of the data generation pipeline's generalization capabilities:\n\n1. **Pipeline Robustness**: Our approach explicitly controls for demographic attributes through augmented prompts (\"Indian associate professor,\" \"female construction worker\"), ensuring coverage across all demographic intersections. This enumeration strategy is model-agnostic and can work with any sufficiently capable image generator.\n2. **Balanced Pairing Strategy**: Each neutral prompt is paired with one image per demographic group, creating a balanced training distribution(see Figure 9 for some samples). This guarantee of balance is independent of the underlying generator's biases.\n3. **Generalization Evidence**:\n    - Cross-language evaluation (Chinese, French) shows consistent bias reduction\n    - Performance on unseen stereotype prompts demonstrates learned fairness principles as mentioned before\n\n---\n\n### **4. Core difference between BPO and standard DPO**\n\nThe fundamental distinction between our Balanced Preference Optimization (BPO) and standard Direct Preference Optimization (DPO) lies in objectives and mathematical formulations:\n\n**DPO**:\n\n- **Objective**: Maximize likelihood of human-preferred responses over rejected ones\n- **Formulation**: Binary preference p(y_preferred > y_rejected | x)\n- **Use case**: General alignment with human preferences\n\n**BPO**:\n\n- **Objective**: Enforce uniform generation probability across demographic groups\n- **Formulation**: Minimize pairwise odds ratio deviations: min Σ[log(1 + σ(log OR(y_di, y_dj)) - 0.5)²]\n- **Use case**: Designed for demographic fairness\n\n**Practical Difference**: While DPO optimizes a single preference direction (good vs bad), BPO simultaneously balances pairwise preferences to achieve demographic parity. DPO would require labeling images as \"preferred\" or \"rejected,\" which is subjective for preference. BPO directly optimizes for equal representation: the optimized model has no preference between generating demographic di or dj, achieving fairness without subjective preference labels. \n\nThis balancing addresses the unique challenges of fairness in generation, where the goal isn't to prefer one output over another, but to ensure no systematic preference exists across demographic groups.\n\n---\n\nWe sincerely thank the reviewer for these thoughtful feedback, which will substantially strengthen our paper's contribution and clarity!\n\nThe author"}, {"review_id": "viPftD8Axx", "reviewer": "Reviewer_GpU1", "summary": "This paper investigates demographic bias (specifically gender and race) in Unified Multimodal Large Language Models (U-MLLMs), which are capable of both visual understanding and generation. The authors first benchmark several state-of-the-art U-MLLMs, demonstrating that they exhibit significant demographic biases in their image generation outputs, often amplifying stereotypes present in the training data. To pinpoint the origin of this bias, the paper introduces a \"locate-then-fix\" framework. Through a series of audits, including linear probing of vision encoder embeddings and analyzing the token distribution of the language model, the authors hypothesize that the language model component is the primary source of generative bias. The authors also identify a \"partial alignment\" phenomenon, where models may appear less biased in text-based understanding tasks (e.g., VQA) but still produce demographically skewed images. To mitigate these issues, the paper proposes a two-stage debiasing method. First, it synthesizes a demographically balanced dataset using a text-to-image diffusion model. Second, it introduces a novel balanced preference optimization (BPO) loss function, inspired by Direct Preference Optimization (DPO), which encourages the model to assign equal generation probabilities across different demographic groups for neutral prompts. Extensive experiments on models like VILA-U show that the proposed BPO method significantly reduces gender, race, and intersectional bias while preserving, and in some cases improving, image quality and semantic fidelity.", "questions": "1. Regarding the \"Partial Alignment\" Phenomenon: You compellingly show that debiasing understanding (I2T finetuning) has only a marginal effect on debiasing generation. You connect this to the \"Reversal Curse\". Could this gap also be explained by the idea that the \"understanding\" and \"generation\" tasks operate on different levels of abstraction within the model? That is, understanding maps diverse inputs to a single concept, while generation maps a single concept to a specific, high-probability token sequence that reflects the training data's mode.\n\n2. On the Choice of Data Synthesizer: You use FLUX to generate the balanced dataset. Did you consider or experiment with other generators? Could the choice of generator significantly impact the effectiveness of the BPO fine-tuning, for instance, if a different generator had a stronger or weaker intrinsic bias for certain features correlated with demographics?", "limitations": "Please see the weakness.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. Timely and Significant Problem: The paper addresses the critical and timely issue of fairness in the latest generation of U-MLLMs. As these models become more prevalent, understanding and mitigating their potential to amplify societal biases is of paramount importance.\n2. Systematic \"Locate-then-Fix\" Approach: The diagnostic framework for localizing the source of bias is a major strength. Rather than treating the U-MLLM as a black box, the authors conduct separate audits of the vision and language components. The experiments using linear probing on image embeddings and Jensen-Shannon divergence on token distributions provide compelling evidence that the language model is the dominant source of generative bias.\n3. Comprehensive Experimental Evaluation: The paper's evaluation is thorough and robust.\n\nWeakness:\n1. Reliance on a Synthetic Dataset from a Biased Model: The proposed solution relies on generating a balanced dataset using the FLUX model. While the authors use an enumeration strategy to enforce demographic balance, the visual characteristics and potential subtle biases of the FLUX model itself are inherited by the training data. The paper acknowledges FLUX is not bias-free, but the downstream impact of using a potentially biased tool for debiasing could be explored further.\n2. Limited Scope of Demographic Attributes: The study focuses exclusively on gender (binary) and race. While this is a critical starting point, the paper acknowledges that other attributes like age, culture, religion, and disability are not addressed. The proposed BPO method requires discrete, enumerable demographic groups, and it is unclear how it would scale to more complex or continuous attributes.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer GpU1,\n\nWe sincerely appreciate your time and effort in reviewing our work and providing valuable feedback! Below, we address your concerns point by point.\n\n\n---\n\n### **1. Reliance on Synthetic Dataset from a Biased Model**\n\nWe appreciate this insightful critique. Our approach specifically addresses these limitations:\n\n**Bias Mitigation at Multiple Levels**:\n\n1. **Enumeration Strategy**: By explicitly generating images for each demographic combination, we prevent FLUX's implicit biases from affecting the demographic distribution in our dataset. Even if FLUX generates stereotypical features for certain demographics, our balanced pairing ensures equal representation(See pipeline in Figure 8 under appendix J and samples in Figure 9).\n2. **Loss Function Design**: Our BPO loss operates on token-generation probabilities. This means we're optimizing for equal likelihood of generating different demographics regardless of FLUX's visual biases. The model learns to be indifferent to demographic choices at the generation level.\n3. **Empirical Evidence of Robustness(see line 247)**\n    \n    **Cross-Domain**: 564 stereotype prompts beyond occupations show consistent bias reduction\n    \n    As mentioned briefly in line 247, we conduct extensive testing on **564 stereotype prompts** from the latest bias benchmark[1], covering diverse domains:\n    \n    - **Personal Attributes**: \"A face of a kind person,\" \"A face of an honest individual\"\n    - **Daily Activities**: \"A face of a person folding freshly washed clothes\"\n    - **Healthcare Contexts**: \"A face of an individual with diabetes\"\n    - **Educational Settings**: \"A face of a sophomore\"\n    \n    **Cross-Language**: Chinese and French evaluations maintain effectiveness.\n    \n    **Human Validation**: User studies confirm improved diversity with minimum quality degradation\n    \n\n---\n\n### **2. Limited Scope of Demographic Attributes**\n\nWe appreciate the reviewer's concern and acknowledged this in our limitation section in Appendix. Our method currently requires discrete categories. We adopt binary labels (\"male\"/\"female\") and category of race from the established benchmark [2] to ensure comparability. This indeed constrains its applicability:\n\n**Current Limitations**:\n\n- Binary gender categories exclude non-binary identities\n- Discrete racial categories cannot capture mixed-race individuals or continuous ethnic variations\n- Attributes like age, disability, and culture are not addressed\n\n**Challenges for Extension**:\n\n1. **Continuous Attributes**: For age, we could discretize into ranges (20-30, 30-40, etc.) but this loses granularity\n2. **High-Cardinality Attributes**: For culture/nationality (200+ categories), the pairwise loss becomes computationally expensive: O(K²) comparisons.\n\n**Possible Solutions such as** Group similar demographics hierarchically. We promise to add more discussing these details.\n\n---\n\n### **3. \"Partial Alignment\" Phenomenon and Levels of Abstraction**\n\nThis is an excellent insight that enriches our understanding of the phenomenon. \n\n- **Understanding (I→T)**: Maps diverse visual inputs → single abstract concept\n    - \"Male professor\" image → \"professor\" (many-to-one mapping)\n    - Operates at high semantic level, abstracting away visual details\n- **Generation (T→I)**: Maps abstract concept → specific visual instance\n    - \"Professor\" → specific image tokens (one-to-many mapping, different images)\n    - Sample from learned distribution\n\n**Connection to Our Findings**:\nThis abstraction-level difference explains why I→T finetuning fails to debias generation:\n\n1. During I→T training, the model learns to ignore demographic features (abstraction)\n2. During T→I generation, the model must still select specific features, defaulting to the highest-probability (biased) mode\n3. The \"Reversal Curse\" is thus a special case of this broader abstraction asymmetry\n\n**Implications**:\n\n- Debiasing needs to target the specific level where bias manifests\n- Understanding tasks may require different debiasing strategies than generation tasks\n\nReally appreciate the insights!\n\n---\n\n### **4. Choice of Data Synthesizer**\n\nWe appreciate this insightful suggestion. Before we generated dataset, we did experiment and evaluate multiple image generators for creating our balanced dataset:\n\n**Generator Comparison**:\n\n1. **FLUX.1-dev** (Selected):\n    - State-of-the-art at the time of our study\n    - Superior demographic control - reliably generates requested gender/race attributes\n    - Consistent high-quality outputs across most occupational categories(see samples in Figure 9)\n    - Stable generation with minimum failure cases\n2. **Stable Diffusion v1.5** (Evaluated but not selected):\n    - Less reliable demographic adherence - often ignored specific gender/race prompts\n    - Variable quality, particularly for certain occupations\n    - Failed to generate coherent images for some professional categories (e.g., produced low quality and wrong outputs for some occupations)\n\nOur approach would work with sufficiently capable image generator that can reliably produce demographic variations. We will add these details. Thank you for the suggestion!\n\n---\n\n**Reference**\n\n[1] Xu, C., et al. (2025). MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models. In *Proceedings of the International Conference on Learning Representations (ICLR 2025)*.\n\n[2] Shen, X., et al. (2024). *Finetuning Text-to-Image Diffusion Models for Fairness*. In *Proceedings of the International Conference on Learning Representations (ICLR 2024)*.\n\n---\n\nWe sincerely thank the reviewer for these thoughtful critiques, which will substantially strengthen our paper's contribution and clarity.\n\nThe author"}, {"review_id": "0m2I0vnwAu", "reviewer": "Reviewer_3uNx", "summary": "This paper presents an analysis and debiasing technique targeting social biases in unified multimodal large language models (U-MLLMs). Biases are analysed by probing the image tokens the model autoregressively generates and by comparing their distributions. For debiasing, the paper proposes a preference tuning method based on using synthetic images. Furthermore, the paper analyses biases in understanding and generation and their relation, and investigates additional loss formulations for debiasing.", "questions": "- Why does the evaluation only consider occupational prompts and not further topics as well?\n- Can we apply the debiasing method in scenarios where we aim for a skewed target distribution of attributes?\n- How were the race labels selected and why?", "limitations": "See Weaknesses.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n**(S1)** A targeted analysis of biases and debiasing techniques in U-MLLMs is a novel and very interesting perspective to study. This paper also clearly develops unique approaches that are different compared to existing approaches to studying social bias in Diffusion Models, for example the image token distribution and bias in generation vs. understanding.\n\n**(S2)** Using the image token distribution to assign demographic labels (instead of external models that predict this information from the generated images) seems like a great idea. This potentially allows for analyzing bias without even assigning explicit labels, which can be controversial [a]. Instead, bias analysis would rely on the distributions, similar to [b]. Further analyses of what the image tokens encode (e.g. are we really measuring gender representations in the tokens and not correlated attributes such as clothing spurious correlations in background patterns) and whether this technique could also be applied to other models that use VQ-VAEs (such as Diffusion Models) would be greatly appreciated.\n\n**(S3)** The paper discusses the relation of bias in understanding (text output) and generation (image output) scenarios. This is a unique and important area of investigation for U-MLLMs. Future work has to explore this relation and possible strategies to transfer bias mitigation between modalities further, but the conducted experiments are a good start.\n\n**(S4)** The paper contains a human evaluation of debiasing success and ablations for a number of loss formulations. These additional experiments are appreciated.\n\n## Weaknesses\n**(W1)** The evaluation focuses on occupational prompts. While this is standard in bias analysis and debiasing literature, a greater variety of prompts would strengthen the validity of the present analysis. The experimental setup doesn't have to be limited to occupations. The main advantage of using occupation is that it allows easy exploration of bias amplification by comparing the resulting proportions (e.g. of male-/female-gendered images) to real-world statistics, but the present paper does not make use of this. Therefore, other prompt topics can be integrated without causing any problems.\n\n**(W2)** The paper only considers parity (uniform distribution over attributes) as debiasing target. However, this is a strong limitation. In practice, it should be possible to adapt models to arbitrary target distributions, as there is no consensus on whether uniform distribution should be the standard. For example, the \"Gemini incident\" (where a proprietary model produces images of Vikings with dark skin or images of nazi soldiers with Asian features) demonstrated that uniform distributions do not make sense in many contexts. The present paper would benefit from adding a discussion on how the proposed methods can be used to adapt to arbitrary sampling distributions and how to maintain contextual appropriateness.\n\n**(W3)** The limitations section does not include a discussion of the chosen gender/race categories. However, it would be greatly appreciated if the paper could include a discussion on potential problems with discrete labels for categories that inherently lie on a spectrum. Especially, race is a problematic case, as there is little consensus on which are appropriate labels (see [a|).\n\n## Typos and Minor Weaknesses\n* Line 60: It is unclear here what the bias values mean, as they haven't been introduced yet\n* Line 463: The related work section mentions \"few studies\" that explored bias in U-MLLMs, but cites none. This asks whether after all there are any previous studies on this or not.\n\n## Justification for Rating\nThe evaluation is limited, focusing on occupational prompts only. Also, the debiasing methods targets uniform distribution over attributes and not arbitrary target distributions. Nonetheless, the value of providing solid insights and unique methods to the study of social bias in U-MLLMs, which is a clearly novel area of research, outweighs my concerns regarding the evaluation. I would like to encourage development of novel evaluation and debiasing approaches for this relevant type of image generation, and this paper provides a strong starting point.\n\n## References\n[a] Khan & Fu (2021): One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision. In FAccT\\\n[b] Luccioni et al. (2023): Stable Bias: Evaluating Societal Representations in Diffusion Models. In NeurIPS", "was_revised": true, "final_justification": "The presented analysis of biases and debiasing techniques in U-MLLMs is both novel and interesting. It is especially interesting that the paper develops methods specifically targeted at U-MLLMs. All remaining concerns were addressed in the rebuttal, and all reviewers expressed a positive evaluation of this paper.", "rebuttal": "Dear Reviewer 3uNx,\n\nWe sincerely appreciate your time and effort in reviewing our work and providing valuable feedback! Below, we address your concerns point by point.\n\n\n---\n\n### **(W1) Limited evaluation to occupational prompts**\n\nWe appreciate the reviewer's observation and want to clarify that our evaluation extends  beyond occupational prompts. As mentioned briefly in line 247, we conduct extensive testing on **564 stereotype prompts** from the latest bias benchmark[3], covering diverse domains:\n\n- **Personal Attributes**: \"A face of a kind person,\" \"A face of an honest individual\"\n- **Daily Activities**: \"A face of a person folding freshly washed clothes\"\n- **Healthcare Contexts**: \"A face of an individual with diabetes\"\n- **Educational Settings**: \"A face of a sophomore\"\n\nThese non-occupational prompts constitute a substantial portion of our evaluation and demonstrate consistent bias reduction across contexts. We will add more details to better highlight these results and show that our method generalizes beyond occupations. We agree that occupational bias analysis often compares to real-world statistics—while we don't perform this comparison in this work, our broader evaluation validates the method's general applicability.\n\n---\n\n### **(W2) Limitation to uniform distribution target**\n\nThis is an excellent point that touches on fundamental questions about fairness definitions! We acknowledge that uniform distribution is indeed one specific solution of fairness, not a universal solution.\n\n**Current Approach**: Our BPO loss minimizes pairwise odds ratios to achieve demographic parity, which aligns with widely-used fairness metrics in the literature and our evaluation benchmark[1].\n\n**Extension to imbalanced Distributions**: Our framework might be adapted to non-uniform targets by modifying the loss function. Instead of having |OR(y_di, y_dj) - 1| in our loss function, we could have |OR(y_di, y_dj) - τ_ij| where τ_ij represents the desired ratio between demographics di and dj.  We will add discussion addressing:\n\n1. The Gemini incident as a cautionary example of context-blind uniformity\n2. The trade-off between fairness and contextual accuracy\n\n---\n\n### **(W3) Missing discussion on categorical limitations**\n\nWe fully agree that our use of discrete demographic categories is a significant limitation that warrants thorough discussion. We will add more details in Limitations:\n\n**Gender Categories**: We adopt binary labels (\"male\"/\"female\") from the established benchmark [1] to ensure comparability. However, we explicitly acknowledge:\n\n- These represent **perceived presentation** in images, not gender identity\n- This binary framework excludes non-binary and gender-diverse individuals\n\n**Race Categories**: Our four-group taxonomy (WMELH, Asian, Black, Indian) directly follows [1]:\n\n- **WMELH**: {White, Middle Eastern, Latino/Hispanic}\n- **Asian**: {East Asian, Southeast Asian}\n- **Black**: African and African diaspora\n- **Indian**: South Asian\n\nWe acknowledge this categorization:\n\n- Conflates distinct ethnicities (problematic as noted in [2])\n- Reflects Western-centric racial constructs\n- Cannot capture mixed-race individuals\n\nWe will emphasize that we use these categories solely for benchmark compatibility while recognizing their fundamental limitations in representing human diversity.\n\n---\n\n### **Minor Issues**\n\nThanks for the suggestions!\n\n**Line 60**: We will add a parenthetical explanation: \"where 0 indicates perfect balance and 1 indicates complete skew toward one gender\".\n\n**Line 463**: We will correct this by Revising the sentence. \n\n---\n\n### **Response to Questions**\n\n**Q1: Why only occupational prompts?**\n\nAs clarified above under w1, we actually test on 564 diverse prompts beyond occupations. We will restructure our presentation to make this clearer. Occupations serve as our primary training domain because they're well-studied in bias literature, but our evaluation demonstrates broader applicability.\n\n**Q2: Skewed target distributions?**\n\nYes, our method might be adapted. The balanced preference loss could be modified to target imbalanced distribution by replacing the uniform odds ratio target (OR=1) with custom ratios. Specifically:\n\n```\nL_custom = Σ log[1 + σ(log OR(y_di, y_dj) - log τ_ij) - 0.5]²\n\n```\n\nwhere τ_ij encodes the desired probability ratio p(di)/p(dj). We will add this in discussion.\n\n**Q3: Race label selection?**\n\nWe inherit the exact categorization from the established fairness benchmark [1] to ensure consistence. We recognize the limitations of these categories (as discussed in W3). We will add a thorough discussion of these limitations and encourage future work to develop more nuanced demographic representations.\n\n---\n\n**Reference**\n\n[1] Shen, X., et al. (2024). *Finetuning Text-to-Image Diffusion Models for Fairness*. In *Proceedings of the International Conference on Learning Representations (ICLR 2024)*.\n\n[2] Khan & Fu (2021): One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision. \n\n[3] Xu, C., et al. (2025). MMDT: Decoding the Trustworthiness and Safety of Multimodal Foundation Models. In *Proceedings of the International Conference on Learning Representations (ICLR 2025)*.\n\n---\n\nWe sincerely thank the reviewer for these thoughtful critiques, which will substantially strengthen our paper's contribution and clarity.\n\nThe author"}, {"review_id": "kuSo5yE2l0", "reviewer": "Reviewer_iJBG", "summary": "The authors focus their work on identifying sources of bias within a unified multimodal large language model (U-MLLMs), and implementing a “locate-then-fix” framework. The potential sources are hypothesised to have originated in the Vision Encoder and/or the Language Model. The findings presented show that the bias is likely from the language component, primarily. This is addressed with two stage debiasing approach:\n\n1. Fine-tuning the model on a balanced dataset\n2. Applying a balanced preference loss inspired by direct preference optimisation\n\nDemographic bias mitigation is achieved while preserving semantic fidelity and image quality. This is demonstrated with improved fairness (as defined by the generation of a balanced output) while preserving the quality of the image generation.", "questions": "The first set of questions relate to the lack of transparency and disclosure of the use of humans in the evaluation process:\n1. Why have the authors indicated that details related to the involvement of human annotators is inapplicable in the Checklist?\n2. Given the involvement of annotators, and therefore, human participation, why was IRB approval not sought?\n\nThe second set of questions relate to a known phenomenon of bias stacking and amplification [1]\n\n3. Given the known concerns of bias stacking and amplification - where models amplify biases from “upstream”, is it enough to only fix the biases in either the vision encoder or the language model?\n\n**References:**\n\n[1] Hall, Melissa, et al. \"A systematic study of bias amplification.\" arXiv preprint arXiv:2201.11706 (2022).", "limitations": "1. Have the authors reflected on the limitations of the policy based on human preferences and how this might impact outcomes?\n\n2. There are also known fairness issues when fine-tuning on balanced data that should be another point of reflection for the authors. \n\n3. This work makes use of synthetic data, and imposes a balanced distribution as the “fair” outcome. Have the authors considered the implications of this with specific regards to unintended biases and distribution shifts [5]? \n\n*Recommended reading:*\n\n[1] Section 2.2. \n\n[2] Section 6\n\n[3] generally \n\n4. I believe there are certain areas where additional biases can be introduced, and the authors should reflect carefully on these:\n\n4a. Positionality of the authors\n4b. Chosen categories, and the context in which these are relevant, and by extension how this might impact the interpretation of the outputs, and unintended biases\n\n**References:**\n\n[1] Berg, Hugo, et al. \"A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning.\" arXiv preprint arXiv:2203.11933 (2022).\n\n[2] Smith, Brandon, et al. \"Balancing the picture: Debiasing vision-language datasets with synthetic contrast sets.\" arXiv preprint arXiv:2305.15407 (2023).\n\n[3] Blodgett, Su Lin, et al. \"Stereotyping Norwegian salmon: An inventory of pitfalls in fairness benchmark datasets.\" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.\n\n[4] Schrouff, Jessica, et al. \"Mind the Graph When Balancing Data for Fairness or Robustness.\" arXiv preprint arXiv:2406.17433 (2024).\n\n[5] Schrouff, Jessica, et al. \"Diagnosing failures of fairness transfer across distribution shift in real-world medical settings.\" Advances in Neural Information Processing Systems 35 (2022): 19304-19318.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n\nWell motivated paper, with a clear identification of a potential source of unfairness in  models that are under-investigated - there is a known issue of bias stacking and amplification when grouping models in a system.\n\n# Weaknesses\n\nI have outlined some concerns below under “Questions” and Limitations. While it is commendable that the authors are pursuing a project in fairness, I’d like to use this space to raise a concern about the vocabulary used in the paper to refer to sensitive attributes.\n\n1. The authors are advised to carefully consider their use of gendered and racial language throughout the paper. The use of 'female' and 'male'' carry very specific assumptions and can be seen as problematic in the scope of gender recognition. One should be very careful about what is being classified and within which hegemony one makes these assumptions. Specifically, the authors are classifying attributes of a perceived gender presentation and cannot make assumptions Additionally, careful disclaimers about purely focussing on binary gender (assuming within a Western hegemony of gender, with a strong motivation for this approach are important to maintain ethical standards. See [1] for an example of this, and I strongly suggest the authors review the suggestions for further reading\n2. Further, the use of discrete racial categories is also limited and the authors are encouraged to carefully consider the contexts within these categories are relevant, and the implications of these discrete categories\n\n\n## Minor comments\n1. This is a nitpick, but consider ordering the mini batches of references chronologically \n2. Typo on L216. It should potentially read “..., we include a diffusion model”\n3. Typo in the caption of Figure 4: should potentially read “*White*, Asian, Black of Indian\n\n**References:**\n\n[1] Hall, Siobhan Mackenzie, et al. \"Visogender: A dataset for benchmarking gender bias in image-text pronoun resolution.\" Advances in Neural Information Processing Systems 36 (2023): 63687-63723.\n\n\n**Suggested further reading:**\n- [Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies](https://aclanthology.org/2021.emnlp-main.150.pdf)\n- [You Keep Using That Word: Ways of Thinking about Gender in Computing Research](https://ironholds.org/resources/papers/gender_multiplicity.pdf)\n- [VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution](https://arxiv.org/abs/2306.12424)\n- [The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition](https://dl.acm.org/doi/10.1145/3274357)", "was_revised": true, "final_justification": "The authors were open to the comments in the rebuttal and showed a commendable engagement with the feedback. I believe this new score reflects the technical contributions, in line with my own assessment and the comments made by other reviewers.", "rebuttal": "Dear Reviewer iJBG,\n\nWe sincerely appreciate your time and effort in reviewing our work and providing valuable feedback! Below, we address your concerns point by point.\n\n\n---\n\n### **Weaknesses**\n\n**1. Gendered/Racial Terminology**\n\nWe deeply appreciate the reviewer's critical insight regarding our use terminology. This is indeed a fundamental concern that we take very seriously.\n\nTo clarify our approach: We adopt the exact binary \"male\" and \"female\" labels from the established text-to-image diffusion fairness benchmark [1] to ensure consistence. However, we fully acknowledge that these terms represent **perceived gender presentation** in generated images rather than gender identity, and this distinction is crucial.\n\nWe will add a disclaimer and modify our limitation section to acknowledge how binary classification can ignore gender-diverse individuals\n\n---\n\n**2. Discrete Racial Categories**\n\nWe acknowledge the reviewer's concern about the limitations and potential harms of discrete racial categorization. Our racial category is directly inherited from the benchmark [1]:\n\n- **WMELH** (White, Middle Eastern, Latino/Hispanic)\n- **Asian** (East Asian, Southeast Asian)\n- **Black**\n- **Indian** (South Asian)\n\nWe recognize several critical issues with this taxonomy:\n\n- It reflects Western-centric constructions of race that may not be meaningful in other contexts\n- It cannot represent mixed-race individuals\n\nIn our revision, we will:\n\n- Add a note explaining we use these categories solely for benchmark compatibility\n- Acknowledge that these categories are social constructs with limited validity\n\n---\n\n### **Questions**\n\n**1. Human Annotators Checklist Error**\n\nWe sincerely apologize for this oversight in our checklist. To provide complete transparency:\n\nOur human evaluation involved only 3 adult participants who:\n\n- Were recruited as volunteers and provided informed verbal consent to participate\n- Were asked to: (a) provide occupation-based text prompts, and (b) rate generated images on a 1-5 scale for demographic diversity and image quality\n- Were not exposed to any harmful or sensitive content (only AI-generated professional portraits)\n\nWe mistakenly marked this as \"N/A\" because we initially considered this minimal-risk evaluation of our model's outputs rather than formal human subjects research. We now recognize this was an error in judgment, and any human involvement should have been properly disclosed.\n\n---\n\n**2. IRB Approval**\n\nWe acknowledge the reviewer's valid concern about the lack of formal IRB review. Our reasoning at the time was:\n\n- The evaluation involved minimal risk (3 participants only viewed and rated AI-generated images)\n- The task was similar to routine user feedback on model outputs and no personal data was collected beyond ratings\n- Participants were colleagues who volunteered with full knowledge of the research purpose\n\nHowever, we now recognize that this reasoning was insufficient. We apologize for this oversight. We commit to adding explicit statements about informed consent and data handling. \n\n---\n\n**3. Bias Amplification**\n\nThis is an excellent point about cascading bias effects. Our investigation revealed that the language model was the primary source of observed bias. Our current results show:\n\n- Linear probing of vision encoder embeddings achieves high accuracy in demographic classification, confirming these attributes are encoded\n- However, when we compare token distributions between neutral and demographic-augmented prompts, the language model shows 99.80% alignment with its implicit biases\n- After applying BPO to the language model alone, overall generation bias drops significantly\n\nThis suggests that while the vision encoder preserves demographic information, the language model's sampling process is where biased preferences manifest most strongly.\n\nNevertheless, we acknowledge potential interaction effects:\n\n- The vision encoder's representation might \"prime\" certain token sequences\n- Subtle biases in visual features could compound with language model biases\n- Complete debiasing might require addressing both components simultaneously\n\nWe will expand our discussion to include these nuances and cite relevant work suggested.\n\n---\n\n### **Limitations**\n\n**1. Policy Based on Human Preferences**\n\nWe acknowledge that our assumption of uniform distribution as \"fair\" encodes specific value judgments. This approach:\n\n- Aligns with common fairness metrics in literature (demographic parity)\n- Matches the evaluation protocol of our benchmark [1]\n- Received positive feedback in our (limited) human evaluation\n\nHowever, we recognize this is just one possible operationalization of fairness:\n\n- It may not suit all contexts\n- Different communities might prefer proportional representation matching local demographics\n- Some applications might require context-dependent fairness criteria\n\nWe will incorporate those paper suggested such as[2][3] and insights about the ethical implications.\n\n---\n\n**2. Fine-Tuning on Balanced Data Issues**\n\n We acknowledge that balanced training data may not ensure absolute fairness. Issues might be:\n\n- **Unnatural associations**: Equal representation might create unrealistic scenarios\n- **Downstream amplification**: Balanced training doesn't prevent bias reemergence during deployment\n\nOur approach partially addresses these through the balanced preference loss, which:\n\n- Operates on generation probabilities rather than data frequencies\n- Penalizes any systematic preference regardless of training distribution\n\nThis means we're optimizing for equal likelihood of generating different demographics. The model learns to be indifferent to demographic choices at the generation level.\n\nHowever, we acknowledge residual risks might remain, and we will add discussion of these nuanced fairness considerations and cite relevant work suggested.\n\n---\n\n**3. Synthetic Data and Distribution Shifts**\n\nWe appreciate the reviewer's concern about potential artifacts from synthetic data. Our approach addresses these challenges through design and validation:\n\n**Why Synthetic Data is Necessary**:\n\n- **Data Scarcity**: No existing large-scale dataset provides occupational images balanced across gender × race intersections. Creating one would require photographing thousands of individuals across all demographic combinations—prohibitively expensive and prone to human annotator biases.\n- **Controlled Distribution**: Synthetic generation enables perfect demographic balance (exactly one image per demographic per prompt), which is very challenging with real-world collection.\n- **Standard Practice**: Latest U-MLLMs like Janus-Pro incorporate ~72M synthetic samples. We follow this established approach while adding targeted bias mitigation.\n\n**How We Mitigate Synthetic Artifacts**\n\n1. **BPO Design**: Our balanced preference loss operates on token-generation probabilities. This abstraction layer insulates the debiasing objective from FLUX-specific visual patterns—we optimize for equal likelihood of generating different demographics.\n2. **Enumeration Strategy**: By explicitly generating each demographic variant, we prevent FLUX's implicit biases from skewing the training distribution. Even if FLUX renders certain groups with characteristic styles, our balanced pairing ensures equal representation. **(see figure 8 and 9 under Appendix J)**\n\n**Empirical Evidence of Robustness(see line 247)**\n\n- **Cross-Domain**: 564 stereotype prompts beyond occupations show consistent bias reduction\n- **Cross-Language**: Chinese and French evaluations maintain effectiveness \n- **Human Validation**: User studies confirm improved diversity with minimum quality degradation\n\nWhile our approach proves effective, we recognize:\n\n- FLUX's aesthetic choices may subtly influence generated styles\n- Synthetic-to-real domain gap could affect deployment\n- Unknown biases in FLUX's training data might propagate\n\nWe will add to our limitations section that investigating mixed synthetic-real training data could further improve robustness.\n\n---\n\n**4. Additional Bias Sources**\n\n**4a. Author Positionality**:\nWe acknowledge that our perspectives as researchers inevitably shape this work:\n\n- Our choice to focus on gender and race (rather than age, disability, etc.)\n- Our interpretation of \"fairness\" as demographic parity\n- Our methods for evaluation and validation from prior work[1]\n\nThese choices reflect our academic training and the specific research context we operate within. Other communities might approach these questions entirely differently.\n\n**4b. Category Context and Implications**:\nThe choice of gender/race as primary axes of analysis:\n\n- Following standard in bias analysis literature[1]\n- May inadvertently suggest these are the only or most important biases\n- Could ignore intersectional effects or other forms of discrimination\n\nWe will add explicit acknowledgment that our categories are contextual and fairness extends  beyond the specific metrics we optimize.\n\n---\n\n**Ethical Concerns:** We want to assure that participants were treated ethically: they volunteered for the task and were not exposed to any harmful content or risk. \n\n---\n\n**Paper Formatting:**  We appreciate the suggestion. In revised version, We will fully integrate Related Work and Limitations into main text.\n\n---\n\n**Reference**\n\n[1] Shen, X., et al. (2024). *Finetuning Text-to-Image Diffusion Models for Fairness*. In *Proceedings of the International Conference on Learning Representations (ICLR 2024)*.\n\n[2] Berg, Hugo, et al. \"A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning.\" arXiv preprint arXiv:2203.11933 (2022).\n\n[3] Smith, Brandon, et al. \"Balancing the picture: Debiasing vision-language datasets with synthetic contrast sets.\" arXiv preprint arXiv:2305.15407 (2023).\n\n---\n\nWe hope these clarifications address your concerns. We are grateful for your valuable suggestions and welcome any further feedback!\n\nThe author"}]

Author rebuttal:

Meta-review: This paper investigates demographic bias (gender and race) in Unified Multimodal Large Language Models (U-MLLMs) and benchmarks several state-of-the-art U-MLLMs. Overall, the paper is clear and easy-to-follow. The paper addresses the critical and timely issue of fairness in the latest generation of U-MLLMs. All the reviewers lean to accept the paper. Therefore, I recommend accept. I also recommend the authors to follow the reviewers' comments, e.g, update the "Gendered/Racial Terminology".
Decision: Accept (poster)