Y in Y ang-Align: A new Benchmark for Competing Objectives and Introducing Multi-Objective Preference based Text-to-Image Alignment (2025)

Abstract: Precise alignment in Text-to-Image (T2I) systems is crucial for generating visuals that reflect user intent while adhering to ethical and policy standards. Recent controversies, such as the Google Gemini-generated Pope image backlash, highlight the urgent need for robust alignment mechanisms. Building on alignment successes in Large Language Models (LLMs), this paper introduces YinYangAlign, a benchmarking framework designed to evaluate and optimize T2I systems across six inherently contradictory objectives. These objectives highlight core trade-offs, such as balancing faithfulness to prompts with artistic freedom and maintaining cultural sensitivity without compromising creativity. Alongside this benchmark, we propose the Contradictory Alignment Optimization (CAO) framework, an extension of Direct Preference Optimization (DPO), which employs multi-objective optimization techniques to address these competing goals. By leveraging per-axiom loss functions, synergy-driven global preferences, and innovative tools like the Synergy Jacobian, CAO achieves superior alignment across all objectives. Experimental results demonstrate significant improvements in fidelity, diversity, and ethical adherence, setting new benchmarks for the field. This work provides a scalable, effective approach to resolving alignment challenges in T2I systems while offering insights into broader AI alignment paradigms.

## yinyangalign: dataset and annotation we have adopted a hybrid annotation pipeline for the development of yinyangalign benchmark. ## contradictory alignment optimization the yinyangalign framework models the challenge of balancing inherently contradictory objectives. for instance, prioritizing faithfulness to prompt can restrict artistic freedom, while emphasizing emotional impact may reduce neutrality. to address these conflicts, we propose contradictory alignment optimization (cao), which employs a per-axiom loss design to explicitly capture competing goals. cao dynamically weights sub-objectives within each axiom, enabling fine-grained trade-offs and adaptive optimization. additionally, it integrates pareto optimality with the bradley-terry preference model, introducing a global synergy mechanism that unifies contradictory objectives into a cohesive strategy. this multi-objective synergy is the core innovation of cao, distinguishing it from existing t2i alignment methods. ## axiom-specific loss function design we now expand each of the axiom-wise losses introduced previously: l artistic , l faith , l emotion , l neutral , l originality , l referentiality , l verifiability , l cultural . l artistic . note that l artistic appears in four of the six axioms, but the core design of the artistic loss remains consistent across all such instances. cf appendix l. ## empirical evaluation evaluation setup and insights: we assess the limitations of optimizing dpo models for individual alignment objectives by training six models, each targeting a single axiom: artistic freedom, faithfulness to prompt, emotional impact, originality, cultural sensitivity, and verifiability. the broader impact is measured by evaluating how optimizing one axiom affects the remaining five, with percentage changes reported relative to a baseline (fig. 5 ). ## conclusion we introduce yinyangalign, a benchmark for evaluating t2i systems across six contradictory alignment objectives, highlighting key trade-offs in ai image generation. to address these conflicts, we proposed contradictory alignment optimization (cao), a novel extension of dpo that integrates synergy-driven global preferences, axiom-specific regularization, and the synergy jacobian. using sinkhorn-regularized wasserstein distance, cao ensures stability and scalability, achieving state-of-the-art performance across all objectives. empirical results validate cao's effectiveness in balancing competing alignment goals. yinyangalign serves as a critical resource for advancing fairness, creativity, and cultural sensitivity in generative ai. future work will explore dynamic weight tuning and extend the framework to emerging alignment challenges.