PairUni: Pairwise Training for Unified Multimodal Language Models
Authors: name: Jiani Zheng; name: Zhiyang Teng; name: Xiangtai Li; name: Anran Wang; name: Yu Tian; name: Kunpeng Qiu; name: Ye Tian; name: Haochen Wang; name: Zhuochen Wang
Published: 2025-10-29 16:47:02

Abstract: Unified vision-language models (UVLMs) must perform both understanding and
generation within a single architecture, but these tasks rely on heterogeneous
data and supervision, making it difficult to balance them during reinforcement
learning (RL). We propose PairUni, a unified framework that reorganizes data
into understanding-generation (UG) pairs and aligns optimization accordingly.
We first use GPT-o3 to augment single-task data, generating captions for
understanding samples and question-answer (QA) pairs for generation samples,
forming aligned pairs from the same instance. Additionally, for each generation
sample, we retrieve a semantically related understanding example to form a
retrieved pair, linking different but related data points. These paired
structures expose cross-task semantic correspondences and support consistent
policy learning. To leverage this structure, we present Pair-GPRO, a pair-aware
variant based on Group Relative Policy Optimization. It assigns a similarity
score to each pair to modulate the advantage, strengthening learning from
well-aligned examples and reducing task interference. We curate a high-quality
dataset of 16K UG pairs named PairUG for RL fine-tuning and evaluate PairUni on
the powerful Janus-Pro UVLMs. Our approach achieves balanced improvements on
various UVLMs, outperforming strong UVLM RL baselines. Code:
https://github.com/Haochen-Wang409/PairUni{github.com/Haochen-Wang409/PairUni}