UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddings
Authors: name: Zhibin Lan; name: Liqiang Niu; name: Fandong Meng; name: Jie Zhou; name: Jinsong Su
Published: 2025-11-01 05:04:23

Abstract: The remarkable success of multimodal large language models (MLLMs) has driven
advances in multimodal embeddings, yet existing models remain inherently
discriminative, limiting their ability to benefit from reasoning-driven
generation paradigm. In this work, we pioneer the exploration of generative
embeddings, unifying embedding tasks within a generative paradigm. We propose
UME-R1, a universal multimodal embedding framework consisting of a two-stage
training strategy: a cold-start supervised fine-tuning equips the model with
reasoning capabilities and enables it to generate both discriminative and
generative embeddings; a subsequent reinforcement learning enhances reasoning
and further optimizes generative embedding quality. This pioneering work
reveals four key insights: 1) generative embeddings unlock substantial
performance gains over conventional discriminative embeddings by leveraging the
powerful generative reasoning capabilities of MLLMs; 2) discriminative and
generative embeddings are complementary, whose combined oracle performance far
exceeding that of either alone; 3) RL can effectively enhance generative
embeddings, establishing a scalable optimization paradigm.; 4) repeated
sampling at inference boosts downstream task coverage (pass@k), highlighting
the inference-time scalability potential of generative embeddings. Evaluated on
the MMEB-V2 benchmark across 78 tasks spanning video, image, and visual
documents, UME-R1 significantly outperforms conventional discriminative
embedding models and offers a foundation for more interpretable,
reasoning-driven generative multimodal embeddings. Our code, models, and
datasets will be publicly available at https://github.com/XMUDeepLIT/UME-R1.