Title: MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO
Venue: NeurIPS 2025
Accepted: yes
Keywords: Unified MLLM

Reasoning Generation

Reinforcement Learning
Abstract: Recent text-to-image systems face limitations in handling multimodal inputs and complex reasoning tasks.
We introduce MindOmni, a unified multimodal large language model that addresses these challenges by incorporating reasoning generation through reinforcement learning. MindOmni leverages a three-phase training strategy: i) design of a unified vision language model with a decoder-only diffusion module, ii) supervised fine-tuning with Chain-of-Thought (CoT) instruction data, and iii) our proposed Reasoning Generation Policy Optimization (RGPO) algorithm, utilizing multimodal feedback to effectively guide policy updates.
Experimental results demonstrate that MindOmni outperforms existing models, achieving impressive performance on both understanding and generation benchmarks, meanwhile showcasing advanced fine-grained reasoning generation capabilities,  especially with mathematical reasoning instruction. All codes will be made public.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents MindOmni, a unified multimodal large language model that enhances reasoning generation in vision-language models via the proposed Reasoning Generation Policy Optimization (RGPO) algorithm. It employs a three-phase training strategy, combining a unified vision-language model with a decoder-only diffusion module, supervised fine-tuning with Chain-of-Thought data, and RGPO to leverage multimodal feedback for policy updates. Experiments demonstrate that MindOmni achieves impressive performance on both understanding and generation benchmarks, particularly in mathematical reasoning and spatial-temporal tasks, demonstrating its advanced fine-grained reasoning generation and image editing capabilities. The work advances the field by enabling deeper reasoning-driven generation in unified multimodal models, bridging the gap between vision understanding and complex reasoning tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n* This paper is clearly written and easy to follow.\n* The core contribution of this work lies in the application of a class of policy gradient reinforcement learning algorithms—specifically, the proposed RGPO—to unified multimodal generation models. By explicitly modeling and optimizing the CoT reasoning process, the authors provide a strong baseline for enhancing the controllability and complex instruction-following abilities of multimodal models. Furthermore, this approach offers potential insights for the development of o4-like models and world models.\n\n**Weaknesses**\n* The consistency reward based on CLIP similarity represents a major limitation of RG-VLM. Extensive prior work has demonstrated that CLIP performs poorly in evaluating complex spatial, physical, or logical relationships. For example, while CLIP can recognize objects within an image, it often fails to verify the correctness of their spatial arrangements. As a result, the reward signal serves as a very coarse and unreliable proxy for reasoning ability. An unhackable and trustworthy reward is arguably the most crucial factor for RLVR training effectiveness and stability.\n\n* In the critical second stage of the pipeline, the paper uses another model (Qwen2.5-VL) to synthesize CoT data, yet provides no description of its quality control, diversity assurance, or bias filtering mechanisms. This raises concerns that the model’s reasoning ability may be inherently limited by the teacher model’s upper bound, and that it may end up merely imitating the specific \"thinking style\" of the teacher model rather than acquiring generalized reasoning skills. In current industry practice, there are primarily two cold-start strategies: distillation or zero-shot self-bootstrapping followed by filtering. If the CoT data is generated by a model lacking genuine reasoning capacity, it is unlikely to exhibit robust reasoning patterns.\n\nQUESTIONS:\n* The claim that the model can \"accurately play chess\" is a particularly strong statement. However, the paper presents no experimental results, evaluation protocols, or qualitative/quantitative evidence to support this assertion. Additional experiments and analyses are needed to substantiate this claim.\n\n*  Policy collapse, entropy collapse, and reward collapse are common issues during RLVR training. Although the authors propose a dual KL-divergence regularization technique as an interesting stabilization method, the paper lacks an in-depth analysis of its effectiveness, such as a thorough comparison with standard single-KL regularization. Moreover, the phenomenon of \"overthinking\" revealed in the results is not discussed in the context of existing literature.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper introduces RG-VLM, an MLLM model that relies on reinforcement learning in visual reasoning generation process. The authors propose Reasoning Generation Policy Optimization (RGPO) algorithm to effectively guide multimodal policy update. RG-VLM is compared with several counterpart approaches, showing its clear reasoning advantage.

STRENGTHS AND WEAKNESSES:
Strengths:
1. Introduction of RG-VLM, and MLLM dedicated to solving visual reasoning tasks and showing strong performance across diverse benchmark datasets.
2. Proposal of RGPO algorithm that has proven very effective in the Chain-of-Thought reasoning. 
3. The research is very well motivated and the main ideas are clearly presented.

Weaknesses:
1. The experimental evaluation completely omits abstract visual reasoning benchmarks, i.e. Raven’s Progressive Matrices, Bongard Problems, Visal Analogy Problems, and other.
2. A discussion on RG-VLM limitations is extremely brief.

QUESTIONS:
1. Are there any other limitations of RG-VLM besides the one mentioned in Appendix A3?
2. It will be interesting to see how effective would RG-VLM be in solving „classical” abstract reasoning tasks, such as Raven’s Progressive Matrices or Bongard Problems.
3. How the RG-VLM Chain-of-Thought reasoning conceptually differs from that of DeepSeek?

LIMITATIONS:
Discussion of limitations is extremely brief (just one sentence).
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes RG-VLM, a unified VLM.  RG-VLM attaches an OmniGen diffusion decoder to a Qwen-2.5 backbone via lightweight connectors and fine-tunes it with Reasoning-Generation Policy Optimisation (RGPO): an RL scheme that extends GRPO with a CLIP-based “consistency” reward and dual KL regularisers. RG-VLM delivers SoTA or near-SoTA scores on six benchmarks spanning multimodal understanding (MMMU, MMBench, RealWorldQA) and reasoning-centric generation (GenEval, DPG-Bench, WISE), while preserving baseline comprehension performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Brings chain-of-thought (CoT) reasoning into image generation/editing, advancing multimodal agent capabilities.\n- RGPO’s extension over GRPO is well documented; ablations (Tables 6-9) isolate each reward/regulariser’s effect.\n- Six quantitative benchmarks plus qualitative comparisons with GPT-4o and Gemini 2.5 demonstrate versatility.\n- Appendices list datasets, hyper-parameters, and training stages, making replication easier.\n\nWeaknesses:\n- RGPO adds a CLIP reward and dual KL loss to GRPO; similar CLIP-guided RL appeared in SimpleAR (2025), not much has changed conceptually.\n- All alignment metrics are CLIP-based; no user study confirms that generated images faithfully reflect the model’s textual reasoning.\n- Consistency reward is computed on low-resolution latents, potentially encouraging adversarial CLIP features rather than genuine semantics.\n- The number of available examples is limited. Including more would help reinforce the paper's claims.\n\nQUESTIONS:\nThe output images from RG-VLM appear to have simpler backgrounds, saturated colors, and reduced aesthetic appeal. Could this be due to the reward design? What might improve this?\n\nLIMITATIONS:\nThe limitations described in the paper seem more like future work. Insufficient attention is paid to the real limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces RG-VLM, a unified multimodal large language model that advances reasoning generation in vision-language tasks by integrating reinforcement learning via the Reasoning Generation Policy Optimization (RGPO) algorithm. The approach involves a three-phase training strategy: the creation of a decoder-only diffusion-based unified architecture, supervised fine-tuning using Chain-of-Thought (CoT) instruction data, and further training via RL with multimodal feedback rewards. RG-VLM demonstrates strong performance in both image understanding and generation tasks, outperforming recent unified models on established benchmarks, and excels on complex reasoning and editing scenarios. The paper includes empirical evidence from several public benchmarks and qualitative results across a spectrum of vision-language scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Reinforcement Learning for Reasoning Generation: The integration of RGPO, a reinforcement learning algorithm with multimodal feedback and explicit Chain-of-Thought supervision, is well-motivated and distinguishes RG-VLM from models relying solely on supervised or prompt-based approaches. The reward shaping strategy is explained in detail, offering insights into how multimodal consistency and CoT structure are operationalized during policy optimization.\n2. Image Editing/Generation with Reasoning: Figure 1 and Figure 6 visually demonstrate RG-VLM’s strengths in both plain text reasoning (with CoT) and complex image editing/generation. For instance, Figure 1’s concrete reasoning scenarios and vision editing examples illustrate capabilities not present in competitor models.\nWeaknesses:\n1. Limited Multimodal CoT: The current CoT process is stated (Appendix A.3, but addressed in main text as well) to operate only in plain text; true multimodal reasoning chains (i.e., with explicit image intermediate steps or edits) are left for future work. This constrains the novelty and applicability of the reasoning generation claim, as current generation is essentially textual CoT guiding generation, rather than fully realized multimodal reasoning sequences.\n\nQUESTIONS:\nHow does RG-VLM compare (quantitatively) with the very latest closed and proprietary models (e.g., Gemini-2.5, GPT-4o) on publicly available benchmarks, given that only qualitative results (Figure 6) are provided? Are there any plans or ongoing work to provide more direct numerical comparisons with these models?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}