Title: Vinci: Deep Thinking in Text-to-Image Generation using Unified Model with Reinforcement Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image generation; Image understanding
Abstract: With the continuous development of large language models and reasoning chain technologies, the potential of deep reasoning based on reinforcement learning has shown remarkable promise in multi-task scenarios.  However, existing unified models have yet to achieve end-to-end integration in image generation and understanding tasks, limiting the model’s self-reflection ability and the realization of cross-modal reasoning chains.  To address this, we propose Vinic, a novel framework designed to enable interleaved image generation and understanding through deep reasoning capabilities.  We leverage a small amount of multimodal chain-of-thought (MCoT) data for cold-start and employ reinforcement learning to guide the integration of image generation and understanding tasks.  Additionally, we introduce a momentum-based reward function, which dynamically adjusts the reward distribution by considering historical improvements, ensuring the stability of the model across multiple generations.  Experimental results demonstrate that integrating MCoT can achieve a +22% improvement over the base model on Geneval, effectively enhancing both image generation quality and instruction alignment capabilities.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a method to enhance the text-to-image generation capabilities of multimodal models that utilize discrete tokens with causal attention for both image and text generation, such as Emu3-Gen. Specifically, the authors incorporate multimodal chain-of-thought (MCoT) reasoning and self-reflection capabilities, enabling the model to progressively refine image generation during inference. Initially, the model undergoes supervised fine-tuning on synthetically generated MCoT data, followed by reinforcement learning guided by three reward functions: a program-based reward leveraging detection models for image evaluation, a question-answering reward using multimodal large language models (MLLM) to assess generated textual reasoning, and a momentum process reward that dynamically encourages incremental improvements based on the improvements across images in the MCoT.\n\nThe authors demonstrate the effectiveness of their method by achieving a significant improvement of +22 absolute points (from 54% to 76%) on the GenEval benchmark. The key contributions include the focus of MCoT and self-reflection into text-to-image generation, developing a pipeline for synthesizing MCoT training data, designing reward functions for reinforcement learning in text-to-image, and providing ablation studies to quantify the impact of each reward component.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The primary research question, which explores the use of multimodal chain-of-thought (MCoT), is compelling and worthwhile to investigate.\n\n2. The pipeline developed to synthesize MCoT data for supervised fine-tuning (SFT) can be beneficial for future research.\n\n3. The insights regarding the contribution of the text-based reward model to image generation performance are valuable and interesting, and highlight the potential of multimodal models.\n\n4. The ablation study, which analyzes the impact of different reward model combinations, is informative.\n\nWeaknesses:\n\n1. The authors do not clearly present results immediately following the cold-start phase. Given the substantial high-quality SFT dataset collected for the specific domain, one would expect a significant performance improvement after SFT alone. Therefore, a more appropriate baseline to clearly demonstrate the impact of the reward models would be the model’s performance after the SFT phase.\n\n2. Despite being central to the paper's focus, the efficacy of MCoT as a method remains somewhat uncertain, particularly considering its computational demands. A simpler baseline comparison could involve selecting the best image from multiple generations using image-based reward models.  Additionally, a valuable comparison would involve performing SFT on top-scoring images and then selecting the best result from several generations, to directly assess whether MCoT offers significant advantages over simpler alternatives.\n\n3. The generalization capability of the model is unclear. A significant advantage of chain-of-thought methods in large language models is their ability to learn reasoning processes that generalize to novel tasks and scenarios. It remains uncertain whether this model can generalize beyond its specific training data. For example, if the SFT data and prompts during RL would not include prompts with color attribution, it is unclear if the model will learn to adjust color attributes effectively during inference.\n\n4. The implementation details provided in the paper are somewhat limited and could benefit from further elaboration to facilitate clearer understanding and reproducibility.\n\nQUESTIONS:\nQuestions and Suggestions for the Authors:\n\n1. Could you provide detailed performance metrics of the model immediately after the supervised fine-tuning (SFT) phase, prior to applying reinforcement learning (RL)? Including this information would significantly clarify the contribution of the RL phase.\n\n2. How does your MCoT-based method compare to a simpler baseline strategy that involves selecting the best image from multiple generations based solely on the image-based reward models? The baseline should be tuned on high quality generations that were collected and used during the cold-start phase to level the playing field of collecting generations using flux and the scoring and filtering them. Including this information can clarify the benefits of using the MCoT framework.\n\n3. Can you provide evidence or additional experiments demonstrating the model's ability to generalize its reasoning capabilities to novel tasks or attributes not explicitly covered during training? For example, if specific reasoning tasks (such as color attribution) were excluded from training, does the model effectively adapt and correctly handle these tasks during inference? This again, can highlight the benefits of using the MCoT framework.\n\n4. Could you expand upon the implementation details provided in the paper? Clarifying specific technical choices (e.g., hyperparameter selection, computational resources, runtime considerations) would improve reproducibility.\n\nLIMITATIONS:\nThe limitations section could be expanded further by explicitly discussing specific scenarios or failure cases where the proposed method and model might underperform or fail entirely.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Vinci, a unified autoregressive text-to-image model that interleaves image generation and understanding through a Multimodal Chain-of-Thought (MCoT). After a small “cold-start” finetuning with 20 k synthetic MCoT traces, the authors apply reinforcement learning guided by three rewards—program-based visual checks, QA-based textual assessment, and a novel momentum reward—to let the model iteratively critique and refine its own outputs. On the object-focused GenEval benchmark, Vinci improves the Emu3-Gen base by +22 % overall, narrowing the gap with much larger systems while remaining fully end-to-end.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. Conceptual novelty – The work realizes interleaved MCoT inside a single autoregressive generator, enabling the model to “look at” intermediate images and revise them on-the-fly. This directly tackles the reflection bottleneck of previous unified methods.\n\n2. Reward design – The momentum-based reward elegantly couples short-term quality with long-term improvement stability, an idea reminiscent of adaptive optimisers yet new to RLHF for vision–language models.\n\n3. Thorough empirical evidence – Results span benchmark scores, qualitative rollouts and an ablation that disentangles each reward’s contribution, convincingly supporting the claimed +22 % gain and showing Vinci to be competitive with much larger models.\n\nWeaknesses\n1. Limited evaluation scope – All quantitative claims rest on GenEval; there is no open-domain or human preference study, leaving generalisation and perceptual quality uncertain.\n\n2. Heavy reliance on automatically-synthesised MCoT traces. The “cold-start” stage uses just 20 k traces whose images come from Flux and captions from Qwen-VL. Because neither source is curated or human-checked, the traces may carry style biases or captioning errors that the policy then imitates, limiting robustness outside the synthetic data distribution.\n\n3. Both the program-based visual check and the QA-based textual check depend on external detectors or MLLMs. Any failure—e.g., detector mis-counting objects in cluttered scenes—directly contaminates the reinforcement signal, encouraging reward hacking rather than genuine image-text alignment.\n\n4. While the paper defines α, λ, γ for the momentum term, it never reports the chosen values or provides a sensitivity study.\n\nQUESTIONS:\nWhat is the training cost and inference cost of the model compared with direct or simple cot?\n\nLIMITATIONS:\nSee weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

comments: SUMMARY:
The authors introduce Vinci, a novel framework that integrates multimodal Chain-of-Thought (MCoT) reasoning with reinforcement learning (RL) to enhance text-to-image generation. Unlike existing unified models that treat image generation and understanding as separate tasks, Vinci enables iterative refinement of generated images through real-time self-reflection.

STRENGTHS AND WEAKNESSES:
Strengths
1) The integration of MCoT with RL for iterative image refinement is novel. The momentum-based reward function is a creative solution to stabilize multi-round generation.
2) Comprehensive experiments on GenEval show significant gains (+22% over baselines), with detailed ablation studies validating each component’s contribution.
​​3)  The framework addresses a critical gap in unified models by enabling real-time self-correction, as shown in qualitative results.

Weaknesses
1) While the authors compare with unified models (e.g., Janus-Pro, GPT-4o), it omits comparisons to recent RL-based T2I models like DALL-E 3 or Stable Diffusion 3, which are industry standards.
​​2) The MCoT data pipeline lacks details on object detection models and GPT-4o filtering criteria. Hyperparameters for RL training are not justified.
3) The momentum reward’s theoretical guarantees (e.g., convergence) are not discussed, despite its central role.

QUESTIONS:
1) Why exclude comparisons to RL-based T2I models like DALL-E 3? Including these would strengthen claims of superiority.

2) Provide specifics on the object detection model and GPT-4o filtering thresholds for MCoT data construction.

3) Derive or cite theoretical guarantees for Equation 5’s stability.

4) Quantify the computational overhead of iterative refinement, e.g., FLOPs per iteration.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to apply Multimodal Chain-of-Thought (MCoT) reasoning and Reinforcement Learning (RL) to text-to-image generation within a unified multimodal model, specifically Emu3. It follows the recent RL techniques in large-language models (LLMs), especially the Deepseek-R1. Specifically, it first collects a small amount of multimodal chain-of-thought data from GPT-4o, used for cold-start of the RL training. Then it introduces a momentum-based reward function that distributes process rewards based on historical improvements, which improves the long-term stability of RL optimization. This method is based on Emu3 because of its unified image representation. Experiments on GenEval have demonstrated +22% improvement compared to the base model.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. This method explores applying Multimodal Chain-of-Thought (MCoT) and RL to cross-modal multimodal models, enabling iterative refinement over both image and text modalities. This is a step beyond prior unimodal CoT approaches, which focus only on textual prompt refinement.\n\n2. It proposes a momentum-based reward function, which is a meaningful contribution that addresses the common instability issues in RL finetuning.\n\n3. The results on GenEval have shown good improvement compared to Emu3.\n\nWeaknesses\n\n1. The method design depends on a unified autoregressive model that has unified text and image representations, which limits its application to different MLLMs.\n\n2. The MCoT data is only collected by GPT-4o without real-world data, which will possibly be limited by GPT-4o's capabilities.\n\n3. The evaluation is limited to GenEval, which contains short prompts and may not be enough to evaluate the CoT capabilities. Experiments on other benchmarks such as DPG-Bench from ELLA and T2I-CompBench should be included.\n\nReference: ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment\n\n4. The ablation study only evaluates the effectiveness of reward functions. However, the alterations with/without the MCoT data should also be included to validate the effectiveness of the cold start phase.\n\nQUESTIONS:\nIn addition to the Weaknesses, I have additional questions about the paper details.\n\n1. This paper does not compare with other CoT & RL image generation models such as T2I-R1. While they may be different in the overall architecture, they address a similar problem. A direct comparison, possibly by applying T2I-R1 to Emu3 under similar settings, would strengthen the results.\n\n2. Are there any methods that can be used to analyze or visualize the effectiveness of the momentum-based reward function and show that it stabilizes the training process?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nVinci is a framework for training a multimodal, reasoning style model for generating images through iterative refinement of the image in the reasoning process. The paper first generates a cold-start dataset of reasoning chains and then use reinforcement learning to optimize according to a number of reward functions based on object detection models, or other suite if image quality evaluators. The authors finally introduce a process-based reward function based on the Adam optimizer. Vinci demonstrates strong performance across a variety of tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- This paper is well written and easy to understand.\n- Aligning models in image generation to match prompt and user intent is a significant problem that Vinci addresses.\n- Vinci shows strong results on most tasks, performing well compared to leading closed source image generation models.\n\n**Weaknesses**\n- The CoT cold-start generation seems sub-optimal. Dataset collection has significant room for error after relying on the concatenation of so many models calls. Further, the authors have the model output a score from MLLM, which could be less grounded than head to head comparison, a common method for LLM as a judge comparison RLHF. Absent of the authors releasing the dataset, reproducibility may be difficult.\n- Scalability of the method is limited as reasoning chains for these models can become very long due to image tokens.\n\nQUESTIONS:\n- What is the significant of having R_i and then momentum reward function? Doesn't the moment reward function also encompass R_i? I find this was the interesting ablation that was missing.\n- Why does table 2 compare against the base model instead of the SFT'd model? It is unclear the benefit from SFT vs RL here.\n- What is the benefit of Vinci over, say, using a fixed image generation model and doing a similar method but to rewrite the prompt instead of training the image generator.\n- In the 3 generation scenario, do we actually see improvement across steps? I think this point is important to verify.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}