Title: GoT: Unleashing Reasoning Capability of MLLM for Visual Generation and Editing
Venue: NeurIPS 2025
Accepted: yes
Keywords: Reasoning-based Visual Generation and Editing

MLLM
Abstract: Current image generation and editing methods primarily process textual prompts as direct inputs without explicit reasoning about visual composition or operational steps. We present Generation Chain-of-Thought (GoT), a novel paradigm that empowers a Multimodal Large Language Model (MLLM) to first generate an explicit, structured reasoning chain in natural language—detailing semantic relationships, object attributes, and, crucially, precise spatial coordinates—before any image synthesis occurs. This intermediate reasoning output directly guides the subsequent visual generation or editing process. This approach transforms conventional text-to-image generation and editing into a reasoning-guided framework that analyzes semantic relationships and spatial arrangements. We define the formulation of GoT and construct large-scale GoT datasets containing over \textbf{9M} samples with detailed reasoning chains capturing semantic-spatial relationships. To leverage the advantages of GoT, we implement a unified framework that integrates Qwen2.5-VL for reasoning chain generation with an end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance Module. Experiments show our GoT framework achieves excellent performance on both generation and editing tasks, with significant improvements over baselines. Additionally, our approach enables interactive visual generation, allowing users to explicitly modify reasoning steps for precise image adjustments. GoT pioneers a new direction for reasoning-driven visual generation and editing, producing images that better align with human intent. We will release our datasets and models to facilitate future research.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Generation Chain-of-Thought (GoT), a new paradigm for visual generation and editing that addresses the failure of current models to understand complex scenes with specific object relationships and spatial arrangements. Instead of directly mapping a text prompt to an image, the GoT framework utilizes a Multimodal Large Language Model (MLLM) to first generate an explicit, structured reasoning chain that details semantic relationships, object attributes, and precise spatial coordinates before any image synthesis occurs. \n\nThe authors' core contributions include the GoT paradigm itself, the construction of the first large-scale datasets containing over 9 million samples with these detailed reasoning chains , and a novel, unified end-to-end framework for implementation. This framework integrates an MLLM with a diffusion model via a Semantic-Spatial Guidance Module (SSGM), which ensures the generated image precisely follows the logical reasoning steps. The GoT framework demonstrates competitive results on benchmarks for text-to-image generation and image editing , and its explicit process enables interactive generation, allowing users to directly modify the reasoning chain for precise control over the final output.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. High Originality and Significance: The paper's core concept, Generation Chain-of-Thought (GoT), is highly original and significant. It introduces a novel reasoning-guided paradigm for visual synthesis, moving beyond direct mapping to address key limitations in compositional generation.\n\n2. Substantial and High-Quality Contributions: The creation of a massive, first-of-its-kind dataset with over 9 million samples is a high-quality contribution that will be a valuable resource to the community. This was a major effort, consuming over 3000 NVIDIA A100 GPU days to produce detailed semantic-spatial annotations.\n\n3. Clarity Empirical Results: The proposed end-to-end framework is presented with excellent clarity and is technically sound. Its effectiveness is studied through extensive experiments across multiple generation and editing benchmarks.\n\n\n\nWeaknesses\n\n1. Incomplete Experimental Comparison: The evaluation in Table 1 lacks a direct comparison with RPG [52], a highly relevant baseline that also employs a recurrent planning and generation methodology for compositional scenes. Given the conceptual similarities in using an explicit planning stage, its omission makes the assessment against layout-guided methods less complete.\n\n2. Unconvincing Ablation Study Design: The setup of the ablation study in Table 3 does not fully isolate the impact of the core contribution—the GoT reasoning. The baseline is trained for significantly fewer steps (10,000) than the full framework (70,000 total). This makes it difficult to distinguish the performance gains from the GoT reasoning itself versus the substantial effect of extended pretraining. A more persuasive ablation would use a baseline with the same amount of pretraining but without the GoT reasoning tokens to provide a more direct comparison.\n\n3. Lack of Statistical Significance Reporting: The quantitative results presented in Tables 1, 2, and 3 do not include error bars or any other measure of statistical significance (e.g., standard deviation across multiple runs). As acknowledged in the paper's checklist, this absence makes it difficult to rigorously assess the stability and significance of the reported performance improvements over other methods.\n\nQUESTIONS:\nSee weaknesses first.\n\nAdditional questions:\n\n1. Impact on Core VLM Capabilities: Does joint training on GoT and diffusion generation degrade the base MLLM's original vision-language understanding abilities? Please provide benchmark results on standard VLM tasks to clarify the trade-offs of your unified training approach.\n\n2. Model Efficiency: Could you provide details on the model's efficiency, specifically its parameter count and average inference latency compared to other models in Tables 1 and 2? This information is crucial for assessing the framework's practical viability.\n\nLIMITATIONS:\nNo. While the authors commended \"Yes\" in the NeurIPS checklist, I didn't see substantial limitations are discussed in the paper.\n\nSuggestions for Improvement:\n\nI suggest the authors add a dedicated \"Limitations\" section to the main paper. This section should transparently discuss the limitations, for example, issues of model efficiency/latency, the high computational costs, etc.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Generation Chain-of-Thought (GoT), a novel framework that incorporates explicit semantic-spatial reasoning chains into visual generation and editing. GoT first produces a step-by-step structured reasoning chain in natural language, which guides the image synthesis process through a Semantic-Spatial Guidance Module (SSGM) that injects semantic embeddings, spatial layouts, and visual references into the diffusion model.\n\nThe framework integrates a multimodal large language model (Qwen-VL) with a diffusion model in a unified, end-to-end trainable pipeline. To support this paradigm, the authors construct a large-scale dataset comprising over 9 million reasoning-chain-annotated samples for both image generation and editing tasks. Extensive experiments demonstrate that GoT achieves state-of-the-art performance on benchmarks such as GenEval, with significant improvements in controllability, transparency, and user interaction. Overall, the work represents a substantial technical contribution, with a large scope and strong empirical results.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths:\n    1. Introduces a reasoning-driven generation framework by integrating chain-of-thought (CoT) into visual generation, enabling explicit modeling of object relationships and spatial layouts.\n    2. Combines a multimodal LLM and a diffusion model with a Semantic-Spatial Guidance Module to inject semantic and spatial control directly into the generation process.\n    3. Achieves state-of-the-art performance on both text-to-image generation and multi-step image editing benchmarks.\n    4. Scalable data contribution: Releases a large-scale reasoning chain dataset (>9M samples) to support further research and reproducibility.\n- Weaknesses:\n    1. Slightly underperforms on fine-grained spatial alignment and attribute binding compared to some task-specific baselines.\n    2. The paper claims that “in GenEval, only the position category is notably affected by SSGM,” but does not report concrete performance numbers for the position category.\n\nQUESTIONS:\n1. While GoT achieves overall strong performance, it underperforms on certain fine-grained metrics such as spatial relation and attribute binding (compared to models like JanusFlow). Could the authors provide more discussion on this limitation and potential reasons behind it?\n2. Could the authors provide the exact performance numbers for the position category with and without SSGM, as mentioned in the GenEval analysis? Also, have the authors considered including a dedicated evaluation (e.g., object position deviation) to further verify the effectiveness of spatial guidance?\n3. It would be helpful if the authors could elaborate on the future development of the GoT paradigm. Do you believe this reasoning-guided framework is easily generalizable to broader multimodal generation tasks? What are the main challenges or limitations that might arise in scaling or adapting it further?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper argues for chain-of-thought reasoning in image generation - e.g., given an user input prompt, generate a more detailed intermediate prompt (that often includes spatial layout coordinates) that produces higher-quality generated images. Such generative chain-of-thought traces are used to improve both image generation and image editing tasks. To do so, this work builds a large-scale dataset of such reasoning chains, with 8 million image-generation examples and 1 million image-editing examples. The datasets is automatically built by running captioning VLM models on source images, to produce such detailed chain-of-thought prompts. Architecturally, this paper also  method a novel end-to-end approach that does not explictly condition image generation on the chain-of-thought prompts, but instead conditions image generation on 64 learnable embeddings (that a VLM is trained to produce). The final diffusion-based image generation module conditions generative denoising on the learnable embeddings along with a spatial image / crop / bounding box masks (derived from the chain-of-thought prompts).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- This paper is reasonably executed and well written.\n- The dataset of 8 million examples would be valuable if released, but this isn't offered as a contribution in the paper.\n\nWeaknesses\n- The overall approach (of generative chain-of-thought) seems to be widely used in industrial models; many frontier models make use of LLM-based prompt engineering to improve results. That said, this paper does compare to a variety of such published prompt-enhancement approaches in Table 1. But I am missing a concise description of the key innovations that allow the proposed approach to work well (since similar prompt-enhancing approaches have been previously explored).\n- Building on the above, I do not quite follow the ablations in Table 3 and their description in L343-346. The natural baselines would seem to be off-the-shelf image generation and off-the-shelf chain-of-thought prompting. But I do not see how to perform \"off-the-shelf\" learnable embeddings. I'm guessing that the key innovation is training the model on the collected 9 million examples. I would have liked to see an ablation that (post) trains a traditional chain-of-thought model on such data, without the learnable embeddings.\n\nQUESTIONS:\n- I would like to see the detailed ablations listed above. This would reveal if the improvement is really coming from the data or from the architectural modifications. Traditional chain-of-thought prompts have the added benefit of being interpretable, which the learned embeddings do not. So it would be useful to better justify the proposed approach.\n- Are the authors planning on releasing their data? I suspect the real innovation comes from this data, and so I'd be more willing to fight for acceptance if that could also be regarded as a contribution.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}