GoT: Unleashing Reasoning Capability of MLLM for Visual Generation and Editing
Authors: Rongyao Fang
Chengqi Duan
Kun Wang
Linjiang Huang
Hao Li
Hao Tian
Shilin Yan
Weihao Yu
Xingyu Zeng
Jifeng Dai
Xihui Liu
Hongsheng Li
Venue: NeurIPS 2025 poster ()
Keywords: Reasoning-based Visual Generation and Editing
MLLM
Primary area: deep_learning

Abstract: Current image generation and editing methods primarily process textual prompts as direct inputs without explicit reasoning about visual composition or operational steps. We present Generation Chain-of-Thought (GoT), a novel paradigm that empowers a Multimodal Large Language Model (MLLM) to first generate an explicit, structured reasoning chain in natural language—detailing semantic relationships, object attributes, and, crucially, precise spatial coordinates—before any image synthesis occurs. This intermediate reasoning output directly guides the subsequent visual generation or editing process. This approach transforms conventional text-to-image generation and editing into a reasoning-guided framework that analyzes semantic relationships and spatial arrangements. We define the formulation of GoT and construct large-scale GoT datasets containing over \textbf{9M} samples with detailed reasoning chains capturing semantic-spatial relationships. To leverage the advantages of GoT, we implement a unified framework that integrates Qwen2.5-VL for reasoning chain generation with an end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance Module. Experiments show our GoT framework achieves excellent performance on both generation and editing tasks, with significant improvements over baselines. Additionally, our approach enables interactive visual generation, allowing users to explicitly modify reasoning steps for precise image adjustments. GoT pioneers a new direction for reasoning-driven visual generation and editing, producing images that better align with human intent. We will release our datasets and models to facilitate future research.

GoT: Unleashing Reasoning Capability of MLLM for Visual Generation and Editing
Rongyao Fang
Chengqi Duan
Kun Wang
Linjiang Huang'
Hao Li
Hao Tian'
Shilin Yan
Weihao Yu!
Xingyu Zeng
Jifeng Dai
Xihui Liu Hongsheng Li
CUHK MMLab
CPII under InnoHK
HKKU
Teme Shanghai Lab
SUAT
Task
Input
Prompt
GoT Reasoning
Generated Images
121 Promph:
Generated GoT:
modern living with touch shabby chic, featuring furniturt, chandelier, and decorative elements like mirrors and wreath.
The image shows beautifully curated living lorge, chandelier (372.0 (37200/6133254)
T2)
adamned with a fromed mirror (1822573 pas, 519) above while sofa
wall are
T2/ Prompt:
Generated
backpock and apple.
gray backpack 6OU 925) green apple (431,844)/81442220 925)
Modify: green
Inferactive Generation
T21 Prompt:
Madified GoT:
backpack and apple.
gray backpack (19.A0LI111. apple (133,86),1136920
Generated GoT:
1. The source image
2. The edited object is
3.
Editing
5. The edited image will show holding dank blue umbrella...
Replace the giant leaf with umbrella.
"Ciloo elenwen STTapalall restesii gesmocted cochest
Figure 1: Generation Chain-of-Thoughh (GoT) with Semantic-Spatial Reasoning. Our approach transforms input prompts into explicit reasoning chains with coordinates (middle), which guides vivid image generation and precise editing (right). This reasoning-basee generation paradigm unifies spatial understanding across visual tasks: semantically-grouss groundedd visual generation (top), controllable interactive generation (middle), and localized image editing (bottom).
Abstract
Current image generation and editing methods primarily process textual prompts as direct inputs without explicit reasoning about visual composition or operational steps. We present Generation Thaiu-of-Thought Thought (Go'T), novel paradigm that em-
powers Multimodal Large Language Model (MLLM) to first generate explicit, semantic relationships,
structured reasoning chain in natural
Equal Contribution
Corresponding Authors
39th Conference Neural Information Processing Systems (NeurIPS 2025).

---

object attributes, and, crucially, precise spatial coordinates- any image synthesis occurs. This intermediate reasoning output directly guides the subsequent visual generation or editing process. This approach transforms conventional text- to-image generation and editing into reasoning-guided framework that analyzes semantic relationships and spatial arrangements. We define the formulation of GoT and construct large-scale GoT datasets containing over 9M samples with detailed reasoning chains capturing semantic-spatial relationships. To leverage the advantages of GoT, we implement unified framework that integrates Qwen2.5- VL for reasoning chain generation with an end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance Module. Experiments show our GoT framework achieves excellent performance on both generation and editing tasks, with significant improvements over baselines. Additionally, our approach enables interactive visual generation, allowing users to explicitly modify reasoning steps for precise image adjustments. GoT pioneers a new direction for reasoning- driven visual generation and editing, producing images that better align with human intent. We will release our datasets and models to facilitate future research.
1 Introduction
Language provides the primary interface for expressing human intent in visual content generation. Traditional image generation systems [37 6 21], particularly diffusion models, process textual prompts by mapping semantic concepts to visual elements without explicit reasoning. These ap- proaches struggle with complex scenes requiring precise spatial arrangements and object interactions that humans naturally consider when constructing scenes. Meanwhile, multimodal large language models (MLLMs) 2 excel at sophisticated reasoning tasks, including amalyzing semantic struc- tures, inferring relationships, grounding visual concepts, and processing detailed contexts through explicit reasoning chains. This gap between MLLMs' advanced reasoning capabilities and the limited reasoning in current generation systems raises a key question: How can we integrate the reasoning mechanisms that have revolutionized language understanding into visual generation and editing?
Prior work attempted to leverage LLMs for image generation from different perspectives. One line of research 22 (55] leverages L.MSs as text encoders for better prompt interpretation. However, the reasoning capabilities of LLMs are not introduced. Another line of work develops multimodal
LLMs to unify understanding and generation [4 47 50 7 Although they present unified models for different tasks, there is no evidence that generation benefits from strong understanding and reasoning abilities of the models. They merely combine independent tasks rather than truly fusing language reasoning with visual generation. Additionally, layout-based methods like GLIGEN (2],, LayoutGPT [9. and RPG [52] incorporate LLMs for layout planning and diffusion models for layout-guided generation. However, these methods treat planning and generation as separate stages rather than integrating reasoning throughout the end-to-end process. Consequently, current image generation methods lack reasoning capabilities, emphasizing the need for framework that seamlessly combines reasoning with visual generation and editing.
Inspired by -f-thought (CoT) reasoning of the LLMs, we introduce Generation Chain-of- Thought (GoT), a novel paradigm that enables visual generation to first output step-by-step reasoning in natural language before producing images. However, implementing GoT poses two significant challenges. First, different from CoT LLMs, the reasoning chain for visual generation and editing requires both semantic and spatial information. It requires a new formulation and collecting training data in this new format. Second, cxisting diffusion- -assd models cannot leverage explicit language reasoning chains during visual generation. We need to design a framework supporting end-to-end language reasoning and visual generation.
To address the first challenge, we formulate GoT as multimodal reasoning chain that integrates semantic and spatial analyses enhance image generation and editing tasks. For visual generation, GoT provides precise control over object layout, relationships, and attributes, while for editing, leverages semantic and spatial understanding to decompose user requests into coherent grounding and modification steps. We utilize advanced MLLMs and LLMs construct complex annotation pipelines, which capture semantic-spatin interactions across diverse visual contexts. We assembled extensive datasets comprising 8.4M images for text-to-image generation (from Laion-Aesthetics 39) Jo (4). and FLUX (21) and 920K examples for image editing (from OmniEdit [48] and
SFFD--diitMulttuur 12). This computationally intensive effort produced the first large-scale dataset of reasoning chains for image generation and editing.

---

To tackle the second challenge of architecture design supporting reasoning and generation. we construct unified end-to-end framework. Our GoT framework integrates the reasoning capabilities of MLLMs with the high-fidelity generation qualities of diffusion models. The proposed framework leverages an MILM to generate reasoning steps and visual tokens, providing explicit guidance that incorporates semantic relationships and spatial configurations. This guidance flows into our novel Semantic- Spatticc--aaallll Guidance Module (SSGM), which conditions the diffusion process to ensure that generated images are closely guided by the reasoning process. This design supports end-to-end training and inference for visual generation and editing guided by explicit reasoning chains.
By effectively integrating reasoning into visual generation, our GoT framework demonstrates signifi- cant improvements in both text-to-image generation quality and image editing accuracy. Additionally, GoT enables interactive generation, allowing users control the generated image by directly mod- ifying the explicit reasoning process according to their preferences. These advantages represent a substantial advancement in sessoningggudee visual synthesis.
The main contributions can be summarized as follows:
We propose Generation Chain-of-Thought (GoT),, a paradigm where an MILM first generates an explicit semantic-spatial reasoning chain- comppising textual descriptions of objects, their attributes, inter-relationships, and precise geometric coordinates- plan visual content before generation or editing.
We define the formulation of semantic and spatial reasoning chains for visual generation and editing, and constructed the first large-scale GoT datasets, encompassing 8.4M image generation, 920K image editing samples. Creating this dataset, with its semantic-spatial annotations derived from complex MLL.--rriven annotation pipelines, consumed over 3000 NVIDIA A100 GPU days.
We develop a unified end-to-end framework that leverages multimodal language models and diffusion models, with a novel Semantic-Spatia Guidance Module that ensures generated images follow the reasoning process.
Our experimental results demonstrate significant improvements in both text-to- -m-age generation and editing.
2
Related Work
2.1 Diffusion Models
Diffusion models have revolutionized visual content creation. Early approaches 33 30 demonstrated this paradigm's 's potential, while Stable Diffusion (37 improved efficiency through latent space com- pression. Recent models [35] 38 32
21 have further advanced photorealism through architectural innovations and larger-scale training. Various efforts to extend diffusion models' capabilities include controllable generation methods [54] 28) and instruction -ased editing frameworks (0) While some researchers have explored unifying vision tasks [I] 8) these primarily focus on traditional computer vision tasks rather than general image generation. Despite these advances, current models typically process prompts through direct mapping, using text encoders like CLIP [33 or T5 [34] to condition the diffusion process via cross-attention (45) This approach treats text as a static represen- tation without explicit reasoning about scene composition or object relationships. The fundamental limitation becomes evident when generating complex scenes with multiple objects and specific spatial arrangements, necessitating more sophisticated aasonnn--based approaches.
2.2 Large Language Models and Reasoning
Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities through chain-of-thought (4), enabling complex problem decomposition. This paradigm extends to MLLMs 21. which integrate visual and textual understanding. Some advanced works [25] B1 have enhanced spatial understanding by grounding textual concepts to image regions, enabling analysis of object relationships. Despite these capabilities, MLLMs remain underutilized for visual generation. While models like Chameleon [44] and Emu2 44 incorporate image generation, they lack mechanisms to decompose user intent into explicit semantic-spatial reasoning steps that guide the synthesis process. More recently, [15] explored applying Co CoT-inspired strategies to autoregressive image generation, focusing on verifying and reinforcing the iterative token prediction steps of the generation model itself, which differs from our GoT paradigm where an explicit, structured comantic-spation reasoning chain with coordinates is generated as a guide for image synthesis.
3

---

2.3 Layout-guided Image Generation and Editing
Recent research has explored layout- approaches for spatial control in visual synthesis. GLI- GEN [22] incorporated bounding boxes through gated cross-attention layers, enhancing object placement. LPy 99] proposed a two-stage pipeline converting text into scene layouts before generation. RPG 122 advanced this through recurrent planning, alternating between layout refine- ment and synthesis. SmartEdit [17] adapts the LLaVA (26) model to specialize in image editing tasks. FlexFdit [29] employs an MILM to comprehend the image content, mask, and user instructions. Despite these advances, existing approaches treat layouts as static constraints or sequential plans generated before synthesis, disconnecting spatial planning from generation.
3 Generation Chain-of-Thought (GoT)
During visual generation and editing, humans naturally reason about object relationships and spatial arrangements. In contrast, most current models process prompts without explicit reasoning, making it difficult to interpret complex human intentions for generating scenes with detailed object relationships and spatial configurations.
Motivated by chain-of-thought (CoT) in language models, we propose Generation Chain-of-Thougse (GoTT,, shifting the visual generation from direct mapping to reasoning- -uiiidd process. Unlike language generation, which operates primarily within a semantic space, visual generation requires an integrated understanding of both semantic relationships and spatial configurations. To address this complexity, GoT employs multi-modal reasoning formulation that bridges conceptual understanding and spatial reasoning. This formulation incorporates explicit coordinate information in format (x1, (x),y1), (x2, y2) with range [0, 1000), ensuring precise management of visual element placement. This unified semantic-spatial reasoning chain enables fine-grained control of object placement, attributes, and inter-object relationships, ultimately supporting robust and coherent visual generation.
To illustrate GoT's formulation, Fig. D presents examples of both text-to-image generation and editing tasks. For text-to-image, GoT generates detailed reasoning chain specifying precise coordinates of elements. This explicit spatial reasoning enables proper arrangement of all constituents while maintaining their semantic relationships, resulting in a coherent and visually appealing composition.
The image editing example in Fig. [ demonstrates how GoT handles manipulation tasks through structured reasoning. When tasked with replace the giant leaf with an umbrella, GoT first analyzes the scene and then plans edits with precise coordinates. Finally, GoT describes the edited image. This decomposition into sequential steps with explicit spatial reasoning streamlines complex manipulations, contrasting with traditional editing methods that lack spatial awareness and reasoning.
GoT endows image generation and editing with reasoning benefits. By decomposing complex instructions into clearly defined, sequential steps, GoT delivers results that more accurately fulfill human requests. Its transparent process explains the intermediate reasoning behind each change and enables both image generation and editing within unified system.
Implementing GoT requires two key components:
This dataset must consist of detailed reasoning chains that align with visual content, capturing both semantic relationships and spatial configurations. Such data provide the necessary foundation for the reasoning process.
A Comprehensive Dataset:
The model needs to accommodate chain input to integrate semantic analysis and spatial reasoning, ensuring effective execution of the reasoning steps derived from the dataset.
A Compatible Visual Generation Model:
In the following sections, we elaborate on these components and discuss how they contribute to the robust performance of the GoT framework.
4
GoT Dataset: Semantic-Spatial Reasoning Chains for Visual Generation and Editing
Based on the formulation presented previously, we construct large-scale training datasets using ad- vanced LLMs and MLLMs. Our GoT dataset features meticulously crafted semantic-spatial reasoning chains for generation/editing tasks, each sample containing instructions, reasoning chain annotations, and corresponding images. Construction requires careful design of task-specific annotation pipelines to ensure quality. The prompts used in the pipelines are attached in Appendix Sec. E

---

Text image GoT Annotaton
Ediiing GoT Annotatin
laag
Cophoning
loakiing Food palter anda
Soutte
Depobbe Stance longe
diik liigg raves kike wil
Fiely
Libeaned hoopping
Owen2.5
Hoorrtt , bacipadi 202
bedlpact (193 (Rt2 9281
toled eeleettyy
Del: 1, the sevce erepe drne 201. 3. the esieel esieel mage vil thow
Pare fully
Assebble
procena Froeping
Semontic
Component
form patteme iderfica unton
Coop E Droeke
vi wil YOuR BADOI
SKEP
Sputao
Coordinates
Text-to-image GoT annotation pipeline that labels detailed GoT with semantic content and spatial coordinates. Right: Editing GoT annotation pipeline that processes source image, target image, and instruction to generate entity-aware reasoning GoT with precise spatial grounding. Both pipelines leverage Qwen2- VL [46] and Qwen2.5 [51 models for various stages of the annotation process.
Figure 2: GoT Dataset Construction Process. Left:
4.1 Automated Data Creation Pipeline
As illustrated in Fig. 2 our annotation pipeline demonstrates the multiple stages of processing required generate these high-quality annotations. For text-to-image, we utilize Qwen2-VL [46] to generate concise prompts that serve as text-to-image generation prompts and detailed visual descriptions that form the semantic component of GoT. Qwen2.5 [51] then performs object entity extraction, followed by Qwen2-Vi VL. establishing spatial relationships through object grounding. The detailed visual descriptions merged with precise object groundings together constitute the complete GoT annotation for text-to-imanl to-image generation.
For the image editing pipeline, we employ Qwen2-VI. LL to generate comprehensive descriptions of source and target images, precisely localize editing regions through bounding boxes, and generate detailed descriptions of edited objects after cropping. We then leverage Qwen2.5 with carefully designed in-context prompting and completeness of the editing process. From this pipeline, we derive concise editing instructions as editing inputs while using the detailed semantic-spatie -ssaana reasoning steps as GoT annotations. For the complex multi-turn editing dataset, we developed related but more sophisticated protocol with Qwen2-VL. and Qwen2.5 to obtain intricate step-by-step reasoning chains with multiple spatial coordinates and transformation descriptions, capturing complex editing sequences.
synthesize coherent GoT reasoning chains, ensuring logical flow
4.2 Dataset Construction
For text-to-image generation, we construct dataset from three sources: Resolution (LAHR) [39 with 3.77M samples filtered for images larger than 512 pixels. Jour- neyDB 41
with 4.09M samples, and 600K S.UUX.I-generaeee (1 images using LAHR prompts.
For the single-turn image editing dataset, we build on OmniEdit [48) premier open-source image editing dataset with high-fidelity images, processing 736,691 samples covering editing operations (addition, removal, swap, changing turn image editing dataset is built upon SEED- Eddt-Multitun [2]. resulting in 180,190 samples.
and style transfer). The multi-
GoT Framework: Reasoning-guided Visual Generation and Editing
We present the GoT framework, novel architecture that integrates a semantic-spatial aware Multi- modal Large Language Model (MLLM) with multi-guided diffusion model. The core structural innovation lies in its unified, end-to-end design where explicit reasoning chain (GoT) is structured intermediate representation, directly bridging the MILM's 's reasoning capabilities with the diffusion model's 's synthesis process. This design ensures generated images precisely follow logical reasoning steps, allowing detailed control over both semantic content and spatial relationships.
5.1 Semantic-Spatial MILMM Design
Our framework utilizes state-of-the-art MILM,, 2wen2.5-VL-3BB as the backbone, chosen for its outstanding visual understanding and grounding capabilities. This MI.MM functions as a reasoning engine, handling both generation and editing tasks through unified architecture.
As illustrated in Fig. 3 the process begins with task-specific inputs. For editing tasks, reference images are processed through the vision encoder. For both generation and editing, the MLLM first produces the detailed GoT reasoning chain, which captures object attributes, relationships, intended modifications, and bounding box information.
5

---

Editing T21
Reference image Guidonce
GoT
SSGM Diffusion Module à
Framework
Spatial Guidance
Semastic Gsidance
Crocoietraay Token lost Geseration Choinol
Diffusion Modules O
MLLM Decoder
Porch Embed
Gr
Ediing
(rutrochion
Generation ChaiselT
N-64 Embodeinge
Vision Encoder
Randon
Edingg: hat from
For Edinng
Gr
Referance image
121: A dag wvering top hat and a bow Ii.
For 721 and Editig
Grodiest Rowing
SSGM Diffusion Module
Figure 3: GoT Framework with Semantic-Spatial Guidance. Left: Our dual-task framework handling both text-to-image generation (T2I) and image editing. Right: The SSGM Diffusion Module, which combines spatial layouts guidance G,, reference image guidance Gy. and semantic guidance G, to generate the final image with precise content and spatial control.
A key aspect of our approach is the generation of semantic guidance Ge for the diffusion model. To achieve this, we introduce sequence of N = 64 dedicated learnable embeddings. Following the generation of the GoT reasoning chain, the MILM's decoder, which inherently employs causal attention, processes these N learnable embeddings. The sequence of N output vectors from the MILMM decoder, corresponding to these initial learnable embeddings and now richly conditioned by the preceding GoT reasoning chain, forms the semantic guidance G.. These G, embeddings thus encapsulate detailed semantic information derived directly from the GoT, ready to steer the diffusion process. Concurrently, the spatial guidance G, is derived by parsing and converting the explicit coordinate information present in the generated GoT reasoning chains.
This semantic-spatial aware design enables the MLLM to direct the SSGM Diffusion Module with precise control over content and layout. During training, the MLLM receives supervision through two pathways: cross-entropy loss on GoT reasoning tokens and gradient signals backpropagated from the end-to-end SSGM diffusion module through semantic guidance G₁-
5.2
Semantic-spatial Guided Diffusion Generation
The end-to-end diffusion module builds upon SDXL's [32) architecture, incorporating an innovative triple-guidance mechanism. At the heart of this is our Semantic-Spatial Guidance Module (SSGM), novel architectural component specifically designed to interpret the structured outputs of the GoT (both semantic and spatial) and seamlessly inject these directives into the diffusion module.
In SSGM, the semantic guidance pathway enhances the diffusion model by channeling N = 64 MLLM-generated embeddings G, through cross-attention layers. This represents a significant struc- tural departure from conventional methods: instead of relying on generic text encoder embeddings (e.g., from CLIP), our architecture uses ei-eemeddiings directly reasoned and generated by the MLLM based on the explicit GoT. This structural choice provides a more nuanced, contextually grounded, and GoT-aligned semantic control signal for the diffusion process.
For spatial guidance in SSGM, we extract coordinate information from the generated GoT to create color-coded masks where each object or editing region receives distinct color based predefined order the GoT sequence. These colored masks are processed through a VAE encoder [18] and averaged to produce spatial latent features G., which are concatenated with the diffusion model's latent representations, enabling precise spatial control during both generation and editing tasks.
crucial structural aspect of the SSGM is its ability synergistically leverage both the semantic guidance G, and the spatial guidance Gg. which are notably derived from the same unified GoT reasoning chain. This co-derivation and joint integration within SSGM ensure high degree of coherence between the generated content's semanties ('what') and its spatial arrangement ('where'), a hallmark of our framework's 's design.
Following InstructPix2Pix (L we incorporate reference image guidance as the third SSGM pathway. For editing tasks, the source image serves as reference, while for text-to-image generation, use black reference image for architectural consistency. This design enables a seamless transition between generation and editing tasks without architectural modifications, All references are processed through the VAE encoder to extract visual features G,,.

---

Method
Architecture
Single Obj.
Obj.
Counting
Colors
Position
Binding
Fruzen Text Encuder Mapping Methods
SDvL.5
37
Unet+CLIP
0.43
0.97
0.38
0.35
0.76
0.04
0.06
SDv2.1 37
Unet+CLiP LIP
0.50
0.98
0.51
0.44
0.85
0.07
0.17
SD-XL
Unet+CLIP
0.98
0.74
0.39
0.85
0.15
0.23
DALLE-2 2-222
35
Une++CLIP
0.52
0.94
0.66
0.49
0.77
0.10
0.19
(d=24)
MMDIT+CLIP-TS
0.98
0.74
0.63
0.67
0.34
LLMsALLLMy Enhanced Methods
LayonGPT
Unetel luma
0.41
0.97
0.51
0.26
0.56
0.11
0.07
LlamaGen 42
Autoregressive
0.32
0.71
0.34
0.21
0.58
0.07
0.04
Chameleon
Autoregressive
LWM 24
Autonegressive
0.47
0.93
0.41
0.46
0,79
0.09
0.15
SEED-X 13
Lnma
0.58
0.80
Emu3- Gen
+7
Autoregressive
0..4
0.98
0.71
0.34
0.81
0.17
0.21
Janus
Autoregressive
0.84
Flow 27
Autoregnessive
0.63
0.97
0.59
0.45
0.83
0.53
0.42
GoT Framework
Unet+Qwen2.5-VL
0.99
0.69
0.67
0.85
0.34
0.27
Table 1: Evaluation of text-to-image generation on GenEval benchmark [14] Attribution.
Obj.: Object. Attr.
5.3
Guidance Strategy
We employ classifier-fres guidance strategy integrating semantic, spatial, and reference image guidance. During diffusion, the score estimation Eg is calculated as a weighted combination of conditional and unconditional scores. Let Eo 50(2,, Ø, Ø, 2) denote the unconditional score estimate, & E0(2,, Ø, Ø, G+) the estimate conditioned only on reference image features Gr, Er,r = Eœ (2,,,, Gr, Ø, Gr) the estimate conditioned on semantic guidance G2 and reference features Gr, and Ets,r (2,, G., G., G,) the estimate conditioned semantic guidance G,, spatial guidance Gs, and reference features G,. The combined score then:
Eo ar(E,, e,) + ag Ct.r) + ar(er - Eo)
Here, 2, is the noisy latent. Guidance scales ae Ca. and a, control the strength of each respective guidance type (semantic, spatial, and reference), while Ø denotes null conditioning. During training, we randomly sample conditioning combinations with probability of 5%, excluding the fully- conditioned case Et,,r, to enhance robustness. Guidance parameters are introduced in Sec. 6
6 Experiments
We evaluate GoT framework on text-to-image generation, interactive image generation, and image editing. Experiments show quantitative improvements and qualitative benefits of our reasoning-guiche approach, with ablation studies validating our design choices.
6.1
Text-to-Image Generation
6.1.1 Quantitative Results
Tab. presents evaluation of text-to-image generation (T2I) on GenEval 14 The comparison spans two main categories of models: those employing frozen text encoders for direct comnpt-to-imad generation (primarily diffusion-based approaches) and those leveraging LLMs or MILMS to enhance the generation process. On T2I task, GoT framework adopts a, = 7.5 and a, 4.0, and more discussions a tuning are shown in Appendix Sec. [C22 The results show our framework achieves the highest overall score of 0.64, outperforming both Trozen text encoder methods and L.M/MLLM- enhanced approaches. GoT excels particularly in single object (0.99), counting tasks (0.67), and color tasks (0.85), demonstrating the effectiveness of our reasoning guiided generation paradigm. While methods like JanusFlow (27] perform better in position and attribute binding tasks, GoT's balanced performance across all metrics validates that incorporating explicit reasoning mechanisms enhances compositional generation abilities.
Among the WOMM/LLMMenhaann methods, our approach outperforms recent systems like Janus [50] and JanusFlow (27 in overall performance despite their advantages specific areas. This suggests that while autoregressive models excel in certain spatial tasks, our GoT framework's 's structured reasoning provides more consistent performance across diverse generation requirements.
6.1.2 Qualitative Results
In addition to the outstanding compositional text-to-image generation capability, GoT framework also exhibits high generation quality. In Fig. 4 we showcase the generation results of model across
7

---

spocnuct Booling picheesque Milly with
holés a basket rouss while attnng alone banch in look garder.
sovered treea and rodky shones Lake Tohox.
anowboard
Figure 4: Text-to- Image samples generated by model. The GoT framework can plan object placement based on the imput caption and generate highly aligned and aesthetic images accordingly.
ckinet neal brawn lore
Amal dage
boll reponniirm....
nacoer opple
allend energe
Figure 5: Samples on interactive generation with GoT framework. By modifying GoT content (description and bounding box position), user can customize their text-to-image process with: 1. Object replacement Object position adjustment Object attribute modification.
diverse set of prompts. We present samples from compositional prompts containing multiple objects, incorporating object attributes, relationships, and relative spatial positions. Our model effectively plans the placement of different objects, producing coherent and aesthetically pleasing images.
6.2 Interactive Generation
In our experiments, we further demonstrate the interactive capabilities of the GoT framework, illus- trated Fig. [ This approach enables user control over the generation process by modifying the GoT content, including both textual descriptions and bounding box positions. Users can customize their text-to-image generation through three primary interaction types: object replacement, object position adjustment, and object attribute modification. The examples showcase how the framework maintains overall scene coherence while precisely implementing the requested changes. This flexibility provides an interpretable and manipulable interface for text-to-image generation that traditional black-box systems lack, allowing precise control over the output without requiring expertise.
6.3 Image Editing
Method
Params.
Emu-Fdit
ImagenHub
Reason-Edit
CLIP-1
CLIP-T
GPTT-4o Eval.
GPTT-4o Eval.
6.3.1 Quantitative Results
IP2P
0.9B+0.11 1B
0.834
0.219
0.308
0.286
As shown in Tab. [2] we evaluate
MagicBrush
MGIE [10
53
0.90+00.1 IB
0.9B+7B
0.838
0.783
0,253
0.222
0,392
0.513
0.334
0.264
our GoT framework against state- of-the-art image editing methods across multiple benchmarks. On Emu-Edit benchmark [40]. GoT framework achieves the highest scores for both CLIP-I (0.864) and CLIP-T (0.276) metrics, out- performing previous methods including CosXL-Edit [4] and
Eanu-Edit HD
0,859
0.231
SEED X [13
2,8B+14B
0.825
0.272
0.166
0.239
SmanFFii 17
0.9B+7R
0.572
CosXL-Edit 4
0.860
0.274
0.464
0.325
GoT Framework 2.8B+3B
0.864
0.276
0,533
0.561
Table 2: Quantitative comparison on image editing benchmarks. denotes that SmartEdit mainly supports removing and replacing operation and is not designed for general editing operations.
Emu-Edit [40] Since CLIP-I and CLIP-T cannot fully reflect editing accuracy, also evalu-
ated using GPT-4o [I], which aligns better with human evaluation [19]. On ImagenHub 20), our
approach attains the highest score of 0.533. On the reasoning- based Reason Maason-EEitt benchmark [17]. model achieves strong score of 0.561, second only SmartEdit (0.572) [7], which is specially designed for reasoning removing and replacing operations. This demonstrates our method's strong editing ability, especially in complex reasoning settings. GoT framework shows consistently superior performance while maintaining competitive parameter efficiency (2.8++3B) compared to approaches like SEED-X (B.8BB+14B [3]. In the editing task, GoT framework adopts 04 4.0, a = 3.0, and
Or = 1.5. The evaluation prompt of GPT-4o is shown in Appendix Sec. E.T
8

---

6.3.2 Qualitative Results
IP2P
MagitBrash
We present qualitative comparison of image edit- ing with other models in Fig. 6 Our approach demonstrates superior performance across di- verse editing scenarios requiring semantic- spatial reasoning. Examples highlight our frame- work's distinctive capabilities: First, model accurately identifies and localizes objects ref- erenced through indirect descriptions. Second, our approach handles complex spatial instruc- tions effectively, such as removing specific sig- nage adding delicate elements to precise loca- tions. Third, our framework excels multi-step editing operations, shown in the bottom exam- ple. The red bounding boxes visible in our re- sults indicate the coordinates predicted by the MLLM within the GoTT framework, providing in- terpretable insight into how our system reasons about spatial relationships during editing.
Fumove o "huup luft aign the pole the foregroand.
Add snal delccte purple Rowar the hop the plane the foreground
mol used in Remove this tool.
6.4 Ablation Study on Framework Design
Replace the desset dunes with a lake and than culd binds tie sky.
Qualitative results of image editing. Our GoT framework demonstrates superior per- formance in settings that require comantic-spatinn reasoning. Red boxes indicate the coordinates pre- dicted by MLLM within the GoT framework.
Figure 6:
We conduct ablation study to analyze the im- pact of different components in our framework. Tab. 3 presents the results of our study, where we progressively integrate different components into the baseline and evaluate their effects on GenEval and ImagenHub benchmarks.
The baseline model leverages Owen2.5-VL-33 and SDXL but does not incorporate GoT reasoning chains. trained with FLUXX and OmniEdit-Go's for 10,000 steps. Adding GoT reasoning chains to the baseline model enables the LLM to achieve stronger semantic guidance capabilities. The reasoning process helps LLM plan for guidance in generation.
Introducing the Semantic-Spatial Guidance Module (SSGM) further enhances model performance. par- ticularly in image editing. SSGM provides spatial control over the diffusion model, ensuring object placement aligns more accurately with the reasoning process. This enables fine-grained editing, reflected by the significant improvement in the ImagenHub evaluation. However, in GenEval, only the position category is notably affected by SSGM, which explains the minor performance gain.
Method
Go'T
SSGM
Pretrain
GenEval
ImagenHub
Baseline
0.176
+ GoT
x
0.40
0.181
SSGM
x
0.370
GoT Framework
0.64
0.533
Table 3: Ablation study of our GoT framework on GenEval overall and ImagenHub GPT-4o eval.
Our final framework, which includes GoT reasoning, SSGM, and an extensive 60,000-step pretraining phase, achieves the highest scores, demonstrating the significant benefits of prolonged pretraining and the full model design. The ablation study confirms that each added component contributes positively to the overall performance, validating our framework design choices.
7 Conclusion
We introduced Generation Chain-of-Thought (GoT), a paradigm integrating MLLM reasoning into visual generation via explicit semantic-spatial chains, overcoming limitations in understanding object relationships. Our approach transforms visual generation from direct mapping into reasoning- guided process with precise spatial control, addressing limitations in existing methods that lack explicit understanding of object relationships and arrangements. Leveraging large-scale datasets Guidance Module, Go'l achieves state-of-the-art interactive visual
and novel Semantic-Spatie synthesis aligned with human cognition. This advancement promises enhanced creative workflows and more controllable AI; however, the increased power to generate and manipulate visual content also demands robust ethical frameworks to mitigate misuse, such as creating misleading media.

---

8 Acknowledgement
This study was supported in part by National Key R&D Program of China Project 2022ZD0161 100. in part by the Centre for Perceptual and Interactive Intelligence, a CUHK-led InnoCentre under the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Government, in part by NSFC-RGC Project 4UHK498/24, and part by Guangdong Basic and Applied Basic Research Foundation (No. 2023B1515130008, XW).
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Igge Akkaya, Florencia Lconi Aleman, Diogo Almeida, Janko Altenschmidr, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXtn:2303.08774 2023.
[2] Jinze Bai, Shuai Bai, Shusheng Yang. Shijic Wang. Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile and beyond. arXiv preprint arXiv:2308 12966, 2023.
model for understanding, localization, text reading,
[3] Shuai Bai, Kegin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang. Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint wrXiv:2502.13922 2025.
Frederic Boesel and Robin Rombach. Improving image editing models with generative data refinement. In The Second Tiny Papers Track ICLR 2024, 2024.
[5] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and puttern recognition, pages 18392-18402, 2023.
[6] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[7] Rongyao Fang, Chengqi Duan, Kun Wang, Hao Li, Hao Tian, Xingyu Zeng. Rui Zhao, Jifeng Dai, Hongsheng Li, and Xihui Liu. Puma: Empowering unified milm with multi-granular visual generation. arXiv preprint arXiv:2410. 13861, 2024.
[8] Rongyao Fang, Shilin Yan, Zhaoyang Huang, Jingqiu Zhou, Hao Tian, Jifeng Dai, and Hongsheng Li. Instructseq: Unifying vision tasks with instruction-comniiionn multi-modal sequence generation. arXiv preprint arXiv:331 1. 18835, 2023.
[9] Weixi Feng, Wanrong 7hu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuchai He, Sugato Basu, Xin Fric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems, 36: 16:18225-11 18250, 2023.
[10] Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang. and Zhe Gan. Guiding instruction- based image editing via multimodal large language models. arXiv preprint arXiv:2309. /7102, 2023.
[11] Yulu Gan, Sungwoo Park, Alexander Schubert, Anthony Philippakis, and Ahmed M Alaa. Instructev: Instruction iatruction-tuned text-to-image diffusion models as vision generalists. arXiv preprint ar000:2300000390 2023.
[12] Yuying Ge, Sijie Zhao. Chen Li, Yixiao Ge and Ying Shan. Seed-data-edit technical report: A hybrid dataset for instructional image editing. arXiv preprint arXiv:2405.04m0 04007, 2024.
[13] Yuying Ge, Sijic Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song. Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi- culti-granularitty comprehension and generation. arXiv preprint arXiv:2404.1 14396, 2024.
[14] Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132-52155, 2023.
[15] Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint 139966 2025.
[16] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
10

---

[17] Yuzhou Huang, Liangbin Xie. Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge. Jiantao Zhou, Chao Dong, Rui Huang. Ruimao Zhang, el al. Smartedit: Exploring complex instruction-base image editing with multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8362 8371, 2024.
[18] Diederik P Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
[19] Max Ku, Dongfu Jiang. Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation. arXiv preprint arXiv:2312. 2023.
[20] Max Ku, Tianle Li, Kai Zhang, Yujie Lu, Xingyu Fu, Wenwen Zhuang, and Wenhu Chen. Imagenhub: Standardizing the evaluation of conditional image generation models. arXiv preprint a:X1v:2100001556 2023.
[21] Black Forest Labs. Flux.
http://llckkk-ae http:/llacc-ooeeeaaaa//iial/llie 2024.
[22] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang. Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to -oitto-image generation. In Proceedings of the IEEE/CVF conference on compuder vision and pattern recogmition, pages 22511-22523 1-22521, 2023.
[23] Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. Lim-grounded diffusion: Enhancing prompt under- standing of text-to-iine to-image diffusion models with large language models. arXiv preprint arXiv:2305.1365: 2023.
[24] Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with ringattention. arXiv e-prints, pages arXiv 2402, 2024.
[25] Haotian Liu. Chunyuan Li, Yuheng Li, Bo Li. Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, oct, and world knowledge, January 2024.
[26] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892 34916, 2023.
[27] Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Liang Zhao, et al. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. arXiv preprint arXiv:2411.07973 2024.
[28] Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-iman -mage diffusion models. In Proceedings of the AAAI conference on artificial intelligence, pages 4296 4304, 2024.
[29] Trong-Tung Nguyen, Duc-Anh Nguyen, Anh Tran, and Cuong Pham. Flexedit: Flexible and controllable diffusion-based object-centric image editing. arXiv preprint arXiv:2403. 18605, 2024.
[30] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, llya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:21 1074), 2021.
[31] Zhiliang Peng, Wenhui Wang. Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306. 14824, 2023.
[32] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arxiv:3307.01952 2023.
[33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PmLR, 2021.
[34] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, 21(140):1-67, 2020.
[35] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint 220.:2204.0612,, 1(2):3, 2022.
[36] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning. pages 8821-88 8831. Pmir, 2021.
11

---

[37]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[38] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, al. Photorealistic texi-to- image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479- 36444,,44494 2022.
[39] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-Sb: An open large-scale dataset for training next generation image-text models. Advances in newral information processing systems, 35:25278- 252944, 2022.
[40] Shelly Sheynin, Adam Polyak, Uriel Singer. Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 8871-8879, 2024.
[41] Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, al. Joureydh:: A benchmark for generative image understanding. Advances in neural information processing systems, 36:49659-49673 2023.
[42] Peize Sun, Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model heats diffusion: Ilama for scalable image generation. arXiv preprint arXiv:2406.06522 2024.
[43] Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[44] Chameleon Team. Chamelcon: Mixed-modal carly-fusion foundation models. arXiv preprint vrXiv:2405.09811 2024.
[45] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
[46] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Kegin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge. et al. Qwen2-vl: Enhancing sssion-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 121111 2024.
[47] Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li. Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 188699999889..888 2024.
[48] Cong Wei, Zheyang Xiong, Weiming Ren, Xinrun Du, Ge Zhang, and Wenhu Chen, Omniedit: Building image editing generalist models through specialist supervision. arXiv preprint 2rX1v:2411.07199, 2024.
[49] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, al. -ftin-of-thought prompting clicits reasoning in large language models. Advances in neural information processing systems, 13:24424-24837 2022.
[50] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xic, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation, 2024a. URL https://arxiv. org/abs/2410 10..144411....... 13848, 2024.
[51] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 2rXi:22412.15113 2024.
[52] Ling Yang. Zhaochen Yu. Chenlin Meng. Minkai Xu, Stefano Ermon, and CUI Bin. Mastering text-to- image diffusion: Recaptioning, planning, and generating with multimodal llms. In Forty-first International Conference on Machine Learning, 2024.
[53] Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for cnrruction-uudderr image editing. Advances in Neural Information Processing Systems, 06:31128-31444 2023.
[54] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVE international conference on computer vision, pages 3836-3847, 2023.
[55] Le Zhuo, Ruoyi Du Han Xiao, Yangguang Li, Dongyang Liu, Rongje Huang, Wenze Liu, Lirui Zhao, Fu-Yun Wang. Zhanyu Ma, et al. Lumina-next: Making lumina-12x stronger and faster with next-dit, arXiv preprint 4rXiv:2406. 18583, 2024.
12

---

A Training Details
Our training process implements a two-phase approach: pretraining using LAHR-GoTE JourneyDB- GoT, and OmniEdit-Goo datasets (60,000 steps), followed by finetuning with FLUX-GoTT, OmniEdit- GoT, and GGED-Edi-MulliTum-Goo to efficiently update the Qwen2.5-VV decoder's parameters while fully optimizing the SDXL-based diffusion module. The process operates end-to-end, jointly optimizing the MILM Go'T' cross-entropy token loss and diffusion MSE loss with equal weighting 1.0, demonstrating robustness without complex hyperparameter tuning. We adopt cosine learning rate scheduler with 500 warmup steps and maximum learning rate of x 10-a.
(10,000 steps). We employ low-rank adaptation (LoRA) [66]
During the fine-tuning stage, we train the model on FLUX-GoT, OmniEdit- GoT, and SEED-Edi MultiTurn- GoT for 10,000 steps. In this phase, we the warmup steps to 200 and the maximum learning rate to 10 10-
For both stages, we use the Adam optimizer with 3| = 0.9, 8n = 0.98, and c = 1 x 10-6. We also apply weight decay of 0.05 during training. The number of batch size is set to 128.
The LLM is fine-tuned using LoRA with = 32, LoRA alpha set to 32, and LoRA dropout rate of 0.05. For diffusion, we introduce noise offset of 0.1.
B Statistics of GoT T21 Datasets
The GoT T2I datasets yield rich annotations: LAHR-GoT samples with prompts averaging 110.81 characters, GoT descriptions averaging 811.56 characters, and 3.78 bounding boxes per image. Similarly, JourneyDB-Grr GoT annotations average 149.78 characters for prompts, 906.01 characters for GoT descriptions, and 4.09 boxes image.
C
Visualization Results
C.1 Qualitative Analysis of Image Editing and Interactive Generation
We provide additional examples to demonstrate the capabilities of the GoT framework. Fig. illustrates the image editing performance of our model. Additionally, we present the corresponding GoT content generated alongside each sample. Further examples of interactive generation using model are shown Fig.
C.2 Visualization of Multi-Guidance Strategy Hyperparameter Selection
We analyze the effect of hyperparameter selection in the Aulti-Guidance Strategy on the generated images, depicted in Fig. 9 The definitions of these hyperparameters provided in Sec. 5.3
D GoT Format and Examples
This section presents examples of the GoT format in our dataset. The GoT structure varies across different tasks, including text-to-image (T21) generation, single-turn editing, and multi-turn editing.
For text-to-image generation, Fig. 10 showcases examples from FLUX- ToT, JourneyDB-Gol and LAHR-GoT, Our GoT format represents the structured planning process of the upstream model in generating image content. It provides detailed breakdown of the various components within an image and their spatial relationships. To enhance spatial understanding, we append location information to key objects within the GoT representation.
Fig. iillstrates the GoT format for image editing within our dataset. For single-turn editing, GoT represents the reasoning plan of the upstream model for specific editing action. It consists of description of the source image, the object to be modified, the specific editing operation, and the resulting edited image. This structured process ensures step-by-step transformation, beginning with the original image, identifying the target object, applying the specified modification, and generating the edited image.
For multi-turn editing, GoT follows more complex structure, as it must encapsulate the breakdown of an instruction into sequence of consecutive steps. In practice, we first generate description

---

of the source image, then decompose the multi-turn instruction into series of step-by-step editing commands. At each step, GoT operates single-turn editing process, specifying the object to be modified along with the corresponding transformation. Finally, the process concludes with description of the fully edited image.
Furthermore, for image editing tasks, positional information is appended to each object to enhance spatial comprehension.
E
Prompts for Evaluation and Dataset Construction
E.1 Prompts for Evaluating Image Editing Performance
We provide the prompts used for evaluating image editing performance with GPT-4o in Fig. 12 We are using GPT-40-2024-11-20. The final score is the average of the minimum value of the scores for each sample.
E.2 Prompts for Text-to-Image Data Construction
Fig.
and 16 present the key prompts utilized in exx-tto-image data preparation.
E.3 Prompts for Image Editing Data Construction
Fig.
15 20
illustrate the key-step prompts employed in image editing data preparation.

---

Generated GoT from our framework
clasding Arred dighely to sile teem amiss and ween eppecoosne The barkgrosd is berred green, sageesing
The wrling
adited woidle deso meerist steling spright keod sightly
the sile"
3. spenlic
de adted
by bounding coardinates (367
which ascompasres
4. Asplaze the isidde reserket with block and wiile pasquss slanding dn rock facing left web in dattectve bieck and
The pleaag worge cles
steodagg
medh Aear badkyy focing forwoed and thir
hevth Armed sightly the itds, givin thes curiess aud obit appeemsecn The lackgro ofd remain bilred grewn the rook and facing lsft with is disvecive black and white planage and back decrly vilii.
paopuct
Cherge vidbs pangeis
Generated GoT from our framework:
fucure beis fap, and chassecoks
sosvee image show
and acsompasted
chanseccka cresey
pizced rable.
the obiec to be aditdd fhe fork, specheully deanbedd fock wit decorctive hasele" The speciic orea to be acited
After remsonng the fank, fhe will recrlealy istegrated lroo fhe bookground asariig the chassecahe and the plat weee odied wirdas nob remsies ciuld craayy fetore will weel bakes top, and
white
The olesseosle
eeppy. with fork presest
What oblect ceed
mllll Brewwe this
Genorated GoT from our framework:
and seak i Alge diiteees. Teo coustyrids feedingee with grees
depect
feveriomd, and the pertly clonly
costig soft light oser the scane. J. The specifc proeecouuss postte where
and
feid
be calted is delnsd by the beseding for coondneres (785, 6401,6600/900 6601, which
lut and cost postioeed the sseeiled area, esaring he bleeds naluaaly with the eatting lordccap and ligttng condiicee edited imnge depict serene mastrsid rows peoter cow graws the mndble ground The thy rensio party cloudyy casteg soff light over the landicip..
the speciic changer be eadi: Add
isll, greas
wille
ArM Aui
The ouante ksugs deyids covared brus and a cleer live she the foreground, a poir in abote ploced on rhe ice odding
wister
ficase (ake ralleckin the usossedng
2 The abjed
hed per of skates
3. ssec fo area acheed a deired the bounding box coardhater (309 658.6854..8880000 9511, which
d The sieie will treed papll evectsiing that skeps and from hor thangin fleir color eater alo adjuted aemooloyyyy
pards
of parple dotte
5 oetee
chow
rellecing
connre bees and clow like dy is fouguosdd a poir people ice slates planed ine adding a pop color otherwae mococfronctc lancheope
Turn the color o of dkates puople
Figure 7: More samples on image editing with the GoT content generated by our model.
Modby fre inck
Proephh live
Modty: change postios
by iropical semiiie.
orange bls
veliclee paried by road.
Modioal splor Mise..
Genenste
Modiied Goli
Gosernee Gol (432,711/779
Modkted (193577,
Figure 8: More examples on interactive generation.
3

---

=1.0
0, =2.0
a.=3.0
-4.0
-100
4.0
a, -7.5
a piano and a chair
a
=4.0
a, =7.5
A bride white dress stands garden, holding bouquet of flowers, with greenery and trees the background.
Figure 9: Visualization Multi-Guidance Strategy dyper-paraamter Selection. The above
text-to-image samples generated by GoT framework under different hyper-parameters.

---

FLUX-GoT
Prompt:
A yoing gir dressed as Frozen, wearing blue dress adomned with mowellkee pattems, matching crown, and holding o wand, stonds a snowy forest selling.
Gol:
young girl (262.40), (8662.299 stands is movv forest (9,0,,198,6611 dramsd as from Disnay's "Froaen." She wears vibrant furquoise dress adorned intricate silvar mnowlloke patters (36),914)/418897777 that cascades bahind hec blonde crows (578) III) leeluring lorge gon center right hand, halds wond (235, (23), 600) with blue handle e glittringg siver seor top, resunscent Esa's magical scepter, bookground sottl landscape snowcovered encharting atnosphare.
natching siver bei: dress fentures long sleeves a flowing
slyled
braids, secured
sporkling
showcosing serene light fihuring through the branches, creating magical and
JournayD8.GoT
Prompt:
A yoing finemon in a helmet and underr stands with on ase, firing a lorgs, arange floms againet maetly white background, rendered in drown, colorful style reminscent of Colvin ard Hobbes.
GoT:
The image depids o young firemon (244, 299,16613 902) standing confidently in front of large, billowing fire (479 01, 582). The firemoo is dressed in detaled, hand fireman's unforn, complete with holmet (269 (00,)00))1 4744) bools and on axe (44, 616).668 646) . The unform is rendered in a worm, carthy color palatto, with fhe holnot featuring prominent visor. The five isell, is a dranohi orangerre red spectado, with snoke rising in chootic, blobilke form the background, The scone set against a mostly white background, which helps to highlght dynanic, fhe firemss pastionnd the foreground, ready chootic nature
whrrarr colors and intricate detail the fire mas and challenge the fire, while sense argency intencity to sceae. The image reminisren the etle Calvin and Hobbes, is detailed, expreisive work and sense of norralive lonsion. The use wotercofor in backgraund adds painterly tauch, enhonting visual impact scene.
The overall compostion is
LAHR-GoT
Prompt:
Shil life painting featuning bottle of wine, glan filld with red wine, and bax of calarful chocolates on table.
GoT:
The painting "Indvlgonce" by Cory Chiffrrd captures sall life arrongement of laserious confections and a glas of red wine, evaking o sence of opulence and relosation. A bas of assorted chocolates, each wrapped in vibront foil open, revealing o variety of shopes and colora, incluelin gold, grewn, red, and yellow. The chocolates (119, (60, 508) are scattered ground the box, come sill their wrapper, whis others have besn enjoyed, A bottle of red wine (751, 49), 779) stands toll, its grees gloss rellecting the ombeal light eccompanied by a fillod wine glass (629. (611,803, 838) that rests nearby. The background is deep, rich blue, contrasting with the while toblecloth that spors the lover partion of the painting creating o serse of depth and focus on the central objects. The overall compostion balanced, with the chocolates and wise glass posiioned cantrolly, drawing the viewer's eye to the indblgunt scone.
Indion Scout matorcycle with brown leather saddleboge and and black color scheme, parked in front of nusic horn.
Gol:
The inage showsoses Indian Scout motorcycle (325, 123) prominently postioned in the foreground. The motorcycle features sleek, vintage design with grean and cream color schame, accented by chrome datalingg The fromt wheel is sightly turned to left, and the bike is equipped with a dlerr windshield and a par of round headights. The seat is brows loother (S35,,50)||(55, d] complementing the andelebogs (335, 449) 619) attached the swar. The motorcycle is parked on gravel surfoce, with rustic barnilke dtucture (0.0).197 (P97/699) in the background, choracterized by corrugated motal siding and wooden door. The overall compositon places the motorcydl as the central focus, with the bam serving as contreating backdrap that amphasizes the bite's clasic asathatic.
Figure 10: Examples of GoT dataset for eext-to-image generation, including FLUX-GoT, JourneyDB- GoT, and -ineeetics-----h-eee Seiih-RResoltion--GG GoT.
5

---

OmniEdii-Go
lestuctis::
langiases parple.
parson
loldng leather nestral rightly meed.
sungleme the rediviivll spesery
abject edited defose the
brows easgleuse case.
coeediaates (361.69).1655,
The spealle chaspes be mods include changing the coler of the zusgleues case papil, emaring target image description.
raker and nuglumee, halding sanglouse case with pai aung annr iaade. The beokgrousd will recon vechonged bat light
adhed
skoar
Intrucioon
kot
foreground.
and
foregroaed, rimoandd by old weathard wasted poster for the Dolon Goug and Jacs beriground prity spectically ckject adited
vittge decorbhed having brown
bat red poecko, and apor-
defived (Pa, (555, 599) which axcompeues extrr comboy fiarn.
bounding
Remove coukay coople lotely from the ivage masisliiiig the samoanding weeted
the old waeterre posters and the vettgge Westers theee.
image show collection vistage
posters iscluingg thase The
Dolin Gong
"WANTEO" the
roclground will remain a prasgy, aged well with sous eution, zuggeetig a chootic area
WANTED
cowboy was oppear sromles Blending butencel pasturs and Kaskgroud.
SEED-EdinMMulTTrn-Goo
Chinge the and judke on the midele in light grees, resoe the bonk os the right ool food
bodigroadd pure
Galt
source image shows paren waering jocket sooding a wallway aurounded Ablidd
wilk derid unter doady shy.
the the Sounding
coordivate (35, (917550)) Change the color the jocket light green and esuere the sumser and reesin stible
defond by the hounding box condirale (545.0), (999,552) Assore the Wook of ssugals from this
adited defines the houwding coandioates (0,0) (p99, P1a), Raplace the basiggound with forest featuring diudy day body of walee and distast
thid
gpesfic
creater grese eanepy overhead.
editdd the fourth ctop, the abject edited leit side
The defined by the bounding box conrliette 10, (243 1392 Add one grees with o shrey. The adted image depect
light joslar with the nember 15 the bock, walking down poveel path Reeted by frees with fesh green fallge cur period the the
cloudy day body of water and distet Mlddd with avildings the baskgraund.
Figure 11: Examples of GoT dataset for image editing, including OmniEdit-GoT for single-turn editing and SEAD--EEi--Mutituunn for multi-turn editing.

---

Human:
You professional digital artist. You will have evaluate the effectiveness of the AI- generated image(s) based on the given rules, You will have to give your output in this way (Keep your reasoning concise and short.): "score": [...]. "reasoning" " and don't I output anything else.
Two images will be provided:
The first being the original Al-generated image and the second being an edited version of the first. The objective is to evaluate how successfully the editing instruction has been executed in the second image. Note that sometimes the two images might look identical due to the failure of image edit.
From a scale 0 10:
A score from 0 to 10 will be given based on the success of the editing.
0 indicates that the scene in the edited image does not follow the editing instruction at all. 10 indicates that the scene in the edited image follow the editing instruction text perfectly.
If the object the instruction is not present in the original image at all, the score will be
A second score from 0 to 10 will rate the degree of overediting the second image.
0 indicates that the scene in the edited image completely different from the original.
10 indicates that the edited image can be recognized as minimal edited yet effective version of original.
Put the score in a list such that output score [scorel, score2], where 'scorel' evaluates the editing success and 'score2' evaluates the degree of overediting. Editing instruction: <instruction> <Image> Source Image </Image>
<Image> Edited Image < </maaee> Assistant:
Figure 12: Prompt for GPT4-o image editing evaluation. We are using GPT-40-2022-11122 The final score is the average of the minimum value of the two scores for each sample.
Human:
<Image> Image </Image>
You are an advanced AI visual assistant specializing in highly detailed and comprehensive visual analysis for image. Your role is to generate a single, descriptive paragraph that encapsulates all relevant details about an image. Here is the provided image prompt for this image: <prompt>.
If the provided prompt aligns with the image, enhance it by adding detailed observations about the objects, their colors, shapes, textures, numeracy, and spatial relationships. If the provided prompt does not match the image content, disregard it and craft a complete description based solely on the visual elements you observe. Consider the 2D-spatial relationships (e.g.. "to the left of," "near," " "aligned with") and 3D-spatial relationships (e.g., "in front of," "above," "at distance from") when describing the scene. Include details about the overall composition, highlighting how elements are arranged relative to each other, their groupings, and any complex interactions or dynamic elements within the scene. Pay close attention to the interplay of colors, textures, and shapes, ensuring that the description reflects both the visual richness and structural composition of the image. Ensure to provide the description as one single paragraph, without preamble or additional explanation. Assistant:
Figure 13: Prompt for detailed recaption for -otttto-imae data.
7

---

Human:
You tasked with identifying and extracting all the real object names from detailed caption. An object name refers to any tangible or physical entity mentioned the caption that can visually grounded in the image. Ensure not to include any adjectives or single-word descriptions that do refer specific object, such "background." Please follow these instructions:
Identify all object names in the caption in the order they appear. Maintain the exact wording of each object name is in the caption, including case consistency. Output the object names Python list format. For example, consider the following caption: Example 1:
"In the image, person is prominently featured a vibrant pride parade, exuding confidence and pride. They are adorned in an extravagant outfit that mirrors the rainbow flag, with deep V-neck top in bold, colorful stripes of red, orange, yellow, green, blue, and purple. The person's 's hair styled in striking rainbow color, complementing their outfit. They are surrounded by a lively crowd, with individuals wearing various colors and accessories, adding to the festive atmosphere. The background reveals bustling street scene with buildings and trees, suggesting an urban setting. The overall composition dynamic, with the person at the center, drawing attention to their vibrant attire and the energetic parade around them." Your output should be list of object names like this:
['person', 'pride parade', 'outfit', 'V-neck top', 'The person's hair', 'a lively crowd', 'individuals', 'street', 'buildings', "trees'] Example 2:
"The image depicts a young boy with slender features and pale complexion, exuding an of arrogance and coldness. His white-blonde hair is slicked back, adding to his composed demeanor. The boy's eyes intelligence. He dressed in a white shirt with a blue and white patterned collar, which contrasts with his pale skin and adds touch of elegance to his appearance. The overall composition is balanced, with the boy centrally positioned against a dark background that accentuates his features and the sharpness of his expression. The interplay of colors, textures, and shapes creates visually striking and emotionally charged image."
striking shade of cold grey, reflecting sense of detachment and
Your output should be list of object names like this: [' y'uung boy', 'white- sehite-blonde hair', "The boy's eyes", 'shite shirt'] Now, given the following caption, extract the object names in the same format: <caption> Assistant:
Figure 14: Prompt for identifying objects in text-to-image caption.
Human:
Please tell me according to the instruction: <instruction> Which object is being replaced with another object? Please only answer the exact name of the two objects using the same words from the instruction. Use the format of Python list including the two object names. The first is the object' and the second is the "another oojjct''object.. Assistant:
Figure 15: An example of prompt for parsing the edited object. This is used when the task type is 'replace'.
Human:
<Image> Image < </Image>
Please provide the bounding box coordinates of this sentence describes: <object_name Assistant:
Figure 16: Prompt for grounding object. This works for both text-to-image and image editing data.

---

Human:
<Image> Image </Image>
You are an Al visual assistant, and you are seeing single image. Please describe this image in one paragraph using no more than two sentences. Always remember to include describing <object_namme name> in the image.
Assistant:
Figure 17: Prompt for image description for image editing data.
Human:
<Image>
Cropped Image </Image>
Please describe <object_nnae __ame>>>ae>ee briefly in several words no more than one sentence. Assistant:
Figure 18: Prompt for cropped image object description for image editing.
Human:
You are a helpful visual assistant. I have an image editing data with the original instruction: <instructions>. want to augment the instruction to obtain more free-language format instructions.
Your task is to rewrite this original instruction in English into three distinct, human-like, free- form instructions that convey the same meaning but varied language and phrasing. The instructions should reflect how humans might naturally request image edits.
Please provide me with three more instructions that have the same meaning as the original instruction but in more free-language format. The new instruction can be in any format that human might input as an editing instruction. The first instruction should be relatively concise. Use the format of Python list which includes three items strings. Assistant:
Figure 19: Prompt for reinstruction for image editing data.

---

Human:
You are a helpful assistant for designer. have image editing data with the following information: instruction: <instructions>, description of source image: <source _ecc_> description of target image: saaree_ddssc2 <coord> poojjct_desee Assume you to the edit instruction and the source image. Your task to provide a step-by-step chain of thought for the image editing process which only includes the image editing processes. The chain of thought can includes the following several type steps (can not in this order, not includes these words in the answer): Describe the source image; the object to be edited; the specific area to be edited; Identify the specific changes to be made; Describe the image after the edit. All information besides the instruction should be considered derived from the source image. The output is meant to train a multi-modal large language model that takes the source image and instruction as input, generates the editing chain of thought, and then outputs the edited image. Therefore, your response should consider this application and provide clear, concise reasoning numbered steps (1. 2. 3. ... etc). The response should be purely reasoning text and formatted succinetly. Ensure your answer be brief and few steps.
visual assistant with access
In context learning, example 1:
1. The source image shows a grand, classical building with intricate stone carvings and statues. One prominent statue, a female figure, stands a pedestal, holding, a torch and book. The building features arched windows and sign that reads "Learning Center."
2. The object to be edited is a statue of a woman holding a torch and a book.
3. The specific area to be edited is defined by the bounding box coordinates ((554, 166), (768, 711)), which encompasses the statue.
4. Remove the statue completely from the image while maintaining the surrounding architectural details and other elements like the building's s facade, arched windows, and the "Learning Center" sign. 5. The edited image will show the grand, classical building with intricate stone carvings and the "Learning Center" sign. The statue, a female figure holding a torch and a book, will no longer be present, and the area where the statue was will appear seamless with the surrounding architecture. The building's s arched windows and stone facade will remain intact.
In context learning, example 2:
1. The source image depicts a snowy mountain slope with a ski board in the foreground, indicating skiing or snowboarding activity. The background features clear blue sky and rocky terrain, suggesting a high-altitude or alpine setting.
2. The inserted object is skier in a black jacket, complete with goggles, sitting on a snowboard.
This skier will be positioned in the center of the slope, facing downhill, sitting on a snowboard.
3. The specific area to be edited is within the bounding box ((882, 303), (782, 813)). where the current object (a ski board) is located. This area needs to be replaced with the new skier.
4. The image now shows a skier dressed in a black jacket and goggles, sitting on snowboard on snowy slope. The background features a clear blue sky and rocky terrain, with other skiers and equipment visible in the distance. The skier is positioned in the middle of the slope, looking downhill, seamlessly blending with the existing scene.
In context learning, example 3:
1. The source image depicts group of women and child standing beach, all dressed in vibrant, summery outfits. The scene bright and cheerful, with the ocean and sky forming a picturesque backdrop. The style of the image is casual and candid, capturing a moment of joy and togetherness. 2. The edited area is ((O, O), , (999, , 999)), which is the whole image. The object to be edited is the group of women and the child, along with the beach and the background elements. These need to be transformed into traditional Chinese ink painting style.
3. After the edit, the image will depict group of women and child standing in traditional Chinese ink painting style, dressed in elegant, flowing garments. They will be positioned against a backdrop of serene mountains and a tranquil sea, with the overall composition reflecting the classical and detailed style of traditional Chinese ink paintings.
Assistant:
Figure 20: In-context assembling GoT prompt for image editing data.
10

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction, including the explicit list of contributions, clearly articulate the main claims: the Generation Chain-of-Thouge (GoT) paradigm, the construction of large-scale GoT datasets, a unified framework with novel Semantic-Spatie Guidance Module, and demonstrated performance improvements. These claims well- supported by the experimental results detailed in Section 6.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
The paper has discussed limitations in the experiment and the conclusion section.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tens system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?

---

Answer: [NA]
Justification: The paper's primary contributions are empirically driven, focusing on the novel GoT paradigm, a framework, and large-scale datasets. It does not introduce new theoretical results that would necessitate formal mathematical assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The paper provides extensive details for reproducibility. Sections 3, 4, and 5 describe the GoT paradigm, dataset creation (including pipeline details and prompts in Appendix Sec. 12), and the framework architecture (including SSGM and guidance strategy).
Justification:
Appendix Section 8 ("Training Details") further specifies training parameters.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. e For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 5. Open access to data and code
12

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification:
The paper does not provide open access the data and code,
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc
public//uiide/CodeSitiee itw.oodeeesmmiiisooooooooi.e.e for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
criips.cc/pubiic/ggg wwwp:///wweeeeeeeeeeeeee.eeeeeee /at////ooeeSeeeeeeeeeeeeee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 5.3 ("Guidance Strategy"), Section 6 (Experiments, e.g.. ). and Ap- pendix Section ("Training Details") provide crucial training and testing details. These include information on dataset usage for different training phases, LoRA parameters, opti- mizer settings (Adam with specific betas, epsilon, weight decay), learning rates, batch sizes, guidance scales, and some notes on hyperparameter selection (e.g., Appendix Sec. 10.2).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper presents quantitative results in Tables 1, 2, and 3 using various metries but does not report error bars, standard deviations from multiple experimental runs, or formal statistical significance tests these scores.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
13

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтroг rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides substantial detail on the compute resources for dataset creation ("over 3000 NVIDIA A100 GPU days").
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
The research focuses advancing visual generation and editing through explicit reasoning, aiming for outputs better aligned with human intent. We have reviewed the NeurIPS Code of Ethics and believe our work, including dataset creation methodologies and model development, aligns with its principles.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The paper discuss the broader impacts the conclusion section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
14

---

If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: The paper states an intention release datasets and models but does not currently describe specific safeguards (e.g., usage policies, content moderation mechanisms) mitigate potential misuse of the advanced generative capabilities.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [No]
The paper properly cites the original sources for existing assets such as datasets (c.g., waiion-nstheeeee JourneyDB, OmniEdit and foundational models (c.g.. Qwen2.5- VLL SDXI.).
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
15

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, palaaets.paperssiii paperswi thcode. cotastts
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces new large-scale GoT datasets (over 9M samples ). These datasets documented within the paper regarding their construction methodology (Section 4. 1),, sources and statistics (Section 4.2. Appendix Section 9), and GoT format with examples (Appendix Section 1 1). This documentation is intended to accompany the assets upon their stated release.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
16

---

We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: The use of LLMs and MLLMs is fundamental to this research. The core GoT paradigm involves an MLLM generating explicit reasoning chains. MLLMs (Qwen2- VI., Qwen2.5) extensively used in the automated dataset creation pipeline detailed in Section 4.1. The proposed GoT framework itself utilizes an MLLM (0wen2.5-VL-31 as its reasoning engine (Section 5.1). Furthermore, GPT-4o is used for evaluation in image editing experiments (Section 6.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (bttps://neur)) wwt.c///eeeeeeeee/222555555555 for what should should not be described.
17


Reviews:
[{"review_id": "lM2163rizO", "reviewer": "Reviewer_mkjA", "summary": "This paper argues for chain-of-thought reasoning in image generation - e.g., given an user input prompt, generate a more detailed intermediate prompt (that often includes spatial layout coordinates) that produces higher-quality generated images. Such generative chain-of-thought traces are used to improve both image generation and image editing tasks. To do so, this work builds a large-scale dataset of such reasoning chains, with 8 million image-generation examples and 1 million image-editing examples. The datasets is automatically built by running captioning VLM models on source images, to produce such detailed chain-of-thought prompts. Architecturally, this paper also  method a novel end-to-end approach that does not explictly condition image generation on the chain-of-thought prompts, but instead conditions image generation on 64 learnable embeddings (that a VLM is trained to produce). The final diffusion-based image generation module conditions generative denoising on the learnable embeddings along with a spatial image / crop / bounding box masks (derived from the chain-of-thought prompts).", "questions": "- I would like to see the detailed ablations listed above. This would reveal if the improvement is really coming from the data or from the architectural modifications. Traditional chain-of-thought prompts have the added benefit of being interpretable, which the learned embeddings do not. So it would be useful to better justify the proposed approach.\n- Are the authors planning on releasing their data? I suspect the real innovation comes from this data, and so I'd be more willing to fight for acceptance if that could also be regarded as a contribution.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- This paper is reasonably executed and well written.\n- The dataset of 8 million examples would be valuable if released, but this isn't offered as a contribution in the paper.\n\nWeaknesses\n- The overall approach (of generative chain-of-thought) seems to be widely used in industrial models; many frontier models make use of LLM-based prompt engineering to improve results. That said, this paper does compare to a variety of such published prompt-enhancement approaches in Table 1. But I am missing a concise description of the key innovations that allow the proposed approach to work well (since similar prompt-enhancing approaches have been previously explored).\n- Building on the above, I do not quite follow the ablations in Table 3 and their description in L343-346. The natural baselines would seem to be off-the-shelf image generation and off-the-shelf chain-of-thought prompting. But I do not see how to perform \"off-the-shelf\" learnable embeddings. I'm guessing that the key innovation is training the model on the collected 9 million examples. I would have liked to see an ablation that (post) trains a traditional chain-of-thought model on such data, without the learnable embeddings.", "was_revised": true, "final_justification": "I appreciate the author's rebuttal. After reading it as well as the other reviews, I have raised my score.", "rebuttal": "We sincerely thank the reviewer for their insightful feedback. We appreciate this opportunity to provide a more comprehensive explanation, supported by new experimental evidence, to clarify these important points.\n\n---\n\n> **Question 1:** The overall approach (of generative chain-of-thought) seems to be widely used in industrial models... I am missing a concise description of the key innovations that allow the proposed approach to work well (since similar prompt-enhancing approaches have been previously explored).\n\nTo distinguish our **Generation Chain-of-Thought (GoT)** paradigm from existing prompt engineering techniques, we first conducted a new experiment to provide a quantitative comparison. This is followed by a explanation of the fundamental architectural differences that define our key innovations.\n\n**1. Quantifying the Effect of Prompt Engineering**\n\nTo empirically ground the discussion, we conducted a new experiment to isolate the effect of pure prompt engineering. **Specifically, we used GPT-4o to rewrite the original GenEval prompts to be more descriptive, and then fed these enhanced prompts directly into the standard SDXL model to generate images.**\n\nThe results show that this approach improved the overall GenEval score for SDXL from 0.55 to **0.58**. This confirms that high-quality prompt engineering provides **an incremental performance gain**. However, the resulting score of 0.58 for SDXL (the foundation model of our diffusion block) is still substantially lower than our GoT framework's score of **0.64**. This significant performance gap underscores that our approach contributes far more than what can be achieved by simply refining the input prompt. The reasons for this lie in the following core distinctions.\n\n**2. Core Distinctions of the GoT Framework**\n\nOur GoT framework is not a pre-processing step but a novel, deeply integrated reasoning-guided generation paradigm.\n\n*   **GoT is a Semantic-Spatial Reasoning Plan, Not Just Enhanced Text:** Prompt engineering focuses on refining the *semantic* content of a prompt. It enriches the textual description but lacks any explicit understanding over the *spatial* arrangement of the scene. In contrast, our GoT generates a structured plan that explicitly combines semantics with spatial information (e.g., `...a **red** apple **(133,684),(336,928)**...`). This is not merely descriptive text; it is an **actionable blueprint** for object placement. This plan is then directly consumed by our novel **Semantic-Spatial Guidance Module (SSGM)** to enforce layout control during the diffusion process. This capability for joint semantic and spatial reasoning is a key innovation entirely absent in standard workflows.\n*   **GoT is Part of a Unified, End-to-End Trained Framework:** Prompt engineering is typically an external, decoupled pre-processing step using a separate LLM. The image model consumes the modified text without any awareness of the rewriting process, and the two models are not jointly optimized. Our GoT framework, however, is a **unified, end-to-end system**. The *same* MLLM (Qwen2.5-VL) that serves as the reasoning engine first generates the GoT chain and subsequently produces the guidance embeddings ($G_t$). Our framework is trained end-to-end, meaning that gradients from the diffusion loss are backpropagated to the MLLM decoder. This **closed-loop learning process** allows the MLLM to learn how to generate *better reasoning chains that directly lead to higher-quality images*. This is a fundamental architectural advancement over the open-loop nature of prompt engineering.\n\n---\n\n> **Question 2:** I do not quite follow the ablations in Table 3... I'm guessing that the key innovation is training the model on the collected 9 million examples. I would have liked to see an ablation that (post) trains a traditional chain-of-thought model on such data, without the learnable embeddings... This would reveal if the improvement is really coming from the data or from the architectural modifications.\n\nWe acknowledge that our description of the ablation study in L343-346 was too brief. We provide a detailed clarification here to demonstrate that the performance gains stem directly from our architectural innovations, not just the data.\n\n**1. Detailed Clarification of the Ablation Study (Table 3)**\n\nTo ensure a fair comparison, every model in the ablation study (except the final fully-trained model) was trained for the **exact same number of steps (10,000)** on identical data subsets. The only difference was the architectural components being used.\n\n*   **Baseline:** This model consists of our base MLLM and diffusion module. To create the most direct comparison, it was trained on our FLUX-GoT and OmniEdit-GoT datasets. However, we **discarded the GoT reasoning chains** and only trained the model on the raw **(instruction, image) pairs**. This setup isolates the performance of the base architecture *without any form of reasoning*, mimicking a standard text-to-image pipeline.\n*   **+ GoT (Semantic Reasoning Only):** This setting builds upon the baseline. Here, the model is trained to first **generate the GoT reasoning chain** from the instruction. This chain is then used to produce semantic guidance ($G_t$), but the spatial coordinate information within the GoT is explicitly ignored. This experiment is designed to precisely measure the performance gain from introducing **semantic reasoning**.\n*   **+ SSGM (Joint Semantic-Spatial Reasoning):** This is the most critical step. Building on the \"+ GoT\" setting, we now activate the spatial guidance pathway. The model parses the **explicit coordinates** from the generated GoT chain to create spatial guidance ($G_s$), which is then fed into our novel SSGM. The significant performance jump observed here (e.g., on ImagenHub, from 0.181 to 0.370) directly validates the effectiveness of our SSGM architecture, as it is the only component changed from the previous step.\n\nThis step-by-step ablation demonstrates that the performance improvements are additive and directly attributable to our architectural additions, first semantic reasoning, then joint semantic-spatial guidance.\n\n**2. The Role of \"Learnable Embeddings\" and Interpretability**\n\n*   **Why Traditional CoT is Insufficient:** We'd like to clarify the role of the `N=64` learnable embeddings, as they are not an \"off-the-shelf\" component but an integral part of our guidance mechanism. They function as a **dynamic query mechanism**. After the MLLM generates the full GoT reasoning chain, these `N` embeddings are passed as input to the MLLM decoder. Conditioned by the preceding GoT chain via causal attention, they effectively \"summarize\" the rich, contextual information from the entire reasoning process into the final guidance $G_t$. This architecture is necessary because a \"traditional chain-of-thought model\" that produces only text lacks the **explicit spatial coordinates** that are a cornerstone of our GoT formulation. Without these coordinates, our SSGM would have no spatial plan to execute, rendering our core architectural innovation unusable. Our GoT formulation and SSGM architecture are thus intrinsically linked and co-designed.\n*   **Enhanced Interpretability:** We argue our framework offers a *higher* degree of interpretability than traditional methods. The interpretability lies not in the `Gt` embedding vectors themselves, but in the **explicit, human-readable GoT reasoning chain that generates them**. This chain provides a transparent, step-by-step \"thought process\" of the model. More importantly, as shown in Figure 5, it makes the model's reasoning **interactive**. Users can directly inspect and **edit the GoT chain** (e.g., changing object names or modifying coordinates) to precisely control the final output, offering a level of transparency and control not possible with opaque, black-box text encoders.\n\n---\n\n> **Question 3:** Are the authors planning on releasing their data? I suspect the real innovation comes from this data, and so I'd be more willing to fight for acceptance if that could also be regarded as a contribution.\n\nWhile our datasets are a key contribution, our primary innovations are the **paradigm, format, and architecture** that this data enables. Specifically, this includes: \n1.  **Proposing the Generation Chain-of-Thought (GoT) paradigm** as a new way to approach visual synthesis; \n2.  **Defining the structured GoT format** that makes reasoning explicit and actionable; and \n3.  **Designing the unified model architecture (including the SSGM)** that can interpret this format and be trained end-to-end. \n\nThe dataset was created to facilitate and validate this new framework. That said, we absolutely agree that the dataset itself is a valuable contribution that will benefit the community.\n\n**We are fully committed to open-sourcing all our contributions, including all datasets, upon acceptance.** We believe that sharing these resources is crucial for fostering reproducible research and accelerating progress in the field.\n\nOur comprehensive release is planned to include:\n\n*   **The Complete Datasets:** This includes the full **LaionAesthetics-GoT, JourneyDB-GoT, FLUX-GoT,** and **OmniEdit-GoT** datasets, encompassing over 9 million examples.\n*   **All Data Components:** For each example, we will provide the source prompt, the target image, and crucially, the structured **GoT reasoning chain**.\n*   **Code and Models:** To ensure full reproducibility, we will also release our full **source code** and the **pre-trained model weights**.\n\nTo affirm our long-standing commitment to this, we would like to gently highlight that this intention was stated in our original submission in the **Abstract (Line 21)**.\n\n---\n\nWe trust these detailed explanations and new experimental results have fully addressed the reviewer's concerns. We will revise the manuscript to integrate these clarifications, particularly in the methodology and ablation study sections."}, {"review_id": "AdDoboDUaW", "reviewer": "Reviewer_YwDX", "summary": "This paper introduces Generation Chain-of-Thought (GoT), a novel framework that incorporates explicit semantic-spatial reasoning chains into visual generation and editing. GoT first produces a step-by-step structured reasoning chain in natural language, which guides the image synthesis process through a Semantic-Spatial Guidance Module (SSGM) that injects semantic embeddings, spatial layouts, and visual references into the diffusion model.\n\nThe framework integrates a multimodal large language model (Qwen-VL) with a diffusion model in a unified, end-to-end trainable pipeline. To support this paradigm, the authors construct a large-scale dataset comprising over 9 million reasoning-chain-annotated samples for both image generation and editing tasks. Extensive experiments demonstrate that GoT achieves state-of-the-art performance on benchmarks such as GenEval, with significant improvements in controllability, transparency, and user interaction. Overall, the work represents a substantial technical contribution, with a large scope and strong empirical results.", "questions": "1. While GoT achieves overall strong performance, it underperforms on certain fine-grained metrics such as spatial relation and attribute binding (compared to models like JanusFlow). Could the authors provide more discussion on this limitation and potential reasons behind it?\n2. Could the authors provide the exact performance numbers for the position category with and without SSGM, as mentioned in the GenEval analysis? Also, have the authors considered including a dedicated evaluation (e.g., object position deviation) to further verify the effectiveness of spatial guidance?\n3. It would be helpful if the authors could elaborate on the future development of the GoT paradigm. Do you believe this reasoning-guided framework is easily generalizable to broader multimodal generation tasks? What are the main challenges or limitations that might arise in scaling or adapting it further?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths:\n    1. Introduces a reasoning-driven generation framework by integrating chain-of-thought (CoT) into visual generation, enabling explicit modeling of object relationships and spatial layouts.\n    2. Combines a multimodal LLM and a diffusion model with a Semantic-Spatial Guidance Module to inject semantic and spatial control directly into the generation process.\n    3. Achieves state-of-the-art performance on both text-to-image generation and multi-step image editing benchmarks.\n    4. Scalable data contribution: Releases a large-scale reasoning chain dataset (>9M samples) to support further research and reproducibility.\n- Weaknesses:\n    1. Slightly underperforms on fine-grained spatial alignment and attribute binding compared to some task-specific baselines.\n    2. The paper claims that “in GenEval, only the position category is notably affected by SSGM,” but does not report concrete performance numbers for the position category.", "was_revised": true, "final_justification": "I have read the rebuttal and other reviewers' comments. Based on these, I keep my original rating.", "rebuttal": "We sincerely thank the reviewer for their thorough and positive evaluation of our work. We are encouraged that the reviewer recognized the novelty of our reasoning-driven framework, the strength of our empirical results, and the value of our data contribution. We appreciate the opportunity to address the insightful questions raised, which will help us further strengthen our paper.\n\n---\n\n> **Question 1:** While GoT achieves overall strong performance, it underperforms on certain fine-grained metrics such as spatial relation and attribute binding (compared to models like JanusFlow). Could the authors provide more discussion on this limitation and potential reasons behind it?\n\nWe acknowledge that on certain fine-grained metrics, our model does not surpass all task-specific baselines. We believe there are two primary reasons for this, which also highlight the unique contribution and future potential of our framework.\n\n1.  **The Importance of the Baseline for Comparison:** The core of our work is the introduction of the GoT paradigm, which integrates a reasoning module (MLLM) with a generation module (diffusion model). To fairly evaluate the contribution of *our paradigm*, the most direct comparison is against the foundational model we build upon. Our diffusion block was initialized from **SDXL**. When compared to the baseline performance of SDXL, our GoT framework demonstrates a **substantial improvement** in spatial control and attribute binding, as evidenced by our ablation studies. This shows that our reasoning-guided approach is highly effective.\n\n2.  **The Role of the Generative Backbone:** The remaining performance gap between our model and models like JanusFlow is largely attributable to the different generative backbones used. Our model employs an SDXL-based U-Net architecture, whereas JanusFlow leverages a more recent and powerful **Flow Matching DiT (Diffusion Transformer)** architecture. This highlights a key strength of our approach: **the GoT paradigm is designed to be modular and can be readily extended to different diffusion model backbones.** The Semantic-Spatial Guidance Module (SSGM) can be seamlessly integrated with any diffusion U-Net or a Flow Matching DiT. We view this as a significant strength and are confident that by combining our reasoning-guided GoT framework with a more advanced generative backbone, we can close this performance gap. We will add this important discussion to the limitations section of our paper.\n\n---\n\n> **Question 2:** Could the authors provide the exact performance numbers for the position category with and without SSGM, as mentioned in the GenEval analysis? Also, have the authors considered including a dedicated evaluation (e.g., object position deviation) to further verify the effectiveness of spatial guidance?\n\nWe apologize for omitting these specific numbers in the main text and are happy to provide a detailed breakdown of our ablation study on the GenEval benchmark. These results directly validate the effectiveness of our SSGM.\n\nThe table below shows the performance of our model at different stages of the ablation, with each model trained for the same number of steps for a fair comparison.\n\n| GenEval Category  | Baseline | +GoT (Semantic Only) | +SSGM (Full) |\n| :---------------- | :------: | :------------------: | :------------: |\n| `single_object`   | 95.3%   |        94.3%        |     94.3%     |\n| `two_object`      | 24.4%   |        28.3%        |     32.7%     |\n| `counting`        | 23.7%   |        24.6%        |     26.8%     |\n| `colors`          | 78.4%   |        80.0%        |     80.4%     |\n| **`position`**    | **4.5%**|      **7.4%**       |   **9.1%**    |\n| `color_attr`      |  1.7%   |        5.2%         |     5.7%      |\n| **Overall Score** | **0.380** |      **0.400**      |   **0.418**   |\n\nAs the results clearly show:\n*   The **Baseline** model achieves a score of only **4.5%** on the `position` task.\n*   Introducing semantic reasoning (**+GoT**) provides a notable lift to **7.4%**.\n*   Activating our proposed **+SSGM** further improves performance, **doubling the baseline score to 9.1%**. This directly validates our claim that the SSGM is effectively translating the explicit spatial plan from the GoT into improved object placement.\n\nRegarding a dedicated evaluation, we agree that incorporating a more direct metric like mean Intersection-over-Union (mIoU) or Object Position Deviation would offer an even more precise measure of spatial guidance. This is a valuable direction for future benchmarking, and we will add this to our discussion on future work.\n\n---\n\n> **Question 3:** It would be helpful if the authors could elaborate on the future development of the GoT paradigm. Do you believe this reasoning-guided framework is easily generalizable to broader multimodal generation tasks? What are the main challenges or limitations that might arise in scaling or adapting it further?\n\nWe are very optimistic about the future of the GoT paradigm and believe its core principles are highly generalizable.\n\n**Generalizability of the GoT Paradigm:**\n\nThe fundamental idea of GoT is to **decompose a complex, monolithic generation task into an explicit, intermediate reasoning step**. This \"show your work\" approach is not limited to images. We envision its application in several other domains:\n\n*   **Video Generation:** A GoT chain could represent a storyboard or a script (e.g., `Scene 1: A cat walks in from the left. Scene 2: The cat sits on a mat.`).\n*   **3D Asset Generation:** The reasoning chain could be a sequence of constructive solid geometry (CSG) operations or a programmatic description of a 3D model.\n*   **Robotics and Embodied AI:** A GoT chain could serve as a high-level action plan for an agent to execute in an environment (e.g., `1. Go to the kitchen. 2. Find the red apple. 3. Place it on the table.`).\n\n**Main Challenges and Limitations for Future Scaling:**\n\nScaling and adapting the GoT paradigm will present several key challenges:\n\n1.  **Reasoning Chain Fidelity:** The quality of the final output is fundamentally capped by the quality of the generated reasoning chain. Improving the reasoning capabilities of the MLLM for complex, multi-step plans remains a primary challenge.\n2.  **Data Scaling and Annotation:** Creating large-scale datasets with reasoning-chain annotations is a significant undertaking. Scaling to new modalities will require developing new, scalable methods for annotating high-quality reasoning chains.\n3.  **Computational Cost and Latency:** The sequential process of reasoning then generating can introduce latency. Optimizing this pipeline for efficiency, especially for high-resolution or long-form content like video, will be crucial for practical applications.\n\nWe trust this detailed response fully addresses the reviewer's questions. We thank the reviewer again for their positive assessment and for providing valuable suggestions that will help us improve the clarity and impact of our paper."}, {"review_id": "yvw7dQqbaF", "reviewer": "Reviewer_oQpg", "summary": "This paper introduces Generation Chain-of-Thought (GoT), a new paradigm for visual generation and editing that addresses the failure of current models to understand complex scenes with specific object relationships and spatial arrangements. Instead of directly mapping a text prompt to an image, the GoT framework utilizes a Multimodal Large Language Model (MLLM) to first generate an explicit, structured reasoning chain that details semantic relationships, object attributes, and precise spatial coordinates before any image synthesis occurs. \n\nThe authors' core contributions include the GoT paradigm itself, the construction of the first large-scale datasets containing over 9 million samples with these detailed reasoning chains , and a novel, unified end-to-end framework for implementation. This framework integrates an MLLM with a diffusion model via a Semantic-Spatial Guidance Module (SSGM), which ensures the generated image precisely follows the logical reasoning steps. The GoT framework demonstrates competitive results on benchmarks for text-to-image generation and image editing , and its explicit process enables interactive generation, allowing users to directly modify the reasoning chain for precise control over the final output.", "questions": "See weaknesses first.\n\nAdditional questions:\n\n1. Impact on Core VLM Capabilities: Does joint training on GoT and diffusion generation degrade the base MLLM's original vision-language understanding abilities? Please provide benchmark results on standard VLM tasks to clarify the trade-offs of your unified training approach.\n\n2. Model Efficiency: Could you provide details on the model's efficiency, specifically its parameter count and average inference latency compared to other models in Tables 1 and 2? This information is crucial for assessing the framework's practical viability.", "limitations": "No. While the authors commended \"Yes\" in the NeurIPS checklist, I didn't see substantial limitations are discussed in the paper.\n\nSuggestions for Improvement:\n\nI suggest the authors add a dedicated \"Limitations\" section to the main paper. This section should transparently discuss the limitations, for example, issues of model efficiency/latency, the high computational costs, etc.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. High Originality and Significance: The paper's core concept, Generation Chain-of-Thought (GoT), is highly original and significant. It introduces a novel reasoning-guided paradigm for visual synthesis, moving beyond direct mapping to address key limitations in compositional generation.\n\n2. Substantial and High-Quality Contributions: The creation of a massive, first-of-its-kind dataset with over 9 million samples is a high-quality contribution that will be a valuable resource to the community. This was a major effort, consuming over 3000 NVIDIA A100 GPU days to produce detailed semantic-spatial annotations.\n\n3. Clarity Empirical Results: The proposed end-to-end framework is presented with excellent clarity and is technically sound. Its effectiveness is studied through extensive experiments across multiple generation and editing benchmarks.\n\n\n\nWeaknesses\n\n1. Incomplete Experimental Comparison: The evaluation in Table 1 lacks a direct comparison with RPG [52], a highly relevant baseline that also employs a recurrent planning and generation methodology for compositional scenes. Given the conceptual similarities in using an explicit planning stage, its omission makes the assessment against layout-guided methods less complete.\n\n2. Unconvincing Ablation Study Design: The setup of the ablation study in Table 3 does not fully isolate the impact of the core contribution—the GoT reasoning. The baseline is trained for significantly fewer steps (10,000) than the full framework (70,000 total). This makes it difficult to distinguish the performance gains from the GoT reasoning itself versus the substantial effect of extended pretraining. A more persuasive ablation would use a baseline with the same amount of pretraining but without the GoT reasoning tokens to provide a more direct comparison.\n\n3. Lack of Statistical Significance Reporting: The quantitative results presented in Tables 1, 2, and 3 do not include error bars or any other measure of statistical significance (e.g., standard deviation across multiple runs). As acknowledged in the paper's checklist, this absence makes it difficult to rigorously assess the stability and significance of the reported performance improvements over other methods.", "was_revised": true, "final_justification": "4 Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.", "rebuttal": "We sincerely thank the reviewer for their detailed and critical feedback. The concerns raised are important, and we appreciate this opportunity to address them directly. We believe that by clarifying our experimental design, providing additional comparisons, and committing to revisions, we can resolve these issues and demonstrate the soundness of our work.\n\n---\n\n### **Weaknesses & Corresponding Questions**\n\n> Weakness 1: Incomplete Experimental Comparison\n> The evaluation in Table 1 lacks a direct comparison with RPG [52], a highly relevant baseline...\n\nWe thank the reviewer for pointing out this omission. RPG is indeed a relevant baseline, and we agree that a direct comparison is crucial. We have now run RPG on the GenEval benchmark and will include these results in the final version of our paper. This comparison further underscores the advantages of our GoT paradigm.\n\nHere is a direct comparison of our final GoT framework with RPG on GenEval:\n\n| GenEval Category | RPG [52] | **GoT (Ours, Full Model)** |\n| --- | --- | --- |\n| `single_object` | 98.2% | **99.2%** |\n| `two_object` | 66.1% | **69.0%** |\n| `counting` | 16.8% | **67.5%** |\n| `colors` | 85.1% | **85.7%** |\n| **`position`** | 7.5% | **34.2%** |\n| `color_attr` | 27.4% | 27.1% |\n| **Overall Score** | **0.502** | **0.643** |\n\n**Analysis:**\nThe results demonstrate that our **GoT framework significantly outperforms RPG**, achieving a **27% higher overall score**. This advantage is driven by massive gains in the most challenging compositional tasks:\n\n- **Spatial Control (`position`):** Our model achieves a score of **34.2%**, which is over **4.5 times higher** than RPG's 7.5%. This validates our core hypothesis that an explicit semantic-spatial reasoning chain, executed by our SSGM, provides vastly superior control over object placement.\n- **Counting (`counting`):** Our framework's score of **67.5%** is nearly **4 times higher** than RPG's 16.8%, showcasing a much stronger ability to handle numerical concepts.\n\nThese dramatic improvements in complex spatial and numerical reasoning highlight the fundamental strengths of our approach. We will prominently feature this analysis in our revised manuscript.\n\n---\n\n> Weakness 2: Unconvincing Ablation Study Design\n> The setup of the ablation study in Table 3 does not fully isolate the impact of the core contribution... A more persuasive ablation would use a baseline with the same amount of pretraining but without the GoT reasoning tokens...\n\nWe acknowledge that our description in the paper was too brief. We would like to clarify the settings of our ablation study, as the \"more persuasive ablation\" the reviewer suggests is, in fact, what we performed.\n\n**Detailed Clarification of the Ablation Study (Table 3)**\n\nTo ensure a fair comparison, every model in the ablation study (i.e., \"Baseline\", \"+GoT\", and \"+SSGM\") was trained for the **exact same number of steps (10,000)** on identical data subsets. The only difference was the architectural components being used. The 70,000-step model was listed only to report the final performance of our fully trained system, not as a point of comparison for the ablation.\n\nHere is a step-by-step breakdown:\n\n- **Baseline:** This model was trained on our datasets, but we deliberately **discarded the GoT reasoning chains** and only used the raw **(instruction, image) pairs**. This setup isolates the performance of the base architecture *without any form of reasoning*, matching the reviewer's request.\n- **+ GoT (Semantic Reasoning Only):** This setting builds upon the baseline. The model is trained to first **generate the GoT reasoning chain** and use it for semantic guidance, while the spatial coordinate information is ignored. This measures the gain from introducing **semantic reasoning**.\n- **+ SSGM (Joint Semantic-Spatial Reasoning):** Building on the \"+ GoT\" setting, we now activate the spatial guidance pathway. The model parses the **explicit coordinates** from the GoT chain to create spatial guidance ($G_s$) for our novel SSGM. The significant performance jump here directly validates the SSGM's effectiveness.\n\nThis step-by-step ablation clearly demonstrates that the performance improvements are additive and directly attributable to our architectural additions. We will revise the manuscript to incorporate this detailed explanation.\n\n---\n\n> Weakness 3: Lack of Statistical Significance Reporting\n> The quantitative results... do not include error bars... this absence makes it difficult to rigorously assess the stability and significance of the reported performance improvements.\n\nWe agree that reporting statistical significance is best practice. However, in the domain of large-scale generative model training, there are two factors that make this challenging:\n\n1. **Computational Cost:** As noted, our data annotation process alone consumed over 3000 A100 GPU days. Training our full model multiple times from scratch to obtain a standard deviation is computationally prohibitive.\n2. **Community Norms:** The current accepted practice for reporting results on benchmarks like GenEval is to report scores from a single, full training run. This practice is consistent with the reporting standards in other recent large-scale generative model papers, such as those for **Stable Diffusion 3, Janus, and Emu3**, where results are also reported from a single training run due to the extreme resource requirements.\n\nWhile we cannot provide error bars, we argue that the performance gaps we report are substantial enough (e.g., a 27% overall improvement over RPG) to demonstrate significance well beyond the margin of random noise. We will add a note acknowledging this limitation in our experimental setup.\n\n---\n\n### **Additional Questions**\n\n> Impact on Core VVLM Capabilities: Does joint training on GoT and diffusion generation degrade the base MLLM's original vision-language understanding abilities?\n\nThis question highlights a key distinction of our work. Our goal is **not** to build a single, generalist model that balances both generation and understanding tasks. Instead, our framework is intentionally designed to **specialize** an MLLM's powerful reasoning capabilities to serve as a high-precision controller for the generative process.\n\nOur end-to-end training fine-tunes the MLLM decoder to become an expert \"reasoning-to-guidance\" module. This focused training means that a degradation in its original, general-purpose VLM capabilities is an **expected and accepted trade-off**. Our contribution lies in demonstrating how to achieve a new level of generative control *through* this specialization, rather than pursuing a unified model for all VLM tasks. We will clarify this specific focus in the limitations section.\n\n> Model Efficiency: Could you provide details on the model's efficiency, specifically its parameter count and average inference latency?\n\nWe provide the requested details on model efficiency below.\n\n**Table A: Text-to-Image Generation Efficiency**\n\n| Model | Parameters | Inference Latency (Original resolution on H100) |\n| --- | --- | --- |\n| SD 1.5 | 1B | 2.1s |\n| SDXL | 5B | 3.5s |\n| SD3-M | 7B | 6.1s |\n| Emu3-Gen | 8B | 313.0s |\n| Janus | 1.3B | 7.5s |\n| JanusFlow | 1.3B | 0.8s |\n| **GoT (Ours)** | **5.8B** | 12.1s |\n\n**Table B: Image Editing Efficiency**\n\n| Model | Parameters | Inference Latency (on H100) |\n| --- | --- | --- |\n| InstructPix2Pix | 1B | 1.3s |\n| MagicBrush | 1B | 2.2s |\n| MGIE | 8B | 10.9s |\n| SEED-X | 17B | 3.1s |\n| **GoT (Ours)** | **5.8B** | 12.9s |\n\nThe data in the tables shows that our GoT framework introduces a latency overhead compared to single-step generative models. This is an inherent consequence of our two-stage design, which first generates an explicit GoT reasoning chain before commencing the guided diffusion process.\n\nWe posit that this is a deliberate and valuable architectural trade-off. The additional inference time is exchanged for the significant and quantifiable gains in controllability, spatial precision, and numerical reasoning. Furthermore, this framework provides unique interpretability and interactivity through the editable GoT chain, a capability absent in faster, single-step models. For applications demanding high-fidelity and complex compositional generation, we believe this trade-off is highly favorable.\n\n---\n\n### **Limitations**\n\n> While the authors commended \"Yes\" in the NeurIPS checklist, I didn't see substantial limitations are discussed in the paper. I suggest the authors add a dedicated \"Limitations\" section...\n\nWe agree with the reviewer. A dedicated \"Limitations\" section is essential for a transparent scientific paper. We will add one to the main paper in the final version. This section will discuss:\n\n1. The computational cost and latency overhead of our approach.\n2. The challenges of scaling GoT data annotation to new domains.\n3. The performance trade-offs with other architectures, as highlighted by the RPG comparison.\n4. The expected trade-off between specialized generative control and general VLM capabilities.\n\nWe trust that these responses and our planned revisions fully address the reviewer's concerns. We are grateful for the rigorous feedback, which has been invaluable in helping us improve the clarity and completeness of our paper."}]

Author rebuttal:

Meta-review: This paper introduces generation chain of thought (GoT). This is a new paradigm for visual synthesis where a Multimodal Large Language Model (MLLM) first generates an explicit, structured reasoning chain before any image is created. This chain details semantic relationships, object attributes, and precise spatial coordinates. This reasoning output then guides a diffusion model via a Semantic-Spatial Guidance Module (SSGM) in an end-to-end framework. The authors' central claim is that this reasoning-first approach improves performance on complex compositional generation and editing tasks, supported by a new dataset of over 9 million reasoning-annotated samples.

In terms of strengths, reviewers noted the original conceptual contribution of the work in tackling complex compositional generation (YwDX, oQpg). Another noted contribution is the creation of a large scale dataset with detailed reasoning chains (mkjA, YwDX, oQpg). Finally, the work was commended for its strong execution, achieving SOTA empirical results on both generation and editing tasks (mkjA, oQpg, YwDX).

As for weaknesses, concerns centered on evaluation and novelty. Reviewers mkjA and oQpg found the ablation study's design not convincing, arguing it didn't isolate the impact of the proposed approach from the dataset. Reviewer oQpg also noted the missing comparison to RPG. Reviewer mkjA also questioned the novelty, comparing the approach to existing prompt engineering techniques, suggesting the gains might primarily come from the new dataset.

The authors' rebuttal was effective, clarifying the ablation study's fairness and adding a new SOTA baseline (RPG) that showed a 27% performance gain. This resolved most concerns. However, Reviewer YwDX, kept their score because the paper's claim of being modular and easily extendable to other backbones was not supported by experimental evidence. While the authors provided a technical explanation, this lack of empirical proof on other architectures is a valid limitation. Reviewer mkjA also noted a missing ablation that would reveal if an end2end-trained latent embedding is really necessary. The AC urges the authors to adress remaining concerns.

Overall, the AC finds the work's contribution interesting and potentially impactful, introducing a novel paradigm and a dataset that will be useful for the community. The author's rebuttal addressed most concerns, and a unanimous positive consensus was achieved by the reviewers. The AC finds the paper passes the bar for acceptance.
Decision: Accept (poster)