GoT: Unleashing Reasoning Capability of Multimodal Large Language Model
  for Visual Generation and Editing
Authors: name: Rongyao Fang; name: Chengqi Duan; name: Kun Wang; name: Linjiang Huang; name: Hao Li; name: Shilin Yan; name: Hao Tian; name: Xingyu Zeng; name: Rui Zhao; name: Jifeng Dai; name: Xihui Liu; name: Hongsheng Li
Published: 2025-03-13 17:59:59

Abstract: Current image generation and editing methods primarily process textual
prompts as direct inputs without reasoning about visual composition and
explicit operations. We present Generation Chain-of-Thought (GoT), a novel
paradigm that enables generation and editing through an explicit language
reasoning process before outputting images. This approach transforms
conventional text-to-image generation and editing into a reasoning-guided
framework that analyzes semantic relationships and spatial arrangements. We
define the formulation of GoT and construct large-scale GoT datasets containing
over 9M samples with detailed reasoning chains capturing semantic-spatial
relationships. To leverage the advantages of GoT, we implement a unified
framework that integrates Qwen2.5-VL for reasoning chain generation with an
end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance
Module. Experiments show our GoT framework achieves excellent performance on
both generation and editing tasks, with significant improvements over
baselines. Additionally, our approach enables interactive visual generation,
allowing users to explicitly modify reasoning steps for precise image
adjustments. GoT pioneers a new direction for reasoning-driven visual
generation and editing, producing images that better align with human intent.
To facilitate future research, we make our datasets, code, and pretrained
models publicly available at https://github.com/rongyaofang/GoT.