Thinking with Generated Images
Authors: name: Ethan Chern, name: Zhulin Hu, name: Steffi Chern, name: Siqi Kou, name: Jiadi Su, name: Yan Ma, name: Zhijie Deng, name: Pengfei Liu

We present Thinking with Generated Images, a novel paradigm that
fundamentally transforms how large multimodal models (LMMs) engage with visual
reasoning by enabling them to natively think across text and vision modalities
through spontaneous generation of intermediate visual thinking steps. Current
visual reasoning with LMMs is constrained to either processing fixed
user-provided images or reasoning solely through text-based chain-of-thought
(CoT). Thinking with Generated Images unlocks a new dimension of cognitive
capability where models can actively construct intermediate visual thoughts,
critique their own visual hypotheses, and refine them as integral components of
their reasoning process. We demonstrate the effectiveness of our approach
through two complementary mechanisms: (1) vision generation with intermediate
visual subgoals, where models decompose complex visual tasks into manageable
components that are generated and integrated progressively, and (2) vision
generation with self-critique, where models generate an initial visual
hypothesis, analyze its shortcomings through textual reasoning, and produce
refined outputs based on their own critiques. Our experiments on vision
generation benchmarks show substantial improvements over baseline approaches,
with our models achieving up to 50% (from 38% to 57%) relative improvement in
handling complex multi-object scenarios. From biochemists exploring novel
protein structures, and architects iterating on spatial designs, to forensic
analysts reconstructing crime scenes, and basketball players envisioning
strategic plays, our approach enables AI models to engage in the kind of visual
imagination and iterative refinement that characterizes human creative,
analytical, and strategic thinking. We release our open-source suite at
https://github.com/GAIR-NLP/thinking-with-generated-images.