MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal
  Mathematical Reasoning
Authors: name: Weikang Shi; name: Aldrich Yu; name: Rongyao Fang; name: Houxing Ren; name: Ke Wang; name: Aojun Zhou; name: Changyao Tian; name: Xinyu Fu; name: Yuxuan Hu; name: Zimu Lu; name: Linjiang Huang; name: Si Liu; name: Rui Liu; name: Hongsheng Li
Published: 2025-10-16 17:58:58

Abstract: While Large Language Models (LLMs) have excelled in textual reasoning, they
struggle with mathematical domains like geometry that intrinsically rely on
visual aids. Existing approaches to Visual Chain-of-Thought (VCoT) are often
limited by rigid external tools or fail to generate the high-fidelity,
strategically-timed diagrams necessary for complex problem-solving. To bridge
this gap, we introduce MathCanvas, a comprehensive framework designed to endow
unified Large Multimodal Models (LMMs) with intrinsic VCoT capabilities for
mathematics. Our approach consists of two phases. First, a Visual Manipulation
stage pre-trains the model on a novel 15.2M-pair corpus, comprising 10M
caption-to-diagram pairs (MathCanvas-Imagen) and 5.2M step-by-step editing
trajectories (MathCanvas-Edit), to master diagram generation and editing.
Second, a Strategic Visual-Aided Reasoning stage fine-tunes the model on
MathCanvas-Instruct, a new 219K-example dataset of interleaved visual-textual
reasoning paths, teaching it when and how to leverage visual aids. To
facilitate rigorous evaluation, we introduce MathCanvas-Bench, a challenging
benchmark with 3K problems that require models to produce interleaved
visual-textual solutions. Our model, BAGEL-Canvas, trained under this
framework, achieves an 86% relative improvement over strong LMM baselines on
MathCanvas-Bench, demonstrating excellent generalization to other public math
benchmarks. Our work provides a complete toolkit-framework, datasets, and
benchmark-to unlock complex, human-like visual-aided reasoning in LMMs. Project
Page: https://mathcanvas.github.io/