MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning
Authors: Xinyan Chen
Renrui Zhang
Dongzhi Jiang
Aojun Zhou
Shilin Yan
Weifeng Lin
Hongsheng Li
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal Large Laguage Models
Chain-of-Thought
Mathematics
Primary area: applications

Abstract: Chain-of-Thought (CoT) has widely enhanced mathematical reasoning in Large Language Models (LLMs), but it still remains challenging for extending it to multimodal domains. Existing works either adopt a similar textual reasoning for image input, or seek to interleave visual signals into mathematical CoT. However, they face three key limitations for math problem-solving: *reliance on coarse-grained box-shaped image regions, limited perception of vision encoders on math content, and dependence on external capabilities for visual modification*.  In this paper, we propose **MINT-CoT**, introducing **M**athematical **IN**terleaved **T**okens for **C**hain-**o**f-**T**hought visual reasoning. MINT-CoT adaptively interleaves relevant visual tokens into textual reasoning steps via an Interleave Token, which dynamically selects visual regions of any shapes within math figures. To empower this capability, we construct the MINT-CoT dataset, containing 54K mathematical problems aligning each reasoning step with visual regions at the token level, accompanied by a rigorous data generation pipeline. We further present a three-stage MINT-CoT training strategy, progressively combining text-only CoT SFT, interleaved CoT SFT, and interleaved CoT RL, which derives our MINT-CoT-7B model. Extensive experiments demonstrate the effectiveness of our method for effective visual interleaved reasoning in mathematical domains, where MINT-CoT-7B outperforms the baseline model by +34.08% on MathVista and +28.78% on GeoQA, respectively.

MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning
Xinyan Chen"l, Renrui Zhang "111 Dongzhi Jiang', Aojun Zhou' Shilin Yan, Weifeng Lin', Hongsheng L,
CCHH MMLab Shanghai Al Laboratory SCPII under InnoHK
{chenxyxy06,
wwg.aal..commmmmmmmm....o....m.. com
hsli@ee cuhk. ok
Equal Contribution
Project Leader
*Corresponding Author
Abstract
Chain-of-Thought (CoT) has widely enhanced mathematical reasoning in Large Language Models (LLMs),, but it still remains challenging for extending it to multimodal domains. Existing works either adopt similar textual reasoning for image input, or seek to interleave visual signals into mathematical COT. However, they face three key limitations for math problem-solving: reliance on coarse- grained box-shaped image regions, limited perception of vision encoders on math content, and dependence on external capabilities for visual modification. In this paper, we propose MINT-CoT. introducing Mathematical INterleaved Tokens for Chain-of of-Thought visual reasoning. MINT-CoT adaptively interleaves rel- evant visual tokens into textual reasoning steps via an Interleave Token, which dynamically selects visual regions of any shapes within math figures. empower this capability, we construct the MINT-CoT dataset, containing 54K mathemati- cal problems aligning each reasoning step with visual regions at the token level, accompanied by a rigorous data generation pipeline. We further present a three- stage MINT-CoT training strategy, progressively combining text-only CoT SFT, interleaved CoT SFT, and interleaved CoT RL, which derives MINT-CoT-7B model. Extensive experiments demonstrate the effectiveness our method for effective visual interleaved reasoning in mathematical domains, where MINT- CoT-7B outperforms the baseline model by +34.08% on Math Vista, +28.78% on GeoQA, and +23.2% MMStar, respectively. Our code and data available https: //gi trib.uu..aa www.xwowaonoooeeeeeeeo cxy www.ayyynncxyyMMIT--CC.. COTNT-
Introduction
Chain-of-Thought (CoT) [66, 32] has emerged as an effective strategy for enhancing the reasoning
capabilities of Large Language Models (LLMM) [49, 51, 62, 72, 81, 39] by generating sequential rationales in their responses. In Multimodal Large Language Models (MLLMs) [50, 33, 88, 18, 20], CoT also plays significant role [84] across various tasks involving image [41, 86, 40, 22, 17, 25], video [38, 4, 71, 14], and 3D [70, 24, 58, 21]. It enables MLLMs to reason over both textual and visual inputs, serving as bridge that connects visual perception with abstract reasoning tasks.
However, despite these advances, applying CoT in mathematical reasoning with visual contexts remains challenging. Existing MLLMs mainly generate text-only reasoning steps for multimodal math problems [84, 85, 60, 79]. simply adopting similar textual reasoning for image input. Nevertheless,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Question:
given diagram, circle line segment as its diameter CD as a chord. A
langent passing through point C intersects the extension of AB at point E, and angle
measures 42". What is the measure of angle CDB? Choices: A: 22"
B: 24"
C: 28"
D: 48
Text-only CoT Reasoning:
Lack of visual information leads to perception error.
Step 1: Since angle
therefore angle 42". X
Step 2: Since AB is the diameter of circle O, angle ACB 90". Therefore, angle B 180* 42" 90' 48'.
Step Since AB the diameter of circle O, angle CDB angle 48'.
Answer: D X
Box-shaped Visual CoT Reasoning:
Box-level too coarse for math concepts.
Step 1:
Since angle E 42", angle CEB = 42" /
Step
Since is the diameter circle O, angle ACB Therefore, angle CDB angle CEB 42". X
Step 3:
Answer: D x
Visual Interleaved CoT Reasoning (Ours):
Token-level interleaved CoT is fine-grained and effective.
Step 1:
Connect OC, OC /
Step 2:
AngleCOE 180" 90" 42" 48".
Step 3
Since OC OD, ongleCDB angleODC 1/2 angleBOC 24
Answer:
Figure 1: Comparison of three CoT reasoning methods: text-only CoT reasoning, box-shaped visual CoT reasoning and visual interleaved CoT reasoning methods. (1) Text-only CoT lacks visual information, causing perception errors mathematical reasoning. (2) Box-level cues are too coarse to capture complex visual structures in mathematical images. (3) Token-level interleaved CoT accurately identifies fine-grad mne-grained visual regions to support reasoning.
due to the limited capability in perceiving math images, this strategy often fails to accurately interpret visual information within the CoT process, leading to reasoning errors.
Recent approaches have attempted to interleave visual content within reasoning steps through mecha- nisms such as bounding box selection and image cropping [55, 26, 76]. While effective in general visual scenarios, these methods still face three key limitations when extended to multimodal mathe- matical reasoning:
1. Reliance on coarse-grained box-shaped image regions: Recent advances introduce visual information into the CoT process by selecting image regions through bounding box-based methods. Visual-CoT [55], Visual SKETCHPAD [26], and VPT [76] all operate box- shaped image regions, employing strategies such as bounding box generation, iterative masking, cropping. re-encoding. However, shown in Figure 1, these approaches all rely on bounding box-based cropping. While such box-level cues are effective in domains like object detection, where objects typically isolated, they too coarse grarseegriinedd to capture the complex structures in mathematical images, where visual information is not discrete but highly interconnected. As result, box-shaped selection tends interleave many irrelevant or misleading visual tokens, impairing the accuracy mathematical reasoning.
2. Limited perception of vision encoders on math content: Some methods, like ICoT [16], adopt attention-based token selection to identify relevant visual tokens during reasoning without requiring additional training. These approaches rely heavily on visual features extracted by the vanilla vision encoders without specific tuning. However, as noted in MAVIS [83], mainstream vision encoders, which are primarily based on CLIP [54] or SigLIP [78], are pre-trained on natural images with general scenes, making mathematical a result, such methods often struggle to accurately locate relevant visual regions in complex mathematical tasks.
images duttor-diitribbution.

---

3. Dependence on external capabilitiess for visual modification: Other approaches attempt to enhance visual reasoning by dynamically generating new visual content or modifying existing images. M [36] is built upon a unified autoregressive MILLM [59] to generate images as part of the CoT process, but it is only applicable to spatial planning tasks. Meanwhile, Visual SKETCHPAD requires external tools to draw on the original image in scometry-related tasks. These approaches depend on external capabilities, either requiring large-scale data to train the understanding model for generation, relying on external tools with additional inference over the modified images, which leads to numerous extra costs.
Therefore, to address these challenges, we aim to propose a fine-grained, efficient visual interleaved CoT method to enhance the mathematical reasoning capabilities of MLLMs. In this paper, we introduce MINT-CoT, an approach of Mathematical INterleaved Token selection for Chain-of- Thought reasoning, which facilitates multimodal reasoning by interleaving relevant visual regions within reasoning steps. At the core of the MINT-CoT is the Interleave Token, special token generated through the next-token prediction process. During reasoning, MINT-CoT automatically identifies and incorporates the most relevant visual tokens from the original image at each reasoning step. This is achieved by computing similarity scores between the output hidden states of the Interleave Token and all visual tokens, in order to identify the tokens most relevant to the mathematical concept the current step. These selected visual tokens are then dynamically integrated into the textual reasoning steps, enabling the flexible selection of visual regions throughout the CoT process. In this way, the interleaved regions of mathematical images are not restricted to box-shaped areas but can flexibly include geometric shapes, line segments, coordinates, and other elements.
To enable effective training of MINT-CoT, we construct the MINT-CoT dataset, a 54K visual interleaved reasoning dataset. Each data point contains reasoning steps paired with the indices of selected tokens corresponding to the mathematical concepts involved in each step. We source mathematical problems from the Mulberry-260K dataset [74] to construct text-only CoT reasoning format, then annotate the reasoning steps with corresponding image regions through a four-step pipeline: (1) dividing images into grid-indexed regions, (2) mapping recognized text elements to grid indices via OCR-based text localization, (3) extracting key words, and (4) assigning visual regions to these key words using an advanced MLLM. This process creates a visual interleaved CoT reasoning dataset providing token-level supervision for training models to interleave visual content into reasoning steps.
Building on the MINT-CoT framework and MINT-CoT dataset, we design a progressive training strategy, the MINT-CoT training strategy, that incrementally improves MLLMs' ability with three training stages: (1) Text-only CoT Training, (2) Interleaved CoT SFT, and (3) Interleaved CoT RL. Through this training strategy, we train a MINT-CoT-7B model with the capability of mathematical visual interleaved CoT reasoning. Extensive experiments demonstrate the superiority of our proposed approach. Specifically, our method achieves absolute improvement of +32.59% on Math Vista [43], +26.92% on GeoQA [5], and +23.2% on MMStar [7] benchmark compared to the baseline model.
Our main contributions are follows:
We propose MINT-CoT, which uses the Interleave Token to interleave fine- fne-grained visual tokens within reasoning steps, enhancing multimodal mathematical reasoning.
We construct the MINT-CoT dataset, 54K dataset for multimodal mathematical reasoning. offering fine-grained alignment between textual rationales and visual inputs. We develop an automated pipeline to generate visual interleaved CoT data annotated with token indices.
We develop progressive three-stage MINT-CoT training strategy, to improve interleaved mathematical reasoning. Extensive experiments validate the efficiency of our method.
2
Related work
MILMS for Mathematics. Recent advancements in MLLMs [50, 2,, 31] have shown impressive capabilities in various vision-language tasks. However, even powerful models like GPT-4V [50] and Qwen2-Ve VL [63] fail to demonstrate satisfying performance on existing visual mathematical benchmarks [5, 44, 43], as highlighted by Math Verse [82]. Various specialized approaches [15, 83, 28, 9, 45, 57, 53] have emerged to enhance visual mathematical reasoning. Current approaches mostly focus on enriching the multimodal math data. G-LLaVA [15] extends the LLaVA architecture
3

---

Similarity
Similarity
Scores
Scores
Post Visual Projector
Post Interleave Projector
Selected
Selected Visual Tokens
Large Language Model
Vision Encoder
Question and
Rationales
Interleave Token
Step1:
Interleave
Token
Step2:
2080
Input Image
Figure 2: Overview of the MINT-CoT framework. During CoT reasoning, MINT-CoT generates an Interleave Token before each reasoning step and computes the similarity scores between embeddings projected by the decoder-side visual projector and the interleave projector. Based these similarity scores, relevant visual tokens are selected, and the model inferences with these selected visual tokens.
with geometric reasoning capabilities by augmenting the current dataset. Math-ILLaA [57] enlarges the data scope with the introduced Math V360K dataset. MAVIS [83] first identifies the critical issue of the vision encoder and empowers with the mathematical capability. Then it further develops an automated system for generating mathematical visual datasets at scale. Reverse Chain-of-Thought (R-CoT) [9] introduces the Geometry Generation Chain for creating geometric images with more accurate descriptions.
Visual Chain of Thought. With advancements of various visual reasoning tasks [43, 77, 30, 75, 69), visual chain of thought has been emerging as effective method for both image generation [23, 29, 61, 87] and understanding [52, 74, 60] tasks. Our work focuses on leveraging it for reasoning on images, where two distinct methods have emerged. One line of the method relies on textual CoT to conduct multimodal analysis [11, 46, 6, 79, 10, 73]. For example, R1-V [6] extends the paradigm of DeepSeek R1 [19] to generate comprehensive text CoT analyze the visual information before providing the final answer. Another line of method explicitly incorporates multimodal elements in the rational [55, 47, 67, 26, 35]. Visual CoT [55] and Chain-of-Spot [42] propose to crop the region of high interest the image and integrate it into the CoT process. Chain-of-Imane [47] and Visual SKETCHPAD [26] introduce auxiliary tools to generate helpful diagrams for mathematical or geometric problem-solving. sing. Although these methods demonstrate competitive performance, they limited to rigid image cropping or dependence on external tools. Recently, ICoT [16] leverages attention map of the MLLM to select the relevant visual tokens to compose the multimodal rational. However, this approach relies solely on attention scores on the image feature maps, which have been shown to be insufficiently informative for mathematical scenarios [83].
Method
To address the challenges of multimodal CoT in mathematical reasoning, we propose MINT-CoT. In this section, we first introduce the framework of MINT-CoT in Section 3.1. Then introduce the MINT-CoT dataset and provide a detailed discussion of the dataset generation method in Section 3.2. Finally, we present the progressive MINT-CoT training strategy Section 3.3.
3.1 MINT-CoT
Previous CoT approaches in MILMSs mainly generate text-based reasoning steps, which are not explicitly grounded in visual features and therefore struggle with mathematical reasoning that involves

---

visual details. We formulate this CoT reasoning process as:
{s())
{( answer LLM(VV TextEncoder(T)),
where V VisionEncoder([] (1) {ur 1 denotes the visual feature extracted from the input image I, and each E, represents the 5-th visual token generated by the vision encoder. T denotes the input mathematical question and instructions, { is the sequence of textual reasoning steps generated by the model, and answer the final answer. Recent advancements attempt incorporate multimodal reasoning steps in the CoT process. However, current coarse-grained methods only focus on selecting box-shaped visual regions; how adaptively select the visual content in alignment with each textual reasoning step remains an open question. We thus propose the MINT-CoT framework and introduce an Interleave Token help MLLMs select visual tokens from the visual feature V. The overview of the MINT-CoT framework is illustrated in Figure 2.
Interleave Token. An Interleave Token is special token generated prior to each reasoning step. is used to select visual tokens that are relevant to the mathematical concepts involved in that step (c.g.. "line segment AB", "angle DOC thereby facilitating the reasoning process. When an Interleave p Token is output in step i, its output hidden state "pustiitt is projected via post interleave projector
Ppostinivv while all the output hidden states of the visual tokens Apust are projected via post visual projector post_ vis. The cosine similarity between the two projected embeddings is first computed and then scaled by a learnable parameter 7;
a (6) y cos (Ppol (h (ost Ppost_vis (hosssiishppsssvi (Ppostt
-cos
(2)
Each tokens' similarity score ( is then compared against predefined threshold and visual tokens with scores above this threshold are selected:
{u'0} (
0}.
The selected tokens {u(0} } are interleaved into the reasoning process at step i. In this way, the important visual regions are interleaved into the model, prior to each textual step, enhancing visual perception and improving reasoning accuracy.
Inference with Interleaved Visual Tokens. With the selected visual tokens {u(0} obtained at each reasoning step, MINT-COT interleaves both visual content and text-based reasoning steps throughout the inference process, ultimately producing the final answer. Formally, this process extends the standard CoT formulation in Eq. as:
(1) (1) (2)
(k),, answer LV,(VV (extEncoder(T)).
This interleaved token selection mechanism enables the model to explicitly ground visual evidence throughout the reasoning chain, thereby facilitating visual interleaved CoT reasoning for solving multimodal mathematical problems.
3.2 Dataset Curation
To empower MINT-CoT capabilities for MILMs, develop data generation pipeline that au- tomatically generates mathematical visual interlcaved data annotated with selected token indices, and obtain 54K samples for model training. To construct the text-only cot format of our dataset, we begin by selecting mathematical problems from the culberryy2600 dataset [74], which was created using Collective Monte Carlo Tree Search and demonstrates strong performance on reasoning tasks. Specifically, we extract the ###f Rationale" and "#fff Steps" sections from the dataset as the reference reasoning steps for our task. Using these sections alongside the corresponding images, we follow four-step data construction process, as shown in Figure 3:
1. Grid Images. To obtain the indices of visual tokens for subsequent token index annotation in textual reasoning steps, we divide the original images into grid cells. Following the patch-splitting strategy used vision encoders such as Vision Transformer [12], each image is partitioned into grid, and unique index is assigned to each cell. These grid cells and their indices are subsequently overlaid onto the original images to produce grideindexed images.
5

---

Step 1: Grid Images
Step Apply
MINT-CoT Dataset Exomple:
Question:
Weat the legth of BC thangle AtC. gemale
AB 3.0 with reho 4,27 Ouicee Vov 10 19 121b0:
DCR
Oatguts:
ese Rettender The preziem instrna a circ's with chord chend, creates right org oi teargls CBS, wnich con be ond of ds (denchud and Sisan sots includee the lingth of the cherd oegment
Grid-indesed Images
Original Iniages
a not
Step Key
#ddt trop iieerse
Sokurion
GPT-Ao
"Toen 48
ewee ESSane depeniial RC, srlnnlee ABC
iinterieave ciniir by Rodi Propertieallt Thancers Thaie"
map by
Priaglan ADE A&C
Srop tn
OCR Text index Pairs
wase Shep interrieeee 3.4 8,8
natto of Ab All sun ADXABI LD300,
dda Shop frddiwen be 4.0 which sagles, sit the propertion OE
"Stup 2": lraato AD ho AIT),
Go
Annotate Key Words
Stop 4: Align and
wisie 3 vnturriaee &.0,
netto AD AR Given AQuA8 1
sites
24 Vistedeseeeiiiee DE whthh comosponde
"Step 3h: Iarr,, "sc"). "step ["cc]]
30 aC
ajiu shop todktt the lingth ac. Ther, (1/3je 4.
84
dete The fire anverr is... Text-only CoT
Key Words
"DE" 14.1011111111 "BC" 28.22282
Les be tha launn of BC Than, (1/3]x 14.
multraly
by giinn
4.0 3
enee fiul inswer
Visual Interleaved CoT Reasoning:
Figure 3: Data generation pipline. Step I: Grid Images. We divide each image into grid cells and assign index values to each cell. Step 2: Apply OCR. We use PaddleOCR to recognize textual elements and associate them with corresponding grid indices. Step 3: Extract Key Words. We employ GPT-4o to extract key words from each reasoning step. Step 4: Align and Annotate Key Words. We GPT-44 annotate each key word with the grid indices, and get the final visual interleaved CoT reasoning steps.
2. Apply OCR. Then, to more accurately map token indices onto textual reasoning steps, we apply PaddleOCR [37] to recognize textual elements in the original images. And align the bounding boxes of the detected text with their corresponding grid indices, thereby constructing "OCR text-index" pairs.
3. Extract Key Words. Certain mathematical concepts often play significant role in each reasoning step. Selecting visual tokens closely related to these concepts can improve reasoning accuracy. Therefore, we employ GPT-4o [12] to extract key words from each reasoning step. Since the extracted key words are used in the subsequent annotation with visual indices, they are extracted only when a reasoning step contains links visual tokens.
4. Align and Annotate Key Words. Finally, given the grid-indexed images, the "### Rationale" and "### Steps" sections, the "OCR text-index" pairs, and the extracted key words, we prompt GPT-4o to annotate each key word with the corresponding grid indices. These annotated indices are subsequently inserted into the reasoning steps associated with their corresponding key words, resulting in a visual-interleaved CoT reasoning dataset.
Through this process, construct dataset of 54K samples, where the reasoning steps annotated with corresponding grid indices. As shown in the right column of Figure 3, each data point consists of mathematical problem and an image input, with the corresponding visual interleaved CoT response as output. This dataset serves as the foundation for training the MINT-CoT models. Further details are provided in Appendix A.2.
3.3 Training strategy
Building on the previously introduced MINT-CoT framework and dataset, we now describe the corresponding MINT-CoT training strategy, which consists of three stages: (1) Text-only CoT Training, (2) Interleaved CoT SFT, and (3) Interleaved CoT RL.
Stage 1: Text-only CoT SFT, To enable the MILM to adopt general reasoning format, we first train the base model using the text-only CoT reasoning data in MINT--CoT dataset, without visual interleaving. This stage serves as foundation for subsequent interleaved training.
Stage 2: Interleaved COT SFT. In the second stage, we aim to train the model to select visual tokens using the Interleave Token and adapt reasoning with interleaved visual content. The model

---

is tine-tuned with a loss that jointly optimizes both textual reasoning and visual alignment. As introduced Eq. 4, the output sequence of MINT-Col alternates between sets of selected visual tokens (8)) and textual reasoning steps s!t), followed by the final answer:
(5) We first apply a cross-entropy loss to textual tokens at positions T C (1, T} covering all segments { and the answer, while conditioning on the full preceding sequence. Let Y {y1, 1/.............................. .r } denotes the full sequence of output tokens. Specifically, the loss for predicting the next textual token is defined as:
{v()) s(), w222, s(2) ((k), (())), answer Po(- I.T),
C===
Σ log Po 1ET
Vet. I.T)
We do not supervise the cross-entropy loss for predicting the Interleave token. Instead, we manually concatenate it at each step, and during inference, we concatenate the Interleave Token whenever the "### Step" marker is generated. To supervise the interleaved visual tokens, we apply binary cross-entropy loss on the scaled cosine similarity scores introduced in Eq. 2 with ground-truth labels X E 10,
N
L
LBCE =
Σ
EEO (=10=1
Xis ((a)) + (1 Xiz) log(1 - (aa)))
where is the number of Interleaved Tokens in a batch, is the length of input visual tokens, and a()) denotes the sigmoid function. The final training objective is defined as the sum of both losses:
L = LCr + LACE.
(8)
This combined loss guides the model jointly align visual tokens and perform interleaved reasoning.
Stage 3: Interleaved CoT RL. To move beyond supervised annotations, aim to enable the model to autonomously explore more flexible and effective selection of visual tokens guided by reasoning objectives, and enhance its ability to perform interlcaving CoT reasoning. Reinforcement learning provides a natural framework for this goal. To this end, we extend the Group Relative Policy Optimization (GRPO) [56] framework to our MINT-COT training strategy. For group of reasoning chains with group size G, we compute answer correctness as the reward r E {0, 1} and define the
std(r) where ry indicates if the j-th chain of steps in group yields the correct answer. The policy loss for the generated tokens then formulated as:
advantage via group-wise comparison as A₁ 5i =r-menn(r)
G Σ
Po(Y,) A PPo. || Prot] 1=1 Po (Y,)
LGROO = BB
G
where Pier reference policy that serves as regularization target. This stage further strengthens the model's reasoning ability with visual interleaved content, ultimately resulting in MINT-C6T-7BB Additional theoretical details of this training stage are provided in Appendix A.3.
4 Experiments
In this section, we first introduce the experimental settings in Section 4.1. Then, we discuss the quantitative results and ablation study in Section 4.2 and Section 4.3 respectively. Finally, we present the qualitative results in Section 4.4.
4.1 Experimental Settings
Implementation Details. We build on Qwen2-VL- [64] and train our model in three stages with combination of SFT and RL the MINT-CoT dataset. All model parameters except the vision encoder are updated. Full implementation details are provided Appendix A.4.
Test Benchmark. We evaluate MINT-CoT on three mathematical benchmarks: GeoQA [5], Math- Vista [43] and MMStar [7]. GeoQA is benchmark of geometric problems with annotated solution programs. To evaluate on GeoQA, we follow R1-V [6] and Hint-GRPO [27] using the Geol 170K test set [15], the English version of the GeoQA benchmark. Math is benchmark designed to
7

---

Table 1: Combined quantitative results on Math Vista. We evaluate MINT-CoT-7B, the baseline model, and state-of-the-ar general and reasoning MLLMs on the mathematical subset of Math Vista. MINT-CoT significantly outperforms the baseline model and achieves superior performance compared to open-source reasoning models. Bold and underlined results indicate the best and second-best among open-source models, respectively.
Math Vista-Math
Model
#Params
All
GEO
ALG
GPS
TQA
Closed-Source Model
GPT-4o [48]
66.67
63.68
67.04
63.46
77.42
Claude-3.5 Sonnet [1]
67.41
65.09
67.79
65.38
74.19
Open-Source General Model
[34]
7B
67.04
69.34
67.04
69.71
58.06
Lntern [8]
8B
62.59
62.26
62.92
62.50
62.90
Le2mn--2-8B-MPPP [65]
8B
68.52
68.87
68.91
69.71
64.52
DeepSeck-V VL2 [68]
4.5B
65,56
63.68
65.54
63.94
70.97
Qwen2.5-VL--BBIInttuee [3]
7B
66.66
65.56
66.29
65.87
69.35
Open-Souree Reasoning Model
Open-RI-Multimdda [13]
7B
54.81
52.36
54.68
53.37
59.68
RI-VL-7B [80]
7B
69.63
68.87
69,66
69.71
69.35
Mulberry [74]
7B
68.52
67.92
68.54
68.75
67.74
MM-Eureka [46]
7B
72.59
71.22
72.66
72.60
72.58
Owen2-VL-7B-Instteee [64] (Baseline)
7B
41.11
35.85
41,57
36.54
56.45
MINT-CoT-7B
7B
73.70
74.53
73.78
75.00
69.35
A over the Baseline Model
+32.59
+38.63
+32.21
+38.46
+12.9
Table 2: Combined quantitative results on
Table Combined results on the mathematical subset of MMStar. We evaluate MINT-CoT--B, the baseline model and the state-of-the-arts.
GeoQA. We evaluate MINT-CoT-7B. the baseline model and the saate-of-thh-ars
Model
GeoQA
Model
MMStar-Math
Qwen2.5-VL LL--B-----ttnee [3]
43.50
Swen2.5-VL-7BB-iee [3]
66.8
RI-V [6]
59.00
IntemVVL2-88 [8]
66.8
[13]
48.67
R1-VL-7B [79]
68.4
Mulherry [74]
66.8
Hint-GRPO [27]
55.31
Open Open-Ri-MMulimnnn 1-Multmodal [13]
59.2
Bann2-V--BB-Isstiin [64] (Baseline)
37.80
Dwen2-VL-7B-Inan [64] (Bascline)
46.4
MINT-CoT-7B
64.72
MINT-CoT-7B
A over the Baseline Model
+26.92
A the Baseline Model
+23.2
integrate challenges from diverse mathematical and visual tasks. As our paper targets specifically mathematical problems, we extract the mathematical subsets (FunctionQA, Geometry3K, GeoQA+, GEOS, and UniGeo), i.c., 'Math Vista-Math" Table 1, and report accuracy scores across four primary tasks: geometry reasoning (GEO), algebraic reasoning (ALG), geometry problem solving (GPS), and textbook question answering (TQA), MMStar a multi-modal benchmark covering dif- ferent capabilities and detailed axes. For evaluation, also extract the mathematical capability dimension, referred to as "MMStar-Math"
4.2 Quantitative Results
Comparison with the Baseline. As shown in Table 1 for the results of mathematical subsets of Math Vista, our MINT-CoT-7B achieves an improvement of up to +32.59% over the baseline, and improves a lot on all four primary tasks. This strongly demonstrates the effectiveness of our MINT- CoT framework and training strategy. Table presents the results on the GeoQA benchmark, where our MINT-CoT-7B outperforms the baseline model by +26.92%. Similarly, in Table 3, MINT-CoT-7B outperforms the baseline model by +23.2% on MMStar-Man validating the efficiency of MINT-CoT on geometry problems.

---

Table Ablation study on different training stages. We evaluate the three progressive training stages on different benchmarks.
Math Vista Matth
Model
MMStar-Math
GeoQA
All
GEO
AL.G
GPS
TQA
Baseline [64]
46.4
37.80
41.11
35.85
41.57
36.54
56.45
Text-only CoT SFT
67.6
59.02
64.07
64.15
64.04
64,42
62.90
Interleaved CoT SFT
68.0
62.07
67.78
66.51
67.79
67.31
69.35
Interleaved CoT RL
69.6
64.72
73.70
74.53
73.78
75.00
69.35
Figure 4: Fl score plot of
Table 5: Ablation study of different interleaving methods GeoQA and Math Vista-Math. Our Interleaved CoT SFT achieves the highest improvement both benchmarks, demonstrating the effectiveness of our interleaved token selection method.
visual token selection dur-
ing Interleaved CoT SFT.
3.6
F1 Soore Trend
25
Model
GeoQA
Math Vistaista- Vista-Math
3.
MMM
All
GEO
ALG
GPS
TQA
23
Original
37.80
41.11
35.85
41.57
36.54
56.45
Are
Text-only SFT
59.02
64.07
64.15
64.04
64.42
62.90
32
Original Image CoT SFT
61.41
40.37
38.68
40.82
39.42
43.54
24
Interlcaved CoT SFT (Ours)
Bounding Box CoT SFT
61.80
62.07
65.56
67.78
63.21
66.51
65.54
67.79
67.31
63.94
70.97
69.35
3.0
500
1000 1300 2000 2003
Comparison with State-of-the-arks. We also compare our model with toate--ffthe-arr MLLMs, including closed-source model, open-source models, and open-source reasoning models. Specifically, for open-source reasoning models, we choose recent works like R1-VL-7B [79]. MM-Eureka [46] and Open-RI RI-Multimddal [13]. As shown Table our model achieves the highest overall accuracy on the Math mathematical subsets, outperforming both open-source reasoning models and general models, and surpassing the best-performing open-source MI.LM by +1.11% as well as closed-source models, demonstrating strong capabilities in mathematical reasoning. On geometry reasoning, geometry problem solving and algebraic reasoning, MINT-CoT-7B outperforms state- of-the-art models by +3.31% +1.1 and +2.4% respectively. However, for textbook question answering, our performance slightly below MM-Eureka. On the GeoQA benchmark, as shown in Table our model outperforms the state-of-the of-the-ar models by +5.72%. In Table 3, MINT-CoT-77 also outperforms the tfate-of-the-- art by +1. 1.2% on MMStar-Mant Math, further demonstrating its capability in geometry reasoning.
4.3 Ablation Study
Training Stage Ablation. We conduct an ablation study on the different training stages of MINT- CoT, as described in Section 3.3. The results different benchmarks are presented in Table 4. The Text-only CoT SFT stage improves performance by +21.2% on MMStar-Math +21.22% on GcoQA, and +22.96% on Math Vista-Math, as it helps the model learn the general reasoning format illustrated in the left column of Figure: 3. The Interleaved CoT SFT stage further boosts performance by +0.4% on MMStar- Math, +3.05% GeoQA, and +3.71% % Math Vista-Math Math across all primary tasks by enabling the model to interleave visual tokens into textual reasoning steps. Finally, the Interleaved CoT RL stage enhances performance by an additional +1.6% on MMStar-Math, +2.65% on GeoQA, and +5.92% on Math Mitt through reinforcement leaming, which enables the model to reason more effectively with interleaved tokens.
Interleaving Method Ablation. We conduct an ablation study on the interleaving method used in the Interleaved CoT SFT stage, with the results presented in Table 5. Starting with the model trained in the Text-only CoT SFT stage, we simply interleave the original image into each reasoning step without the use of projectors or the Interleave token structure, which we refer to as "Original Image CoT SFT".. We find that, on Math Vista-Math, the performance of Original Image CoT SFT significantly decreases compared to Text-only CoT SFT. On the GeoQA benchmark, it also underperforms our Interleaved CoT SFT. This decline likely due to the interleaving of excessive unrelated visual tokens during reasoning. Furthermore, train model that uses the Interleave token to select

---

Question:
Given triangle ABC where AB = AC and angle BAC measures 100.0 degrecs, and the perpendicular bisector of AB intersects AB and BC at points und E, what is the meusure angle BAE? Represent the measure of angle BAE as 0, Choices: A: 80" B: 60" C: 50 D: 40
Qwen2vi--BB-Isstee
To solve this prablem, first need visualize the triangle ond the given information Vn'vn, Triangle ABC isosceles with AB = AC and angle BAC= 100.0 degrees.\n? Ln2. The perpendicular bisector of AB intersects AB ot point D and intersects BC at point E.An'nSiice median to the hypotennuse of triangle ABC (because D is the midpoint of AB). An'nAlso, in an isosceles triangle, if a median Due the
AC, triangle ABC isosceles
well. The perpendicular bisector
implies that
drown the hypotenuse, creates two right-angled triangles. Let's denote the measure angle BAE properties of the isosceles triangle and the perpendicular bisector, angle BAE (9) should be half of the opex angle BAC, An\nSince BAC 100.0 degrees, 9 100.0 50.0 degrees. ThenTheeee,,,, the correct answer is
MINT-CoT-BB
The problem involves geometry question about ongles an isosceles triangle with certain intersections, To find angle BAE, consider properties of isosceles triangles and angle bisectors. ### Let's think step by step.
Rationales:
### Step 1:
Since triangle ABC isosceles with AB AC, ongles ABC and ACB are equal.
### Step 2:
Calculate angle ABC ACB. The of angles triangle ABC degrees. Therefore, angle ABC angle ACB = (180 angle BAC) / 2 (180 - 100) / 2 degrees.
### Step 3:
The perpendicular bisector of AB implies that point is equidistant from and making AE. The bisector makes angle BAE an angle in triangle BAE.
Apply angular considerations: Consider that both AE and BE are equal, and with angle BAC 100 degrees and angle ABC 40 degrees, angle BAE will share properties of isosceles triangle and angle relationships.
### Step 4:
Cosine Similarity Scores
### The final onswer is:
Figure 5: Qualitative results of --------V---------- and MINT-CoT-7B, MINT-CoT-7B demon- strates improved CoT reasoning capability by interleaving fine-grained visual tokens. There is also a visualization of the similarity scores for the Interleaved Token generated during Step 4.
rectangular region of visual tokens at each reasoning step, referred to as "Bounding Box CoT SFT". As shown in the table, this approach underperforms our Interleaved CoT SFT on both benchmarks, except for the TQA task, and even underperforms the Text-only CoT SFT on GEO and GPS tasks in Math Vista-Math. These results demonstrate the effectiveness of our token selection method for mathematical reasoning tasks.
4.4 Qualitative Results
We present the qualitative results of the baseline model Brren--VL-7B-Instee and our proposed model MINT-CoTE Co,-BB, as shown Figure 5. Compared to the baseline, MINT-CoT-77 demonstrates more coherent reasoning format and is capable of selecting and interleaving relevant visual tokens during inference. More qualitative results of our model are shown in Appendix A.6. Moreover, we provide a plot of the average F1 score between the selected visual tokens and ground truth visual tokens in each reasoning step during the Interleaved CoT SFT stage, as shown Figure 4. For the Interleaved CoT RL stage, we do not report an F1 score plot due to the absence of ground truth visual token indices for online inference. As shown the plot, the score exhibits a fluctuating upward trend during training, demonstrating that the accuracy of visual token selection is increasing during the Interleaved CoT SFT training strategy.
5 Conclusion
In this paper, we first propose MINT-CoT, a method for enhancing multimodal mathematical reasoning by interleaving fine- tne-ggrined visual tokens into COT. We use the novel Interleave Token automatically select visual tokens for each reasoning step. Then, we introduce the MINT-CoT dataset and a four-step dataset generation pipeline. Finally, we present
MINT-CoT training strategy, which includes Text-only CoT Training, Interleaved CoT SFT and Interleaved CoT RI., enhancing the MILMS ability to reason over interleaved visual tokens. Our experiments with the obtained MINT-CoT-7B model demonstrate significant improvements across various benchmarks.
10

---

Acknowledgements
This study was supported in part by National Key R&D Program of China Project 20222D0161 100, in part by the Centre for Perceptual and Interactive Intelligence, a CUHK-led InnoCentre under the InnoHK initiative of the Innovation and Technology Commission of the Hong Kong Special Administrative Region Govermment, in part by NSFC-RGC Project N 4UHK498/24, and in part by Guangdong Basic and Applied Basic Research Foundation (No. 202381515130008, XW).
References
[1] Sonnet Anthropic. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet.
[2] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. ArXiv, abs/2308.12966 2023.
[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang. Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
[4] Guo Chen, Yin-Dong Zheng, Jiahao Wang, Jilan Xu, Yifei Huang. Junting Pan, Yi Wang, Yali Wang, Yu Qiao, Tong Lu, et al. Videollm: Modeling video sequence with large language models. arXiv preprint arXiv:2305. 13992, 2023.
[5] Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. ArXiv, abs/2105. 14517, 2021.
[6] Liang Chen, Lei Li. Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generaliza- tion ability in vision-languan models with less than $3. https: stttps://githie thub. com/Deep-Agent/ R1-V, 2025. Accessed: 2025-02-02.
[7] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, al. Are the right way for evaluating large vision-language models?
arXiv preprint arXiv:2403.20338 2024.
[8] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang. Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic lisual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185-24198, 2024.
[9] Linger Deng, Yuliang Liu, Bohan Li, Dongliang Luo, Liang Wu, Chengquan Zhang, Pengyuan Lyu, Ziyang Zhang, Gang Zhang, Errui Ding, al. R-cot: Reverse chain-of-thought problem generation for geometric reasoning large multimodal models. arXiv preprint a4Xiv:2410.1 10..:/80..778..... 2024.
[10) Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open- vlthinker: early exploration complex vasion-language reasoning via iterative self- improvement, 2025.
[11] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv preprint (tt::2411..44433 2024.
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021.

---

[13] EvolvingLMMM-Lab. openorl-mulimodal:: A fork to add multimodal model training to open- rl. attpss//gitthu..com//aaa wow//vvoviggLMM LLC////M---LL////A Lab/open--ooeeeeeeeeoe r1-multimodal. 2025. Accessed: 2025-05-13.
[14] Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal Ilms in video analysis. arXiv preprint arXiv:2405.21005, 2024.
[15] Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, et al. G-llava: Solving geometric problem with multi-modal large language model. arXiv preprint arXiv:2312.1 137.1130,, 2023.
[16] Jun Gao, Yongqi Li, Ziqiang Cao, and Wenjie Li. Interleaved chain-of-thought, 2025.
[17] Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, and Yu Qiao. Llama-adapter v2: Parameter-effiiient visual instruction model. arXiv preprini arXiv:2304. 15010, 2023.
[18] Google Gemini Team. Gemini: family of highly capable multimodal models. arXiv preprint vrXiv:2312.11800 2023.
[19] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint (rrxii:2501.1244: 2025.
[20] Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng. Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seedl. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025.
[21] Zilu Guo, Hongbin Lin, Zhihao Yuan, Chaoda Zheng, Pengshuo Qiu, Dongzhi Jiang, Renrui Zhang, Chun-Mei Feng, and Zhen Li. Pisa: A self-augmented data engine and training strategy for 3d understanding with large models. arXiv preprint arXiv:25033 10529, 2025.
[22] Ziyu Guo, Ray Zhang, Hao Chen, Jialin Gao, Dongzhi Jiang, Jiaze Wang, and Pheng-Am Ann Heng. Sciverse: Unveiling the knowledge comprehension and visual reasoning of Imms on multi-modal scientific problems. arXiv preprint arXiv:2503.10627, 2025.
[23] Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint arXiv:2501. 13926, 2025.
[24] Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang. Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li, Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with modti-modality for 3d understanding, generation, and instruction following. arXiv preprint arXiv:2309.00615 2023.
[25] Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. Worldsense: Evaluat- ing real-world omnimodal understanding for multimodal llms. arXiv preprint arX1v:2502.043366 2025.
[26] Yushi Hu, Weijia Shi. Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. arXiv preprint arXiv:24006. 0946609903,0 2024.
[27] Qihan Huang, Long Chan, Jinlong Liu, Wanggui He, Hao Jiang, Mingli Song, Jingyuan Chen, Chang Yao, and Jie Song. Boosting milm reasoning with text-debiased hint-grpo, 2025.
[28] Zihan Huang, Tao Wu, Wang Lin, Shengyu Zhang, Jingyuan Chen, and Fei Wu. Autogeo: Automating geometric image dataset creation for enhanced geometry understanding. arXiv preprint arXiv:2409.0900 099399999999 2024.
[29] Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng- Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703, 2025.
12

---

[30] Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li. Yu Qi, Xinyan Chen, Liuhui Wang. Jianhan Jin, Claire Guo, Shen Yan, Bo Zhang. Chaoyou Fu, Peng Gao, and Hongsheng Li. Mme-cot: Benchmarking chain-of-thoughy in large multimodal models for reasoning quality, robustness, and efficiency, 2025.
[31] Dongzhi Jiang, Renrui Zhang. Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, al. Mmsearch: Benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409. 02200./2990 /2959, 2024.
[32] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-2222 222112 2022.
[33] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang. Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0333 03322, 2024.
[34] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.033266 2024.
[35] Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-ttouggu arXiv preprint arXiv:2501.077 075522 2025.
[36] Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-ttougl 2025.
[37] Chenxia Li, Weiwei Liu, Ruoyu Guo, Xiaoting Yin, Kaitao Jiang, Yongkun Du, Yuning Du, Lingfeng Zhu, Baohua Lai, Xiaoguang Hu, Dianhai Yu, and Yanjun Ma. Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system, 2022.
[38] KunChang Li, Yinan He, Yi Wang. Yizhuo Li, Wenhai Wang. Ping Luo, Yali Wang, Limin Wang. and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355 2023.
[39] Pengxiang Li, Shilin Yan, Joey Tsai, Renrui Zhang, Ruichuan An, Ziyu Guo, and Xiaowei Gao. Adaptive classifier-free guidance via dynamic low-confidence masking. arXiv preprint wwXiv:2505.20/99 2025.
[40] Ziyi Lin, Chris Liu, Renrui Zhang. Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. ECCV 2024, 2023.
[41] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurlPS, 2023.
[42] Zuyan Liu, Yuhao Dong, Yongming Rao, Jie Zhou, and Jiwen Lu. Chain-of-spot: Interactive reasoning improves large vision-language models. arXiv preprint arXiv:2403. /2966, 2024.
[43] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang. Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning foundation models in visual contexts. In International Conference on Learning Representations (ICIR), 2024.
[44] Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-chum Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. In Annual Meeting of the Association for Computational Linguistics, pages 6774- 6786, 2021.
[45] Ruilin Luo, Zhuofan Zheng, Yifan Wang, Yiyao Yu, Xinzhe Ni, Zicheng Lin, Jin Zeng, and Yujiu Yang. Ursa: Understanding and verifying chain-of-thougee reasoning in multimodal mathematics. arXiv preprint 2rX4:2250..4666, 2025.
13

---

[46] Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365 2025.
[47] Fanxu Meng, Haotong Yang, Yiding Wang, and Muhan Zhang. Chain of images for intuitively reasoning. arXiv preprint arX1v:2311.09241, 2023.
[48] OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tul- loch, Andrey Mishchenko, Angela Baek, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang. Benjamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugarssi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger. Charlotte Barette, Chelsea Voss. Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czamecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin, Daniel Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong, Ehsan Asdar, Blizabeth Proehl, Elizabeth Yang. Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Eugene Brevdo, Evan Mays, Farzad Khorasani, Felipe Petroski Such, Filippo Raso, Francis Zhang. Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao. Haitang Hu, Hannah Wong, Haoyu Wang. Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner. Henrique Ponde de Oliveira Pinto. Hongyu Ren. Hui- wen Chang, Hyung Won Chung, Ian Kivlichan, Ian O' O'Connell, Ian O' O'Connell, Ian Osband, Ian Silber, Ian Sohl, Ibrahim Okuyucu, Ikai Lan, Ilya Kostrikov, Ilya Sutskever, Ingmar Kanitschei- der, Ishaan Guirajani, Jacob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shieh, Ji Lin, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beutler, Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Workman, Leher Pathak, Leo Chen, Li Jing, Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang, Louis Feuvrier, Lu Zhang, Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang, Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yathaz, Meng Jia Yang. Mengchao Zhong, Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe. Michael Petrov, Michael Wu, Michele Wang, Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro. Mikhail Pavlov, Miles Brundage, Miles Wang, Minal Khan, Mira Murati, Mo Bavarian, Molly Lin, Murat Yesildal, Nacho Soto. Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige, Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofir Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Comphell-Moore, Patrick Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming
14

---

Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang. Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miserendino, Sandhini Agarwal, Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balaji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman. Thomas Degry, Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng. Todd Underwood, Todor Markov, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Eloundou, Valerie Qi. Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojeiech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Gpt-4o system card, 2024.
[49] OpenAl. Chatgpt. https:// /ctttt.ccctt///// openai. com, 2023.
[50] OpenAl. GPT-4V(ision) system card, 2023.
[51] OpenAl. Hello gpt-4o. https:// cow.//nnee//helllgggt--ee 2024.
[52] OpenAL. Introducing openai o1, 2024., 2024.
[53] Shuai Peng, Di Fu, Liangeai Gao, Xiuqin Zhong, Hongguang Fu, and Zhi Tang. Multi- math: Bridging visual and mathematical reasoning for large language models. arXiv preprint arXiv:2409.00144 0444744477 2024.
[54] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 7748.8763. PmLR, 2021.
[55] Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with comprehen- sive dataset and benchmark for chain-of -ffin--ffii sii--fff--ooe thought reasoning, Advances in Neural Information Processing Systems, 37:8612-8642, 2024.
[56] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024.
[57] Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See- See-Kiong Ng. Lidong Bing, and Roy Ka-Wei Lee. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. arXiv preprint 240.:2200..7294, 2024.
[58) Yiwen Tang, Zoey Guo, Zhuhao Wang, Ray Zhang, Qizhi Chen, Junli Liu, Delin Qu, Zhigang Wang, Dong Wang, Xuelong Li, et al. Exploring the potential of encoder-free architectures in 3d Imms. arXiv preprint 25020222000006200, 2025.
[59] Chameleon Team. Chameleon: Mixed-modal carly-fusion foundation models. arXiv preprint arXiv:2405.09878. 2024.
[60] Qwen Team. 72g-72b--r 2025. Accessed: 2025-05-13.
[61] Chengzhuo Tong, Ziyu Guo, Renrui Zhang, Wenyu Shan, Xinyu Wei, Zhenghao Xing, Hong- sheng Li, and Pheng-Ann Ann Heng. Delving into rl for image generation with cot: A study on dpo 2025.
vs. grpo. arXiv preprint arXiv:2505.
[62] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 2rXiv:2302.13971 2023.
15

---

[63] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, al. Qwen2-vl: vl: Enhancing lision-langued model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[64] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model's s perception of the world at any resolution, 2024.
[65] Weiyum Wang. Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv preprint arXiv:2411 I. 10444, 2024.
[66] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thoughe prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
[67] Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. Mind's eye of Ilms: of ootthuught elicits spatial reasoning large language models. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[68] Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang. and Chong Ruan. Deepseek-v12: Mixture-of-experts vision-language models for advanced multimodal understanding, 2024.
[69] Han Xiao, Guozhi Wang, Yuxiang Chai, Zimu Lu, Weifeng Lin, Hao He, Lue Fan, Liuyang Bian, Rui Hu, Liang Liu, al. Ui-genie: A self-improving approach for iteratively boosting malm-based mobile gui agents. arXiv preprint arXiv:2505.214 1449, 2025.
[70] Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointlim: Empowering large language models to understand point clouds. arXiv preprint arXiv:2308.1691 2023.
[71] Shilin Yan, Jiaming Han, Joey Tsai, Hongwei Xue, Rongyao Fang, Lingyi Hong. Ziyu Guo, and Ray Zhang. Crosslmm: Decoupling long video sequences from Imms via dual cross-attention mechanisms. arXiv preprint arXiv:2505. 17020, 2025.
[72] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang. Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan. Yunfei Chu, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint 4xXiv:2407.10671, 2024.
[73] Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. Rl-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.. 10615 2025.
[74] Huanjin Yao. Jiaxing Huang. Wenhao Wu, Jingyi Zhang, Yibo Wang. Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with -ike reasoning and reflection via collective monte carlo tree search. arXiv preprint 18X1122412.18319 2024.
[75] Junyan Ye, Dongzhi Jiang, Jun He, Baichuan Zhou, Zilong Huang, Zhiyuan Yan, Hongsheng Li, Conghui He, and Weijia Li. Blink-twice: You see, but do you observe? a reasoning benchmark on visual perception. arXiv preprint arXiv:2510.09361, 2025.
16

---

[76] Runpeng Yu, Xinyin Ma, and Xinchao Wang. Introducing visual perception token into multi- modal large language model. arXiv preprint arXiv:2502.17425 2025.
[77] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
9567, 2024.
[78] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training, 2023.
[79] Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. RI-vI: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503. 222.::2503./997,,, 2025.
[80] Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vI: Learning to reason with multimodal large language models via step-wise group relative policy optimization, 2025.
[81] Renrui Zhang, Jiaming Han, Chris Liu, Aojun Zhou, Pan Lu, Yu Qiao, Hongsheng Li, and Peng Gao. Llama-adapter: Efficient fine-tuning of large language models with zero- aartiiniiiaizzed attention. In ICLR 2024, 2024.
[82] Renrui Zhang, Dongzii Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal Imm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169-186. Springer, 2024.
[83] Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Ziyu Guo, Shicheng Li, Yichi Zhang, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, Shanghang Zhang, Peng Gao, Chunyuan L.,, and Hongsheng Li. Mavis: Mathematical visual instruction tuning with an automatic data engine, 2024.
[84] Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Mul- timodal chain-of-thour reasoning in language models. arXiv preprint 2rXi::2302.00223, 2023.
[85] Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Yu Zhou, and Sibei Yang. Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models. Advances Neural Information Processing Systems, 36:5168-511 119118559911110 2023.
[86] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing usiio-language understanding with advanced large language models. arXiv preprint ar04v:2304. 10592, 2023.
[87] Le Zhuo, Liangbing Zhao, Sayak Paul, Yue Liao, Renrui Zhang, Yi Xin, Peng Gao, Mohamed Elhoseiny, and Hongsheng Li. From reflection to perfection: Scaling inference-tine optimization for text-to-image diffusion models via reflection tuning. arXiv preprint 1600002504 2025.
[88] Zhuofan Zong, Bingqi Ma, Dazhong Shen, Guanglu Song, Hao Shao, Dongzhi Jiang, Hongsheng Li, and Yu Liu. Mova: Adapting mixture of vision experts multimodal context, arXiv preprint 2404 13046, 2024.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The three main conclusions and the scope of our paper are detailed in the final section of the introduction and summarized in the abstract.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

Justification:
All the asuumptions and theorems are clearly stated.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The architecture details are provided in the Methodology and Experiment Section for reproducing the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
The code and data are currently available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /Codeeuomiisseeeeeeee
/nips. wwtpccccpppcccaecccccccccccccclt /tneeeeeueeeeeeeeemiiiiiii ssssssoliccyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The training and test details are provided in the Experiment Section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We conduct experiments only once and report the accuracy of the best model, and would be too computationally expensive to conduct the pre-training multiple times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The compute resources are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //neer/inesseeee http://urrrpp...pppiii/.//////// httpp//www.tttttt.cscteeeeeeeeee ctwpiiiisceielennecc ines?
Answer: [Yes]
Justification:
Our research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
We cite the original papers or websites that produced the code package or dataset.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We details the data and model the experiment section, appendix, and GitHub.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
23

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https http://ips.c//ooofeeeeeeeeeeeeee for what should or should be described.
24

---

A Appendix
A.1 Overview
We organize our supplementary material as follows.
Dataset Details
Dataset Example
Dataset Statistic
Theoretical Details of Interleaved CoT RL
Additional Implementation Details
Additional Ablation Study
Text-only Method Ablation
Results on Other Benchmark
Projector Ablation
Additional Qualitative Results
Limitations
A.2 Dataset Details
Dataset Example We present examples from our MINT-CoT Dataset in Figures 8, where the yellow highlights indicate the interleaved grid indices, and the blue highlights denote the key words in each reasoning step.
Dataset Statistic We provide the key statistics of MINT-CoT Dataset in Table 6. This dataset comprises 54,031 data points derived from the mathematical portion of the Mulberry-260k dataset.
Table 6: Key statistics of the MINT-CoT dataset.
Statistic
Value
Total data points
54,031
Data points containing Interleave Tokens (interleaved data points) Average number of Interleave Tokens per interleaved data point
52,142
2.80
Maximum number Interleave Tokens in a single interleaved data point
12
Average number of selected indices per interleaved data point Average number selected indices per Interleave Token Minimum number of selected indices in single Interleave Token
19.91
7.10
1
Maximum number selected indices in a single Interleave Token
140
A.3 Theoretical Details of Interleaved CoT RI.
Following the standard GRPO framework [56], we integrate GRPO into our approach. Specifically, similar to LCE in Stage 2, we apply policy loss LGRPO_ieN to textual tokens:
Paly Ày, 3Dxu Pad }}. (10) (P
LaRVOLIe text -BB (1.r))
t E |.. EieT,
where AS. is the advantage detailed in Section 2.3, Pier reference policy that serves as regularization target, and [P Piet] penalizes deviation from this reference distribution to encourage stable updates. The min and clip operations are omitted for brevity.
To enable more flexible and effective selection of visual tokens, we further apply a LaRPO_vi to the scaled similarity scores f, which are derived from the interactions between Interleave tokens and input visual tokens in the the j-th chain of reasoning steps. Let N, denote the the number of
25

---

Table 7: Ablation study on the post interleave projector and the post visual projector. We compare three configurations: without projectors, with single-layer linear projections, and with two-layer MLPs.
Configuration
Layer Number
All
GEO
ALG
GPS
TQA
w.o, projectors
64.44
63.68
64.42
63.94
66.13
67.78
66.51
67.79
67.31
69.35
w. projectors
2
65.18
63.21
65.54
63.94
69.35
reasoning steps in j-th chain, and M denote the number of visual tokens interleaved the i-th reasoning step in the j-th chain. Formally, the loss is defined as:
CaRro Eoss (1r)
E
+
Polalllone-d. - 3Daa |I P H|
(11)
Pet Pe
The final policy loss is defined as the sum of both losses, with the LoRPO vis rescaled by a weighting factor A;
CORPO LaRPO_Iet A LaRPO_vis
(12)
By computing this combined loss, we enhance both token selection and inference capabilities using Interleave tokens.
A.4 Additional Implementation Details
We Qwen2-VI.-7B [64] the base MILM model in experiments. Each of the projectors, Pinterleave and Pvis. is implemented as a single linear layer. We uniformly set the threshold / = 0.7 to filter the similarity scores. The hyper-paramest
y to scale the similarity is set to 1/0.07 following CLIP [54]. The training procedure consists of three stages: (I) Text-only CoT Training, where we train for 2 epochs on the MINT-CoT dataset without applying the interleaving strategy, using a learning rate of 5.0e-6 and a batch size of 64, following the configuration of Mulberry [74]; (2) Interleaved COT SFT, where
train for epochs the MINT-CoT dataset with learning rate of le-6 and batch size of 64; and (3) Interleaved CoT RL, where we train for 700 steps on the MINT-CoT dataset, using a group size == 4, a weighting factor 0.02, learning rate of 1e-6 and a batch size of 16. During training, all model parameters, including the Interleave Token and projector layers, are unfrozen, except for the vision encoder, which remains fixed. Finally, the resulting model is named MINT-CoT-7BB
For Bounding Box CoT SFT, rectangle that covers the index positions of all labels as the ground truth bounding box to train the model. We train epochs with learning rate of le-6 and batch size of 64. And during inference, interleave the minimal enclosing rectangle that covers all the seleted tokens. For Original Image CoT SFT, however, we enforce the concatenation of the entire image at the beginning of each step during both training and inference. We train only 1 epoch with learning rate of le-6 and batch size of 64,
the MINT-COT dataset and extract the minimal enclosing
A.5 Additional Ablation Study
Text-only Method Ablation We conduct an ablation study using text-only training pipeline. This involves first applying a Text-only CoT SFT stage to the baseline model, followed by Text-only CoT RL stage using the GRPO framework. As shown Table 8, MINT-CoT still outperforms this setting by 2.96% on the Math Vista mathematical subsets, showcasing the superiority of our Interleaved CoT approach.
Results on Other Benchmark In addition to the quantitative results in Section 3.2, we provide results of the different training stages our method on the vision-only input setting the Mathemat- ics section of MMMU-Pro [77] benchmark. As shown in Table 9, MINT-CoT-7B outperforms the baseline model by +5%. and the performance improves with the Interleaved CoT approach, validating the efficiency of our method.
26

---

2,22,30, 30,33, interleave Assign the given natio the triongle's sides: cornesponds (which corresponds to Table Ablation study of the text-only training method.
Table 9: Ablation study of different training stages on the Mathematics section of the MMMU-Pro benchmark.
Model
Math Vista-Math
Model
MMMU-Pro-V
Baseline
41.11
+ Text-only CoT SFT
64.07
Baseline
25
Text-only CoT RL
70.74
Text-only CoT SFT
25
Interleaved CoT SFT
67.78
Interleaved CoT SFT
30
Interleaved CoT RL
73.70
Interleaved CoT RL
Question:
Input Image:
Grid Image:
The Municipal Planning Bureau designing cross street bridge with height of 'z' units, where = 6.0. the slope of the overpass AC follows a 3:4 ratio, what is the length of the slope A AC/mChoices''nA 10minH: 8minC: 6minl): 6v3m
Outputs:
The problem involves calculating the hypotenuse (slope AC) of a right triangle where one side (the height AB) is given, and the ratio of the legs (AB:BC) is provided. Using these details, the Pythagorean theorem
### Rationales:
will help find the length of the hypotenuse.
Let's
step by step.
the values and natios given. The height of the bridge (AB) is units, with slope ratio of 3:4, indicatina AB:BC.
### Step 1: <interleaaee 9,116, 9.16,23, 23,32,,00 iennnOnereeoeeeeeeee
### Step 2:<interleave=2
parts, and BC corresponds To parts.
Step 3:xinterleave>2 23, 26, 26,,0,334in vinterleeter intlulaave-Clllllate the length BC. Since
parts), then each part units. BC, having parts, will be 4 2 units.
### Step 4:Use Pythogorean theorem, VV AC Msgrrt(AB22 BC*2) \1).
### Step 5:Substitute the values the equation: VV AC Visgrt(6 22 8"2) Visgrt(36 64) Vlsgrt(100) W..
The enswer
Figure 6: An example from MINT-CoT dataset.
Projector Ablation We conduct an ablation study on the post interleave projector Posttimnte and the post visual projector P Pis on the Interleaved CoT SFT stage. Both projectors were initially implemented as single-layer linear layers. We first remove both projectors entirely, and then replace them with two-layer MLPs using GELU activation. Both configurations are trained for three epochs. The results on the mathematical subset of Math Vista are shown in Table 7, in which we find that the initial configuration as single-layer linear layers performs the best over all primary tasks.
A.6 Additional Qualitative Results
In addition to Section 3.4, we provide more qualitative results of the baseline model Qwen2-VL-7B- Instruct and our proposed model MINT-CoT-7B in Figures 11.
A.7 Limitations
In our dataset curation pipeline, the use of GPT-4o for dataset annotation still has some computa- tional cost. Moreover, alternative reinforcement learning strategies beyond our method now remain underexplored.
27

---

Question:
Input Image:
Grid Image:
Find the measure of the altitude drawn to the hypotenuse. Choices: A: 11 B: (29T) C: 27 1): 297
Outputs:
The question asks for the measure of the altitude drown to the hypotenuse of right triangle. Given the lengths of segments \V( MP \\) and VV PO V)), we can use properties of right triangles involving altitudes.
### Rationales:
#*## Let's think step by
### Step 1: <interleave 22 23 25 29 30,32 interleave seterllavveIdeettii Identify the known segments: VV MP 11 \V). VV : 27 \1).
#### Step 2:<interleave 21,22,27 28,2 28,29,34,444 inntelveeeee The hypoTenuse VV MO MP PO 11 27 38 VV).
### Step 3:Use the geometric mean theorem (altitude-on -nfitude-nn-yypateuuss theorem), which states that the altitude equal to the product the segments divides hypotenuse into: VV NPN222 MP Vitimes
squored 2P"222
(N)
#*## Step 4:Calculate the product: VM NP"2 Wttimes
11).
#### Step 5:Find VV NP \V) taking the square root: VM( (1agrt(297) WV.
### The final answer is:
Figure 7: An example from MINT-CoT dataset.
Input Image: Grid Image:
Question:
Determine the degree measure ungle the figure, where circle inseribed right triangle ABC. Angle ACB measures degrees angle CBO measures degrees. 60"
Choices: 30" C: 45"
Outputs:
### Rationales: To solve problem, need determine angle in the right triangle ABC with 90-degree ongle at C Cadd a 30-degree angle B.
### Let's think step by stee.
#### Step bxiinterleave-..1 0,1,00,,,,31111.1.. 3..33...34iinneeettet..e interleaave Recognize that triangle ABC night triangle with angle ACB measuring 90 degrees.
###t Step 2:<interleave> 31.344interleavee Ideettff ### Step 3:Use the fact that the sum of angles in triangle ABC is 180 degrees.
angle
part triangle OBC, measures 30 degrees.
### Step 4:<interleave i.344interleave Calculaae angle ABC, Since angle CBO 30 degrees and angle ACB degrees, angle ABC 90 30 dearees.
### Step 5: 0 1,,30,33,33 iineereeneeeevee Find angle A using the angle sum property of emiangle:\nAng. A (8Aglee
Angle ABC Angle ACB degrees
30 degrees
### The final onswer is: B
Figure 8: An example from MINT-CoT dataset.
28

---

Question:
As shown in the figure, in diamond ABCD, the diagonals AC and BD intersect ut O. It is known and AC The perimeter of diamond ABCD O).
Choices: A:
C:: 1>:5
Qwen2vi-B--Iustruc
The quadrilateral ABCD is rhombus since its diagonals perpendiculor. Vr\nn[ss]\Vvpaii A.B.C.D. (0, (6,0): = (-3*sg) 3)AnD (0, 4):\nO (0, - htwp://iwwiiiiiiiiiddiniaoooooaa (egeeeiinewit(())))eee (ndrowww))) black-linewidth((): Vndrew(O--A,, blcckeiinewidhhh))))dddroooooo
block-linewidth(()) Indraw(O--C, black-linewidth(()) Androw)w)dOOO-- blcck-linewidhh()))dddeeed
NJ; \nlobe(($$05 w.t../////////////////aaaaa of the diogonals O. An\nSince OA OC and OB OD = 3, we have OB and BD 6.AnnnSo, the perimeter of the rhombus ABCD
E): En:lbbel("$CS dirr(31)))\llobee(("
label the intersection point
28 Therefore, for the perimeter of the diamand ABCD, the answer 20 \nVnTherefore, the answer is
408 4
28. VninSince the question
MINT-CoT-7B
### Rationales:
To find the perimeter the rhombus, the length of all sides needs to be determined. Since the diagonals bisect each other at right angles in rhombus, the sides can be found using the Pythagorean theorem. An \n Let's 'ss think step by step.
#### Step I:
Identify the properties of the rhombus. In rhombus, the diagonals bisect each other right ongles. Therefore, AO OC AC/2 BD/2 6/2
Cosine similarity
BO OD
Scores
Use the Pythogorean theorem calculate the side length (AD) of the rhombus. The hypotenuse AD, with AO and 3. V'sgrt(AO"2 BO"2) *Visgr((422 3*2)= V1sgrt(16 9) Vasrr(25)) 5\n\
### Step 2:
Similarity Scores
### Step 3: Calculate the perimeter of the rhombus. Since all sides are equal in rhombus, the perimeter 4 times the \V)
side length. AnVnrerimeter VV Vttimes ### The final answer is:
Figure 9: Comparison between Dwwen2-VL-7B-Inssse and MINT-CoT-7B.
Question:
As shown in the figure, the rudius of circle O is 4.0, BC is the diameter, AC is the tangent of circle O, and AC - 6.0, then AB
Choices: A:4 B: 6 C: 10 D: 12
Qwea2vi-7B-Instruct
Becouse AC is tongent to circle O, perpendicular OC, defining the length of AB using the Pythogorean theorem in a right triangle:Vn'ni. Start by understanding the Pythogorean theorem. In2. Apply Pythogorean theorem right triangle with segments AC hypotenuse, An'nnGven that AO is the rodius (4.0), we use the Pythagerean theorem triangle OAB:An\AAB S(OA" ) \n\nAB /(16 16) \nAB
solve for segment
the legs segment
the
O8*)= /(4.0* no exact option with this value, but the closest would be C) However, given the calculation, the length AB should be bit more than the radius of circle O. U.TTTherreeee, the answer is C:6. X
6.4\nnnnS answer is approximate
seems there
MINT-CoT-7B
#### Rationales: To determine AB, understand the properties of the circle and the right-angled triangle formed the tongent AC and the diameter BC, Use the Pythagerean theorem in triangle ABC. #### Let's think step by step.
### Step 1:
Identify the known lengths: radius of the circle (OC) is 4, and AC, the tangent,
Cosine Similarity Scores
### Step
Recognize that is diameter of the circle. Since the rodius 4, BC (being the diameter) is twice the radius: BC 8.
Cosine Similarity
### Step
Since AC is tangent and BC diameter, consider triongle ABC, Triangle ABC is right triangle with AC perpendicular to BC.
Cosine Similarity Scores
### Step Apply the Pythagorean theorem in triangle ABC: VV 4B"2 AC"2 BC V)).
### Step 5: Substitute the known values into the Pythogorean theorem: \n\ A V.). and W 8-2 \)).
### Step Calculate
squares: \nV 622
### Step 7: Sum the squores: \n\V AB 36 64 100 V\).
### Step Take the square root to find AB: \n\ AB \\sgrt(100) = VV).
### The final answer is: C
Figure 10: Comparison between Dwwn2-VL-7B--nneee and MINT-CoT-7B.
29

---

Question:
In ABCD, given that AB BC, the perpendicular bisector of diugonal AC intersects AD point E. Connect CE. the perimeter of ABCD is 200em, what is the perimeter of triangle CDE?
Choices: A: 200.0 B: 400.0 C: 150.0 D): 100.0
Qwen22--7B-Instrueet
The correct answer is D. By symmetry, BC 50 em, AD 50 cm, so AE = cm. From the right triangle AEC, AC AE"2 CE*2, so AC AE"2 Therefore, the perimeter of triangle CDE 75
CE"2 AD 2E"2. Therefore,
MINT-CoT-7B
### Rationales: The problem involves understending the geometric placement of point and relations between various sides and diagonals the quadrilateral. It requires determining the perimeter of the triangle CDE given the perimeter ABCD.
#### Let's think step by step.
### Step 1: Given the perimeter of ABCD 200 cm. we need to explore the relationships among the sides of the quadrilateral and the triangle formed by extending certain segments.
### Step 2:
The perpendicular bisector of AC indicates symmetrical property:
Similarity
EC. The intersection meons that equidistant from and C.
consider potential equalities relationships with the line segment created by intersection of AD and the perpendicular bisector at E.
### Step
### Step Express the perimeter triangle CDE: bisector of AC, explore possible equalities simplifications. ### Step 5: Using the perimeter constraint and the properties outlined, determine the lengths of the sides of triangle CDE and thus its total perimeter.
EC. Given
to being on the perpendicular
### The final onswer is:
/
Figure 11: Comparison between Qwen2-VL Biem2-VL-7B-Instruue and MINT-CoT-7B.
30


Reviews:
[{"review_id": "GmHBwlv0NG", "reviewer": "Reviewer_Ed2K", "summary": "The paper aims at improving the mathematical reasoning capabilities of the VL models using the visual signals in the chain-of-thoughts. Specifically, it argues that the prior work does not focus on the fine-grained visual details (e.g., rely on box-level cues), and require external tools which come with additional inference costs and large-scale training. To resolve this, the paper proposes MINT-CoT framework. In particular, the authors (a) curate 50K problems with fine-grained visual cue annotations, (b) train a vision-language model to focus on the cues via an interleave tokens, and (c) performs progressive training (e.g., text-only, SFT, and RL) to further refine the model performance. Ultimately, the authors show that the method achieves large improvements in the performance on MathVista and GeoQA dataset.", "questions": "Mentioned in strengths and weaknesses", "limitations": "Mentioned in strengths and weaknesses", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengths\n\n1. The experimental design is quite interesting. The idea behind creating a nice dataset with fine-grained visual cues will be a very usual artifact for the community. \n2. The method to train an interleave token that decides the regions to focus on automatically is novel and seems to work well on MathVista and GeoQA.\n3. The three-stage training pipeline is also sensible and the ablation studies showcase the usefulness of diverse design choices (Table 3-4).\n\nWeaknesses\n\n1. The choice of the base model as Qwen-2-VL-7B seems strange to me given Qwen-2.5-VL-7B exists in Table 1. For instance, Qwen-2.5-VL achieves 66.6% on MathVista instead of 40% Qwen-2-VL-7B. It is unclear if the method can be utilized to improve the frontier VLMs or not. I imagine that the gains will not look as large as ones claimed in the paper (30% on MathVista) once you train with Qwen-2.5-VL model.\n2. The evaluation is quite narrow – just MathVista and GeoQA. Infact, GeoQA is a part of MathVista already. There are many more evaluations that are quite popular – MathVerse, MathVision, We-Math, MMMU-Pro that should be tested. There should be some general-purpose evals too like VQAv2, GQA, Blink, ChartQA etc to understand the impact of such training on overall visual capabilities of the model. In addition, it would be better to include the numbers for the entire mathvista data instead of a particular subset.\n3. As suggested by Table 3 and 4, it seems that the biggest gain over the base model is from text-only CoT which is the default method nowadays. In this light, I believe that the paper overclaims its numbers. Instead of saying +34% on MathVista over the base model, I think the right baseline should be text-only CoT (default) method for reasoning training. \n4. It is not clear if text-only SFT followed by text-only RL is worse than MINT-CoT framework or not. It is a valid baseline which seems missing in the current setup.", "was_revised": true, "final_justification": "Rebuttal performs several important comparisons to understand the usefulness of the method.", "rebuttal": "Thank you for your thoughtful review and constructive feedback. We greatly appreciate your recognition of our dataset design, the novelty of the Interleave Token, and the effectiveness of our three-stage training pipeline. We address your concerns below.\n\n ### 1. Choice of Qwen-2-VL-7B as Base Model\nWe choose Qwen-2-VL-7B as the base model to more clearly demonstrate the effectiveness of our method, as it leaves more room for improvement and highlights relative gains. To address your concern, we also conduct additional experiments using the stronger Qwen-2.5-VL-7B model. As shown in the table below, our method still brings consistent improvements on MathVista-Math, confirming its generality across different base models. \n| Model | Base model | Text-only CoT SFT | Interleaved CoT SFT | Interleaved CoT RL | \n|-|-|-|-|-|\n| Qwen2.5-VL-7B-Instruct | 66.66      | 71.85      |  72.22    | **75.93**   |\n| Qwen2-VL-7B-Instruct  | 41.11      | 64.07            | 67.78        | **73.70**   |\n\nDue to the limited time available during the rebuttal period, we did not perform any hyperparameter tuning and directly adopted those from Qwen-2-VL-7B. Despite this, the MINT-CoT framework on Qwen-2.5-VL-7B still shows significant potential for improvement and demonstrates the robustness of our approach.\n\n### 2. Evaluation on other benchmarks\n> The evaluation is quite narrow – just MathVista and GeoQA.\n\nWe appreciate this suggestion and have extended our evaluation to cover additional datasets. Specifically:\n - In our supplementary materials, we have added the evaluation results on the Mathematics section of **MMStar** benchmark (Table 2 in Suppl.) \n - We did not evaluate on **MathVision**, since it was used in constructing our training data (aligned with the training data of Mulberry), which would make evaluation results biased.\n - Due to the time limit, we add evaluation experiments on the Vision Only domain of **Mathverse**, which can well represent the improvement of our method. \n - Moreover, we add experiments on the Mathematics section of the **MMMU-Pro-V**. Although this subset contains only 60 questions, our method still achieved the best performance. \n\nThe combined results of those benchmarks are shown below:\n| Benchmark  | Qwen2-VL-7B-Instruct  | Text-only CoT SFT | Interleaved CoT SFT | Interleaved CoT RL |  \n|----------------------------------|--------------------|----------------------|---------------------|----------------| \n| MathVista-Math | 41.11              | 64.07                | 67.78               | **73.70**       | \n| GeoQA                            | 37.80              | 59.02                | 62.07               | **64.72**          | \n| MMStar-Math                      | 46.4               | 67.6                 | 68.0                | **69.6**           | \n| MathVerse_MINI (Vision Only)     | 23.98        | 32.74              | 33.25            | **36.42**    | \n| MMMU_Pro_V                       | 25                 | 25                   | **30**                  | **30**             | \n\n>There should be some general-purpose evals too like VQAv2, GQA, Blink, ChartQA …\n\n\nThe focus of our research is specifically for multi-modal math problems, including the interleave token for finer-grained selection and a dataset construction pipeline. These designs target math-specific challenges, such as geometric structures and compositional diagrams. Whereas our methodology can also generalize to general-purpose scenarios:\n - Our dataset curation pipeline can be adapted to other tasks. The image gridding strategy, auxiliary OCR tools, keyword extraction, and final visual index annotation can also be applied to general visual reasoning tasks. For math reasoning tasks, we need to align fine-grained visual indices with the reasoning steps. In contrast, general visual reasoning tasks do not require such a high level of granularity, so our pipeline remains suitable for general tasks as well.\n - The introduction of an Interleave Token for grounding reasoning steps in corresponding visual content is a general mechanism that enhances multimodal alignment and interpretability across various tasks.\n - Our three-stage training pipeline can be easily adapted to many other general tasks. \n\nWhile our current work focuses on math-specific challenges, we acknowledge the importance of evaluating our method on broader vision-language tasks. Due to the domain gap between our mathematical reasoning dataset and benchmarks like VQAv2, GQA, BLINK, and ChartQA, we did not include them in our current evaluation. We plan to explore this in future work.\n\n> In addition, it would be better to include the numbers for the entire mathvista data …\n\nWe chose to evaluate only the MathVista-Math subset because our method specifically targets mathematical reasoning tasks. The full MathVista dataset includes general visual question answering tasks (e.g., VQA, TQA), which fall outside the scope of our training data and method design.\n\n### 3. Text-only CoT Baseline\nThanks for your suggestion. We conduct experiments on the text-only SFT followed by text-only RL. As shown below, MINT-CoT still outperforms this stronger baseline:\n| Model | Qwen2-VL-7B-Instruct | Text-only CoT SFT | Interleaved CoT SFT |  Text only SFT + Text only RL | MINT-CoT-7B | \n| - | - | - | - | - | - | \n| MathVista-Math | 41.11    | 64.07                | 67.78               | 70.74 | **73.70** | \n\nThis confirms that our improvements specifically arise from the interleaved visual reasoning enabled by MINT-CoT."}, {"review_id": "ubJsYAo3UE", "reviewer": "Reviewer_PFDq", "summary": "The paper presents a method for visual math reasoning where an interleave token at each reasoning step is used to select additional relevant visual tokens for reasoning.", "questions": "As listed in the \"weaknesses\" section above, copied below\n\n- Need to clarify the use of \"interleave tokens\".  e.g., Note that an interleave token is the output of a reasoning step. (without visual tokens), Do we assume that the interleave token might not be optimal since it is generated without visual info, or do we simply assume that the ground-truth reasoning steps are given -- therefore the proposed method is a data curation method that enhance the reasoning steps with additional visual tokens?\n- the method is task or dataset specific. I am not convinced that the method can be easily applied to other visual reasoning tasks.\n- the evaluation is performed only on MathVista. More, different benchmarks need to be used to validate how general the method can be applied.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: the research is well-motivated. The proposed method is conceptually simple, and its effectiveness is validated on a standard visual math benchmark.\n\nWeaknesses: \n- the method is task or dataset specific. I am not convinced that the method can be easily applied to other visual reasoning tasks.\n- the evaluation is performed only on MathVista. More, different benchmarks need to be used to validate how general the method can be applied.\n- Need to clarify the use of \"interleave tokens\".  e.g., Note that an interleave token is the output of a reasoning step. (without visual tokens), Do we assume that the interleave token might not be optimal since it is generated without visual info, or do we simply assume that the ground-truth reasoning steps are given -- therefore the proposed method is a data curation method that enhance the reasoning steps with additional visual tokens?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and feedback. We sincerely appreciate your recognition of the motivation and simplicity of our approach. We address your concerns in detail below.\n### 1. Application on other visual reasoning tasks:\n> The method is task or dataset specific. I am not convinced that the method can be easily applied to other visual reasoning tasks. \n\nWhile the current focus of our research is specifically on multi-modal math problems, our method can also be applied to other visual reasoning tasks. \n - Our dataset curation pipeline can be adapted to other tasks. The image gridding strategy, auxiliary OCR tools, keyword extraction, and final visual index annotation can also be applied to general visual reasoning tasks. For math reasoning tasks, we need to align fine-grained visual indices with the reasoning steps. In contrast, general visual reasoning tasks do not require such a high level of granularity, so our pipeline remains suitable for general tasks as well.\n - The introduction of an Interleave Token for grounding reasoning steps in corresponding visual content is a general mechanism that enhances multimodal alignment and interpretability across various tasks.\n - Our three-stage training pipeline can be easily adapted to many other general tasks. \n\n### 2. Evaluation on other benchmarks\n> More different benchmarks need to be used to validate how general the method can be applied.\n\nWe appreciate this suggestion and have extended our evaluation to cover additional datasets. Specifically:\n - In our paper, the results of **GeoQA** have been presented.\n - In our supplementary materials, we have added the evaluation results on the Mathematics section of **MMStar** benchmark (Table 2 in Suppl.) \n - Due to the time limit, we add evaluation experiments on the Vision Only domain of **Mathverse**, which can well represent the improvement of our method. \n - Moreover, we add experiments on the Mathematics section of the **MMMU-Pro-V**. Although this subset contains only 60 questions, our method still achieved the best performance. \n\nThe combined results of those benchmarks are shown below:\n| Benchmark  | Qwen2-VL-7B-Instruct  | Text-only CoT SFT | Interleaved CoT SFT | Interleaved CoT RL | \n|----------------------------------|--------------------|----------------------|---------------------|----------------| \n| MathVista-Math | 41.11              | 64.07                | 67.78               | **73.70**       | \n| GeoQA                            | 37.80              | 59.02                | 62.07               | **64.72**          | \n| MMStar-Math                      | 46.4               | 67.6                 | 68.0                | **69.6**           | \n| MathVerse_MINI (Vision Only)     | 23.98        | 32.74              | 33.25            | **36.42**    | \n| MMMU_Pro_V                       | 25                 | 25                   | **30**                  | **30**             | \n\n\n### 3. Clarification of the use of \"interleave tokens\"\n> Note that an interleave token is the output of a reasoning step. (without visual tokens) …\n\nSorry for the confusion caused. We need to clarify that Interleave Token is not *”the output of a reasoning step”*, but a special token concated prior to each reasoning step. It is in the token sequences, so it is not *“without visual tokens”*. Because transformers allow all tokens to attend to prior context, the Interleave Token can access:\n - Previously selected visual tokens;\n - The input image tokens;\n - All prior reasoning steps.\n\nTherefore, the Interleave Token **is not blind to visual input**, but can “see” both the text reasoning steps and visual tokens before that step, to choose the related visual tokens. This process is visualized in Figure 2, where the red token marks the Interleave Token."}, {"review_id": "QpcsKizPGO", "reviewer": "Reviewer_nxuS", "summary": "This paper proposes MINT-CoT, a novel approach to enhancing mathematical reasoning in multi-modal large language models (MLLMs) through interleaved token-level interleaving Chain-of-Thought (CoT). Central to this approach is the MINT-CoT dataset of 54K problems, where each reasoning step is explicitly grounded to relevant visual regions. The authors design a three-stage training pipeline involving text-only CoT supervised fine-tuning (SFT), interleaved CoT SFT, and interleaved CoT reinforcement learning (RL), culminating in the MINT-CoT-7B model. The model achieves strong gains on visual math benchmarks.", "questions": "1. Resolution Dependency of Token Selection: Since the number of visual tokens depends on input image resolution, how do you handle ground-truth labels for active visual tokens when resolution changes? Do you apply any post-processing to align token indices across different resolutions?\n2. Efficiency Concerns: How many visual tokens are input during reasoning on average? Could you provide details about the training time and inference time overhead introduced by interleaving visual tokens compared to the baseline?\n3. Number of Interleave Tokens: How many Interleave Tokens are leant during training? \n4. Design Choice in Token Selection Loss (Eq. 7): Why is the token selection supervised with a binary cross-entropy loss (Eq. 7) rather than integrated directly into the next-token prediction objective (Eq. 6)? For instance, could token selection be formulated as a number secretion over L visual tokens, jointly optimized during sequence modeling? You can number the visual tokens from 1 to L, and predict which token number is selected.\n5. Domain Specificity of Visual Tokens: What justifies the use of selected visual tokens specifically for geometric math reasoning? I think this token election can be generalize to natural images or more general domains.", "limitations": "Yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n1. The paper addresses a fundamental challenge in multi-modal mathematical reasoning.\n2. The introduction of an Interleave Token that dynamically selects arbitrary-shaped visual regions per reasoning step is novel.\n3. The construction of a large-scale dataset (54K samples) with token-level alignment between text and vision is a significant contribution to the community. \n4.  MINT-CoT-7B achieves substantial performance improvements over baselines on two challenging benchmarks (MathVista and GeoQA).\n\n## Weaknesses:\n1. Unclear Visual Perception Capability: While the paper emphasizes fine-grained interleaving of visual tokens, it relies on a math-domained variant of CLIP (MAVIS) for visual grounding. However, CLIP and its derivatives are known to capture coarse-grained semantics, which raises concerns about their ability to accurately perceive fine-grained visual elements such as small geometric cues, labels, or spatial relationships crucial in mathematical diagrams. The paper does not provide direct evidence that MINT-CoT meaningfully overcomes these perceptual limitations.  Although the model shows improved final-answer accuracy on benchmarks like MathVista and GeoQA, it remains unclear whether the interleaved visual steps are always semantically or visually correct. There is no diagnostic analysis or qualitative study examining failures in intermediate reasoning, which could be masked by correct final answers but reflect flaws in the model’s step-wise visual understanding.\n2. Questionable Necessity of Multimodal CoT for Geometric Figures:  The paper assumes that visual interleaving is beneficial for geometric figures. However, many such diagrams consist of simple symbolic elements—lines, points, circles, triangles—which lack rich semantics and can be easily and precisely described in natural language. Given that LLMs are already capable of understanding these concepts from large-scale text-only training, it's unclear whether introducing visual tokens truly adds value. In fact, there is no direct evidence provided to show that the model understands these visual tokens better than their natural language descriptions. For example, does the model truly understand the meaning of the selected visual tokens? what happens if a portion of these selected tokens is randomly dropped or replaced with irrelevant ones?\n3. Unclear Advantage of Token-Level Visual Grounding over Box-Level: While the paper criticizes box-level grounding for including irrelevant visual content, visual tokens are also subject to noise and overlap, especially in complex diagrams. For example, a token covering a \"circle\" may still partially include other elements (e.g., lines or text). Furthermore, the empirical improvement in Table 4 over the box-based OCT baseline is minimal. \n4. Missing Ablation on the Effectiveness of Interleaved CoT: The paper lacks a key ablation study to isolate the impact of the proposed interleaved multimodal CoT reasoning. Specifically, it remains unclear how much of the performance gain comes from the MINT-CoT dataset itself versus the interleaving strategy. A natural baseline would be to train the model on the MINT-CoT dataset without interleaved visual tokens, using only text-based CoT supervision, and compare performance. Without this, it is difficult to determine whether the interleaved design is truly responsible for the observed improvements or whether the gains stem largely from exposure to large-scale curated training data.", "was_revised": true, "final_justification": "**Thanks authors for providing detailed explanations and additional experiments.** I appreciate the contribution of proposing a pipeline that interleaves visual and textual tokens to support reasoning.\n\nHowever, the effectiveness of this pipeline fundamentally relies on the accuracy of the selected visual tokens, which in turn depends on the perception ability of the visual encoder. Perception ability refers to the encoder’s capacity to extract, represent, and highlight key features from raw diagrams, an essential prerequisite for any visual reasoning task. But throughout the rebuttal, the authors repeatedly emphasis that current MLLMs exhibit limited visual perception capabilities and that they do not attempt to enhance the encoder's perception performance. This raises a critical concern: without accurate visual perception, how can one ensure that the interleaved Chain-of-Thought (CoT) reasoning steps are grounded in correct visual understanding?\nIf the visual inputs are poorly encoded, the downstream reasoning, even if structurally improved,  still operate on incorrect or incomplete visual representations. \n\nBased on the authors’ experiments, in the setting where the model is trained without inserted visual tokens, but with visual supervision loss (“Reasoning w.o. Visual Tokens”), the performance drops even below the baseline (Text-only CoT SFT). This result raises a serious concern: the proposed training data may not provide meaningful learning signals for the visual encoder. The quality of the visual token labeling used for supervision has also been questioned by two other reviewers.\n\nOverall, without ensuring accurate visual token selection, the performance improvement observed from interleaving with textual tokens still lacks a clear and convincing explanation. Without deeper insights into the underlying mechanism behind these gains, the proposed pipeline of interleaving visual and textual tokens for reasoning currently appears more like an engineering workaround or technical report, rather than a fully substantiated scientific contribution suitable for a top-tier venue NeurIPS.\nRegarding the questions about the resolution dependency of token selection/ Application on other visual reasoning tasks, I have asked this twice but have yet to receive a clear answer from the authors.\n\n**I keep my score, but lean more toward the borderline.**", "rebuttal": "Thank you very much for your feedback and positive comments. You highly praised the design of our Interleave Token, our large-scale dataset, and the performance of MINT-CoT-7B. Your concerns are addressed below.\n\n### 1. Visual Perception Capability\n> it relies on a math-domained variant of CLIP (MAVIS) for visual grounding …\n\nWe would like to clarify that **our MINT-CoT framework does not rely on MAVIS or any math-domain variant of CLIP**. We apply our MINT-CoT framework on Qwen2-VL-7B-Instruct model. As stated in lines 46-53, models like CLIP and SigLIP are pre-trained on natural images with general scenes, making mathematical images out-of-distribution. So attention-based token selection method to identify relevant visual tokens during mathematical reasoning is difficult.\n\n> The paper does not provide direct evidence that MINT-CoT meaningfully overcomes these perceptual limitations …\n\nWe do not aim to improve the raw perception ability of the encoder, but rather provide an **alternative mechanism** through the interleaving of selected visual tokens at each reasoning step, which helps the model to “watch” these important visual tokens again. \n\n### 2. Necessity of Multimodal CoT for Geometric Figures\n#### 2.1. Disadvantages of text-only reasoning\n> Given that LLMs are already capable of understanding these concepts from large-scale text-only training …\n\nIn practice, text-only captions of mathematical figures are frequently inaccurate. As shown in MAVIS, the visual perception of models like CLIP is inaccurate on mathematical reasoning tasks, so the natural language captions generated from MLLMs are often inaccurate. As a result, we need to interleave the related visual regions during reasoning, which can significantly reduce hallucination and improve the accuracy of mathematical reasoning.\n\n#### 2.2. A training-free experiment\n> In fact, there is no direct evidence provided to show that the model understands these visual tokens better than their natural language descriptions …\nWe have conducted a **training-free** multimodal CoT experiment on our base model: \n - Input question: *As shown in the figure, AB is tangent to circle O at point B, if angle A = 36.0, then angle C is equal to ()\\nChoices:\\nA:36°\\nB:54°\\nC:60°\\nD:27°*\n - Output solution: *Since AB is tangent to circle O at point B, we have angle ABO = 90°. Therefore, angle AOB = 90° - angle A = 54°. Since OA = OB, we have angle OAB = angle OBA = 36°. Therefore, angle C = 180° - angle AOB - angle OAB = 180° - 54° - 36° = 90°. Since angle C is an inscribed angle, we have angle C = 1/2 angle AOB = 1/2 * 54° = 27°. Therefore, the answer is D.*\n\nThe output solution of the base model is wrong when it comes to “Since OA = OB, we have angle OAB = angle OBA = 36°”. So we manually concat the related visual tokens before this reasoning step, and find that the whole output solution will be:\n -  *Since AB is tangent to circle O at point B, we have angle ABO = 90°. Therefore, angle AOB = 90° - angle A = 54°.<selected_visual_tokens> Therefore, angle C = 1/2 angle AOB = 27°. Therefore, the answer is D.*\nSo, the model outputs with correct answer when interleaving correct visual tokens during reasoning, which can proves that MLLMs benefits from Multimodal CoT when solving methematical problems.\n\n#### 2.3. Randomly select visual tokens\n> what happens if a portion of these selected tokens is randomly dropped or replaced with irrelevant ones?\nWe conduct an experiment that randomly selects visual tokens during inference of the Interleave CoT SFT model. The results show that inference with random selections introduced irrelevant visual context in each reasoning step, which deeply hurts the original format and logic.\n| Qwen2-VL-7B-Instruct | Interleave CoT SFT | Interleave CoT SFT (random select) | \n| - | - | - | \n| 41.11 | 67.78 | 31.11 | \n\n\n### 3. Precision of token selection\n> visual tokens are also subject to noise and overlap…\n\nDue to the limitation of the patching method of ViT, which is not fine-grained enough, it’s hard to avoid some noise when selecting visual tokens. However, token-level grounding remains significantly more precise than box-level selection.\n> Furthermore, the empirical improvement in Table 4 over the box-based OCT baseline is minimal.\n\nAs shown in Table 4 in our paper, the Bounding Box CoT SFT is 2.22% lower than Interleaved CoT SFT on MathVista-Math, and even performs worse than Text-only CoT SFT on GEO and GPS domains, which demonstrates that the Bounding Box CoT SFT is worse than our Interleaved CoT SFT method.\n\n### 4. Text-only CoT Baseline\nThanks for your suggestion. We conduct experiments on the text-only SFT followed by text-only RL. As shown below, MINT-CoT still outperforms this stronger baseline:\n| Model | Qwen2-VL-7B-Instruct | Text-only CoT SFT | Interleaved CoT SFT |  Text only SFT + Text only RL | MINT-CoT-7B | \n| - | - | - | - | - | - | \n| MathVista-Math | 41.11    | 64.07                | 67.78               | 70.74 | **73.70** | \n\nThis confirms that our improvements specifically arise from the interleaved visual reasoning enabled by MINT-CoT.\n\n### 5. Resolution Dependency of Token Selection\nWhen gridding the image, we align our gridding method and image resolution with the Qwen image processing method, so the selected token indices are correct during training. For models with different patching strategies, we support token index remapping based on image resize ratios, so the selected visual tokens remain aligned.\n\n### 6. Efficiency Concerns\n> How many visual tokens are input during reasoning on average? \n\n - As presented in Table 1 in the supplementary material, the Average number of Interleave Tokens per interleaved data point is 2.80, and the Average number of selected indices per interleaved data point is 19.91.\n - Importantly, when interleaving the selected visual tokens, we just concat the visual embeddings to the text embeddings, rather than re-encoding the image through the vision encoder (ViT). This design reduces a lot of time.\n - Training time and Inference time:\nWe provide the training time of three training stages on 8*A100 GPUs, and the average inference time per question on GeoQA:\n|  | Text-only CoT SFT | Interleave CoT SFT | Interleave CoT RL | \n| - | - | - | - | \n| Training | 4h | 9h | 25h | \n| Inference | 8.15s | 8.03s | 8.24s | \n\n\n### 7. Number of Interleave Tokens\n> How many Interleave Tokens are leant during training?\n\n\nAs presented in Table 1 in the supplementary material, the average number of Interleave Tokens per interleaved data point is 2.80.\n\n### 8. Design Choice in Token Selection Loss\nThank you for the insightful suggestion. But we concern that it will not be efficient enough to select visual tokens through next-token prediction. Moreover, the order of selected visual tokens is not semantically meaningful, and attempting to predict them autoregressively may introduce unnecessary complexity and training instability. We wil explore this formulation in the future work.\n\n### 9. Application on other visual reasoning tasks:\n> What justifies the use of selected visual tokens specifically for geometric math reasoning? I think this token election can be generalize to natural images or more general domains.\n\nThanks for your insight. While the current focus of our research is specifically on multi-modal math problems, our method can also be applied to other visual reasoning tasks. \n - Our dataset curation pipeline can be adapted to other tasks. The image gridding strategy, auxiliary OCR tools, keyword extraction, and final visual index annotation can also be applied to general visual reasoning tasks. For math reasoning tasks, we need to align fine-grained visual indices with the reasoning steps. In contrast, general visual reasoning tasks do not require such a high level of granularity, so our pipeline remains suitable for general tasks as well.\n - The introduction of an Interleave Token for grounding reasoning steps in corresponding visual content is a general mechanism that enhances multimodal alignment and interpretability across various tasks.\n - Our three-stage training pipeline can be easily adapted to many other general tasks."}, {"review_id": "P7633DI1me", "reviewer": "Reviewer_XQF3", "summary": "This paper proposes a chain-of-thought method, MINT-CoT, for multimodal mathematical reasoning. Its core contribution lies in introducing the Interleave Token mechanism, which dynamically selects visual tokens related to the current mathematical semantics at each reasoning step and interleaves them into the textual reasoning process to achieve collaborative visual-textual reasoning. To support this method, the authors construct a 54K-scale MINT-CoT dataset and design a three-stage training process, gradually transitioning from purely textual supervision to reinforcement learning-based visual interleaving training.", "questions": "1.The paper does not specify whether there is any limit or sparsity constraint on the number of visual tokens inserted. Have there been instances where excessive tokens caused reasoning failures?\n2.Does token selection behavior differ statistically between algebraic and geometric problems? Are there task-specific biases in token filtering?\n3.You use GPT-4o to extract keywords in dataset annotation. Have you evaluated the impact of alternative extraction methods on final performance? There appears to be full reliance on GPT-4o, raising concerns about potential error propagation.\n4.If OCR or keyword extraction fails, would visual token selection break down entirely? Have you incorporated such simulated error scenarios during training to improve robustness?", "limitations": "No. The checklist claims that the limitations of the work are included in the appendix, but in fact, no such discussion could be found.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\t1.The proposed Interleave Token mechanism enables dynamic insertion of visual tokens at each step of textual reasoning.\n\t2.The performance gains are significant, with improvements of 26.92% and 32.59% on GeoQA and MathVista-Math subtasks, respectively.\n\nWeaknesses:\n\t1.The method essentially relies on traditional similarity matching principles; the Interleave Token is effectively a formalized expression of a query-like vector in an attention mechanism, lacking structural novelty.\n\t2.The method heavily depends on high-quality alignment between visual regions and reasoning steps in the dataset. Although the token index annotation process is automated, inaccuracies in the mapping can significantly degrade performance, limiting generalizability in real-world applications.\n\t3.Figure 3 is not clearly presented, with small fonts and repetitive data processing descriptions, making the methodological pipeline appear redundant and less intuitive. The coordination between textual explanation and illustration is suboptimal.\n\t4.Although the reinforcement learning stage adopts the GRPO strategy, there is no description of implementation details, making it difficult to assess the effectiveness of this phase.", "was_revised": true, "final_justification": "The authors have addressed my concerns.", "rebuttal": "Thank you very much for your detailed feedback and constructive suggestions. We greatly appreciate your recognition of the Interleave Token mechanism and the strong performance gains achieved by MINT-CoT. Below, we address your concerns point by point.\n### 1. Interleave Token Structure:\n> the Interleave Token is effectively a formalized expression of a query-like vector in an attention mechanism, lacking structural novelty. \n\nWe would like to reclaim the novelty of our Interleave Token.  While the Interleave Token superficially resembles a query vector, its **design and function are distinct in several important aspects**:\n - The Interleave token is a special token that is specifically designed for selecting visual tokens. It will be appended to the input sequences, which are passed forward through the entire transformer. And then it will compute the similarity between text content and visual tokens. \n - In a highly integrated MLLM, the Interleave Token serves as a bridge that connects high-level CoT reasoning with low-level visual structure.\n - Moreover, the Interleaved token is supervised during training, which is different from the query-like vector in the attention mechanism.\n\n### 2. Mapping accuracy of visual regions in the dataset\nWe agree that accurate alignment is important. To ensure annotation quality, we have designed our dataset curation pipeline with careful consideration and multiple safeguards:\n - We used **state-of-the-art OCR tools** (e.g., PaddleOCR) to extract fine-grained elements such as symbols and labels.\n - We employed GPT-4o, one of the strongest available MLLMs, to generate alignment annotations using specially designed prompts aimed at minimizing noise and ambiguity.\n\nMoreover, to validate the reliability of the automatic annotations, we conducted a **human evaluation**: 10 annotators independently reviewed 100 randomly sampled MINT-CoT examples. They assessed whether the selected visual tokens at each reasoning step were relevant and accurate. The results showed an average step-wise accuracy of **89.49%**, which demonstrates that our pipeline produces high-quality alignments suitable for training multimodal reasoning models. \n\n### 3. Problem of Figure 3 \n> with small fonts and repetitive data processing descriptions, making the methodological pipeline appear redundant and less intuitive. The coordination between textual explanation and illustration is suboptimal.\n\nThanks for your suggestions. Given the complexity of the pipeline, we opted for a compact layout to keep it within a single figure. We encourage readers to zoom in for better readability. And we will refine it in the final version.\n\n### 4. Inplemental details of GRPO strategy\nDue to space limitations in the main paper, we have provided the detailed implementation of the GRPO strategy in Sections A.4 and A.5 of the supplementary material. We will integrate these descriptions into the main text in the final version to improve clarity.\n\n### 5. Limitation or sparsity constraint on the number of visual tokens inserted?\nAfter training, the number of interleaved visual tokens does not pose a risk of inference failure, as the selected tokens generally contain minimal irrelevant information. Furthermore, as presented in Table 1 in the supplementary material, the average number of Interleave Tokens per interleaved data point is 2.80, and the average number of selected indices per interleaved data point is 19.91, which reflects a sparse and controlled selection process, rather than excessive inclusion.\n\n### 6. Behavior difference between algebraic and geometric problems\n> Does token selection behavior differ statistically between algebraic and geometric problems? Are there task-specific biases in token filtering? \n\nWe analyzed whether token selection behavior differs between algebraic and geometric problems, using evaluation results of MINT-CoT-7B on MathVista-Math. Specifically, we compared samples from two distinct sources:\n - UniGeo (geometry questions)\n - FunctionQA (algebra/function questions)\n\nWe found that, on average, the proportion of visual tokens selected during inference is 3.60% for geometry questions, while for algebra/function questions, the rate is 2.48%. This suggests that the model tends to select broader visual regions for geometry problems, reflecting a task-specific preference in token filtering.\n\n### 7. GPT-4o annotation accuracy\nAs discussed in **”2. Mapping accuracy of visual regions in the dataset”**, we conduct a human evaluation on the accuracy of our annotated data generated by GPT-4o. The results show an average per-step accuracy of 89.49%, indicating that the annotation pipeline is reliable and sufficiently precise for the task.\n\n### 8. If OCR or keyword extraction fails, would visual token selection break down entirely?\nOverall, the annotation process handles OCR and keyword extraction failures well, and rarely leads to incorrect visual-token selection.\n - Modern OCR tools (e.g., PaddleOCR) are highly accurate, and in our pipeline, OCR serves only as auxiliary input. If there are no OCR results in a image, the GPT-4o-based annotation process remains unaffected, as it does not depend on OCR outputs exclusively.\n - During keyword extraction, prompts are carefully crafted to ensure that extracted keywords are both (1) highly related to the visual content, and (2) critical for reasoning. If no keyword satisfies these two conditions, no visual tokens will be interleaved for that reasoning step. As shown in Table 1 (supplementary), approximately 2,000 samples in the dataset contain no Interleave Tokens, which reflects this conservative design.\n\n### 9. Limitations\n> The checklist claims that the limitations of the work are included in the appendix, but in fact, no such discussion could be found.\n\nThank you for pointing this out — we apologize for the omission. The limitations of our current work are as follows: In the dataset curation pipeline, the use of GPT-4o for dataset annotation still has some computational cost. Moreover, alternative reinforcement learning strategies beyond our method now remain underexplored. \nWe will include a discussion of these limitations in the final version of the paper."}]

Author rebuttal:

Meta-review: Paper summary:  The paper present MINT-CoT a method to improve mathematical reasoning capabilities of VLMs.  The core contribution is the "interleave token mechanism" which dyanmically selects visual tokens related to the current mathematical semantics in the reasoning process and interleaves them with textual reasoning. The paper also has a curated dataset of 50K problems.  Findings include improvements in MathVista and GeoQA datasets and other results provided during the rebuttal.

Strengths/reasons to accept are: novelty of the approach, dataset contributions, and strong experimental results. 
Salient weaknesses are mentioned by reviewer nxuS which the AC concurs with. The AC asks the authors to address these concerns to improve the paper.
-  "the effectiveness of this pipeline fundamentally relies on the accuracy of the selected visual tokens, which in turn depends on the perception ability of the visual encoder. ... Overall, without ensuring accurate visual token selection, the performance improvement observed from interleaving with textual tokens still lacks a clear and convincing explanation."

Summary of Review and Discussion:  the four reviewers gave scores of 3, 4, 4, 5.  The reviewer who gave "3" did not participate in the discussion and did not respond to the rebuttal -- the AC has looked at the authors' response and deems it to be satisfactory.
Decision: Accept (poster)