VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
Authors: Chaoya Jiang
Yongrui Heng
Wei Ye
Haiyang Xu
Ming Yan
Ji Zhang
Fei Huang
Shikun Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal Chain-of-Thought; Multimodal Large Language Model; Region Recognition;
Primary area: applications

Abstract: Recently, reasoning-based MLLMs have achieved a degree of success in generating long-form textual reasoning chains. However, they still struggle with complex tasks that necessitate dynamic and iterative focusing on and revisiting of visual regions to achieve precise grounding of textual reasoning in visual evidence. We introduce VLM-R³  (Visual Language Model with Region Recognition, Reasoning, and Refinement ), a framework that equips an MLLM with the ability to (i) decide when additional visual evidence is needed, (ii) determine where to ground within the image, and (iii) seamlessly weave the relevant sub-image content back into an interleaved chain-of-thought. The core of our method is \textbf{Region-Conditioned Reinforcement Policy Optimization (R-GRPO)}, a training paradigm that rewards the model for selecting informative regions, formulating appropriate transformations (e.g. crop, zoom), and integrating the resulting visual context into subsequent reasoning steps. To bootstrap this policy, we compile a modest but carefully curated Visuo-Lingual Interleaved Rationale (VLIR) corpus that provides step-level supervision on region selection and textual justification. Extensive experiments on MathVista, ScienceQA, and other benchmarks show that VLM-R$^3$ sets a new state of the art in zero-shot and few-shot settings, with the largest gains appearing on questions demanding subtle spatial reasoning or fine-grained visual cue extraction.

VLM-R Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
Chaoya Jiang',Yongee Heng', Wei Ye- Han Yang", Haiyang Xu²j
Ming Yan?, Ji Zhang", Fei Huang", Shikun Zhang'
1 National Engineering Research Center for Software Engincering, Peking University Alibaba Group
ZEEKR Intelligent Technology Holding Limited
[wye] J @ (ww.@ppk.uccoooo
(shuofeng.xhy allibaba-inc.com
Abstract
Recently, reasoning-basee MLLMs have achieved degree of success in gener- ating long-form textual reasoning chains. However, they still struggle with com- plex tasks that necessitate dynamic and iterative focusing on and revisiting of vi- sual regions achieve precise grounding of textual reasoning in visual evidence. We introduce VLM-R* (Visual Language Model with Region Recognition and Reasoning), a framework that equips an MLLM with the ability (i) decide when additional visual evidence is needed, (ii) determine where to ground within the image, and (iii) seamlessly weave the relevant sub-image content back into an interleaved chain-of-thought. The core of our method is Region- Condit-nnddi Re- inforcement Policy Optimization (R-GRPO), a training paradigm that rewards the model for selecting informative regions, formulating appropriate transforma- tions (c.g. crop, zoom), and integrating the resulting visual context into subsequent reasoning steps. To bootstrap this policy, compile modest but carefully cu- rated Visuo-Lingual Interleaved Rationale (VLIR) corpus that provides step-level supervision region selection and textual justification. Extensive experiments on Math Vista, ScienceQA, and other benchmarks show that VLM-R sets a new state of the art in zero-shot and fho- settings, with the largest gains appear- ing on questions demanding subtle spatial reasoning or fine-grained visual cue extraction.
Introduction
Multimodal Large Language Models (MLLMs) have recently emerged as a powerful paradigm, demonstrating remarkable capabilities understanding and generating content across different modalities, primarily vision and language [G8, 27, X 65, E, M.. Models like O1 [67]], QvQ [3]] and Gemini 2.5 [MI] have showcased impressive performance on wide array of tasks such as MMMU [66], Math Vista [BO]] and ScienceQA [3T]. A key factor contributing to their advanced reasoning abilities is the integration of Chain-of-Thought (CoT) prompting [58], which elicits step-by-step reasoning pathways, often leading to more accurate and interpretable outputs.
Despite these advancements, critical limitation persists the way current MLLMs interact with visual information during complex reasoning processes, Most existing approaches [ 37.
51 62
'These authors contributed equally this work. corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

employing CoT predominantly confine the reasoning steps to the textual domain, with only an initial static grounding in the visual input. This paradigm falls short in scenarios demanding dynamic, iterative, and fine-grained interaction with specific visual regions throughout the reasoning chain. As shown in Figure II examples include se- quentially verifying hypotheses against image de- tails, tracking object states across visual cues, or comprehending intricate spatial relationshipsall of which require a more active and adaptive vi-
What the syadoal first bas counted from top botioes nigh??
sual grounding mechanism. Encouragingly, recent models such as O3 [ which capable of interleav- ing image analysis with text generationinspire new frontier where reasoning is not merely con- ditioned on image, but is continuously inter- twined with ongoing visual perception and local- ization.
-think- seod lo find the aymbe on the dans red box iven the top en the night side the ucage. Fins, Illl get doee look at dua specili srea slentify the bus and muchings "blus, 20 1815,
Developing an MLLM that can look again dur- ing reasoning faces two notable hurdles: Region- grounding learning. The model must learn where to focus and how to transform the grounded re- gion (crop. zoom) based on partial textual delib- eration. Credit assignment. Simply supervising final answers does not teach the model whether a chosen region actually contributed to correct rea- soning, making it hard to refine the visual-query policy.
right very uep daa will zcers lopool contaner, (Mbuu, 24t:: (8n5, 234, 963, 300])
deartgtion teeeuuotoon
Booo an dhe visal stalnisef the inage. hav the skanifictionn of the syndad on the fire rnd hox froe the upp the agat: The tmage dhonsc exds or deppinng contaians on the riglt stde stde The Thociniine el the very kis of sluck rod uppar left comerr the contener
is gybbal syrahal areagge verticllyy 'nt, counted claawerr
Therefore. yorbol
the
cmiii
To bridge this crucial gap, we make two primary contributions. First, we introduce Visuo-Lingual Interleaved Rationale (VLIR), a pioneering dataset meticulously curated to support the development of MLLMs for interleaved text-image CoT reason- ing. VLIR provides explicit annotations for visual region localization, image cropping instructions, and semantie enhancement cues, all embedded within multi-step reasoning narratives. Second, building upon this, we propose VLM-R* (Visual Language Model with Region Recognition and Reasoning), novel framework designed master this intricate reasoning style. VLM-R is trained using distinctive strategy that combines cold- start finetuning on our VLIR dataset with a novel Lagion-Conditionel
Figure 1: This figure visualizes our proposed VLM-R* approach, which integrates region grounding and refinement in an interleaved visual-textual reasoning chain. While conven- tional text-based reasoning fails when analyz- ing scenes that require dynamic, iterative, and fine-grained interaction with specific visual re- gions, our approach succeeds by precisely iden- tifying and focusing on critical visual elements, such as 'the first red box counted from top to bottom on the right' in this example, to de- rive accurate conclusions through targeted vi- sual reasoning.
Reinforcement Policy Opti- mization (R (R-GRPO). This empowers VLM--R* to learn when and where look within an image, how to process the localized visual evidence (e.g.. by cropping requesting enhancement), and how to integrate this dynamically acquired informa- tion into its evolving reasoning chain. Our extensive experiments on diverse multimodal reasoning benchmarks, including MME [144], ScienceQA [5I]] and MathVista [BO], demonstrate that VLM-R* significantly outperforms existing state-of-the-aat models. In summary, our contributions are:
The introduction of VLIR, the first benchmark dataset tailored for training and evaluating MLLMs on interleaved visual-textual CoT reasoning with explicit region-level interactions.
The proposal of VLM-R a novel MILMM framework, and associated R-GRPO training strategy, which enables dynamic visual region localization and evidence integration within the reasoning process.
Comprehensive empirical validation showing that VIM-R3 achieves superior performance on challenging multimodal reasoning tasks, setting a new benchmark for fine-grained, visually-groundn inference.

---

2 Related Work
2.1 Large Language Model Reasoning
evolved substantially with Chain of Thought (CoT) prompting [99. 58, 24, 57 35,, I.l which enables models to break down complex problems into intermediate steps, mimicking human reasoning. This foundational approach has expanded to include diverse structures like grogram-of-thouggee [ table-of toblooos- [099] and tree-of- thoughts [[64]] each offering unique advantages for different reasoning scenarios. Recent advances
Reasoning in Large Language Models [ST, 7 07 [8]
include OpenAl's Ol [37]] which combines reinforcement learning [39,
2EL [6]] with CoT opti-
mize decision-makie
without external guidance, and DeepSeek RI
[12 which employs pure rein-
forcement learning through Group Relative Policy Optimization (GRPO) [!6] to enable autonomous
evolution of reasoning capabilities while incorporating ule-based rewards that significantly improve performance across complex reasoning tasks.
2.2 Multi-modal Large Language Model Reasoning
Multi-modal Large Language Model reasoning research
69
59
54
28] has emerged 21,
focusing on both effective multi-modal chain-of- structures [6]]. 53 51 21 and righ-uuaitty training data construction methods [TE. 97, H]. Mainstream approaches have adapted text-based reasoning paradigms to multi- modal contexts, as seen in Virgo [[S)] which demonstrated that text-only reasoning data can activate certain multi-modal reasoning capabilities, and more structured frameworks like LLaVA-CoT's [6]]] four-stage reasoning process and MM- Verify's [5Il] verificationeenhaneed approach. However, these methods largely inherit reasoning paradigms from text-only models without adequately addressing visual information processing, leading to limitations in Assuall-intensive reasoning tasks.
following the success of text-only reasoning models GV
52
3 Method
We propose a novel framework, VLM-R3 designed to perform visuo- lisuullingual interleaved reasoning with region grounding. This section details the components of our approach, including the con- struction of the Visuo-Lingual Interleaved Rationale (VLIR) dataset used for cold-start supervised fine-tuning, the interactive inference pipeline enabling dynamic visual grounding, and the Region- Conditioned Reinforcement Policy Optimization (R-GRPO) strategy employed to enhance reason- ing capabilities.
3.1 Visuo-Lingusl Interleaved Rationale (VLIR) Dataset
Prior work, such as Visual CoT [45]], introduced the concept of incorporating visual grounding (specifically bounding boxes) into reasoning chains. However, these methods typically suffer from several limitations: (1) They often lack explicit linguistic reasoning steps interleaved with visual actions. (2) The visual grounding actions (e.g.. cropping based on bounding boxes) are predefined or manually specified, rather than being dynamically generated by the model. (3) They are often restricted a limited number of visual interactions, typically a single bounding box selection be- fore providing final answer, lacking the fiexibility for multi-step visual querying. To address these limitations and cultivate the ability for models autonomously and flexibly perform iterative visual retrieval and cropping based on their ongoing reasoning, we introduce the Visuo-Lingual Inter- leaved Rationale (VLIR) dataset. This dataset is specifically curated to provide rich, interleaved sequences of textual reasoning steps interspersed with explicit visual grounding actions and the cor- responding cropped visual evidence.
3.1.1 Data Construction
The construction of the VLIR dataset focuses scenarios that necessitate fine-grained spatial un- derstanding and precise utilization of visual cues. We select data from a diverse set of existing benchmarks to cover wide range of visual reasoning challenges:
Text/Document Understanding: TextVQA [4]], VQA 333 for tasks requiring OCR
and document structure understanding.
3

---

Image token
Query token
Respores token
Meeked loken
<abip-T deterine
camez (aabaa
Large Language Model
Visual Encoder
Whl can IYSTEG...
stadeto coeeet and
Vieual Encoder
sinn C"itee 241
Vieual Encoder
Figure 2: The visualization of the inference pipeline of our proposed methods.
General Visual Question Answering: GQA for complex multistep reasoning over visual scenes.
Chart and Infographic Interpretation: InfographicsVQA 22 for understanding struc- tured visual data.
Spatial Relation Reasoning: VSR [26]] for tasks focused on identifying and reasoning about spatial relationships between objects.
We leverage the advanced capabilities of powerful MILMs, such as Qwen2.5-VL. 72B [8], through sophisticated prompt engineering to generate interleaved image- reasoning chains for data points from benchmarks like GQA and TextVQA, which represent real-world question answering scenarios. We then employ rejection sampling strategy on the generated samples, filtering for those that align with the ground-truth answers.
For tasks where direct prompt engineering on the original image-question pair is less effective, par- ticularly those involving detailed OCR or tabular data interpretation (e.g., data underlying Visual CoT [8SI), we utilize GPT-4o [66]] with tailored prompts that incorporate the metadata provided by the source dataset (e.g., the initial bounding boxes from Visual CoT), This allows us to generate detailed, step-by-step interleaved rationales within these challenging domains.
3.1.2 Data Filtering
To ensure the quality and relevance of the interleaved rationales generated, we apply a rigorous filtering process based on the following criteria:
Semantic Unit Validity of Regions: and semantically meaningful visual unit (e.g.. recognizable object, a block of text, distinct part of a chart). To automate this, we utilize a smaller VLM and prompt it with the cropped image corresponding to the proposed bbox, asking to confirm the presence and identity of a recognizable entity ("Can you identify what is in this image (a specific object or piece of text)? Respond with yes/no."). "). Samples in which the VLM fails to confirm meaningful semantic unit rejected.
Each proposed bounding box must enclose a complete
Logical Coherence and Non-Redunyancy Reasoning: The generated textual reason- ing steps must be logically sound, progressive, and directly contribute to arriving at the final answer, avoiding spurious or redundant text. We employ powerful text-only LLM, such as DeepSeek V3 [2S], through prompt engineering to evaluate the logical flow and relevance of the text rationale preceding each visual interaction and the overall reasoning path. Samples with illogical padded reasoning are rejected.

---

3.2 Interactive Inference Pipeline
The VLM-R* model executes reasoning through an interactive pipeline that enables the model to dynamically select and incorporate visual information during its inference process.
The interaction is initiated by providing the VL.M-R* with an instruction that defines the reasoning task and the available visual interaction tool:
You need to first think about the reasoning process in your mind, and then provide the answer. When thinking, you should call the "crop" tool (format: ["bbox_dd 2d": [x1, y1, x2, y2]}) to focus on the key areas in the image. The reasoning process and the answer are included in the <think> </think> and <answer> </answer> tags respectively.
When the model generates string that matches the specified JSON format, the pipeline intercepts the output. The system parses the coordinates [z1, y1, y2] and performs a cropping operation on the original input image. The resulting cropped image is then zoomed in and encoded into visual tokens and appended to the model's input sequence, effectively providing the model with the requested visual detail as a new context. Following the injection of the cropped image, the model resumes generation, which may involve generating further text or issuing additional "Crop" commands. This interactive loop continues until the model generates the final answer, at which point the process terminates. This pipeline structure shows as Figure D.
3.3 Region-Conditioned Reinforcement Policy Optimization (R-GRPO)
Standard supervised learning on fixed trajectories struggles to optimize the complex state-dependent policy of deciding when and where acquire visual information. Our approach. Region- Conditioned Reinforcement Policy Optimization (R-GRPO), adapts policy optimization frame- work, building upon Group Relative Policy Optimization (GRPO) [+6]. The "Region-Conditoneed" aspect implies that no is explicitly conditioned on the visual state, including dynamically incorpo- rated regional evidence.
To estimate the advantage of each reasoning trajectory, we normalize its reward relative to the group follow:
A' rd mean( (r "r.,,,,..,, rM) std((r) .r?.....,M?? M))
Here, r² is the total reward for the i-th trajectory in group of M trajectories, and Á serves as form of advantage function relative to the group performance.
critical adaptation R-GRPO concerns the computation of the policy gradient and the actions considered in the objective. In our interleaved image-text sequences, the model (textual reasoning. bbox commands), while others (the representations of cropped images) injected by the environment. The policy gradient should only optimize the likelihood of actions generated by the model. Therefore, when calculating the gradient of log To (a, st), we apply a mask: the gradient is computed only for tokens a that text tokens bounding box command tokens, masking out gradients for tokens corresponding to injected image regions. Conceptually, the sum of actions A, in the loss primarily considers the probabilities of generating valid text tokens and bounding box commands, weighted by their advantage. The injected image tokens influence the state $141 but are not actions a for which we compute policy gradient.
tokens are generated by
Following this, we optimize the policy model Ty with the loss function defined as:
Tc(c' To ( Q) no grad
CORPO -EqEDs
A' 3D ( Tret)
where Ds is the dataset of question-state pairs, Q represents a specific question and current visual state, e¹ is the sequence of generated tokens for the i-th trajectory given Q, and B is a coefficient for the KI. divergence term. The first term the sum uses the normalized reward Á to weight the likelihood of the generated sequence, encouraging sequences with higher relative rewards.
5

---

The KL divergence between the policy model and the reference model is estimated as in [+
(Q)) Teer(e" (c) DRI(TellFiat) = (a(ci(Q) Tea (c' (Q)
log
(3)
The total reward r¹ for a trajectory comprises four components that encourage desired VLM-R* behaviors:
Accuracy Reward (Face): A terminal reward: 1 if the final answer is correct, 0 otherwise.
Format Reward (fformat): A terminal reward: 1 if the output uses correct <answer> tags,
0 otherwise.
Region Validity Reward (Tregion): An intermediate reward of 0.5 for each valid, non- redundant bounding box generated (only when the answer is correct), capped at 0.5 per episode.
Reasoning Length Reward (Ntemgeh): An intermediate reward 0.001 per character for reasoning steps, capped 0.25 per episode to prevent verbosity.
The total reward is computed as:
Toverall
Tace + format + Tlength + Tregion I[(race 1.0)
By optimizing this objective, R-GRPO encourages the VLM to learn a policy that not only leads to correct final answers but also involves generating logical textual reasoning and strategically gather- ing the necessary visual evidence.
4 Experiments
4.1 Experiment Setting
We covers several public benchmarks. General visionlanguage understanding is measured MME [[]]]] and MMMU[60]; complex mathematical reasoning on Math Vista [+I]] and Math Vision [53]] scientific question answering ScienceQA [[I]] and document understanding We also assess hallucination rates with HallucinationnBenc([[]] We evaluate our method against three categories of multimodal models. The first category consists of open-source baselines with- out explicit reasoning capability, including Qwen2.5-VL. 7B [8] (also used as our primary baseline), VL2.5-88 [[ and VA-Nett 8B [22]]. The second category comprises closed-source non- reasoning systems, represented by Gemini-2 Flash [M]] and GPT-4o [36]]. The third category contains models equipped with dedicated reasoning modules, namely VA-CoT 1IB [6]]] Mulberry- Qwen2VL 7B [63]], R R1-onevision 7B [62]] To probe the upper bound of performance, we also compare our results with two larger closed-source models ol [GZ]]] Moreover, we also evaluate our model on four vision-centric benchmarks: MMVP [54], V* [[[]], HR-Bende [56]] and MME- Realworld [68] to evaluate the model's capability fine-grained visual understanding and spatial reasoning.
DocQA [33]]
4.2 Dataset Details
Our supervised fine-tuning experiments used the VLIR dataset, which comprises 11,810 samples in total. As shown in figure B. the distribution of crops per image exhibits considerable variation:
11,105 images contain a single crop, 607 images feature two crops, 68 images have three crops, 16 images include four crops, images contain five crops, and 6 images have six or seven crops (3 each). These samples are drawn from five distinct source datasets: GQA (4,057 samples), TextVQA (3,267 samples), DOA (1,497 samples), Infographics VQA (1,497 samples), and VSR (1,492
samples). We categorize the crops based on their relative size, defined as the ratio of the bounding box area to the total image area: "very small" (ratio 0.05) accounts for 5,280 crops; "small" (0.05
< ratio < 0.25) comprises 4,043 crops; "medium" (0.25 & ratio 0.5) includes 1,914 crops; and "large" (ratio 2 0.5) consists of 573 crops.

---

VLIR Dataset Visualiization (11.810 Total Samples)
Distribution of Crops per Image
Distribution by Source Dataset
Distribution of Crop Sizes
044
(LEAM]
tbe soae)
IeforeaphesVA
DeciOn
1009
212
11800 0.B4000S
ratl (382) aintt ( inaio 0.0 Grop Sue Catogaryy
Nu Number Crops
Figure 3: Distribution of the VLIR dataset: (a) number of crops per image, (b) samples across different source datasets, and (c) categorization of crops based on relative size.
Table 1: Performance comparison of various multimodal models across different benchmarks. Our model (in bold) compared against non-reasoning models (both open-source and closed-source) and reasoning-based MILMs. The highest values in each column are highlighted with green back- ground.
Benchmarks
Model
Params
Math MathVision MMMU MME ScienceQA DocVQA HallusionBench
Glosed-Suuree Non-Reasoning MLLMs
Gemini-2 Flash GPT-4o
73.1
41.3
71.7
92.1
63.8
30.4
70.3
2328
66.2
91.1
56.2
Larger Closed Source Models
o1
71.8
63.2
77.6
81.6
Open-Source Source Non- Reasoning MLLMs
V125
58.3
51.8
2210
LexVA-Next
8B
37.5
41.7
1957
72.8
Open- Source Reasoning MLLMS
LLaVA-CoT
11B
54.8
47.8
R1 -1-onevision
7B
64.1
29.9
7B
73.5
2190
49,5
Mulberry
7B
63.1
55.0
54.1
Qwen2.5-VL
7B
68.2
25.1
58.6
2347
73.6
95.7
61.3
Ours
70.4
30.2
62.2
2432
87.9
96.8
62.0
4.3
Main Result
Our VLM-R model, built upon the Qwen2.5-VL. 7B architecture, consistently outperforms its base model across all benchmarks, with particularly significant gains in domains requiring precise visual reasoning and fine-grained understanding. Specifically, we observe a 2.2% improvement on Math- Vista (70.4% vs. 68.2%) and a remarkable 5.1% improvement on Math Vision (30.2% vs. 25.1%), highlighting our method's effectiveness in mathematical reasoning tasks that demand careful at- tention visual details. The substantial performance gain of 14.33% on ScienceQA (87.90% vs. 73.57%) further demonstrates VLM-R3*s superior capability in scientific reasoning, where dynamic grounding of visual evidence is critical. When compared to other open-source cessoning-focuseee models like Vision-R1 and Mulberry, VLM-R3 exhibits competitive performance on Math Vista and surpasses Mulberry on HallusionBBench (62.0% vs. 54.1 1..1% %), indicating enhanced reliability in avoiding visual hallucinations. Our approach also narrows the gap with closed-soure models like Gemini-2 Flash and ol, despite having significantly fewer parameters and being fully transparent in its architecture.

---

Table 2: Performance comparison and ablation study on vision-centio benchmarks. The highest
values in each column are highlighted with green background.
Model
MMVP
V*
HR-4k
HR-8k
MME-Res
Avg.
Open-source General Models
LLaVA-OneVission-221
73.8
66.3
60.9
48.7
VL.3-8B
72.3
70.8
62.0
47.9
Intern VL3-38B
77.5
76.3
67.0
51.0
Qwen2.5VL-7B (Base)
66.7
74.3
69.8
64.6
42.3
63.5
Base + Vanilla Text RL
72.0
78.5
72.9
64.1
46.2
66.7
Base Text-only Bbox
72.4
79.5
73.2
65.2
47.2
67.5
Base + Re-insert Full Image
73.9
77.8
72.5
64.8
45.8
67.0
Ours
75.0
83.8
73.4
66.8
51.6
70.1
A vs. Vanilla Text RL
+3.0
+5.3
+0.5
+2.7
+5.4
+3.4
A vs. Text-only Bbox
+2.6
+4.3
+0.2
+1.6
+4.4
+2.6
A vs. Re-insert Full Image
+1.1
+6.0
+0.9
+2.0
+5.8
+3.1
Table 3: Ablation study on Math Vista, MMMU, ScienceQA, and Doc benchmarks. We evalu- the contribution of each key component: Interleaved Chain-of-Thought, VLIR fine-tuning, and R-GRPO.
Model Variant
MathVista
MMMU
ScienceQA
DocVQA
Avg.
Base Model (Qwen2.5-VL.)
68.2
58.6
73.6
95.7
74.0
w/o Interleaved Chain-of of of----o-TTo- Thought w/o VLIR Fine-tuning
67.1 (13.3)
59.4 (12.8)
75.4 ((12.5)
95.9 (10.9)
74.4 (14.9)
65.8 (14.6)
57.0 (15.2)
72.2 (L15.7)
93.3 (13.5)
72.1 (17.2)
w/o R-GRPO
69.7 (10.7)
60.8 (11.4)
84.6 (13.3)
96.1 (10.7)
77.8 (11.5)
Full VLM-R (Ours)
70.4
62.2
87.9
96.8
79.3
Table 0 presents the performance of our VLM-R* model on vision-centric benchmarks, including MMVP [54], V* [60], HRBench [56], and MME-Realworld [68]. Our model shows superior ca- pability in handling complex visual reasoning tasks. Notably, VLM-R3 attains an average score of 70.1% %, significantly outperforming the base Qwen2.5-VL model (63.5%) and larger open-source
models like LLaVA-One Vision-72B [20] and Intern VL3-38B [71]].
4.4 Ablation Study
To assess the contribution of each component in our VLM-R framework, we conduct comprehen- sive ablation experiments. Table Y and Table Q summarize our findings.
4.4.1 Effectiveness of Interleaved Chain-of-Thoughe
To isolate the impact of our interleaved reasoning approach, we conduct an experiment where we maintain the region localization capabilities (bounding boxes) but remove the associated region im- ages from the reasoning chain. This variant relies solely on textual descriptions of identified regions without visually grounding each reasoning step. As shown in Table 3 removing the interleaved vi- sual evidence leads to consistent performance drop across all benchmarks, with particularly notable decreases ScienceQA (-12.5%) and MMMU (-2.8%). This degradation is most pronounced in tasks requiring fine- fine-grained visual understanding, such as scientific diagrams in ScienceQA, where purely textual descriptions of regions fail to capture crucial visual patterns and spatial relationships.
In Table we further compare this variant against other ablations on vision-centrie benchmarks. The results indicate that the interleaved reasoning chain achieves an average improvement of 3.4% over vanilla text-based reinforcement learning approach. Furthermore, outperforms variants that rely solely on text-based bounding box descriptions (+2.6%) or re-inserting the full image at each reasoning step (+3.1%% 1%). These findings underscore the critical role of dynamically incorporating visual regions into the reasoning process for effective multimodal understanding.

---

Impact of Region Grounding Accuracy on Model Performance
MathVaada
XINMU
SciencesA
Netommee
Region Greanding Accurscy
Begion Crounding Accurncy
Baglon Coozading Accuraac
Figure 4: Impact of region grounding accuracy on model performance across three benchmarks. Each subplot shows the performance trajectory from 40% to 90% grounding accuracy with confi- dence intervals (shaded regions).
4.4.2 Effectiveness of Finetuning on VLIR
Our approach leverages the VLIR corpus to bootstrap the model's ability to identify informative re- gions and incorporate them into coherent reasoning chains. To evaluate the specific contribution of VLIR fine-tuning, we experiment with a variant that skips this initialization phase and proceeds di- rectly to R-GRPO training. The results in Table demonstrate that omitting VLIR fine-tuning leads to performance degradation across all benchmarks, with particularly significant decreases observed in ScienceQA (-15.7%) and MMMU (-5.2%). More critically, observed that ablating VLII capabilities, leading to substantial deficien-
fine-tuning impairs the model's instruction-folloviin cies such failures to adhere to required formatting conventions for bounding box specifications. This accounts for the substantial performance deterioration observed in our experimental results.
4.4.3 Effectiveness of R-GRPO
To assess the impact of our Region-Conditinn Conditioned Reinforcement Policy Optimization (R-GRPO), we evaluate a variant that relies solely on supervised fine-tuning using the VLIR corpus without the subsequent reinforcement learning stage. This allows us to isolate the specific benefits of reinforcement learning approach over purely supervised learning. The experimental results show that removing R-GRPO reduces performance in benchmarks, with the highest decreases observed in ScienceQA (-3.28%) and Math Vista (-0.7%). This suggests that while VLIR fine-tuning provides strong foundation, the reinforcement learning stage is essential for optimizing the model's region selection and reasoning policies beyond what can be achieved through imitation learning alone.
4.5 Discussion
4.5.1
Impact of Region Grounding Accuracy on the Reasoning Chain
The quality of region grounding, represented by the accuracy of bounding boxes (bbox), plays critical role in multimodal reasoning capabilities. Our analysis investigates how varying levels of grounding accuracy impact the performance of the VLM-R model across multiple benchmarks. We systematically evaluated model performance by controlling grounding accuracy from 40% to 90% and measuring outcomes on three key benchmarks: ScienceQA, Math Vistaista, and MMMU. Ground- ing accuracy was manipulated by randomly replacing or perturbing a controlled percentage of bound- ing boxes in the input. As shown in Figure there clear positive correlation between region grounding accuracy and model performance across all three benchmarks. ScienceQA demonstrates the most substantial improvement, with performance increasing from 54.1% at 40% grounding ac- curacy to 87.9% at 90% grounding accuracy. Math Vista shows similar upward trend, rising from 47.9% to 70.4%, while MMMU exhibits more modest but consistent gains from 43.8% 62.2%. These results underscore the fundamental importance of precise region grounding for effective multi- modal reasoning, with higher-level reasoning tasks showing greater sensitivity to grounding quality.

---

Interleaved Reasoning Chain with Visual Region Localization
To identify the blue thing behind the man, I need to focus the area behiidd him. The blue plastic tablecloth is visible on the tablee front of him, need the blue object.
After cropping. can see that the blue ubject umbralla. closed and hanging above the The umbbrella his some white but the not elearly logible in this view The inower is am umbrella
What the thing behind the sitting the table with blue
will crop zoom the behind man to get clearer view of what is blue.
plastic tablecloth""
("bioxx [301, 390, 206)
VLM-R
General Textual Reasoning Chain
To identify the blue thing behind the man. I'll look the objects situated imagee There appears
bohind
veriial blue
thing behind the sitting the with plastic tableclath?*
shipe that somew resembles sign or boand, Since it's standing upright and have café elements visible (like chairs and tables), makes sense to interpoet this object menu board displaying infonmation for customers, The answer
and because there other restourant
User
Goera
Figure 5: Comparison of attention distribution patterns between the interleaved reasoning chain with visual region localization (top) and general textual reasoning chain (bottom).
4.5.2 Why the Interleaved Reasoning Chain with Visual Region Localization Effective?
To understand the efficacy of our VLM-R² approach, we conducted comparative analysis between the interleaved reasoning chain with visual region localization and traditional textual reasoning chains. Figure B visualizes the attention distribution patterss for both approaches when answer- ing the same visual query. Our analysis reveals critical insight: in traditional approaches where the image is positioned at the beginning of the sequence, attention to visual information diminishes significantly
the reasoning chain progresses. As shown in the lower portion of Figure 4, gen- eral MLMs tend to make incorrect inferences (identifying "menu board" instead of an umbrella) as they lose visual context during extended reasoning. In contrast, VLM-R" maintains persistent visual attention throughout the reasoning process by dynamically localizing and incorporating rele- vant visual regions. The attention heatmap demonstrates that tokens generated later in the reasoning process maintain strong attention connections to the cropped visual regions. This region- specific attention enables the model correctly identify the blue object as an umbrella by explicitly fo- cusing on the area behind the person, cropping it for detailed examination, and making accurate observations about its features.
Conclusion
This paper introduced VLM-R3 novel framework enabling MLLMs to perform dynamic visual reasoning through region recognition, reasoning, and refinement. By integrating our custom VLIR dataset and Region-Conditioned Reinforeement Policy Optimization (R-GRPO), demonstrated that interleaved visual-textual chains-of-thouge significantly outperform traditional approaches. VLM-R3 achieves state-of-the-ard results across multiple benchmarks, particularly excelling in tasks requiring fine-grad ane-grained spatial reasoning and visual evidence integration. Our work opens promising directions for developing more sophisticated risually-grundde reasoning systems that can adap- tively focus on relevant regions during multi-step inference processes.
10

---

Acknowledgement
This work supported by the National Natural Science Foundation of China (NO.623B2007), Chi- nese Institute of Electronics (CIE) Tencent Doctoral Research Incentive Program (Hunyuan Large Language Model Special Track) (N0.841010445) and CCF-Zhipu Large Model Innovation Fund
(NO.CCF-2hipu2024155)
References
[1] Gemini
most http:///ppna..ccommmmmc////deeet [2] Introducing openai 03 and o4-mini, 2024.
2.5:
Our
intelligent ai
model,
2024.
mini/.
[3] Qvq: To see the world with wisdom, 2024. http::/wwweeiiniggbhbbbbbbb/g/yy
[4] Seekworld: Geolocation natural rl task for o3-like visual clue-tracking reasoning, 2025.
[5] lean-Baptise Alayrac, Jeff Donahue, Pauline Luc, et al. Flamingo: visual language model for few-shot learning. In NeurIPS, volume 35, 2022.
[6] Gemini Team Google Rohan Anil, Sebastian Borgeaud, Yonghui Wu, can--Bptiste Alayrac, Jiahui Yu, Radu Soricut, and Johan Schalkwyk... Gemini: A family of highly capable multi- modal models. ArXiv, abs/2312. 18805 2023.
[7] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: frontier large vision-language model with versatile abilities. ArXiv, abs/2308.12966 2023.
[8] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jian- qiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. ArXiv, abs/2502. 13923, 2025.
[9] Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. MCcot: A novel benchmark for multi-domain multi-step multi-modal chain-of-thought 2024.
[10] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Trans. Mach. Learn. Res., 2023, 2022.
[11] Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuchui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui Hc. Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong Hc, Zhiy- ong Wu, Huipeng Deng. Jiaye Ge, Kai Chen, Kaipeng Zhang, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling, 2025.
[12] DeepSeck-Ar AI, Daya Guo, Dejian Yang. Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiaoling Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bing-Li Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dong-Li Li Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingehang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Jiong Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean
11

---

Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, M. Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. J. Chen, Ruiqi Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shao-Kang Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wen-Xia Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang. Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li. Xiangyu Jin, Xi-Cheng Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song. Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yi Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yu-Jing Zou, Yujia He, Yunfan Xiong, Yu-Wei Luo. Yu mei You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang. Yao Li, Yi Zheng, Yuchen Zhu, Yunxiang Ma. Ying Tang. Yukun Zha, Yuting Yan, Zehui Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhen guo Zhang, Zhewen Hao, Zhicheng Ma. Zhigang Yan, Zhiyu Wu, Zihui Gu. Zijia Zhu, Zijun Liu, Zi-An Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in ilms via reinforcement learning, ArXiv, abs/2501. (ttp/2501112448,, 2025.
[13] Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji- Ji-Rong Wen. Virgo: A preliminary exploration on repro- ducing ol-like mllm, 2025.
[14) Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 13394, 2023.
[15] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang. Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusion- bench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models, 2024.
[16] Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262, 2025.
[17] Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real- world visual rea- soning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700-6709, 2019.
[18] Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprini arXiv:2303.0553 0333,8 2023.
[19] Ziqi Jin and Wei Lu. Tab-cot: Zero-shot tabular chain of thought. In Annual Meeting of the Association for Computational Linguistics, 2023.
[20] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408. 03326, 2024.
[21] Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vuli, and Furu Wei. Imagine while reasoning in space: Multimodal visualization of of-uthtugt,, 2025.
[22] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. nata--nex-inttereaww Tackling multi-image, video, and 3d in large multimodal models, 2024.
[23] Junnan Li, Dongxu Li, Silvio Savarese, et al. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In ICML, 2023.
[24] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step, 2023.
12

---

[25] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, al. Deepseek-vä v3 technical report. arXiv preprini 0nn.24422.194377 2024.
[26] Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning, 2023.
[27] Haotian Liu, Chunyuan Li, Yuheng Li, et al. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023.
[28] Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement, 2025.
[29] Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong. Yuhang Cao,, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning, 2025.
[30] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical rea- soning of foundation models in visual contexts, 2024.
[31] Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. 2022.
[32] Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V Jawahar. Infographicvqa, 2021.
[33] Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. Doevqa: A dataset for vqa document images, 2021.
[34] Minheng Ni, Yutao Fan, Lei Zhang, and Wangmeng Zuo. Visual-ol: Understanding am- biguous instructions via moltil multi-turn chain-of of-thoughts reasoning. arXiv preprint arXiv:2410.03321, 2024.
[35] Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang. Huazhong Yang, and Yu Wang. Skeleton- of-thhuuht: Prompting llms for efficient parallel generation. arXiv preprint arX1v:2307.15337, 2023.
[36] OpenAl, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mdry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis, Alexis Conneau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein, Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, An- drew Tulloch, Andrey Mishchenko. Angela Back, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang. Ben- jamin Zweig, Beth Hoover, Blake Samic, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hud- son, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czamecki, Colin Jarvis, Colin Wei, Con- stantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin, Daniel Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dim- itris Tsipras, Doug Li, Due Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong, Ehsan Asdar, Elizabeth Prochl, Elizabeth Yang, Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Eugene Brevdo, Evan Mays, Farzad Khorasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Ge- off Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Han- nah Wong, Haoyu Wang, Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner,
13

---

Henrique Ponde de Oliveira Pinto, Hongyu Ren, Huiwen Chang, Hyung Won Chung, lan Kivlichan, Ian O'Connell, Ian O' O'Connell, Ian Osband, Ian Silber, lan Sohl, Ibrahim Okuyueu, Ikai Lan, Ilya Kostrikov, Ilya Sutskever, Ingmar Kanitscheider, Ishaan Gulrajani, Jacob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shich, Ji Lin, Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beutler, Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder, Joshua Achiam, Joy Jiao, Joyce Lee, Jun- tang Zhuang, Justyn Harriman, Kai Fricke. Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Lemberg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Workman, Leher Pathak, Leo Chen, Li Jing, Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang, Louis Feuvrier, Lu Zhang, Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang, Marwan Aljubeh, Mateusz Litwin, Matthew Zeng. Max Johnson, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang, Mengchao Zhong, Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang, Michelle Fradin, Michelle Pokrass, Miguel Cas- tro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang, Minal Khan, Mira Murati, Mo Bavarian, Molly Lin, Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Staudacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury. Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige, Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofir Nachum, Ola Okelola, Oleg Boïko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moore, Patrick Chao, Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng. Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora. Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang, Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miserendino, Sandhini Agar- wal, Sara Culver, Scott Ethersmith. Scott Gray. Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jomoto, Shirong Wu, Shuaiqi, Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Bal- aji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Pat- wardhan, Thomas Cunninghman, Thomas Degry. Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng. Todd Underwood, Todor Markov, Toki Sherbakov. Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters. Tyna Eloundou, Valerie Qi. Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko, Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojeiech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang, Yuchen He, Yuchen Zhang, Yujia Jin, Yunxing Dai, and Yury Malkov. Gpt-4o system card, 2024.
[37] OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Alli- son Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Bar- ret Zoph. Behrooz Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dra- gos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong, Elizabeth Proehl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo
14

---

Raso, Florencia Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guil- laume Leclerc, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman. Hyung Won Chung, Ian Kivlichan, Ian O' Conneell, Ian Osband, Ignasi Clavera Gilaberte, Ilge Akkaya, Ilya Kostrikov, Ilya Sutskever, Irina Kofman, Jakub Pa- chocki, James Lennon, Jason Wei, Jean Harb, Jerry Twore. Jiacheng Feng. Jiahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joaquin Quiñonero Candela, Joe Palermo, Joel Parish, Johannes Hei- decke, John Hallman, John Rizzo, Jonathan Gordon, Jonathan Uesato, Jonathan Ward, Joost Huizinga, Julie Wang. Kai Chen, Kai Xiao, Karan Singhal, Karina Nguyen, Karl Cobbe, Katy Shi, Kayla Wood, Kendra Rimbach, Keren Gu-Lemberg, Kevin Liu, Kevin Lu, Kevin Stone, Kevin Yu, Lama Ahmad, Lauren Yang, Leo Liu, Leon Maksin, Leyton Ho, Liam Fedus, Lilian Weng, Lindee Li, Lindsay McCallum, Lindsey Held, Lorenz Kuhn, Lukas Kondraciuk, Lukasz Kaiser, Luke Metz, Madelaine Boyd, Maja Trebacz, Manas Joglekar, Mark Chen, Marko Tin- tor, Mason Meyer, Matt Jones, Matt Kaufer, Max Schwarzer, Meghan Shah, Mehmet Yat- baz, Melody Y. Guan, Mengyuan Xu, Mengyuan Yan, Mia Glaese, Mianna Chen, Michael Lampe, Michael Malek, Michele Wang, Michelle Fradin, Mike McClay, Mikhail Pavlov, Miles Wang, Mingxuan Wang, Mira Murati, Mo Bavarian, Mostafa Rohaninejad, Nat McAleese, Neil Chowdhury, Neil Chowdhury, Nick Ryder, Nikolas Tezak, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Patrick Chao, Paul Ashbourne, Pavel Izmailov, Peter Zhokhov, Rachel Dias, Rahul Arora, Randall Lin, Rapha Gontijo Lopes, Raz Gaon, Reah Mi- yara, Reimar Leike, Renny Hwang. Rhythm Garg. Robin Brown, Roshan James, Rui Shu, Ryan Cheu, Ryan Greene, Saachi Jain, Sam Altman, Sam Toizer, Sam Toyer, Samuel Miserendino, Sandhini Agarwal, Santiago Hernandez, Sasha Baker, Scott McKinney, Scottie Yan, Shengjia Zhao, Shengli Hu, Shibani Santurkar, Shraman Ray Chaudhuri, Shuyuan Zhang, Siyuan Fu, Spencer Papay, Steph Lin, Suchir Balaji, Suvansh Sanjeev, Szymon Sidor, Tal Broda, Aidan Clark, Tao Wang. Taylor Gordon, Ted Sanders, Tejal Patwardhan, Thibault Sottiaux, Thomas Degry, Thomas Dimson, Tianhao Zheng, Timur Garipov, Tom Stasi, Trapit Bansal, Trevor Creech, Troy Peterson, Tyna Eloundou, Valerie Qi. Vineet Kosaraju, Vinnie Monaco, Vitchyr Pong, Vlad Fomenko, Weiyi Zheng, Wenda Zhou, Wes McCabe, Wojeiech Zaremba, Yann Dubois, Yinghai Lu, Yining Chen, Young Cha, Yu Bai, Yuchen He, Yuchen Zhang, Yunyun Wang, Zheng Shao, and Zhuohan Li. Openai ol system card, 2024.
[38]
OpenAl. Gpt-4v(ision) system card. https://cch. Card. pdf. 2023.
wwt.ccommpppp.. com/papers/GPTV System
[39] Long Ouyang, Jeff WU, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022.
[40] Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based rl, 2025.
[41] Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, and Mao Yang. Mutual reasoning makes smaller llms stronger problem-solvers arXiv preprint 20Xi:2400.066195 2024.
[42] Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Pro- ceedings of the 26th ACM SIGKDD international conference on knowledge discovery data mining, pages
2020.
[43] Daniel Rose, Vaishnavi Himakunthala, Andy Ouyang, Ryan He, Alex Mei, Yujie Lu, Michael Saxon, Chinmay Sonar, Diba Mirza, and William Yang Wang. Visual chain of thought: bridg- ing logical gaps with multimodal infillings. arXiv preprint arXiv:2305.02311 2023.
[44] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint ar arXiv:1007006347, 2017.
15

---

[45] Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. 2024.
[46] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024.
[47] Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang. Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-rl: stable and generalizable r1 r1-style large vision-language model, 2025.
[48] Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317-8326, 2019.
[49] Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time com- pute optimally can be more effective than scaling model parameters. ArXiv, abs/2408.03314 2024.
[50] Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, et al. A survey of reasoning with foundation models, arXiv preprint arXiv:2312. /566, 2023.
[51] Linzhuang Sun, Hao Liang, Jingxuan Wei, Bihui Yu, Tianpeng Li, Fan Yang, Zenan Zhou, and Wentao Zhang. Mm-verify: Enhancing multimodal reasoning with chain-of ohotho verifica- tion, 2025.
[52] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501. 12599, 2025.
[53] Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ah- san, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, Hisham Cholakkal, Ivan Laptev, Mubarak Shah, Fahad Shahbaz Khan, and Salman Khan. Llamav-ol; Rethinking step-by-step visual reasoning in 1lms, 2025.
[54] Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal Ilms, 2024.
[55] Ke Wang. Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset, 2024.
[56] Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. arXiv preprint, 2024.
[57] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in lan- guage models. arXiv preprint 1rXiv:2203.11177, 2022.
[58] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023.
[59] Jinyang Wu, Mingkuan Feng, Shuai Zhang, Ruihan Jin, Feihu Che, Zengqi Wen, and Jianhua Tao. Boosting multimodal reasoning with mcts-automated structured thinking. arXiv preprint arXiv:2502.02339, 2025.
[60] Penghao Wu and Saining Xie. V Guided visual search core mechanism in multimodal Imms. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recogni- tion, pages 1 13084-13094 2024.
16

---

[61] Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025.
[62] Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. RI Rl-onevision: Advanc- ing generalized multimodal reasoning through cross moosl formalization, 2025.
[63] Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. Mulberry: Empowering mllm with ol-like reasoning and reflection via collective monte carlo tree search, 2024.
[64] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. ArXiv, 10000, 2023.
[65] Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang. Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv: 14178, 2023.
[66] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi, 2024.
[67] Eric Zelikman, Yuhuai Wu, Jesse Mu. and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476-15488, 2022.
[68] Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, al. rme-realworld: Could your multi- modal Ilm challenge high-resolution real-world scenarios that are difficult for humans? arXiv preprint arXiv:2408. 13257, 2024.
[69] Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Mul- timodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923. 2023.
[70] Fan Zhou, Haoyu Dong, Qian Liu, Zhoujun Cheng, Shi Han, and Dongmei Zhang. Reflec- tion of thought: Inversely eliciting numerical reasoning in language models via solving linear systems. arXiv preprint arXiv:2210.05073 2022.
[71] Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint urXivv25044 10479, 2025.
17

---

NeurIPS Paper Checklist
The checklist designed to encourage best practices for responsible machine learning research, ad- dressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (12 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evalu- ation. While "[Yes] " generally preferable to "[No] is perfectly acceptable answer "[No] provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased binary way. acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] question, in the justification please point to the section(s) where related material for the question can be found.
used"). In
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Check- list".
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
18

---

Justification: [NA]
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on a few datasets or with few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations their approach ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data provided not)?
Answer: [Yes]
Justification: [NA]
Guidelines:
19

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibilityy which may depend on the nature of the contribution. For example
(a) the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-souree dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors welcome deseribe the partieular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so No is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:)
/ /maps..c/public/guudee Sooeedeuee//oSSSummiioa issiisssinniii Pnolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
20

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer:
[Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
21

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
22

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines or restrictions to access the model or implementing safery filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license. copyright information, and terms of use in the pack- age should provided. For popular datasets, caperswithcode com/datasen has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
23

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, cura- tion, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable). such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy /etees/rneerrpppcccc/oooferereee for what should or should not be described.
24

---

A Experiment Settings
A.1 Pipline Settings
A.1.1 Model Hyperparameter Settings
Our base model is Qwen2.5VL-77II)) which supports dynamic resolution for imput images. In all experiments, we constrained the pixel dimensions of each image to a minimum of 3136 pixels and maximum of 1605632 pixels. Because the value of the bounding box is related to the number of pixels in the input image, the setting of the range pixels needs to be unified.
A.1.2 Zoom Scaling Rule
In our pipeline, when a region selected for closer inspection (e.g., via a "Crop" operation), a zoom operation is applied. The scaling factor for this zoom, denoted as scale, is determined dynamically based on the relative area the selected bounding box (Anoox) compared to the area the original image (Aurig) Letr Auie
Amea this area ratio. The scale calculated using the following piecewise
function:
2.0,
if < 0.125
scale
1.0,
i 2 0.5
(5)
0.125
2.0
otherwise
0.375
This rule implies that smaller selected regions (smaller r)) are scaled up more significantly (up to factor of 2.0), while larger regions (larger r) are scaled up less, not at all if they already occupy a substantial portion of the original image. The intermediate case provides a linear interpolation of the scaling factor.
A.2 Training Setting for Supervised Fine-tuning Stage
In the supervised fine-tuning stage, we used the complete VLIR dataset. Our experiments were con- ducted on NVIDIA A100 GPUs, each equipped with 80GB of memory, leveraging DeepSpeed[42] for efficient training. We used batch size of 2 with gradient accumulation of 8, a learning rate of 2 x 10-7, and trained for 3 epochs. During this phase, the vision encoder and MLP projector were frozen, and only the Large Language Model (LLM) component was trained.
A.3 Training Setting for R-GRPO Stage
For the R-GRPO stage, we sampled approximately 5,000 data points from TextVQA [ GQA
[TZ]], VSR [26]] Doc [+ and C [9]] datasets. Regarding the hyperparameters for the GRPO formulation()), we set M 5. Following the experience of related studies, we set 0.0,
ic., we eliminate the KL divergence constraint.
Our experiments for R-GRPO were performed on NVIDIA A100 GPUs, each with 80GB of memory, also utilizing DeepSpeed(E2). The batch size per device was set to 1, with a gradient accumulation of 16. The learning rate was 10-6, and training continued for 300 steps. We employ rule-based reinforcement learning approach, where the correctness of the final answer was judged using an exact match criterion. Similar to the supervised fine-tuning stage, the vision encoder and MLP projector were frozen, and only the LLM component trained.
B Prompt Templates for VLIR Dataset Construction and Filtering
B.1 Data Construction Prompts
Given an fimage, question, answer) triplet, the following prompt was used to construct the inter- leaved visual-linguistic chain of thought:
You performing "Multimodal Interleaved Reasoning". During the thinking
25

---

process, you need to keep eye the visual cues in the original image, the queation, and the "Crop" zoom for detailed amalysis.
find regions of the image that help tool crop
When using the tool, you must output JSON object in the following format: ("bbox_2d") ; [x1, yl, x2, y2]) Ensure that you "Crop" at least
Continue thinking after each operation until you reach the final
Output the thinking process within pair of <think> </think> tage and then output the final anawer within pair of <anawer> </answer> tags. (question)
Listing 1: Prompt for dataset construction.
Given (image, question, answer, bounding box annotation] quadruplet, the following prompt was used:
will now provide you with an image, a question, and a "Crop" operation string. Your task is to write the reasoning process used to answer the question instructed. During the reasoning process, the respondent utilizes "Crop" operation assist with reasoning. The format of the operation is as follows:
"'bbox_dd". [x1, y1, x2, y2]}
This bounding box indicates the key region that needs to be focused on correctly
the question.
You must think step by step from the perspective the respondent, using the "Crop" operation at appropriate moments in your reasoning process to eventually reach the correct answer. Important notes:
1. You must not modify the content format of the "Crop" operation in any way.
2. real setting, the respondent only has access the image and the question. This bounding box indicates the area where the correct answer information is located. In this task, they are provided to ensure the correctness of your reasoning process. When writing the reasoning. pretend you are the respondent who independently identifies when to use the "Crop" operation and how to reach the answer step by step.
3. Make sure the reasoning is fluent, logical, and concise.
4. Format of the reasoning process: <think>.. w/thinnncansserr.... </ansver)>
Here an example:
Question: Are there any black numbers or letters? "Crop" operation: {"bbox_2d": _d": [247, 384, 307, 444))
Reasoning: <think>
Step To determine if there are black numbers or letters, need to dog is wearing a heart-shaped
focus on the text visible the image. tag that has some text on it. I will crop and zoom in on the tag for a closer look at the text details. {"bbox_2d") 2d": [247, 384, 307, 444]] Step 2: After cropping, I 13" are also in red, There are no black numbers
that the letters "G PLUS" are in red,
and the numbers or letters on the tag. Review the rest of the image, there are no black numbers or letters either. either.</thinl </think> <ansver>no</ansuer>
Question centtion:qquestie (question) "Crop" perration((crop))
Now Dutput the reasoning process:
Listing 2: Prompt for dataset construction.
B.2
Data Filtering Prompts
The prompt for assessing the recognizability of the cropped images is as follows:
26

---

You need to determine whether the content in picture is complete and semantically meaningful visual unit. Please look carefully this cropped image and determine whether contains a recognizable object, block of text, or specific part of a diagram. If it is recognizable, answer 'yes'; if not, answer 'no'.
Now output "yes' 'no' directly.
Listing 3: Prompt for assessing cropped image recognizability.
The prompt for assessing the quality of the reasoning process is follows:
You need make in-depth assessment of this reasoning process. First, determine whether its logic rigorous and whether each step of reasoning leads naturally and smoothly to the next; second, check whether the reasoning process progresses gradually towards arriving at the final answer; and lastly, check whether there is any false information repetitive redundancy in the text that relevant to the reasoning. If this textual reasoning meets the requirements in terms logic, advancement and content streamlining, output 'yes'; whenever one of these is not met, output 'no'. (question)
(ground-truth answer} (reasoning process}
Now output 'yes' or 'no' directly.
Listing Prompt for assessing reasoning process quality.
C More Demonstrations
This section provides qualitative examples illustrating the capabilities of VLM-R
What the total volume the mensuring cup?
What measuring cup?
To determine the tutal vulume meusuring need focus on the murkings on the side the cup. The markings indicate volume milliliters (ml) and grams (g). The maximum volume is clcarly marked at top cup.
<think- I'll apply crup tool to isolate the key of the ('bhox 2dth [208, 641, 482, 983]]
The label clearly states "Chablis Cru" and "William which are French wine appellations. Additionally, the phrase "Produit de France" is prominently displayed the label. This information strongly indicates that the wine from France. The viatage year, 2010, does not affeet country of ohiik Camner--Fance-/aaee
Step I: Use the Crop tnol to zoom in on the markings measuring cup to ensure accuracy.
TAELIS PREMUER
("bbox 2d": (616, 212, 1021, 599]]
WILLIAM PEVIE
Step Analyze markings. maximum volume is indicated by the highest number 1000 mi.
2010
scale, which
</thinl>>
<answer= 1 1000 mi-s/nnwwrr>>
Figure 6: This figure presents how VLM-R captures details in natural image and reasons about them.
27

---

To whom the letter was sent
The name of the professor from India?
<think= First, I'll apply the crop tool focus the key areas of the image: ("bbox_ 2d":: [119, 500, 320, 600))
<think-First, I'll apply the crop tool to fncus on the key areas image related professor India: bbbox 2d": 1694, 1085, 958, 1367];
The clearly stated in the text under the D. Chaudhri India." There ambiguity
Melson Enterprises
440 Park Avenue So
Endia
heading
NNw York, New York
Profeoss Tecti's Jnnedarte Neteut
spe wilh prohian of poverty and
text,
The letter addressed to "Mrs. Berman" Melson Enterprises. The salutation "Dear Mrs. Berman:" the beginning of the letter confirms that the letter is addressed
and the name is directly provided. </think> <answer=Dr. Nag
and indutrry ornteand "Copprs
bes thour wit pean
Chaudhristanswer>
<answer-Mes. Bermon-Sanswer>
Figure 7: This figure presents how VLM-R3 gets the text details from the document image.
Is there green cabinet or chair?
<think-To determine chair areas in the image. Step 1: Examine the first cabinet area. ("bbox_ 2d": [4, 263, 117, 364]]
green cabinet chair, need analyze potential cabinet
There green cabinet. Step 2: Examine second eabinct area. ("bbox_ 2d": [138, 269, 276, 364]3
eabinet also green. Examine chair ["bbox_ 2d" (444, 217, 503, 3241
The cropped image shows a chair that is not green. Final Step: Based the analysis, there green cabinct but green chhirs.>/hinkk- Sanswarpyescaanseee
Figure 8: This figure presents how VLM-R iteratively tracks multiple visual cues in image.
28

---

What old comic book is this?
<think= determine the old comic book, noed to identify key elements such title, churucters, and style.
Step Zoom in on the title text for better clarity. ("bbox_ [377, 553, 126))
COOL TOWAY YOUNE GOT TO TELL WE TRUTH?
The cropped
shows the dialogue but the tidle. Let's try another approach. Step2:
main charncter ("bbox_ 2d" (859, 489, 688, 617]]
analysis.
The character distinetive appearance with hat and unique face. This style is reminiscent of older comics.
Mystic," page numher, and style, this comic appears to be old issue of "Mr. Mystic." Mystic."s/hhnkk "</think> <answer> This is an old comic book titled Mr. Mlyttic./answeer
Figure 9: This figure presents how VLM-R performs complex interleaved text-image CoT reason- ing.
What can be inferred from objects on the Choose from the fullowing uptions:
A. The computer owner is game player
computer
editor
C. The computer owner has diverse repertoire of compater
knowledge
The computer forgetful
<think-To determine correct inference about the computer owner. need analyze the objects on the desk. The could be used for different types of work or activities. Let's focus on the key areas of the image to crop
of multiple monitors, laptops, and various peripherals suggests setup that
identify any specific software applications that might indicate the owner's profession or activities.
gather specific
laptops
("bbox 2d": [17, 131, 156, 255]] ("bbox 2d" [168, 139, 262, 222])
The cropped images show two laptops. The first laptop has a blue with which could indicate boot-up sereen error message. The second lnptop has blank screen, which could mean not use or a sleep mode. Now, let's look the monitors. ("bbox 2d (336, 462, 186])
The first monitor is turned off, and the second monitor displaying desktop vurious open applications, which could indicate the owner engaged in multitasking working multiple projects simultaneously. Based on the unalysis, the owner seems to have diverse various
setup that could general computing. However, there is no specific evidence to suggest that the owner game player; editor, or forgetful. setup more indicative of somenne who has broad range computer knowledge and is likely to be engaged in multiple activities--(think Celanswer
programming. editing,
Figure 10: This figure presents how VLMMR performs complex interleaved text-image CoT reason- ing.
29


Reviews:
[{"review_id": "HpXSMtHNZC", "reviewer": "Reviewer_vKQN", "summary": "The paper introduces VLM-R3, a framework that enhances multimodal large language models with dynamic visual reasoning via region recognition and interleaved chain-of-thought. It leverages a curated VLIR dataset and a novel R-GRPO training strategy to enable adaptive region selection and integration during reasoning. VLM-R3 achieves state-of-the-art performance on benchmarks like MathVista and ScienceQA, especially in tasks requiring fine-grained visual grounding.", "questions": "One potential concern is that the proposed VLIR dataset is custom-built to favor region-level visual reasoning, which may amplify the advantages of VLM-R3. This raises the question of whether the observed gains would generalize to more natural or unbiased datasets with a broader range of reasoning types. Also, in real-world data, how many cases actually require such a technique is questionable. If the authors could evaluate on publicly available benchmarks without custom selection or filtering, the results would be more convincing.", "limitations": "A potential limitation is the scalability of the proposed approach. While the VLIR dataset is partially auto-generated, it still requires careful construction, prompt engineering, and task-specific filtering to ensure high-quality interleaved reasoning chains with region-level supervision. Extending this method to new domains may require significant adaptation effort, which could limit its practicality in low-resource or rapidly shifting scenarios.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros:\n\n1. The writing is clear and easy to follow. The ideas are well explained and the structure makes sense.\n\n2. The motivation is solid and the design is thoughtful. It tackles a real limitation in existing models.\n\n3. The model performs well on key benchmarks, showing both practical value and strong impact.\n\nCons:\n1. The model’s success seems to hinge on domain-specific training using the carefully constructed VLIR dataset. Without it, the model fails to follow the correct formats or issue crop commands properly, even performing worse than the base Qwen model. This makes the method costly to adapt to new tasks and may limit its broader applicability.", "was_revised": true, "final_justification": "I have read the rebuttal and discussion carefully. The authors provided additional experiments showing adaptability to other datasets and scalability of the data generation pipeline, which addresses part of my earlier concern. However, the real-world necessity remains unclear, as no quantitative evidence was given on how often region-level reasoning is required in natural, unbiased settings. Given this, I will keep my original score.", "rebuttal": "Thank you for your exceptionally thorough and positive review of our work. We are very encouraged by your praise for the paper's clarity, motivation, and strong performance. Your critical feedback regarding the model's dependency on the VLIR dataset and its scalability is insightful and has prompted us to clarify our contributions and provide additional evidence.\n\nWe address your concerns in detail below.\n\n---\n\n### **Q1: Over-reliance on the VLIR Dataset and Cost of Adaptation**\n\n**Reviewer's Comment:** *The model’s success seems to hinge on domain-specific training using the carefully constructed VLIR dataset. Without it, the model fails to follow the correct formats or issue crop commands properly, even performing worse than the base Qwen model. This makes the method costly to adapt to new tasks and may limit its broader applicability.*\n\n**Our Response:**\nThank you for this thoughtful analysis. We believe your observation that the model performs \"worse than the base Qwen model\" stems from the \"w/o VLIR Fine-tuning\" entry in our ablation study (Table 2 in the original manuscript). We apologize for the potential ambiguity of this entry and would like to clarify its meaning and implications.\n\nThis ablation does not remove the VLIR dataset entirely, but rather **skips the initial supervised fine-tuning (SFT) stage** before applying our R-GRPO reinforcement learning. The resulting performance drop is not due to the absence of VLIR's specific knowledge, but rather due to the **instability of the RL process when applied to a \"cold\" model.** Without an SFT warm-up, the model has no prior exposure to the pattern of generating {\"bbox_2d\": [x1, y1, x2, y2]}, inserting the cropped image, and then continuing the reasoning process, leading to erratic policy exploration and unstable training, which ultimately harms performance.\n\nThe primary role of the SFT phase is therefore to **bootstrap the model**, teaching it the fundamental mechanics of how to perform region-based actions. To demonstrate that this bootstrapping is not exclusively dependent on our VLIR dataset, we conducted a new experiment. We replaced VLIR in the SFT stage with a different interleaved dataset, **M3CoT** [1], which contains varied text-image sequences but lacks VLIR's explicit region-to-rationale linkage.\n\n**Table 1: Impact of Different SFT Datasets on VLM-R³ Performance**\n\n| SFT Dataset | Commonsense | Science | Mathematics | Average |\n| :--- | :---: | :---: | :---: | :---: |\n| Qwen2.5VL-7B (Base Model) | 81.3 | 66.5 | 49.8 | 67.4 |\n| VLM-R³ (SFT with **M3CoT**) | 82.4 | 67.7 | 51.0 | 68.9 |\n| **VLM-R³ (SFT with VLIR - Ours)** | **84.1** | **69.2** | **52.5** | **70.4** |\n\n**Analysis:**\nThis experiment yields two key insights:\n1.  Bootstrapping with an alternative dataset (M3COT) still allows our R-GRPO framework to **outperform the base model** (+1.5% on average). This confirms that our method is not brittle and its success does not hinge solely on VLIR.\n2.  Using our purpose-built VLIR dataset for SFT yields **superior results** (+3.0% on average), validating its high quality and effectiveness in teaching the intended skills.\n\nIn summary, the SFT stage is a necessary \"warm-up\" for stable RL, and while VLIR is the optimal fuel for it, our framework's core strength lies in the R-GRPO policy, which is adaptable to other data sources.\n\n[1]  Chen, Qiguang, et al. \"M $^ 3$ CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought.\" arXiv preprint arXiv:2405.16473 (2024).\n\n---\n\n### **Q2: Limitations on Scalability and Generalization**\n\n**Reviewer's Comment:** *A potential limitation is the scalability... Extending this method to new domains may require significant adaptation effort... which could limit its practicality in low-resource or rapidly shifting scenarios.*\n\n**Our Response:**\nThis is a critical point regarding the practical utility of our approach. We address it from two angles: the generalization capability of our final model and the low-cost scalability of our data generation method.\n\n**1. Strong Zero-Shot Generalization to New Domains:**\nA key goal of our work was to teach the model a *generalizable skill* of dynamic visual analysis, not just to overfit to the domains within VLIR. To test this, we evaluated our single VLM-R³ model, trained on VLIR, on challenging benchmarks from domains it had **never seen during training**, such as video frames and real-world scenes.\n\n**Table 4: Zero-shot Generalization of VLM-R³ to Unseen Vision-Centric Domains**\n\n| Model | MMVP (Video) | MME-RealWorld (Real World) |\n| :--- | :---: | :---: |\n| Qwen2.5VL-7B (Base) | 66.7 | 42.3 |\n| **VLM-R³ (Ours)** | **75.0** | **51.6** |\n\nThe results clearly demonstrate that our model possesses strong generalization capabilities. The significant performance gains on completely unseen domains indicate that VLM-R³ learns a robust and transferable skill of \"how to look,\" which is not narrowly confined to its training data.\n\n**2. Low-Cost, Scalable Data Generation Pipeline:**\nRegarding the effort to adapt to new domains, we want to highlight that our data creation pipeline is designed for **low-cost scalability**. The entire process, from rationale generation with GPT-4o to multi-stage filtering, is **fully automated**. While initial prompt engineering required careful thought, the established pipeline can be targeted to new domains with minimal human intervention—primarily by supplying new seed questions. This automated approach makes extending our method far more practical and resource-efficient than manual annotation would be.\n\nIn conclusion, we are confident that our approach is both generalizable and scalable, addressing the key concerns about its broader applicability. We will integrate these clarifications and new experimental results into the revised manuscript. Thank you again for your valuable and encouraging feedback."}, {"review_id": "ZpO86J7rcT", "reviewer": "Reviewer_ukEB", "summary": "Introduces a new dataset, Visuo-Lingual Interleaved Rationale (VLIR), for supervised fine-tuning of VLMs. Extends GRPO to Region-GRPO and uses it to train a new model, VLM-R^3, based on Qwen2.5-VL 7B, which can automatically perform image grounding and zooming-in during reasoning. Results show that VLM-R^3 outperforms Qwen across seven different datasets, especially on ScienceQA.", "questions": "1. How does R-GRPO prevent the model from cropping too many regions during reasoning?\n\n2. According to the ablation in Figure 3, random cropping harms performance. As noted in Section 4.5.2, traditional MLLMs tend to lose visual context over long reasoning steps, leading to errors—I agree with this observation. However, to better demonstrate the effectiveness of cropping, it would be helpful to compare against a baseline where the input image is periodically re-inserted, e.g., by adding prompts like “Let’s look back to the input image \\<image_token\\>.”\n\n3. The example in Figure 1 can be better. I can hardly recognize \"HONGDONG\" characters even in the zoomed image", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n* The VIRL dataset will be useful for research in visual reasoning\n* The performance of VLM-R3 is impressive\n* Ablation studies demonstrate the effectiveness of the proposed method\n\nWeaknesses:\n\n* small issues related to formatting and visualizations", "was_revised": true, "final_justification": "The author addressed most of my concerns. I will keep my score.", "rebuttal": "Thank you for your very positive and encouraging review of our paper. We are thrilled that you found the VLIR dataset useful, the performance of VLM-R³ impressive, and our ablation studies effective. We are also grateful for your thoughtful questions and suggestions, which have helped us further refine and validate our work.\n\nWe provide a point-by-point response to your questions and concerns below.\n\n---\n\n### **Question 1: Preventing Excessive Cropping**\n\n**Reviewer's Question:** *How does R-GRPO prevent the model from cropping too many regions during reasoning?*\n\n**Our Response:**\nThis is an excellent question about the self-regulating behavior of our model. The R-GRPO framework has two key mechanisms that inherently discourage the generation of excessive or redundant regions:\n\n1.  **Region Validity Reward:** As described in Section 3.3, our reward function includes a term specifically for \"each syntactically correct and **non-redundant** bounding box command.\" This means the model receives a reward for identifying a *useful* new region, but it does not receive additional rewards for repeatedly cropping the same area or generating superfluous regions that do not contribute to the final answer. This discourages redundancy.\n\n2.  **Task-Completion Reward:** The primary reward signal for the policy is tied to the final task accuracy (i.e., whether the final answer is correct). Through exploration, the model learns that generating an excessive number of regions does not necessarily lead to a correct answer and can even derail the reasoning chain. The RL process optimizes for the most efficient path to the correct answer, which naturally penalizes inefficient strategies like over-cropping.\n\nIn essence, the model is incentivized to be efficient: it learns to crop only when necessary to acquire new information that progresses it toward the correct final answer.\n\n---\n\n### **Question 2: Comparison with Periodically Re-inserting the Full Image**\n\n**Reviewer's Question:** *...to better demonstrate the effectiveness of cropping, it would be helpful to compare against a baseline where the input image is periodically re-inserted, e.g., by adding prompts like “Let’s look back to the input image <image_token>.”*\n\n**Our Response:**\nThis is a fantastic suggestion for a highly relevant baseline. It directly tests whether the benefit comes from simply refreshing the visual context or from our targeted, region-specific refinement. Following your advice, we have implemented this exact baseline: we fine-tuned the model to periodically re-insert the *entire* image into the reasoning context. We evaluated this new baseline against our VLM-R³ on three diverse benchmarks.\n\n**Table 1: VLM-R³ vs. Periodically Re-inserting the Full Image**\n\n| Model | MathVista | MME-RealWorld | MathVision |\n| :--- | :---: | :---: | :---: |\n| Qwen2.5VL-7B (Base) | 50.1 | 42.3 | 41.5 |\n| **Baseline:** Re-insert Full Image | 50.8 (+0.7) | 43.1 (+0.8) | 42.0 (+0.5) |\n| **VLM-R³ (Ours)** | **53.2 (+3.1)** | **51.6 (+9.3)** | **44.8 (+3.3)** |\n\n**Analysis:**\n*   Periodically re-inserting the full image provides only a **marginal improvement** over the base model. This suggests that simply \"reminding\" the model of the image is not sufficient to overcome the visual attention dilution problem for complex tasks.\n*   Our VLM-R³ approach, which intelligently selects and zooms in on specific, relevant regions, yields **significantly larger performance gains**. For example, on MME-RealWorld, our method improves by +9.3 points, whereas the baseline improves by only +0.8.\n\nThis experiment strongly validates our core hypothesis: the key to enhanced reasoning is not just refreshing visual context, but **actively refining it by focusing on the most informative regions**. We will add this new baseline and analysis to our ablation study section.\n\n---\n\n\n\n### **Minor Points:**\n\n*   **Figure 1 Example:** Thank you for the feedback on the \"HONGDONG\" example. We agree it could be clearer and will replace it with a more compelling and easily recognizable example in the revised manuscript.\n*   **Formatting and Visualizations:** We will carefully review the manuscript to correct any formatting issues and improve the clarity of all visualizations.\n\nThank you once again for your constructive and supportive review. We are confident that the revisions and new experimental evidence will further strengthen the paper"}, {"review_id": "ZZFuhkgVxc", "reviewer": "Reviewer_5gUr", "summary": "The main idea of the paper is to select the most relevant image regions to answer questions. Paper uses a region-conditioned reinforcement policy optimization training that rewards the model for selecting informative regions, formulating appropriate transformations (e.g.crop, zoom), and integrating the resulting visual context into subsequent reasoning steps.  The paper also introduces Visuo-Lingual Interleaved Rationale (VLIR), a dataset curated to support the development of MLLMs for interleaved text-image CoT reasoning.", "questions": "Line39-40: Can authors clarify why it is need for a visual model to look at the image after each reasoning step? Can't attention already takes this into account?\n\nIt is unclear whether the paper employs a specific cropping method. Modern vision-language systems, such as Qwen-VL, are capable of returning bounding boxes in response to specified prompts.", "limitations": "There is not much novelty in this work. Section 3.3 and the rest of the method is simple application of existing methods.\n\nAs the VLIR is primarily constructed using automatic annotations, the size of the dataset is really small to count as a contribution. If the data is fully human annotated, the size of the dataset is acceptable. \n\nThere are no meaningful visualizations in the main paper.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "+ Paper addresses an important problem.\n+Automatically generation of bounding boxes for the reasoning steps is interesgting.\n\n- The introduction is too repetitive and their are many over-claims such as  \"pioneering dataset\", \"meticulously curated\" , \"a novel framework\", \"master this intricate reasoning\". some of these claims are not factual.\n- Some of the limitations mentioned in [42] has been addressed by many recent works.\n- Paper does not discuss more recent work on this topic.\n-The data filtering and rejection sampling processes rely on small VLMs and LLMs, whose outputs may not be reliable or verifiable.", "was_revised": true, "final_justification": "After the rebuttal I have changed my rating. However, introduction must be revisited by authors to reflect on facts.", "rebuttal": "Thank you for your detailed feedback on our manuscript. We appreciate your time and the insightful questions you've raised. We have carefully considered all your comments and provide a point-by-point response below, including new experimental data that we believe will address your concerns.\n\n### **Response to Concerns 1: Novelty, Recent Work, and Claims**\n\n**Reviewer's Question:** \n\n*Q1：The introduction is too repetitive and their are many over-claims*，\n\n*Q2：Paper does not discuss more recent work on this topic.*，\n\n*Q3：There is not much novelty in this work. Section 3.3 and the rest of the method is simple application of existing methods.*，\n\n*Q4: Some of the limitations mentioned in [42] has been addressed by many recent works.*\n\n**Our Response:**\n\nWe group these points as they relate to the novelty and positioning of our work.\n\n*   **On Language and Claims (Q1):**\n\n We agree that our language can be more precise. We will revise the manuscript to replace subjective terms with objective, factual descriptions, ensuring our claims are directly supported by evidence.\n\n*   **On Novelty and Comparison to Recent Work (Q2, Q3):** \n\nWe respectfully disagree with the assessment that our work lacks novelty.  \n\nTo our knowledge, **our research was conceived as one of the first systematic explorations into capabilities similar to those in OpenAI's o3 model**, for which we also developed a new, high-quality multimodal reasoning dataset, and we have contributed a high-quality dataset specifically for this purpose. \n\nRegarding recent related works like DeepEyes [1] and GRIT [2], we must highlight their temporal context: these papers were released on arXiv **after the NeurIPS 2024 submission deadline**. They therefore represent concurrent, independent research, and their emergence validates the timeliness and significance of the problem we address. We will add a discussion of these works to our Related Work section to properly situate our contributions. \n    \nFrom a performance perspective，as shown in Table 1, our VLM-R³ framework's superior results against these concurrent methods attest to the novelty and efficacy of our technical design.\n\n**Table 1: Comparison with Concurrent Region-Enhancement Methods**\n| Model & Method | MathVista | MathVision | Avg. Improvement |\n|---|---|---| --|\n| **Qwen2.5-VL 7B (Base)** | 68.2 | 25.1 | - |\n| + DeepEyes | 70.1 (+1.9) | 26.6 (+1.5) | +1.7 |\n| **+ VLM-R³ (Ours)** | **70.4 (+2.2)** | **30.2 (+5.1)** | **+3.65** |\n| | | | |\n| **Qwen2.5-VL 3B (Base)** | 58.5 | 21.2 | - |\n| + GRIT | 59.8 (+1.3) | - | - |\n| **+ VLM-R³ (Ours)** | **61.4 (+2.9)** | **23.3 (+2.1)** | **+2.5** |\n\n\nThe data shows that VLM-R³ consistently outperforms these methods, most notably achieving a **+5.1 point gain on MathVision** over the 7B base model, more than tripling the gain from DeepEyes. This empirically validates the novelty and power of our approach.\n\n[1] Zheng, Ziwei, et al. \"DeepEyes: Incentivizing\" Thinking with Images\" via Reinforcement Learning.\" arXiv preprint arXiv:2505.14362 (2025).\n[2] Fan, Yue, et al. \"GRIT: Teaching MLLMs to Think with Images.\" arXiv preprint arXiv:2505.15879 (2025).\n\n\n### **Response to Concern 2: Reliability of Automatic Annotation for VLIR**\n**Reviewer's Question:** \n\n*Q4：The data filtering and rejection sampling processes rely on small VLMs and LLMs, whose outputs may not be reliable or verifiable.*，\n\n**Our Response:**\n\nWe would like to clarify our data pipeline to address concerns about its reliability. The most critical reasoning tasks were handled by state-of-the-art large models: **GPT-4o** for rationale generation and **DeepSeek-V2** for logical coherence checks. A smaller model, **Qwen-VL-Chat (3B)**, was used *only* for a simple, low-level sub-task: verifying that a cropped region contains a recognizable entity. This was a deliberate design choice to balance cost and efficiency.\n\nTo validate this choice, we compared the 3B model's filtering decisions against much larger models on 1,000 sample images. The results confirm its reliability for this specific task.\n\n**Table 2: Cross-Model Agreement on Cropped Region Verification**\n| Model Pair | Agreement Rate |\n| :--- | :---: |\n| Qwen-VL-Chat (3B) vs. Qwen-VL-Max (72B) | 98.2% |\n| Qwen-VL-Chat (3B) vs. GPT-4o | 97.6% |\n\nThe >97% agreement demonstrates that our cost-effective approach did not sacrifice data quality.\n\n### **Response to Concerns 3: Necessity of \"look at the image\" vs. Attention and Cropping Method**\n\n**Reviewer's Question:** \n\n*Q5：Line39-40: Can authors clarify why it is needed for a visual model to look at the image after each reasoning step? Can't attention already take this into account?*，\n\n*Q6: It is unclear whether the paper employs a specific cropping method. Modern vision-language systems, such as Qwen-VL, are capable of returning bounding boxes in response to specified prompts.*\n\n**Our Response:**\n\nThis is a fundamental question. While powerful, standard attention mechanisms can \"dilute\" over long reasoning chains. Our method solves this with targeted refinement.\n\n*   **\"look at the image\" vs. Attention (Q5):** We direct the reviewer to **Figure 4 in our paper**, which was designed to answer this exact question. It visually demonstrates how a standard MLLM's attention on crucial visual details weakens over time, leading to errors (e.g., mistaking an umbrella for a \"menu board\"). In contrast, VLM-R³'s re-introduction of a cropped region creates a strong, focused \"attentional reset,\" enabling correct, grounded reasoning.\n\n*   **Cropping Mechanism and its Value (Q6):** Our model learns to generate `{\"bbox_2d\": [x1, y1, x2, y2]}` , which is consistent with the default bbox format supported by QwenVL, as an intermediate reasoning step. To prove the value of the subsequent cropping and re-injection of this region, we compare against a strong baseline, **\"Vanilla text-only RL\"**, which generates coordinates but does *not* re-examine the cropped image.\n\n    **Table 3: Ablation Study on Vision-Centric Benchmarks**\n\n    | Model | MMVP | CV-Bench-3D | V* | MME-RealWorld |\n    | :--- | :---: | :---: | :---: | :---: |\n    | Qwen2.5VL-7B (Base) | 66.7 | 72.6 | 74.3 | 42.3 |\n    | Vanilla text-only RL | 72.0 | 80.9 | 78.5 | 46.2 |\n    | **VLM-R³ (Ours)** | **75.0** | **81.7** | **83.8** | **51.6** |\n    | **Δ vs. Vanilla RL** | **+3.0** | +0.8 | **+5.3** | **+5.4** |\n\n    The significant gains of VLM-R³ over \"Vanilla text-only RL\" (e.g., **+5.3 on V* and +5.4 on MME-RealWorld**) provide decisive proof that the **Refinement** step—actively re-examining the selected region—is the key driver of our model's success.\n\n### **Response to Concern 4: Dataset Size and Visualization Quality**\n**Reviewer's Question:** \n\n*Q7：As the VLIR is primarily constructed using automatic annotations, the size of the dataset is really small to count as a contribution. If the data is fully human-annotated, the size of the dataset is acceptable.*，\n\n*Q8: There are no meaningful visualizations in the main paper.*\n\n**Our Response:**\n\n*   **On the VLIR Dataset(Q7):** We acknowledge VLIR's modest size. Its contribution, however, is not scale but **novelty and methodological value**. It is one of the first datasets to provide explicit, step-by-step supervision for both region selection and textual rationale. We also present a robust, semi-automated pipeline to create such high-quality, specialized data efficiently. We are encouraged that other reviewers agree on its utility, with one noting, **\"The VLIR dataset will be useful for research in visual reasoning.\"**\nMoreover, the significant performance gains shown across numerous standard benchmarks (e.g., Table 3) demonstrate that our model generalizes exceptionally well. This powerful generalization from a modest-sized training set is the ultimate validation of its contribution. \n \n*  **On the Visualizations(Q8):** We respectfully suggest that our visualizations are indeed meaningful. As noted above, **Figure 4** is not merely illustrative; it provides direct, empirical evidence for our core claim by visualizing the \"attention decay\" problem and how our method solves it. It is a visual answer to a key scientific question posed by the reviewer."}, {"review_id": "UCfHt5eGoC", "reviewer": "Reviewer_ifQp", "summary": "This paper proposes VLM-R3, a vision-language model framework equipped with an adaptive ROI (region-of-interest) cropping mechanism that enables autonomous selection of optimal visual input regions before actually answering questions. To develop this capability, the authors first constructed the VLIR dataset through multi-stage curation processes, which was subsequently used for supervised fine-tuning (SFT) followed by reinforcement learning (RL) optimization. Comprehensive evaluations, including both quantitative metrics and qualitative analyses, demonstrate the effectiveness of VLM-R3 in achieving accurate question answering while optimizing computational efficiency through intelligent region selection.", "questions": "N/A", "limitations": "The authors **DID NOT** provide discussions on limitations.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. The motivation is clear and quite reasonable.\n2. The overall paper is easy to follow.\n3. The presentation is clear.\n\n**Weaknesses**\n1. Insufficient evaluations. VLM-R3 is mainly evaluated on LLM-centric benchmarks such as MathVista, MathVision, MMMU, and ScienceQA. These benchmarks could not actually reflect the ability of \"grounding then answering\", as vanilla CoT prompting also brings significant improvements on these benchmarks. Therefore, the authors should evaluate on *vision-centric* benchmarks like MMVP, CV-Bench, V*, HR-Bench, and MME-RealWorld. Furthermore, the vanilla text RL baseline should be incorporated.\n2. The definition of the grounding accuracy illustrated in Figure 3 is not clear. A flowchart, pseudo-code, or equation is strongly encouraged.", "was_revised": true, "final_justification": "I really appreciate the rebuttal, especially towards my W1. I am really excited to see the improvements of VLM-R3 against the vanilla text RL baseline (with bounding box generation during reasoning) on several vision-centric benchmarks. \n\nTherefore, I raise my rating to 4. The authors should include these additional experimental results and discussions in their revised version.", "rebuttal": "Thank you for your thoughtful and constructive review of our manuscript. We are grateful for your positive feedback on the clarity of our motivation and presentation.\n\nBelow, we provide a point-by-point response to your concerns, including new experimental results that we believe will fully address the points you've raised.\n\n### **Q1: Insufficient Evaluations on Vision-Centric Benchmarks**\n\n**Reviewer's Comment:** *Insufficient evaluations. VLM-R3 is mainly evaluated on LLM-centric benchmarks... These benchmarks could not actually reflect the ability of \"grounding then answering\"... the authors should evaluate on vision-centric benchmarks like MMVP, CV-Bench, V*, HR-Bench, and MME-RealWorld. Furthermore, the vanilla text RL baseline should be incorporated.*\n\n**Our Response:**\nThis is an excellent point. We agree that demonstrating the effectiveness of our grounding mechanism on vision-centric benchmarks is crucial for a comprehensive evaluation. While our initial submission focused on complex reasoning tasks where grounding is a key sub-problem, we acknowledge that benchmarks designed to directly test visual perception and grounding are essential to validate our claims.\n\nFollowing your suggestion, we have conducted a new set of extensive experiments on the very benchmarks you recommended. We compare our full **VLM-R³** framework against two critical baselines:\n1.  **Qwen2.5VL-7B (Base Model):** The original, pre-trained model without any additional fine-tuning.\n2.  **Vanilla text-only RL:** A strong baseline where we apply our GRPO policy to the base model, but **only reward textual reasoning and bounding box generation, without the crucial 'Refinement' step** of cropping and re-injecting the visual region. This baseline directly tests the impact of our core contribution.\n\nThe results, presented below, unequivocally demonstrate the superiority of our approach on these vision-centric tasks.\n\n**Table 1: Performance on Vision-Centric Benchmarks. VLM-R³ demonstrates significant improvements, validating its \"grounding then answering\" capability.**\n\n| Model | MMVP | CV-Bench-2D | CV-Bench-3D | V* | HR-Bench-4k | HR-Bench-8k | MME-RealWorld |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| Qwen2.5VL-7B (Base) | 66.7 | 74.1 | 72.6 | 74.3 | 69.8 | 64.6 | 42.3 |\n| Vanilla text-only RL | 72.0 | 76.9 | 80.9 | 78.5 | 72.9 | 65.2 | 46.2 |\n| **VLM-R³ (Ours)** | **75.0** | **77.5** | **81.7** | **83.8** | **73.4** | **66.8** | **51.6** |\n| **Δ vs. Vanilla RL** | **+3.0** | +0.6 | +0.8 | **+5.3** | +0.5 | +1.6 | **+5.4** |\n\n**Analysis of Results:**\nThe data clearly shows that VLM-R³ consistently outperforms both the base model and the strong \"Vanilla text-only RL\" baseline across all seven vision-centric benchmarks. The substantial gains, particularly on benchmarks like **V* (+5.3)** and **MME-RealWorld (+5.4)**, which heavily rely on fine-grained perception and spatial understanding, provide powerful evidence for our core claim: the iterative process of **Region Recognition, Reasoning, and Refinement** is highly effective. It allows the model to \"zoom in\" and extract critical visual details that are missed by other approaches.\n\nWe are confident that this new, comprehensive evaluation directly addresses your concern and robustly validates the effectiveness of VLM-R³'s grounding mechanism. We will integrate this new table and analysis prominently into the Experiments section of our revised manuscript.\n\n### **Q2: Clarity of Grounding Accuracy Definition**\n\n**Reviewer's Comment:** *The definition of the grounding accuracy illustrated in Figure 3 is not clear. A flowchart, pseudo-code, or equation is strongly encouraged.*\n\n**Our Response:**\nThank you for pointing out this lack of clarity. We apologize for the ambiguity and agree that a formal definition is necessary. The \"Grounding Accuracy\" reported in our results is a binary metric based on a predefined Intersection over Union (IoU) threshold, which is a standard evaluation practice in object detection and grounding tasks.\n\nTo make this crystal clear, we will add the following formal definition to the manuscript:\n\n> **Definition of Grounding Accuracy:**\n> Grounding accuracy evaluates whether the model's predicted region of interest is sufficiently aligned with the ground truth. For a given reasoning step, let $B_{pred}$ be the bounding box generated by the model, and $B_{gt}$ be the ground-truth bounding box from our VLIR corpus. A prediction is considered **correct** if the Intersection over Union (IoU) between the two boxes exceeds a certain threshold, $\\tau$. We follow common practice and set $\\tau = 0.5$.\n>\n> The IoU is calculated as:\n> $$ \\text{IoU}(B_{pred}, B_{gt}) = \\frac{\\text{Area}(B_{pred} \\cap B_{gt})}{\\text{Area}(B_{pred} \\cup B_{gt})} $$\n>\n> The grounding decision is then determined by:\n> $$ \\text{Grounding Correct} =\n> \\begin{cases}\n> 1 & \\text{if } \\text{IoU}(B_{pred}, B_{gt}) \\ge 0.5 \\\\\n> 0 & \\text{otherwise}\n> \\end{cases}\n> $$\n> The overall \"Grounding Accuracy\" is the average of these binary scores across all evaluation samples.\n### **Q3: Lack of a Limitations Section**\n\n**Reviewer's Comment:** *The authors DID NOT provide discussions on limitations.*\n\n**Our Response:**\nYou are absolutely correct that our initial submission did not include a dedicated limitations section. We thank you for highlighting this.\nNeurIPS2025 guidelines do not strictly mandate a separate limitations section within the main body of the paper; we agree with you that including one provides a more balanced and complete scientific perspective. Your suggestion is well-taken, and we see the value it adds for the reader.\nTherefore, in our revised manuscript, we will add a dedicated **\"Limitations\"** section. This new section will discuss several key points to provide a more nuanced view of our work:\n*   **Potential for Error Propagation:** An incorrect region selection early in the reasoning chain can impact subsequent steps, a challenge inherent in sequential decision-making processes.\n*   **Scope of Visual Actions:** Our current work focuses on cropping. We will discuss how future work could expand the model's toolkit to include more complex visual transformations.\n\nWe believe this addition will significantly strengthen the paper and are grateful for the suggestion."}]

Author rebuttal:

Meta-review: The paper proposes VLM-R³, a framework that equips multimodal LLMs with the ability to iteratively select, crop, and revisit visual regions while reasoning. The approach combines a curated Visuo-Lingual Interleaved Rationale (VLIR) dataset with Region-Conditioned GRPO (R-GRPO), enabling models to decide when and where to ground their reasoning in visual evidence and integrate the extracted content into chain-of-thought. Experiments on MathVista, ScienceQA, and related benchmarks demonstrate state-of-the-art results, particularly on tasks requiring fine-grained spatial reasoning and visual cue extraction.

The work is motivated by a clear gap in existing reasoning-based MLLMs, and the design is thoughtful. The introduction of R-GRPO and the interleaving of region-level reasoning within CoT are novel and effective. Results are strong and supported by ablations, and the VLIR dataset may also provide value to the community. The paper is well written and generally easy to follow.

Weaknesses highlighted by reviewers include limited evaluation scope, as benchmarks are mostly LLM-centric and do not fully capture grounding capability. Some reviewers raised concerns about scalability and adaptability since the approach relies heavily on the carefully curated VLIR dataset. Presentation issues, such as unclear definitions of grounding accuracy, limited visualizations, and some overstated claims, were also noted.

During rebuttal, authors clarified certain issues. R2 remained strongly supportive, while R1 and R3 acknowledged clarifications but maintained caution on generalization, and R4 emphasized evaluation breadth. On balance, I think the contribution technically solid. I recommend acceptance.
Decision: Accept (poster)