VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use
Authors: Mingyuan Wu
Jingcheng Yang
Jize Jiang
Meitang Li
Kaizhuo Yan
Hanchao Yu
Minjia Zhang
ChengXiang Zhai
Klara Nahrstedt
Venue: ICLR 2026 Poster (2026)
Keywords: Reinforcement Learning
Vison Lanaguage Model
Reasoning
Primary area: foundation or frontier models, including LLMs
TL;DR: Reinforcement learning finetuning can enable vision language models to think with intermediate image reasoning steps.

Abstract: Reinforcement learning finetuning (RFT) has significantly advanced the reasoning capabilities of large language models (LLMs) by enabling long chains of thought, multi-turn self-correction, and effective tool use. While recent works attempt to extend RFT to vision-language models (VLMs), these efforts largely focus on text-only reasoning conditioned on original image inputs, and do not incorporate visual reasoning in the response. In contrast, test-time methods like Visual Sketchpad incorporate visual steps but lack training mechanisms.

We introduce VTool-R1, the first RFT framework that trains VLMs to generate multimodal chains of thought by interleaving text and intermediate visual reasoning steps. VTool-R1 integrates Python-based visual editing tools into the RFT process, enabling VLMs to learn when and how to generate visual reasoning steps that enhance the final output quality. Trained with outcome-based rewards, our approach elicits strategic visual tool use for multi-modal reasoning without relying on process-based supervision. Extensive experiments on structured visual reasoning over charts and tables show that VTool-R1 enhances reasoning performance by teaching VLMs to "think with images" and generate multimodal chain of thoughts with tools. To support future research in multi-turn multi-modal reasoning, we open-source our code at https://github.com/VTOOL-R1/vtool-r1.

Published as a conference paper at ICLR 2026
VTOOL-R1: VLMs LEARN TO THINK WITH IMAGES VIA REINFORCEMENT LEARNING ON MULTIMODAL TOOL USE
Mingyuan Wu'; Jingeheng Yang" Jize Jiang', Meitang Li,, Kaizhuo Yan', Hanchao Yu², Minjia Zhang', Chengxiang Zhai Klara Nahrstedt
University of Illinois Urbana-Chaman Champainn
University of Michigan Ann Arbor, 'Independent Researcher (mw34, klara) @cs. illinois.ed.e
ecs..
ABSTRACT
Reinforcement learning finetuning (RFT) has significantly advanced the reasoning capabilities of large language models (LLMs) by enabling long chains of thought, multi-turn self-correction, and effective tool use. While recent works attempt extend RFT to vision-language models (VLMs), these efforts largely focus on text- only reasoning conditioned on original image inputs, and do not incorporate visual reasoning in the response. In contrast, test-time methods like Visual Sketchpad incorporate visual steps but lack training mechanisms.
We introduce VTool-R1, the first RFT framework that trains VLMS generate multimodal chains of thought by interleaving text and intermediate visual reasoning steps. VTool-R1 integrates Python- based visual editing tools into the RFT process, enabling VLMs to learn when and how to generate visual reasoning steps that enhance the final output quality. Trained with outcome- based rewards, our ap- proach elicits strategic visual tool use for multi-modaa reasoning without relying process-based supervision. Extensive experiments on structured visual reasoning over charts and tables show that VTool-R1 enhances reasoning performance by teaching VLMs to "think with images" and generate multimodal chain of thoughts with tools. To support future research in multi-turn multi-moda reasoning, we open-source our code at
INTRODUCTION
Recent large language models (LLMs), notably DeepSeekRI (DeepSeek-AI, 2025) and the GPT4o series (OpenAl 2024), have demonstrated remarkable capabilities in text-based reasoning. Central to this advancement is Reinforcement learning finetuning (RFT), which enables these models to learn to generate long chains of thought and engage in self-correction and verification for complex reasoning tasks (Kumar et al. 2025 Zeng et al., 2025). Beyond boosting intrinsic reasoning capabilities, RFT has also shown promising results in effectively integrating external tool use, such search engines (Tin et al.] 2025 Chen et al. 2025b) and code interpreters (Feng et al. 2025), into the reasoning process of LLMs.
Despite this rapid progress in LLM reasoning brought by RFT, there has not yet been a well- recognized breakthrough in improving multimodal reasoning capabilities of vision-language models (VL.Ms). Modern VLMs (Bai et al. 2025) typically consist of a strongly language-aligned image encoder that maps visual inputs into feature space, then a connector module further projects these visual features into the token space of a decoder-only LLM. On top of these architectures, prior works have attempted to replicate post-training strategies first developed on LLMs, like visual instruction tuning (Xu et al. 2024), to improve VLMs' ability to follow textual instructions.
However, similar attempts by recent works (Zhou al. 2025 Chen al. 2025a Zhang al.. Huang et al.
2025
2025 Liu et al.) 2025 Deng et al. 2025) to replicate the success of RFT on
Mingyuan and Jingeheng contributed equally.

---

Published as a conference paper at ICLR 2026
LLMs onto the VLM domain to enhance multimodal reasoning fall short: these reasoning approaches remain fundamentally text-driven: they handle images only during the initial encoding stage and generate reasoning chains purely in text conditioned on fixed imput image tokens, without involving any intermediate visual reasoning steps.
Why is text-dominant reasoning insufficient? Even the most advanced VLMs may rely language shortcuts if text dominates reasoning. For example, consider prompt where a user shows GPT-5 (OpenAT 2025) an image of hand with six fingers and asks, "How many fingers in this hand?" The model may respond with "five" with high chance, based on the pure text reasoning path -"a hand has five fingers". This illustrates critical failure of purely language- settings: the model can decode a more plausible answer through wrong textual chain of thoughts shortcuts.
reasoning multimodal
To go beyond text- lext- reasoning, early inference frameworks like Visual Sketchpad (Hu et al. 2024) and Refocus (Fu et al. (005)) demonstrate that incorporating intermediate visual reasoning steps during inference can improve multimodal reasoning performance, However, these inference-time methods rely on the usage of highly capable models such as GPT-4o to produce meaningful visual steps, and underperform when used with open-source, less capable models.
In this paper, we present the first RFT frameworks that directly enables VLMs learn to think in a combination of images and texts and to be trained to generate multimodal chains of thoughts. Our framework, called VTool-R1, teaches VLMs to generate intermediate visual reasoning steps through interactions with external image editing tools implemented in Python code. These visual reasoning steps are interleaved with textual chains thought, resulting in multi-turn multimodal reasoning steps during the generation of the response. Following the design of DeepSeekR1, VTool-RI uses only outcome-basse rewards-the model is not explicitly rewarded for generating visual steps, but instead learns when and how to incorporate visual reasoning steps in order to improve final task performance.
We demonstrate the effectiveness of VTool-R1 on challenging structured image reasoning tasks, focusing on table and chart-based reasoning. Our experiments use well-curated dataset and a visual editing tool set from Refocus. This toolset, which directly callable by the VLM, enables selective attention on relevant regions in the table and chart simulating how humans process visual information through attention before forming final reasoning results. Through VTool-R1, the model learns to use these tools strategically to guide its multimodal chain of thought and enhance final reasoning performance.
Our key contributions can be summarized follows:
We present VTool-R1, a novel RFT framework that supports VLM multimodal reasoning with external visual editing tool use. We demonstrate that RFT with outcome based reward design can incentive generation of visual reasoning steps for final reasoning accuracy.
To the best of our knowledge, our work is the first work that successfully enables VLMs to learn integrate intermediate visual reasoning steps into text- chains of thoughts in the generated response (i.e. thinking with a combination of images and text).
We validate our approach through extensive controlled experiments on challenging and well-established structured image reasoning datasets, using a predefined visual toolset in Refocus dataset.
RELATED WORKS
2.1 VISUAL CHAIN OF THOUGHT REASONING
Early works have demonstrated that incorporating visual intermediate steps-ofter generated via external tools or Python scripts-can benefit wide range of visual question answering (VQA) tasks, without any training. Pioneering efforts such as ViperGPT (Surís et al. (023) and Visual Programming without Training (Gupta Kembhavil 2023) utilize Python-basee visual tools to manipulate images during inference. Cache of Thought Wu et al.) 2025) further improves multimodal reasoning by retrieving high-quality rationales from similar queries. Recently, Visual Sketchpad (Hu et al. 2024) introduced framework that equips multimodal language models with sketchpad and drawing
2

---

Published as a conference paper at ICLR 2026
tools. The model is prompted to generate visual artifacts during inference and uses them for iterative planning and reasoning. While this approach successfully introduces visual information into the reasoning process. it operates solely at inference time, with no training involved. Refocus (Fu et al. 2025) takes step forward by prompting the VI.M to invoke visual editing tools for selective attention over images. These modified images does not train the model to reason with tools; instead, it relies on pre-edited images generated by commercially powerful model such as GPT-4o, and hence underperforms in scenarios where only less less-performant, smaller models are available.
then used inputs for further reasoning. However, Refocus
2.2 LLM/VLM, REINFORCEMENT LEARNING AND TooL USE
The trend of RFT for LLMs began with RLHF (Ouyng et al. 2022), using PPO (Schulman et al. 2017), and later evolved into more efficient variants like DPO (Rafailov al.] 2023) and SimPO (Meng 2024). though these often face off-policy limitations. GRPO (Shao et al. 2024) mitigates such issues with critic-free, group-based design that improves both efficiency and stability. RFT has also been used to guide tool-augmente reasoning LLMs (Chen et al. 2025bb Feng et al.) 2025 Jin cct al.) 2025), demonstrating that outcome- based rewards can teach models when and how to use tools during multi-step reasoning.
In the VLM domain, recent works attempt to adapt RFT for VLMs to incentivize multimodal reasoning behaviors (Zhou al. 2023 Chen al. 2025a Zhang al. 2025; Huang al., (025
et
Wang et al.) 2025 but they primarily train VLMs to only generate textual chains of thought from visual inputs. However, training VLMs produce and reason over intermediate visual steps via tool use remains largely unexplored. Concurrent works related to VTool- RI, such Deepeyes (Zheng et al.) 2025) and OpenThink-IMC (Su et al. 2025), also integrate visual steps into reasoning chains, with different tool and task designs. We further demonstrate that VTool-R1 significantly outperforms Deepeyes on structured image data, which we attribute to the intrinsic design of our tools and tasks.
et al. 2025 Deng et al. 2025
VTOOL-R1
VLM Preliminaries. A VLM policy can be denoted as Te parametrized with model weights 0. Given a text prompt sequence x and an image I, the model can generate text response sequence sampled from the (L, .). Some VLMs support multiple image inputs; however, their capabilities in parsing and understanding multiple images vary significantly and are highly dependent on the training procedure (Wang al.] 2024). In our setting, we require VLMs capable of processing multiple images as both the original input image and intermediate visual steps must be processed together as input to the VLMs.
In the following sections, we present the detailed design of VTool-R1, covering inference and training parts. In the Section 3.1 we show that how pre-trained VLM can be prompted to use visual editing tools and generate integrate intermediate visual steps. We then go beyond inference in the Section 3.2 by extending the RFT objective, we train VLMs to use these tools and generate multimodal chains of thought during rollout by themselves. We also introduce outcome-based reward formulation that encourages effective visual reasoning while avoiding the pitfalls of process- based reward hacking.
3.1 VLM INFERENCE AND ROLLOUT WITH VISUAL CHAIN OF THOUGHTS
Refocus (uu et al. 2025) has shown that capable VLMs, such as GPT-4o, can be prompted to generate Python code for calling external tools edit the input image, followed by reasoning over the resulting visual outputs. Our inference and rollout prompting closely follows their format; full prompt templates are included the Appendix.
VLM Inference and Rollout Prompt
<Image Input><Syaten Prompt><Pyshon Codes Templates> CoAD #: Based on the above tools, I want you to reason about how to solve the USER REQUEST and generate actions step by step tesch action python function call) to solve the request. You may need to use the tools above to process the images and
3

---

Published as a conference paper at ICLR 2026
make decisions based on the visual outputs of the previcus code blocks, You should only use tcols above, you should use other functions or executed.
will be
RRCOTRRMENTS #:
...3, If you think got the anawes, ANSNER: syout answers Flease extract the final answer in FINAL, ANSWER: <final answer? and enda with TERMINATE,
you have enough information inages retursed by the Lools, you should directly answer the question based on the original image...
...8, you
questior the
9. Only one turn of action, ACTICN 0, is allowad. You must provide the ansver after maximum one ACTTON call.
In context Examples: <Thought 0><Acttor for O>=0haervation>><ddte trage><Thought, 1><Answer>
Info: x_valuee_bang boex, storing values coordinates. y_values_ storing x values and coerdinates. The x values in the image are: vavauues, The y valces in image are: <y_values>.
USER Bounding
USER TMAGE stored in image_1, ag PTL imago.
As illustrated in the prompt box above, we provide system instructions and task-specific goals to guide the VLM in using visual tools for reasoning. The model given the names and definitions of visual editing functions, along with detailed descriptions of their usage. Through in-context examples, the VLM is prompted to begin its reasoning in Thought 0, which outlines where to focus in the image. It then produces Action O, which is either "no action needed" statement or Python-like pseudocode snippet that invokes an appropriate visual editing tool.
The tool call is executed externally in Python environment to generate modified image, which then fed back into the model as additional input for an additional turn. The VLM continues its reasoning over this generated intermediate visual step, forming richer visual chain of thought that supports the final answer. As specified in the Requirement section of the prompt, the model allowed to either respond directly invoke a tool once to create an intermediate visual step for further reasoning. In this work, we focus on single-turn tool use-i.e., the model may call a tool at most once and reason over the edited image. Extending this to multi-turn tool use, where the model iteratively edits and reasons for multiple rounds, is left for the future, as effective multi-turn design requires stronger VLMs capable of handling multi-image inputs.
This inference and rollout process is inherently iterative and cannot be completed in single-turn VLM call if the model chooses to use tools. Execution must pause for the external Python environment to run the generated code. Once the modified image is available, it has to be reintroduced into the same VLM instance as the second image input, rather than being inserted at the original location in the generated response sequence, as is common in prior tool-use approaches such as Search-R1 (Jin ct al. 2025).
Formally, if the VLM policy my decides to invoke tool from external visual editing toolset T, the inference involves two rounds of model execution. The first round samples an initial response containing tool calls, y' To(- I, where the input text prompt includes tool descriptions. The tool calls are then executed in the Python environment as I' T(V., I) to generate a modified image. In the second round, the VLM performs reasoning over both the original and edited images:
y y Te(- 1,,;
1,2; T) Te(- 1 Ix)) = T T T),)
where ④ denotes the concatenation of the original image I and the updated image as dual image inputs the model. If the VLM chooses not to invoke any tools and instead answers the question directly, the final answer is obtained in the first round without needing a second inference pass: y To I,)).
3.2 RFT VLM TO GENERATE VISUAL CIIAIN OF THOUUITTS
VTool-RI RI adopts RFT to train VLMs to explore flexible reasoning trajectories and learn to invoke visual editing tools effectively. Given the two-stage iterative inference structure, two training objective designs are possible- for instance, optimizing only the final response y that yields the answer, or jointly optimizing both the intermediate tool-invoking output y and the final response y. In this work,
4

---

Published as a conference paper at ICLR 2026
choose to optimize only the final response y, goal is to directly train the model to improve final reasoning quality. We maintain the notation introduced at the end of Section 3.1
Multi-Modal GRPO Tool Use
Trained
Froome
2dd Rollout
1 Rollout
Executor
Tool
Instrccer
L T
y'1
Tool Esscutor
Tool Sucoess
A
Reward
q
xx,7 T
y'2
/ (y' x)xxx
//2
F2
Group Computation
A2
Reference LUM
L,,T
y'a
YG
Tg
AG
Eseraitor
Falled
1,
KL
Policy LLM
Figure 1: Multi-Modal GRPO Tool Use Training Pipeline, where the input is multimodal query
In VTool-R1. we assume reward model T under the Bradley-Terry formulation, and consider the following RFT training objectives:
Optimize the final reasoning response y during RL rollout:
max E Bjr,a)D, (-1l())1 [T, v, BKL][TTl I, Tye T) (2) Xo
where mo is the policy VLM parametrized with model weights 0. Tref is the reference VLM policy, Ta the reward function. DxL is the KL-divergence measure. |3 >> 0 is the KL penalty coefficient. The input [I, denotes multimodal samples drawn from the dataset D. The generated response in the rollout ~ nz(- 1, x; T) (- / l", x)) = Tz( T(y', a)) if the model chooses to tool; otherwise, when tool is invoked, the response simplifies y #T(: T 1, mo(: T.
Unlike prior RFT that simply relies on LLM/VLM policy to generate rollout during training (Ouyang ct al. 2022), VTool-RI explicitly incorporates the visual editing tool use from the toolset T in the rollout, and conditions the model on the edited image input. We refer readers Section the formal definition and intuition of the iterative tool-use rollout policy, which mirrors the model inference pipeline. This iterative pipeline enables more effective step-by-step reasoning across both modalities: the model learns to modify the image using tools to support its reasoning before producing the final answer.
for
Note that we do not directly optimize the intermediate tool-invoking response y' in our RFT process, as our goal is to encourage the model to autonomously decide whether using a tool improves reasoning. This design supports more end-to-end training objective.
Our training approach is built upon well-established policy gradient method: Group Relative Policy Optimization (GRPO) in (DeepSeek-Al) 20255 Shao et al. (024). which offers improved stability and climinates the need for a separate critic model. Unlike Proximal Policy Optimization, which estimates advantages using learned critic, GRPO estimates the baseline from group of sampled responses and reduces training resources. Specifically, for each input [T,z], GRPO samples a group of responses {y, G T-aa-1 I, T) from the old policy model, and then optimize the current policy model by maximizing the following objective equation 3
Here, and B are syper-parameters, and A, F; r (e-mean(r) denotes the normalized relative std(r)
advantage computed within the group of sampled responses. This formulation avoids the need for critic model, while maintaining stable and reward -aliged policy updates by regularizing with the KL divergence between the updated policy my and the reference policy mref-
5

---

Published as a conference paper at ICLR 2026
JCRPO(0) = "/aa):
(3)
min
3Dxx [To]|
We use inference template in the Section 3.] for training rollouts as well. This template structures the model's output, think before actions and let the model decide whether need a tool call, with the system instructions and requirements. We make the template highly formatted and listed clearly thoughts, actions, tool function blocks and final answers. We also include few shot examples for better instruction and format following.
3.3 REWARD MODELING
Following Deepseek-RI, RI, we adopt an outcome- autcome-based reward design that relies solely on the correct- ness of the model's 's final answer. For closed-ended tasks like factual QA, exact string match works well. However, in our setting-sttucturrg structurre visual understanding the answers are more free-form and easily judged by string match. To address this, assess the match between the predicted answer and the ground truth. While not strictly rule-based, this serves pseudo rule-based reward appropriate for open-ended tasks such as ChartQA. We reward score of when the judge thinks it is a match.
lightweight LLM-based judge merely
We also study process- based rewards that penalize incorrect tool reward successful invocations. However, this often results in reward hacking in RFT: models either avoid tools entirely when penalties are applied, or exploit success criteria by generating tool calls that superficially meet expectations without contributing to reasoning.
We do not use format-based rewards, our models already learn to follow the structured for- mat-Thoughts, Actions, Tools, and Final beswer-thanke to clear instruction templates. We leave further exploration of more dedicated format rewards future work, but find our current setup sufficient for reliable rollout behavior.
4 EXPERIMENT
4.1 DATASET
Following Refocus (uu et al. 2025), we evaluate VTool-R1 on structured image reasoning tasks that are particularly suitable for visual editing tool use. Our evaluation focuses chart and table-based reasoning questions, which poses significant challenges for early VLM works (Liu et al.) 2023 (022).
To ensure fair comparison, we strictly adhere to the data splits created in Refocus. These splits constructed from the following sources:
Table Split. We evaluate on three datasets: (1) VWTQ (Pasupat & Liang 2015), 750 QA pairs from Wikipedia tables rendered as screenshots with HTML; (2) VWTQ (Kim et al. 2024). 250 synthetic tables with randomized styles to avoid training data leakage; and (3) VTabFact (Chen et al. 2019), 250 entailment classification pairs rendered from TabFact content. All split 70/30 for train/test, with no validation set due to limited size.
Chart Split. From ChartQA (Masry et al.) 2022) with human written questions grounded in real world charts, Refocus select 444 horizontal and 382 vertical bar chart QA pairs for testing. For training. Refocus collect 14,344 examples and 813 validation examples.
4.2 VISUAL EDITING TOOLSET
While our long-term goal is enable models to invoke arbitrary tools or APIs within sandbox environment using outcome-based rewards, we begin by demonstrating VTool-R1's capabilities with set of simple but effective visual editing tools. These tools implemented in Python and help simulate visual attention by modifying table or chart images. We adopt the same tool set used in Refocus. In experiments for tabular problems, utilize variety of tools as follows:

---

Published as a conference paper at ICLR 2026
Highlight Colunn/Row: overlays semi-transparent red on the selected columns/Rows.
Mask Column/Row: applies white mask over irrelevant columns/rows.
Draw Column/Row: draws solid red bounding box around selected columns/rows.
For charts, we apply analogous operations to highlight or mask individual bars, based on their positions along the x-axis y-axis.
The model is instructed to call one or multiple tools the same time, as many operations (e-g., drawing bounding boxes multiple positions) can be performed in parallel and we involve at most one round tool call in the experiments.
These tools leverage external libraries such OpenCV to perform tasks like drawing bounding boxes and identifying maskable regions based on contours of bars or tables for selective attention in our tasks. Looking ahead, envision integrating more advanced generative models powerful tools that can execute more generalized visual modifications directly from language prompts.
4.3 EXPERIMENT SETUP
We demonstrate the effectiveness of our RFT framework by training the state-of-the-art open-source VLMs, Qwen-VL 2.5 models (Bai et al. 2025) at 3B, 7B, and 32B scales. Training uses the open- source VeRL training infrastructure (Sheng al. 2024). Training conducted with the AdamW optimizer (Loshchilov & Hutter 2019). using an initial learning rate of 1e-6 and weight decay of 1e-2. Due the large image sizes and long prompt sequences (up to 16,384 tokens), we set the global batch size to 32 for table split and 256 for chart split. The rollout group size is set to 5, and the KL divergence coefficient to 1e-2. The 3B/7B models are trained on 8/16 H100 GPUs with 96GB memory; the 32B models are trained 8 H200 GPUs with 141GB memory. We standardize decoding across rollout and evaluation with temperature 1.0 and bf16 precision.
4.4 BASELINE MODELS
3B
7B
32B
Qwen2.5-VL
Pure Run
Tool Use
V'ooolRR1
Pure Run
Tool Use
VTool-RI
Pure Run
Tool Use
VoToolRRI
Chart Split
51.8
24.6
64.0
76.2
53.4
80.7
88.0
85.0
86.7
Table Split
41.3
24.3
57.9
64.7
41.1
71.7
86.2
76.0
84.5
Table 1: Main Results of VTool-R1 and Baselines in Accuracy
VTool-R1
RI-VL
GPT-4o
3B
7B
7B
Pure Run
Tool Use
Chart Split
64.0
80.7
10.4
63.8
82.9
80.5
Table Split
57.9
71.7
8.6
45.4
75.7
77.0
Table 2: VTool-R1 compared to other post-RL reasoning models
We report several baselines contextualize VTool-R1's performance in Table I and Table
2
GPT-4o (OpenAI, 2024): Used as an upper bound across all benchmarks. This is a powerful commercial model that has already shown remarkable inference-only tool use capability for reasoning in Refocus (Fu et al. 2025).
Qwen2.5-VL (3B / 7B 32B) (Bai al. 2025); Included without any RFT in two configurations:
Prompted Tool-Use Inference: The model is prompted to use tools following our inference and rollout template in Section 3.1] However, prior to RFT, these open-source models before our RL fine-tuning struggle to follow tool-use instructions. For fairness, when a tool call fails, we append a prompt indicating the failure and ask the model regenerate its answer.
7

---

Published as a conference paper at ICLR 2026
Direct Inference (No Tools): Only the image and question provided, with tool-related prompts. Surprisingly, this setting of open-source model yields strong results across datasets often for larger models like 32B.
outperforming even GPT-4o-espeeee
These findings about high pure run accuracy suggest that Qwen2.5- VL 3B and 7B may have been post-trained VtA-style tasks or distilled from larger models, enabling strong direct-answering performance, but lacking the general-purpose tool-use capabilities seen in models like GPT-4o. This aligns with observations from Refocus, which found that only commercial models could be prompted to use tools for generating meaningful intermediate visual steps, while open-source models lacked this ability- until the introduction of our VTool-R1 framework.
Weaker Baseline: We include RI-VL (Zhagg al. 2025). an open-source RL-tuned model trained on general visual reasoning tasks, as a weaker academic baseline.
4.5
MAIN RFT RESULTS AND FINDINGS
Mon thes wthert calog depee
Thought D
Sois wiic sriape digets
INACE
Thought
craee ge uee tatl
can
Toods
Taverablr veus Marii and
ans vows Wer to Worre'
Answer
Van io Womer 20)
Request Denry
Exacute Code
Correct Answer
Whyt th
ralion behwcce to
Figure 2: Illustrative Example from VTool-R1 (3B): After RFT, 3B Model Successfully Integrates Intermediate Visual Steps.
Ascuracy
Tool Succes Rate
Toul Call Ratio
CharlQA3B TableVQA 3B
03-
0.2
30
33
10
10
Stops
Stopa
Sleps
Accurney
Tool Success Rate
Call Ratio
600-
0.5
64
20
28
40
Steps
Steps
Skep4
Figure Multi-Modal GRPO w. Tool Use Training Dynamics, for 3B models
Qualitative Tool-use Example. Figure 2 presents qualitative example where the VTToolR1 3B model successfully integrates intermediate visual steps through tool use as part of the reasoning process, ultimately arriving at the correct answer.
RFT makes Better Tool Use for Reasoning. When comparing VTool RI Model reasoning accuracy with non-trained baselines in Table we observe significant improvement in tool use capability, After training, models learn to reason correctly with multimodal tool use, guided solely by outcome- based rewards. Remarkably, the 3B and 7B model, which initially failed with high chance to generate meaningful tool use (thoughts, actions, or tool calls), learns to use tools effectively to generate intermediate reasoning. This indicates that VTool-R1 not only improves final answer accuracy, but
8

---

Published as a conference paper at ICLR 2026
also enables models to internalize structured reasoning patterns, In both the 3B and 7B cases, VTool- R1 significantly outperforms the direct inference baseline, highlighting that RFT-driven tool use as an additional reasoning step brings substantial benefits to the model's overall reasoning capabilities.
Better Tool Use, Not? It's Not Monotonic. Our goal goes beyond improving accuracy-we aim to teach the model when and how to use tools in a way that meaningfully supports reasoning in the RL training. As shown in Figure 3 VTool-RI enables models to make nuanced, context-aware tool-use decisions. Interestingly, tool call frequency and success rate do not increase monotonically when the training proceeds and accuracy goes up. Instead, we observe fluctuations: models tend to overuse tools early in training due to prompt instruction exposure but later learn to invoke them more selectively. The 3B model becomes more cautious with tool use over time, leading to higher reasoning accuracy. Crucially, the model also learns when tools are unnecessary and confidently proceeds with direct reasoning. The 32B model (training curve shown in the Appendix) exhibits a higher overall tool use rate but similarly shows periods of decline, reflecting adaptive behavior. This adaptive tool-use behavior for reasoning is key outcome of our RFT strategy. While the exact trends vary between table and chart tasks, the overall pattern remains consistent.
More Successful Tool Use or Not? Figure 3 also presents the tool call success rates of the most representative 3B model during RFT training both chart and table tasks. It is important note that we cannot evaluate tool call correctness with full precision and recall, as no oracle verifier is available. Instead, rely on proxy metric to approximate success: A tool call is considered successful if the python commands executed did not raise any exceptions inside a sandbox environment with the given functions, and a valid pillow image is returned from the execution through passing the processed image into the display function. According to this metric, the success rate of tool use steadily increases on table tasks, while for charts it fluctuates throughout training. We would like to highlight the need for future work to incorporate human-ane annotated oracle verifier to more accurately evaluate tool-use success.
Training Dynamics. Overall, the model's accuracy steadily improves throughout training, with minor fluctuations. Performance gradually converges and stabilizes around the final accuracy within approximately 50 training steps in around one epoch. The saturated step number varies depending on the training configuration.
Reward Design. We also explore alternative reward settings beyond the standard 0/1 outcome-based reward. When applying process- based rewards, such as penalizing failed tool calls, we observe that the model quickly learns to avoid tool use entirely, driving the tool usage rate to zero. Conversely, when we add extra reward for successful tool use that leads to a correct answer, the model begins to exploit the verifier and hack to triggering "success" signal. This holds even under stricter verifier criteria. These findings support our claims that satcome-based rewards tied solely to final task correctness serve the most reliable and robust reward design for VTool-RI.
Benefits from External Tool Use. As shown in Table 2 VTool-R1 shows clear advantages over R1-VL models trained general visual reasoning data, by incorporating an additional tool-use step to generate intermediate visual outputs. These results underscore the impact of learning to "think with images": the added visual reasoning step significantly enhances final performance.
Experiments v.s. Deepeyes. Additionally, we present results comparing our approach against a concurrent RL-trained tool-use VLM, Deepeyes. Our 7B model trained with VTool-R1 achieves significantly higher performance. 60.0 (deepeyes) v.s. 80.7 (ours), which we attribute to the intrinsic good design of our tools, training recipe, and well-designed structured image tasks.
Failure Analysis, In particular, we highlight several representative categories of failure cases, including: (a). The model correctly generates an intermediate reasoning visual step, but fails to make the correct inference in the second rollout. (b). The model correctly identified the intermediate visual step needed for the reasoning, but the augmentation is slightly flawed (numbers obstructed by the added bounding box). The second rollout fails to read the correct numbers from the edited image. (in Appendix) (c). The model concludes in the first rollout that no image reasoning step is not needed, but fails to provide the correct answer. (in Appendix) (d). The model generated tool use code for but the code execution was unsuccessful, and no intermediate visual step is provided. (in Appendix)

---

Published as a conference paper at ICLR 2026
Failure Case #1
Query: What is the value of largest loE??
Shane glolal domeetle actaties idfembbers cftirie laner cotigge 100e teit praneciees
Augmented Image:
Geer
F" ret Rollout Reaponse: TROUGET 0: The question on *What is the value of the largest baz?" aeks to de:ermine the percentage corsesponding co the largest zevenue passenger k-lometers (RPK) for globsl demestic svistier in specific year, preferably 2018. From the chart, Egypt's percentage bar is the tallest, which indicates the highnst share of global domestio aviation among the seleated antities (Rgypt, Namibfa, Luxembourg)..
0:
" "'yython
inage_ withu_fouused____e focue with (inggeayiimgeeviii___eeeoo
(inage_), ["Egypt"), bbox)
hightighted suscesatully, graph, representing the highest percentage of global domestic aviation RPX among the chosen entities.
OBSERVATTON:
Baypt/s
the tallest
THOUSIT Now that the for Dgypt Cocused, I tell that the value of the largest bar is 0.013.
ANSWER: kgypt's share global domestic aviation RPK 0.018. FINAL ANSWER: 0.01. Second Rollout Response: ThOUGhT 1: After focuaing on the row labeled "sypp"' it closest
the percentage
Egypt
0.0018.
ANSWE3: The largest bar in the image is for Bgyst, which representa the larges: share of global domestic aviation REK (Revenue Passenges Kilometers) in 2018. The percentage corresponding to the lergest bar is approximately 0,0018. FINALLY, the exact ansver Egypt
Ground Truth: 0.51
5 CONCLUSION
VTool-R1 demonstrates that RFT can effectively teach VLMs to reason by interleaving textual and visual steps. By integrating visual editing tools into the RL training loop and optimizing for outcome-basee rewards, VTool-Ri enables models to learn when and how use tools to support their reasoning, without requiring process-level supervision. Extensive experiments visual reasoning tasks show that VTool-RI not only improves final reasoning task accuracy, but also equips models to generate coherent, multimodal chains of thought with intermediate reasoning steps in the image modality.
structured
Broader Impacts, Limitations and Future Directions. VTool-RI is one of the first frameworks to show that RFT can train VLMs to integrate visual reasoning steps by invoking visual editing tools and generating intermediate visual states to support their own reasoning goals. This opens up novel and promising direction for multimodal AI, enabling models to reason more effectively across modalities and potentially unlocking fundamentally new capabilities that go beyond what encoded in model parameters, especially for more tasks. VTool-RI I holds strong potential for scaling to broader range of toolsets and generalizing to more diverse datasets. However, in this work, we focus on straightforward task, selective attention in structured image reasoning, Looking forward, we expect future extensions of the VTool-RI framework to support multi-turn execution over more sophisticated and diverse tools. Beyond external tool calls, future multi-turn post-training framework could also incorporate model internal feedback (Liao et al. 2025 Wu et al. 2026) to further advance multimodal reasoning and agentic systems.
starting point.
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENTS
This research used the Delta advanced computing and data resource which is supported by the National Science Foundation (award OAC 2005572) and the State of Illinois. Delta is a joint effort of the University of Illinois Urbana-Chama Chbana-Champaign and its National Center for Supercomputing Applications.
We would also like acknowledge Bowen Jin (author of Search-R1) and Xingyu Fu (author of Refocus) for their valuable suggestions and contributions to our project.
This work was supported by the National Science Foundation grants NSF 2106592, NSF 1900875, and NSF 1411 Any results and opinions are our own and do not represent views of National Science Foundation.
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, and et al. Qwen2.5-vl technical report, 2025. URL httt:t::///axiv..rrrg httorrgg/bb////21322.....
Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. RI-v; Reinforcing super generalization ability in vision-language models with less than $3, R1-V 2025a. Accessed: 2025-02-02.
com/ Deeppee//eeeeeeeeeeeeeeeeeeeeee Agent/
Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. Research: Learning to reason with search for Ilms via reinforcement learning, 2025b.
http://axxivv... org/
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. Tabfact: A large-scale dataset for table-based fact verification. arXiv preprint 19X902990.0021::: 2019.
DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in ilms via reinforcement learning, 200 /250///aas22111.111 12948
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Openvithinker:
An early exploration to complex lanion-e reasoning via iterative eef--improveeett, 2025.
URL httt/htts///aax/t http://///a.xv..../abbbb/////// wrtg
17352
Jiazhan Feng. Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in Ilms, 2025.
URL https: /ttpii///axivv.. org/ abs /ttp///500000 250.
Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Corring, Yijuan Lu, Jianwei Yang. Dan Roth, Dinei Florencio, and Cha Zhang. ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding. In Proc. of the International Conference on Machine Learning (ICML), 2025.
URL http://wwwooo.a.......ee...a.aaa wwwpeen..cdu/aapeee////C.e...eee
seas. upenn.
pdf
Tanmay Gupta and Aniruddha Kembhavi. Visual programming:
Compositional visual reasoning without training. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pp. 19953-14962, 2023.
Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. arXiv preprint AXXv::2406004403. 2024.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models, 2025.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training llms to reason and leverage search engines with reinforcement learning, 2025. | perttps:/arrii...oo wwtp:/wwww.w.///x////ooe
00t.//55033300133363 09516

---

Published as a conference paper at ICLR 2026
Yoonsik Kim, Moonbin Yim, and Ka Yeon Song. Tablevga-bench: A visual question answering benchmark on multiple table domains. arXiv preprint arXiv:2404.19205 2024.
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq lqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup. Feryal Behbahani, and Aleksandra Faust. Training language models self-correct via reinforcement learning. In Proceedings of the 12th International Conference on Learning Representations (ICLR), 2025. URL https 7arxiv. org/ aba/2409/ 12917
Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, and David Acuna. Can large vision-language models correct semantic grounding errors by themselves? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025. URL https
//arxiv.orq// abs/
06510
Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang. Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, and Julian Martin Eisenschlos. Matcha: Enhancing visual language pretraining with math reasoning and chart derendering, arXiv preprint arXiv:2212.0900 2022.
Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton
Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. Deplot: One-shot visual language reasoning by plot-to-table translation. In Findings of the 6 Annual Meeting of the Association for Computational Linguisties, 2023. URL htRpstts::/ patss///axivvv/ /arxivivvv... wwwxivi.rgg/aooo abs 2212
10505
Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement, 2025. URL https
//axxi.org//ab///5000 06520
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019, URL https
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244 2022.
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference- free reward. Advances in Neural Information Processing 3 37:124198-1242233 2024.
OpenAl. Openai ol system card, 2024. URL
OpenAI. Gpt-5 system card. Technical report, OpenAI, August 2025. URL https://cdn
openai Sttmm/ppt--sssstemmme pd Accessed: 2025-09-24.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022. URL
https::/arxxvv http://wwwwvvo..g.....oo abs/ /bss/223...22 2t////223.0222 02155
Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv: 1508..0500.000 550.000305 2015.
Rafacl Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728 53741, 2023.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063477 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li. Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. ptttpp:://xxii...//
2402. 03300
12

---

Published as a conference paper at ICLR 2026
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv:
2024.
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, and Yu Cheng. Openthinkimg: Learning to think with images via visual tool reinforcement learning, 2025. URL BRLhstps: /ttpes//axivvvvv www.iioww.aaaaaaaaaoooa.oo.rra/a org/ abs /200. 08617
Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. Proceedings of IEEE International Conference Computer Vision (ICCV), 2023.
Fei Wang, Xingyu Fu, James Y Huang. Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image 2024.
understanding. arXiv preprint arXiv:2406.094:
Haozhe Wang, Chao Qu. Zumingg Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. VI-rethinker: Incentivizing self-reflection of vision-languae language models with reinforcement learning, 2025. URL
https://ares /arxiv org/ abs 2504. 08837
Mingyuan Wu, Jize Jiang, Haozhen Zheng, Meitang Li, Zhaoheng Li, Beitong Tian, Bo Chen, Yongjoo Park, Minjia Zhang, ChengXiang Zhai, and Klara Nahrstedt. Cache-of-thought: Shasttr-apprentice framework for cost-effective vision language model reasoning. In Christos Christodoullooollss, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 1909, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176- 332-6. doi: 10. 1//80/////1222.mmmmmmm.m manlp-main... 977]
URL httppsss/accaattooooooooooo/e/ee
Mingyuan Wu, Meitang Li, Jingcheng Yang, Jize Jiang, Kaizhuo Yan, Zhaoheng Li, Hanchao Yu, Minjia Zhang, and Klara Nahrstedt. Aha moment revisited: Are vlms truly capable of self verification inference-time scaling?, 2026. URL httph:tt////aatt wwtpxii.orggabs///00
17417
Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. 2024. URL httl:httss///axi/// wttpo/gggbbbb///aeeeee 10440
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- z00: Investigating and taming zero reinforcement learning for open base models in the wild, 2025.
URL httphttts///xxiii org.
18892)
Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. RI-vI: Learning to reason with multimodal large language models via step-wise group relative policy optimization, 2025. URL https::///// /tttpp://arxi..../
arxiv. wwt.a///.rg/bbb////e//ae aas.ab///333 12937
Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing "thinking with images" via reinforcement learning, 2025. URL
https://arxing:
abs /20...255... 14362
Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsich. R1- zero's "aha moment" in visual reasoning 2b non-sft model, 2025. https: //arxivv
org arg/abs/2503: 05132
13

---

Published as a conference paper at ICLR 2026
A
APPENDIX
LLM Usage: We use LLM for grammar checks and writing polish.
Accuracy
Toal Call Ratio
0.9
ChariQA 328
0.78
Thal Cal Rabo
0.7-
Accuracy
0.76
0.6
0.74
0.72
0.4
0.70
10
30
40
50
60
80
10
30
40
50
00
80
Stop
Stop
Accuracy
Tool Call Ratio
0.78
TableV/QA 328
0.76
Tool Call Ratio
0.9875
Accuracy
0.9850
0.72
0.9825
0.9800
0.9775
10
10
Step
Step
Figure 4: Multi-Modal GRPO w. Tool Use Training Dynamics, for 32B models
Prompts
Here are some tools that can help you. All are python codes. They are in tools.pp py and will be imported for you. You will be given a table figure: image_l and a question. Notice that you, as an AI assistant, are not good at answering questions when there are too many unnecessary and irrelevant information. You should determine which are the relevant columns to the question, and specify them in python list. You should use the given column headers. You should also determine which are the relevant rews to the question, and specify them in a python list. You should use the given row headers. You could select the tools to focus some columns rows, or mask out some columns / rows. Use whichever tool you think is more appropriate. Below are the tools in tools.py:
python
def focus_on_col wtt___ooooollmms_iiiiiaa htw.://wmmmwttthhhhhhhhhggggggg (image,
columns_to_foouss all_collmme columns_______m wtw.ooomm__oooonnnnggoooooo houndinggboxes
\"\"\"
This function is useful when you want to focus on some specific columns of the image.
14

---

Published as a conference paper at ICLR 2026
It does this by adding light transparent red highlight to the columns that need to be focused on.
For example, you can focus on the columns in a table that relevant to your analysis. Return the drawed image.
Args:
image (PIL. Image I the input image columns_to_mas mask (List a list of column names to focus on.
all__ooumns__on
(Dict [Dict]])
dictionary of bounding boxes for all columns in the image key is column name and value is the bounding box of that column. Each bounding box is in the format ('xl': x1 'yl': yi, 'x2': x2, 'y2': y2).
Returns:
(PIL. Image. Image) : the image with specified columns focused on
Example:
image Image. open ("samole_img, ipg") wwwthh_ocused_eoecm
focus_on__colmmm__itth gigh (image, ["Year", "Name" ("Year": ( ('x1': 0.1, 'yl'; 0.1, 'x2'; 0.3, 'y2's 0.9), Team": ('x1': 0.4, 'yl': 0.1, 'x2': 0.6, 'y2': 0.9), Name": ('x1': 0.7, 'yl': 0.1, 'x2': 0.9, 'y2': 0.911)
("""""
def foous__ou_Jowws.ithh.hhhhhhhhhhh (image, rows_to_focus_on, all_rows [oorrss_bounddgg_ooxsss \"\"("
This function is useful when you want focus on some specific rows of the image.
It does this by adding light transparent red highlight to the rows that need to be focused on.
For example, you can focus on the rows in a table that are relevant to your analysis. Return the drawed image.
Args:
image (PIL, Image. Image) the input image rows_to_focus_oo [List[str]] a list of row headers to focus on.
wtwrrrowwwswdnnnee boxess (Dict[Dict)), bounding boxes for all rows in the image. key is row header and value is the bounding box of that Each bounding box is in the format ('xl': x1, 'yl': yl, 'x2': x2, 'y2': y2).
dictionary of
Returns:
(PIL. Image. Image) : the image
with specified rows focused
Example:
image Image. open ("samplejimg ipg")
15

---

Published as a conference paper at ICLR 2026
focus focus_on _ocus_on_rows, otw_ooo___sw_______________s__ee with
(image, [*1972"), ["Year": {' x1': 0.1, 'yl': 0.1, 'x2': 0.9, 'y2': 0.15), "1969": ('x1 : 0.1, 'yl': 0.2, 'x2': 0.9, 'y2': 0.5), "1972": ('x1': 0.1, 'yl' 0.6, 'x2': 0.9, 'y2': 0.911)
display Lmagee \""|""
def Eocus_ on wwwooonnnooooooooooooooooooooeoo wwwh_miatttt (image, columns_te to focus_on, [al_ccolmns_ae ngg_ummss__udddd__beeeeeeeee \"("("
This function is useful when you want to focus on some specific columns of the image. It does this by masking out the columns that are not needed.
For example, you can focus on the columns in a table that are relevant your analysis and ignore the rest. Return the masked image.
Args:
image (PIL, Image. Image) the input image columns_to_mast mask (List(str)) a list of column names to focus on.
wtw.coolss_bouunde boboxes (Dict (Dict]])
dictionary of bounding boxes for all columns in the image key is column name and value is the bounding box of that column. Each bounding box is in the format ('xl': x1 'yl': yl, 'x2': x2, 'y2': y2).
Returns:
(PIL. Image. Image) the image with specified columns focused on
Example:
image Image. open ''sample_img. jpg") mmage_with__ weee_wtth_fccsse__eem____ foh_foocued__ollumt
focus_on_cooluee
( (image,
["Year", , "Name"), ("Year": ('x1': 0.1, 'yl': 0.1, 'x2': 0.3, 'y2': 0.9), "Team": ('xl': 0.4, 'yl': 0.1, 'x2':
0.6, 'y2': 0.9), "Name": ('x1:: 0.7, 'yl': 0.1, 'x2":
0.9, 'y2': 0.911)
display (image_ with cocoused_colunee \"\"("
def Eoous__o_rros__itth_nee (image, rows_to_focus_org all wtl_rowsbbounding _oL_rooo_bounddd___oxxss. \"""("
This function is useful when you want focus on some specific rows of the image.
It does this by masking out the rows that are not needed. For example, you can focus on the rows in table that are relevant to your analysis and ignore the rest. Return the masked image.
Args:
image Image Image): the input image rows_to_focus__or (List list of row headers to focus on.
16

---

Published as a conference paper at ICLR 2026
all_roos__bol howws_bounding_aae boxes (Dict [Dict]): a dictionary of bounding boxes for all rows in the image. key is header and value is the bounding box of that row. Each bounding box is in the format ('x1': x1, 'yl': yl, 'x2' x2, 'y2': y2).
Returns:
image with aatth_fooused_ rows (PIL. Image. Image): the image with specified rows focused on
Example:
image Image. open ("sample_iae iin.. jpg")
fhage_with_focuses_roo
focus_on_rows (image,
["1972"], ["Year": ('x1': 0.1, 'yl': 0.1, 'x2': 0.9, 'y2 :: 0.15), "1969": ('x1': 0.1, 'yl': 0.2, 'x2': 0.9, 1y2':
0.5), "1972": ('xl': 0.1, 'yl': 0.6, 'x2': 0.9, 'y2': 0.91])
display
\"""|"
def focus on focus_on _t__oooonnnooooeeeee_ee_e__eeeee with with__draw (image, columns to focus_on, all _al_cooumns__ _ttpiins__ouddiig_beeeee \"\")"
This function is useful when you want to focus some specifia columns of the image.
It does this by drawing red box around the columns that need to be focused on. For example, you
focus the columns in table that are relevant to your analysis. Return the drawed image.
Args:
image (PIL. Image.Inaee): the input image columns_to mask (sist a list column names to focus on.
ali_columns htwps_uounnnnnggeeeeee (Dict [Dictt]) a dictionary of bounding boxes for all columns in the image key is column name and value is the bounding box of that column. Each bounding box is in the format ('x1': x1 'yl': yl, 'x2'; x2, 'y2': y2).
Returns:
Image_with siitt__focued_coo cold__colmms (PTL. Image. Image) the image with specified columns focused on
Example:
image Image. open ("sample_ing jpg")
bogge_with_fousse__oot
= focus_on, coccus__oocollloooooooo wwwoooooolll umms (image,
["Year", "N ( ("Year": ('x1': 0.1, 'yl's 0.1, 'x2': 0.3, 'y2': 0.9), "Team": ('x1': 0.4, 'yl': 0.1, 122
0.6, 'y2': 0.91, "Name": ('xl': 0.7, 'yl': 0.1, 'x2"
0.9, 'y2': 0.911)
display (
def focus_on_rows_with_.rrr (image, rows_to_focus_ong all_rows_bounddng_booeess
17

---

Published as a conference paper at ICLR 2026
\"("("
This function is useful when you want to focus specific rows of the image.
It does this by drawing red box around the rows that need to be focused on.
For example, you can focus on the rows in table that are relevant to your analysis. Return the drawed image.
Args:
image (PIL. Image. Image): the input image rows_to_focus_en [List[str]] : a list of row headers
to focus on.
(Dict [Dict)) a dictionary of bounding boxes for all rows in the image. key is row header and value is the bounding box of that row. Each bounding box is in the format ('xl': xl, 'yl': yl, 'x2': x2, 'y2': y2).
all_rows_bounding_boe
Returns:
image_widd jocused_rows rows (PIL. Image. Image) : the image with specified rows focused on
Example:
image Image open (neen("aamee ("sampl_imm jpg")
image_with wwitth_ooused_drr rows wtw.:oonoooomssstta___a_____aaae
(image, ["1972"], ["Year":
('x1': 0.1, 'yl': 0.1, 'x2': 0.9, 'y2': 0.15), "1969": ('x1': : 0.1, 'yl': 0.2, 'x2': 0.9, 'y2': 0.5), "1972": ('
x1': 0.1, 'yl': 0.6, 'x2': 0.9, 'y2': 0.911)
display
|"("""
GOAL #: Based the above tools, I want you to reason about how to solve the # USER REQUEST # and generate the actions step by step (each action is python function call) to solve the request. You may need to use the tools above to process the images and make decisions based on the visual outputs of the previous code blocks. You should only use the tools above, you should not use other functions or code which will not be executed.
REQUIREMENTS #:
1. The generated actions can resolve the given user request USER REQUEST perfectly. The user request is reasonable and can be solved. Try your best to solve the request.
2. The arguments of a tool must be the same format specified
in # TOOL LIST #;
3. If you think you got the answer, use ANSWER: <your answer> Please extract the final answer in FINAL ANSKER:
<final answer> and ends with TERMINATE.
stored in PIL Image objects named image_l, image_2, ... image_n. You can use these images in your code blocks. Use display() function to show the image in the notebook for you too see.
4. All images in the initial user request
5. Use as few tools as possible. Only use the tools for the
cases written in the tool description. You can multiple tools in a single action.
18

---

Published as a conference paper at ICLR 2026
6. If you have multiple answers, please separate them with
Il marks. For example, if the answer is 'Alice' and 'Bo'', you should write 'Alicel |Bob'.
7. When you focus on columns in the image, most like you need to look multiple columns instead of single one.
8. If you do not think you have enough information to answer the question on the images returned by the tools, you should directly answer the question based on the original image. Below are some examples of how to use the tools to solve the user requests. You can refer to them for help. You can also refer to the tool descriptions for more information.
9. Only one turn of action, ACTION 0, is allowed. You must provide the answer after maximum one ACTION call.
EXAMPLE: Simple question that does not require any tool USER REQUEST #: <A image here> what is the title of this
table?
USER Bounding Box Info: columns_bbox where keys are column headers and values are column bounding boxes. rows bbox, where keys are headers and values are bounding boxes. The columns in the image are: ["Grade", "Mentor", "Salary"I. ("Grade", "A", "B", "c"].
The
in the image start with:
USER IMAGE stored in image_l, as PIL image. RESULT #:
THOUGHT 0: The question does not require any tcol. can see the title of the table is "Customer Information". ACTION
action needed.
ANSWER: The title of the table is "Customer Information".
FINAL ANSWER: Customer Information. TERMINATE
EXAMPLE: Focus specific columns in the image USER REQUEST #: <A image here> Who had the same game version as John Roth?
USER Bounding Box Info: columns _box, where keys are column headers and values are column bounding boxes. rows_bbox, where keys are row headers and values are bounding boxes. The columns the image are: ['Manager Name', 'Game Version', Game Score'). The rows in the image start with: [ ['Manager Name', 'John Roth', 'Alice Smith', Bob Johnson'
USER IMAGE stored in image_l, as PIL image. RESULT #:
THOUGHT 0: To identify who had the same game version as John Roth, I need to focus on the 'ame Version' column and the MMnaager Name' column. Also, I need to focus on all the rows so do not need to focus on some specific zows. ACTION 0:
"""phtonn
image_wtt withh_oocsee__cocmm columns _ocus_on_columatee withwiithd_rir_ draw (image L,, ["Game Version", "Manager Name"], columns' bboxx display https_iiet_oo_ud___e__________ columns))
OBSERVATION: Execution success. The output is as follows: <the image outputs of the code is added as the second image> THOUGHT 1: Now I can see the Game Version' column and the Manager Name" column more clearly. The game version of John
19

---

Published as a conference paper at ICLR 2026
Roth is 'v..2'. Other people with the same game version 'Alice Smith' and BBb Johnson's ANSWER: 'Alice Smith' and 'Bob Johnson' had the same game version as John Roth are. FINAL ANSWER: Alice Smith Bob Johnson. TERMINATE
EXAMPLE: Focus specific rows in the image USER REQUEST #: < image here> How many games did Josh win after 1996?
USER Bounding Box Info: columns bolumns_bbox, where keys are column names and values are column bounding boxes. rows_bbox, where keys are row headers and values are row bounding boxes. The columns the image are: ("Rank", "Year", "Score", "Month"]. The rows in the image start with: ["Rank", "O", "1", "2", "3", "4", "5"]. USER IMAGE stored in image_l, as PIL image. RESULT
THOUGHT 0: This table is about the games played by Josh that he won. I do not need to focus on any specific columns. need to focus rows have year after 1996: one has year 1997 and this row starts with "3", one has year 1998 and this "4", and one has year 1999 and this row starts with "5" So will focus on the rows with row starters "3", "4", and "5" ACTION 0:
the rows with the year after 1996. Three
starts with
"python
= hoous_onrrow__withhhgggg (image) _1, ["3", "4", "5"], rows zows\_bbox) bbox) display (image (iwage_wwth__aneeeeeeee wwwpwwitttehoeeeeoooooteeee rows)
see_wwtthffocseee
Execution success. The output is as follows: <the image outputs of the code is added as the second image> THOUGHT 1: can see that Josh won 3 games after 1996.
OBSERVATION:
ANSWER: Josh won 3 games after 1996. FINAL ANSWER: TERMINATE
EXAMPLE: Focus on specific columns and speci fic rows in the image
USER REQUEST #: <A image here> what is the sum of annual earnings after 2006?
USER Bounding Box Info: columns_bbor bbox where keys are column names and values are column bounding boxes. rows rbox_bbox, where keys are row headers and values are row bounding boxes. The columns in the image are: ["Index", "Year", "Cost", "Earning"]. The rows in the image start with: ["Index", "O", "1", "2", "3", "4", "5"]. USER IMAGE stored in image_l, RESULT #:
PIL image.
THOUGHT 0:
need to focus on the 'Year' column and the "arniing' column. I also need to focus on the rows with years after 2006. The row with year being 2006 starts with "3" So I will focus on the rows starting with "4", and "5",
ACTION
python image with www.ssee_ooluueeeeeeeeo L,, ["Year", "Earning"), columns [bbox)
www.snsuuunooooooooooooooo.....u with_mask (image
20

---

Published as a conference paper at ICLR 2026
image_with, www.whhhhhsseeeoooooooooooo focus_on wwwooo_wwwwwwooeeeoo _
wwith__ocusee_ooll coluuns,, ["4", "5"], rows bbox)
display wwwps/otcocceeoooorooooooo
Execution success. The output is as follows: <the image outputs of the code is added as the second image> THOUGHT 1 can see that the annual earnings after 2006 are $165, 498 and $198,765, The
OBSERVATION:
of the annual earnings after
2006 is $364,263.
ANSWER: The sum of the annual earnings after 2006 is $364, 263. FINAL ANSWER: 364263. TERMINATE. USER Bounding Box Info: x_values_bbos storing x values and coordinates.
iLvalues_bbox storing values and coordinates. The x values in the image are: <x_values>. The y values in the image are: vy_values>. USER IMAGE stored in image_l, as PIL image.
2. The model correctly identified the augmentation needed for the problem, but the augmentation is slightly flawed (numbers obstructed by the added bounding box). The second rollout fails to read the correct numbers from the augmented graph.
Failure Case #2
Query: Findd the two ama lest values in the givan chart and add them?
Monte bealth Eutto for aloobal deperddecyy abuse
Augmeated Inage:
First Rollout Response: THOUGET 0: To find the two smallest values, I need to focus on rows vs valves are "Generalizar anxiety disorder (GAD) and MMajor deprossion". The values are 1.6 and 1.3, respectively.
contain row headings. From provided, two smallest
ACTION 0:
"'python inage_with
foweeuiseh_fooeeedeeeeeee focused focus_on rowesoonnowwwwoooeeee wtw.:withhdwawwaaaeeeaaae drawgenggeell, ("Geseralized anxiety disorder (SAD) ", "Xajor depression"), roms_bbox)
displayytinge__tt sasage_wwch_foooeeeeee www.hhhhoooueeeooeeeoeoooeoeee
RESULT: The zows with the zow headings "Gnerallizdd anxiety disorder IGAD)' and 'Major depression' highlighted, and the of these two values is 1.6 1.0 2.6. ANSWER: The sum of tha anallest twe values iCeneralizad anxiety disorder (GAD) and Major depressicnt is
Second Rollout Response: THOUGHT 1: can see that the row labels 'Generalized anxiety disorder (GAD)" and 'Major depreastor' have the values 1,6 and 1.3. Adding these Logether gives total of 2.6.
ANSWER: The two smellest values in the chart are 1.5 and 1.0 from "Generalized anxiety disorder (GAD)' and 'Major depression', respectively. Their sum 2.6. FINAL ANSWER: 2.6.
Truth:
The model concludes in the first rollout that augmentation is not needed, but fails to provided the correct answer.
21

---

Published as a conference paper at ICLR 2026
Failure Case #3
Query: What is the pescentage of the highest blue bar?
100
Query Imago:
Fl rst Rollout Reaponse: TROUGET 0: This quest asks for the highest value of the blue baz, which indicates the peccentage of agricmlture. The blue baz cozzeeponding to sgriculture in the graph reaches its highest value in the 2020 21.28t.
ACTION 0: No action needed.
foll The highest value the blue which ind cates the share of agriculture, is in the year 2020 at 21.206.
RESULT: result
blue har, indicating share agriculture, 22.28% in the year 2020. FINAL ANSWER: 21.28.
ANSWER: highast
Second Rollout Response: None
Ground Truth: 31.01
The model generated code for augmentation but the code execution was unsuccessful.
Failure Case #4
Query:
What the everage upper income class population?
2011
Query Image:
First Rollout Reaponse: THOUGHT question
average upper income class populatien. The relevant data is in gray columns of the ber chart. need to up the population years (which 1s 5).
upper income
by the of
ACTION 0: " "'yython
wtwee_iithhooouudddcom slass"), ("Upper
- Locus_og con_oolume wwwuumnnttt__a.a with_maek (image_ ["Upper incone class": :'xl's 0.6, Tylf: 0.44, x2"; 0.3, 'y2': 0.67),
"Lower income class"; ('xl'; 0.4, fyl'; 0.44, (x21 0.3, 'y2': 0.67), *Middle
income class": 1'x1'; 6.4, tyl's 0.44, (x21s 6.6, (y2 0.67)11
for value image image_wishh Focusee columns set_children (chi1 (children-O)) vet_eiements .get_cceeennts (0) get__elemente (3) get_elements get_ggee_eeaeeeeeeeeeeeeeeeeeeee clmentss (0)
sum 1- float
(1)
average sum
Execution succeas. The image has been focused on the 'Upper income Glass* column in preparation for calculating the and average.
OBSERVATION:
THOUGAT 1: Now that we have focused on the 'Upper inceme class' celumn, we can retrieve the values. the number of years to calcuiate the average.
get the
of the valuas in the column and divide it by
class millien. Dividing this by the number of years, which la 6, gives average upper incore class population of 8.5 million.
ANSWER: total papuletion
upper
FINAL ANSKER: 8.5.
Second Rollout Respense: None
Ground Truth: 34.98
22


Reviews:
[{"review_id": "TjYOXP8rqe", "reviewer": "Reviewer_ajqX", "summary": "This paper proposes an RFT (Reinforcement Fine-Tuning) framework that supports VLM multimodal reasoning with external visual editing tool usage. It defines multimodal reasoning as a two-step rollout. The experiment in the table and chart image scenes shows that the RFT training brings improvements for baseline models.", "questions": "1. Refer to the issues raised in the weakness section.\n2. The curve of the model training process shown in Figure 3 has the risk of overfitting. A significant performance improvement was achieved after only 50 training steps. How many steps are generally trained for the model in this article?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This work extends the textual reasoning in multimodal understanding to the multimodal reasoning that involves images and text.\n2. This work successfully enables VLMs to learn to integrate intermediate visual reasoning steps into text-based chains of thought in the generated response.", "weaknesses": "1. It is not recommended to use expressions like \"the first RFT framework that trains VLMs to generate multimodal chains of thought\". I think the expression \"first\" is questionable.\n2. As shown in Figure 1, can only two-step Reasoning be performed during model training? What about during inference after model training? Why wasn't it designed as a reasoning process with a maximum number of rounds? This is more in line with the phenomenon of multiple iterations in multimodal reasoning.\n3. The experiment was insufficient. The experiment was conducted only on the two data Settings of Table Split and Chart Split, and it was only an in-domain experiment. The baseline models and benchmarks covered in the experiment are insufficient, which fails to reflect the effectiveness and universality of this method.\n4. In the results of Table 1 and Table 2, why is there a situation where the result of \"Tool Use\" is worse than that of \"Pure Run\"? For example, in Qwen2.5-VL-32B and GPT-4o.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "f4gW5U5nAE", "reviewer": "Reviewer_Bc6R", "summary": "The paper introduces VTool-R1, a framework that applies reinforcement learning finetuning (RFT) to train vision-language models (VLMs) to generate multimodal chains of thought by interleaving text and intermediate visual reasoning steps. Unlike prior work, which focuses solely on textual reasoning given images, VTool-R1 leverages Python-based visual editing tools during training, allowing the model to learn how and when to use visual steps that can improve structured visual reasoning tasks. The framework uses outcome-based rewards to incentivize improved reasoning. Extensive experiments on chart and table-based reasoning tasks show that VTool-R1 enables VLMs to “think with images”—improving performance, especially with open-source models previously unable to use tools meaningfully.", "questions": "1.  How does VTool-R1 perform when applied to less-structured images (e.g., photographs or scenes)? \n2. What are the key challenges to scaling the VTool-R1 approach to multi-turn, iterative tool use? \n3. Can you provide more detailed behavioral analysis of failure modes? eg. will the model misuse some tool?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. VTool-R1 is the first research work to demonstrate RL can train VLMs to interleave visual steps within a chain of thought using python-based visual editing tools.\n2. Strong, controlled experiments and comparisons with state-of-the-art baselines, including commercial and open-source models, show clear improvement and robust methodology.\n3. The framework and training design have potential to generalize to more diverse tools and reasoning tasks.\n4. Explanations are accessible, with step-by-step rationale for design choices, experimental setup, and limitations.", "weaknesses": "1. Single-Turn Tool Use: The model is only trained/tested with one round of tool invocation, limiting application in multi-step, interactive reasoning.\n2. Limited Task Scope: The experiments focus only on structured chart and table reasoning with a simple, small, predefined set of visual editing tools.\n3. Lack of Extensive Qualitative Comparison: The paper would benefit from further qualitative comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Eua3LAGYMi", "reviewer": "Reviewer_4Mrt", "summary": "This paper proposes VTool-R1, a reinforcement learning finetuning (RFT) framework for vision-language models (VLMs). The method aims to train VLMs to “think with images” — i.e., to generate multimodal chains of thought (CoT) by interleaving text reasoning and intermediate visual reasoning steps.\nUnlike prior work that only uses images as static input, VTool-R1 integrates Python-based visual editing tools into the RFT loop, allowing models to modify images during reasoning. The approach employs outcome-based rewards rather than process supervision, trained with a Group Relative Policy Optimization (GRPO) objective adapted from DeepSeek-R1.\nExperiments on structured visual reasoning tasks (tables, charts) demonstrate improvements in accuracy over both inference-only and non-tool-use baselines, using Qwen2.5-VL models (3B/7B/32B). The authors report that the model learns to selectively invoke visual tools to improve reasoning outcomes.", "questions": "1. How does VTool-R1 generalize to multi-turn tool use or open-ended visual reasoning beyond structured data?\n\n2. Can you provide human evaluation or ground-truth verification for visual tool correctness?\n\n3. How sensitive is performance to reward signal noise (given LLM-judge subjectivity)?\n\n4. What is the computational cost of two-stage inference and RFT rollouts relative to pure text-based RFT?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper presents a clear and technically solid framework that adapts reinforcement finetuning (RFT) to multimodal reasoning. I like that the authors focus on the gap between text-only reasoning and truly visual reasoning, and they provide a well-defined approach—integrating Python-based visual tools into the RL loop—to address it. The method is implemented cleanly and described in enough detail to be reproducible. The experiments, though limited in scope, are consistent and show that the model can indeed learn to use visual tools more strategically after RFT. Overall, the idea of teaching VLMs to “think with images” is appealing, timely, and potentially useful for future multimodal reasoning work.", "weaknesses": "From my perspective, the contribution feels somewhat incremental—the method mainly extends DeepSeek-R1–style RFT to VLMs without introducing new algorithmic ideas. The evaluation is narrow, focusing only on structured tasks like chart and table reasoning, which limits how convincing the results are for general multimodal reasoning. The reward design depends on an LLM-based judge, which is subjective, and the tool-use evaluation metric (simply checking if Python runs) doesn’t truly measure reasoning success. Some claims about being the “first” to enable multimodal chain-of-thought are overstated, given existing work such as Vision-R1 and Refocus.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ED4nT5BOQy", "reviewer": "Reviewer_JqBn", "summary": "This paper presents VTool-R1, a reinforcement learning finetuning (RFT) framework tailored for vision-language models (VLMs) that aims to teach these models to generate truly multimodal chains of thought by effectively interleaving text and intermediate visual reasoning steps using external Python-based visual editing tools. Unlike many recent RFT efforts which only incentivize text-based reasoning conditioned on images, VTool-R1 explicitly integrates visual tool invocation and allows VLMs to \"think with images,\" optimizing for outcome-based rewards tied solely to task accuracy. The work provides comprehensive experimental results across structured visual question answering (VQA) benchmarks using both tables and charts, demonstrates outperforming or on-par tool-augmented reasoning compared to strong baselines, and analyzes dynamics of tool use during training.", "questions": "1. Can the authors provide baseline comparisons and/or qualitative ablations versus directly related, recent tool-use RL works like DeepEyes, ACTIVE-O3, MM-EUREKA, and DINO-R1? This is critical to position VTool-R1’s improvements in context.\n2. Is there evidence that the proposed framework generalizes to more open-ended task domains, (e.g., beyond tables/charts) and to toolsets that provide more than highlighting/masking? Have any experiments been attempted in less-structured or multi-stage reasoning domains?\n3. What was the measured reliability of the LLM-based reward judge? Was there a manual verification of reward assignment on a holdout set?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The manuscript convincingly highlights the gap in existing VLMs' inability to reason with images in a stepwise, multimodal fashion—a limitation ignored by prior RFT works focusing on text-only COTs conditioned on static images.\n2. The paper benchmarks across challenging settings (tables and charts), leverages SOTA VLMs (Qwen2.5-VL), and includes both accuracy results (Table 1) and detailed training dynamics (Figure 3), encompassing accuracy, tool call rates, and execution success. These support the claimed advances.\n3. The VTool-R1 framework extends outcome-based RFT principles to visual tool-use, with an explicit two-stage rollout process and a policy gradient method (GRPO), as visualized in Figure 1. This design enables conditional tool invocation and reasoning directly over modified visual states.", "weaknesses": "1. The paper’s discussion of “tool use” is overly restricted to structured visual tasks (e.g., tables and charts), yet the title and abstract emphasize that the model “learns to think with images.” This framing creates an expectation of broader visual reasoning capabilities, but the actual scope is much narrower, leading to a mismatch between presentation and content. In the Related Work section, the review of prior research on tool-augmented VLMs and visual reasoning chains is present but lacks depth. In particular, the paper omits discussion of recent works on open-domain visual-language tool use, which would provide necessary context for positioning its contribution. Moreover, the paper does not analyze failure cases or qualitatively examine incorrect tool calls. Although the authors acknowledge that “tool use is not always beneficial for accuracy”, they do not provide representative failure examples. As a result, readers cannot assess when and why the proposed method fails. Finally, in the Conclusion, the authors claim that this work “is the first to teach VLMs intermediate visual reasoning steps via RFT”. However, this statement appears overstated—it gives the impression that the method is mature and widely applicable, whereas the actual experimental scope remains limited.\n2. The authors claim that this work is the first reinforcement learning framework that enables vision-language models (VLMs) to learn through intermediate visual reasoning steps combined with tool use. However, the differences from prior work are not clearly articulated. For example, existing methods such as Visual Sketchpad (Hu et al., 2024) and OpenThinkIMG (Su et al., 2025) have already introduced visual sketching or editing steps during reasoning. The core idea—integrating tool invocation and visual editing into VLMs to enhance reasoning—is not fundamentally new in the tool-augmented or multimodal reasoning community. The paper lacks a deep analysis explaining why previous approaches are insufficient and why the proposed method works better. Furthermore, the authors emphasize using outcome-based rewards instead of process supervision. Yet, the absence of process-level guidance (i.e., when to invoke a tool and how to edit an image) may turn the method into a black-box fine-tuning procedure. The paper does not sufficiently justify why this design is preferable to process supervision.\n3.The visual editing toolkit used in experiments is overly simplistic—limited to actions like highlighting, masking, or drawing boxes on rows and columns. These operations apply only to structured visual tasks such as tables and charts, yet the paper generalizes the approach as a universal paradigm for visual reasoning, which is a clear overreach from structured to real-world imagery. The datasets (VWTQ, VTabFact, ChartQA) are small-scale and relatively controlled. They do not reflect the diversity or complexity of open-domain multimodal reasoning, leaving generalization to broader settings unproven. The reward model relies on a lightweight LLM to judge answer correctness, which can be unreliable due to fuzzy matching, semantic variation, and inconsistent tool outputs. The authors themselves admit that tool-call correctness cannot be precisely evaluated, suggesting potential instability in the reward signal. Training only optimizes final answers, without explicitly supervising intermediate tool calls or visual edits. As a result, the model might simply learn to occasionally invoke a tool rather than to use tools meaningfully for reasoning, which undermines the core claim. There is a lack of thorough ablations: while Table 1 compares runs with and without tool calls, the paper does not analyze cases of tool misuse, tool failure, or scaling effects across model sizes (3B / 7B / 32B). Finally, the discussion of generalization and real-world applicability is weak. The authors acknowledge that their work focuses on structured visual reasoning, but this narrow scope significantly limits the claimed impact of the method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)