Grounded Reinforcement Learning for Visual Reasoning
Authors: Gabriel Herbert Sarch
Snigdha Saha
Naitik Khandelwal
Ayush Jain
Michael J. Tarr
Aviral Kumar
Katerina Fragkiadaki
Venue: NeurIPS 2025 poster ()
Keywords: visual reasoning
vision-language models
reinforcement learning
visual grounding
Primary area: applications
TL;DR: ViGoRL is a vision-language model trained with reinforcement learning to ground each reasoning step in image coordinates, improving performance on spatial and web-based reasoning tasks through better attention and visual verification.

Abstract: While reinforcement learning (RL) over chains of thought has significantly advanced language models in tasks such as mathematics and coding, visual reasoning introduces added complexity by requiring models to direct visual attention, interpret perceptual inputs, and ground abstract reasoning in spatial evidence. We introduce ViGoRL (**Vi**sually **G**r**o**unded **R**einforcement **L**earning), a vision-language model trained with RL to explicitly anchor each reasoning step to specific visual coordinates. Inspired by human visual decision-making, ViGoRL learns to produce spatially grounded reasoning traces, guiding visual attention to task-relevant regions at each step. When fine-grained exploration is required, our novel multi-turn RL framework enables the model to dynamically zoom into predicted coordinates as reasoning unfolds. Across a diverse set of visual reasoning benchmarks—including SAT-2 and BLINK for spatial reasoning, V$^\*$bench for visual search, and ScreenSpot and VisualWebArena for web-based grounding—ViGoRL consistently outperforms both supervised fine-tuning and conventional RL baselines that lack explicit grounding mechanisms. Incorporating multi-turn RL with zoomed-in visual feedback significantly improves ViGoRL’s performance on localizing small GUI elements and visual search, achieving 86.4% on V$^\*$Bench. Additionally, we find that grounding amplifies other visual behaviors such as region exploration, grounded subgoal setting, and visual verification. Finally, human evaluations show that the model’s visual references are not only spatially accurate but also helpful for understanding model reasoning steps. Our results show that visually grounded RL is a strong paradigm for imbuing models with general-purpose visual reasoning.

Grounded Reinforcement Learning for Visual Reasoning
Gabriel Sarch
Snigdha Saha
Naitik Khandelwal
Ayush Jain
Michael J. Tarr
Aviral Kumar
Katerina Fragkiadaki
Carnegie Mellon University
disually-grounded weapodrrrrgggggith .igggtthub...... 1o
Question: Are the baskets next to the wooden spoons?
Ours
wooden spoons, consider locations. The wooden spocns are placed on the stove, indicated by the position at (20, 650).
determino batakets
courderr near
baskats appaar posttioned further the (520, 650).
Since the baskets are not the immediate viainity of the wooden spoons, we need chack there's any other possiaie location. (500, 660) thera is bowl, but
beskel and
the wooddnn
Let's evaluate the woodoo spoons are located near other toms that might be matakon for baskets. Al (200. 700), there is containar with a lid, but il not beakel. This confirms that the basshels are noll neat to the wooden spocnns </ninc-
censwers no <lanswer>
worden spcon countertop. positroning and the spoons the image directly corresponds wooden spoons. Therefore, statement accurats. </llinkk
From image,
Vanilla GRPO
baskets
<onswer> yes </omser>>
DiT w/o CoT
Figure 1: Grounded visual reasoning enables interpretable and accurate answers. ViGoRL decomposes the task into sequence of natural language thoughts anchored in image regions. In contrast, Vanilla GRPO and SFT baselines produce ungrounded and incorrect responses.
Abstract
While reinforcement learning (RL) over chains of thought has significantly ad- vanced language models in tasks such as mathematics and coding, visual reasoning introduces added complexity by requiring models to direct visual attention, interpret perceptual imputs, and ground abstract reasoning in spatial evidence. We introduce ViGoRL (Visually Grounded Reinforcement Learning), vision-language model trained with RL to explicitly anchor each reasoning step to specific visual coor- dinates. Inspired by human visual maaking ViGoRL learns to produce spatially grounded reasoning traces, guiding visual attention to task-relevant regions each step. When fine-gree nre-grained exploration is required, our novel multi-turn RL framework enables the model to dynamically zoom into predicted coordinates as reasoning unfolds. Across diverse of visual reasoning benchmarks-inchheee SAT-2 and BLINK for spatial reasoning, V bebcch for visual search, and ScreenSpot and Visual WebArena for web-based grounding ViG consistently outperforms both supervised fine-tuning and conventional RL basclines that lack explicit ground- ing mechanisms. Incorporating multi-tu multi-turn RL. with zoomed-in visual feedback significantly improves ViGoRL's performance on localizing small GUI elements and visual search, achieving 86.4% on V Bench. Additionally, we find that ground- ing amplifies other visual behaviors such region exploration, grounded subgoal setting, and visual verification. Finally, human evaluations show that the model's ss visual references are not only spatially accurate but also helpful for understanding model reasoning steps. Our results show that visually grounded RL is a strong paradigm for imbuing models with general-purpots visual reasoning.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Visual reasoning tasks vary widely in structure and often demand different solution strategies depend- ing on the problem at hand. Some tasks are dominated by salient visual cues, such as recognizing a refrigerator centered in kitchen scene, while others, like locating a pair of scissors in cluttered environment, require sequential visual search and selective attention. Despite this diversity, most tfate-of-the-an vision-language models (VLMs) operate in an end-to-end fashion, predicting answers directly a single forward pass. These models often lack the ability to adapt their computational strategies to different tasks or to expose intermediate reasoning beyond visual attention maps. Prompt- based models, such as ViperGPT [54], VisualProg [22], and V* [66]. explicitly decompose visual tasks into sequences of subgoals or intermediate steps to improve interpretability and performance without additional training. However, such models typically generate fixed reasoning chains that do not adapt to the structure of the input scene.
Recent advances in reinforcement learning (RL) over reasoning chains have significantly enhanced the capabilities of LLMs in text-based domains [21, 57, 28], enabling them to learn diverse reasoning strategies tailored to the context. However, RL can only build upon skills or compose reasoning behaviors that are already latent in the base model's 's sampling distribution [17, 81]. For e.g., Gandhi et al. [17] has identified key cognitive behaviors in text-based domains, such as setting sub-goals, backtracking, verification, that support self-improvement under RL. Models lacking these behaviors often do not benefit from RL and must be bootstrapped via supervised fine-tuning (SFT) on curated reasoning traces before RL is run [17, 81]. However, it remains unclear whether the cognitive behaviors identified in text-based domains similarly support generalization in visual reasoning tasks.
Several recent works have attempted RL fine-tuning directly on base vision-language models (VLMs) [40, 39, 86, 55, 57, 83, 51, 4, 34], implicitly assuming RL alone can induce useful cognitive behaviors. However, our analysis reveals that such naive applications of RL typically yield abstract, ungrounded reasoning rather than richer, visually grounded cognitive behaviors (see Section 3. 1, 5.3). These findings align with prior research showing that explicitly prompting VLMs to reference spatial object locations improves performance and interpretability [66, 72], suggesting that grounding thought in spatial regions may serve as key cognitive behavior for effective visual reasoning. Thus, a critical open question arises: How can we embed useful cognitive behaviors in VLMs before applying RL to achieve robust visual reasoning?
We hypothesize that models both "see better" and "think better" when their textual reasoning steps are explicitly grounded in specific image regions, promoting more targeted and systematic cross- referencing between textual and visual information during reasoning. This hypothesis is inspired by the fact that humans systematically shift their restricted gaze to selectively gather and integrate task-relevant information when reasoning about the world [74, 25, 73]. Grounding may serve a similar role in models, functioning as spatial attention mechanism that enables accurate feature binding [60, 7, 8] and supports deictic reference [6] to simplify multi-step reasoning through localized perceptual anchoring. We move beyond prompt- based reasoning, proposing that learning to compose reasoning steps explicitly anchored in image coordinates induces structured region-level behaviors that support improved generalization in visual tasks.
Our Approach. We introduce a multi-turn RL framework for training VLMs to reason in grounded, visually- aware manner. This stands in contrast to LLM reasoning in math or code, where grounding in external input is not strictly required. Within each reasoning step, the model produces a natural language thought along with corresponding spatial grounding (i.e., an (x. location in the image). This enables it to progressively refine its attention and gather task-relevant visual information as each turn
reasoning unfolds. By incorporating multi-turn interaction into the RI. consists of one or more reasoning steps followed by a query to a visual feedback tool-the model learns to iteratively request zoomed-in views of selected regions when fine-grained visual information is required. Critically, no external supervision or explicit human- human-provided provided grounding cues are used to supervise the spatial grounding of the thought; instead, the model autonomously learns to propose and utilize spatial grounding as an internal cognitive tool.
Current methods for training VLMs to directly produce textual answers from visual inputs inherently bias them toward abstract, ungrounded reasoning, making it fundamentally difficult for RL methods alone to spontaneously discover systematic visual strategies at the region-level. To explicitly inject grounded reasoning behaviors before RL training, we employ Monte Carlo Tree Search (MCTS) to systematically stitch together independently sampled reasoning steps, generating diverse, visually-
2

---

grounded reasoning trajectories. We bootstrap the model via supervised fine-tuning (SFT) on these MCTS-consttucterd paths, thus embedding rich region-level reasoning strategies into the model. We then apply RI., through Group Relative Policy Optimization (GRPO) [50], to further reinforce grounded sequences that lead to correct answers. Finally, we introduce novel multi-turn RI. formulation with visual feedback loops, allowing the model to dynamically zoom into image regions via tool calling for more detailed visual inspection when needed. This multi-turn variant of our method improves the model's capacity to localize and reason about fine-grained visual elements.
Empirical Results. We evaluate ViGoRL across a suite of visual reasoning benchmarks, includ- ing SAT-2 [48], BLINK [15], RoboSpatial [53], ScreenSpot [10, 331, AesualWerArena [3] and V*Bench [66]. Our approach consistently outperforms existing methods on all tasks. Specifically, ViGoRL achieves substantial improvements over vanilla GRPO, with accuracy gains of 12.9 points on SAT-2 and 2.0 points on BLINK. In fine- g-aigaaiid web grounding scenarios, our method surpasses both vanilla GRPO and large-scale web-finetuned models on ScreenSpot-Pin By leveraging multi-turn RL for dynamic, zoomed-in visual feedback, ViGoRL. further improves performance on ScreenSpot- Pro, effectively localizing small elements within high-resolution images. Moreover, multi-turn RL significantly enhances visual search capabilities, allowing ViGoRL to outperform both VLM tool-use pipelines and proprietary VLMs on VeBenc, achieving an accuracy of 86.4%. On VisualWebArena, a benchmark requiring live web interaction from image inputs alone, without access to HTML, ViGoRL outperforms both direct SFT and vanilla GRPO, and surpasses the previous state-of-ther oftt for this model size, ICAL [49], despite using only visual input.
Ablation studies confirm the importance of grounding: models trained without spatial anchoring perform significantly worse. Further, we find that grounding amplifies other visual cognitive behaviors such as region exploration, goal setting, and visual verification. Human evaluations show that our model's 's reasoning is both spatially accurate and helpful to understanding the model's ''ss reasoning steps. Our results point to visually grounded RL as a strong paradigm for general-purpose visual reasoning.
2 Related Work
Programmatic Reasoning in VLMs. Vision-Languag Models (VLMs) [46, 29, 3, 9, 62, 56] excel on multimodal tasks through large-scale pretraining, but struggle with complex reasoning such as counting [47], spatial reasoning [48], and compositional understanding [59]. Prompting strategies
like chain-of-thought (CoT) [64, 32], mm-CoT [85], loT prompting [90], and Mind's Eye [68] guide models to generate explicit reasoning steps grounded in images. Methods such as V* [66], Sketchpad [26]. VisProg [22], REFOCUS [16], and ViperGPT [54] use language models to produce executable visual plans but rely frozen backbones and band-crafted prompts.
Distillation and Supervised Fine-Tuning. To robustly instill reasoning skills, supervised fine-tuning (SFT) methods train on curated reasoning trajectories. For text tasks, STaR [82] generates reasoning via few-shot prompting and selects based on correctness, while S1 [41] distills reasoning chains into smaller models. Similar approaches in VLMs include LLAVA-CoT [71], which distills CoT reasoning from GPT-4o [1], and ICAL [49], VPD [27], and Mulberry [76], which employ LLMs or MCTS to generate reasoning data. Methods like VOCOT [35] improve grounding of entities via SFT. However, distillation methods rely solely on positive examples, neglecting failed paths. RL addresses this by learning from both successes and failures, outperforming SFT alone in experiments.
Reinforcement Learning on Chains of Thought. Applying RL to chains of thought has improved reasoning in verifiable domains like math and coding [28, 57]. Farly methods [84, 43, 14, 67, 20] iteratively refine reasoning using preference-basee
approaches, while recent efforts like DeepSeek- R1 [21] and Kimi-1.5 [57] leverage online RL with outcome-based rewards. While initially believed to induce novel cognitive behaviors, analyses [17, 81, 38, 78] suggest RL primarily amplifies existing capabilities already found in the base model. Most prior work focuses on text-only domains, leaving visual reasoning behaviors largely unexplored. Visual-RFT [39] applies RL to textual reasoning in VLMs without incentivizing new visual behaviors. In contrast, explicitly ground reasoning steps visually, amplifying exploration, verification, and backtracking via learned visual interactions.
3

---

3 Preliminaries
Reasoning in Vision-Language Models. The overarching objective of our work is to improve the reasoning capabilities of vision-language models (VLMs). We consider reasoning tasks defined by dataset 'D of problem instances q,g,a"), where I is a visual input (e.g., an image), q is natural language query about the image, and a" is the correct, verifiable answer (e.g., a class label, bounding box, or discrete actions such as click [element]). The goal is to train a vision-language policy ma parameterized by that outputs a reasoning trace т consisting of sequential textual reasoning steps $1; $2...... ST culminating in an answer a. This policy factorizes autoregressively:
T
To (T I,9) = (II no(s 119,8e)) Ta I, Igq.))))
While supervised fine-tuning can teach models to mimic reasoning chains provided in training data, RL offers the potential to directly reinforce reasoning behavior sampled from the base model based on correctness or other reward signals [42, 50, 17]. Specifically, RL allows us to optimize policies over reasoning traces r that maximize expected returns based on task performance and adherence to desired format structure. Formally, the RL objective can be expressed as:
Ru..aa""> [RTW, [(T)]], s.t. T satisfies format constraints,
max 9
where the reward R(T) typically includes correctness of the final answer, and proper adherence to structured reasoning formats.
3.1 Do Current RL Recipes Amplify VLM Behaviors That Support Visual Reasoning?
has been shown that RL on chains of thought alone cannot necessarily induce new behaviors from scratch; it can only amplify chain reasoning primitives that are already present in the base model's sampling distribution [17, 81]. Do current base VLMs exhibit desirable visual reasoning behaviors and can RL amplify these behaviors to improve performance? Following previous work from LLMs [17]. we categorize the behaviors of base VLMs when tasked with the Spatial Aptitude Test [48] spatial reasoning benchmark, which requires the synthesis of evidence in multiple regions of one or multiple images to answer. Our complete analysis and experimental protocol is detailed in Section 5.3 and Appendix A6. Our analysis reveals two key insights:
Takeaway 1: Current VLMs often fail to reference fine-grained image inputs; their reasoning is largely ungrounded. Without explicit grounding, models treat vision as static context rather than actively referenced input, spending tokens on abstract thought instead of analyzing visual regions. Dwen2.55VVL-31 examines only 1.44 regions per task with minimal visual verification (0.14) and no backtracking. We provide an examples of such an output in Figure 2.
Takeaway 2: Standard RL optimization exacerbates ungrounded reasoning. RL-tuning with task- level rewards slightly increases region exploration (1.8) but eliminates visual subgoal setting (0.00) and still shows no backtracking (0.00). Figure and Appendix 19.1 illustrates typical outputs under this setup. This demonstrates that optimizing for correctness without encouraging grounded reasoning fails to instill visual reasoning skills. Furthermore, prompt engineering for broader visual behaviors is ignored during RL training (Section 5.2, Table 4).
Queetion The image
Standard CoT
Vanilla GRPO
Visually Grounded RL
video and the swcand image a trom the end low dil the camera liaaly mowe when ahocting vidke?
ctinko
<thiko
The first imago shows a room
Giion that the shair and the bod
To determine how the cancra moved, le's analypo the objoots the wage (1a. 350) while the
Imoge:
Image 2
image
image th's canthally loceled (roo. 400 The tilk the first image is more abrited skde (200.
telecistooa mounted
vait.
Imsge
The second mmge shcwes the
the ourrere must have moved
some room poedioss. This suspesta that the currers porspection
offeren:
right. pointing Image"
image. nenn, nesy. Consktenng poetionn the door 1800, 4028 the firtt image, it posiiooeet lowards right longer <rthrioo
momi
becasoe the cher's new position
and changed
suggesto wabble
the
framc in the image. the door netarton
wheer video <Ihh rico
carawar- jotated right c/kraeeers
car aerannrs rstated righti---taanne -larowerr>
car aecann rotited lefi -olmmwere
Figure 2: Without actively reinforcing visually grounded behaviors, RL collapses onto shortcuts that maximize immediate rewards at the expense of richer visual reasoning. Standard CoT and Vanilla GRPO (left and center) exhibit visually ungrounded reasoning, relying vague references scene elements (shown yellow), which often results in incorrect answers (marked in red). In contrast, Visually Grounded RL (right) explicitly references object positions, demonstrating precise spatial grounding (shown in blue) and more often producing correct reasoning outcomes (marked in green). See Section 5.3 for further analysis.

---

Why does standard RL fail here? We hypothesize that the failure mode of RL comes down two interrelated issues: (1) the initial sampling distribution of pretrained VLMs heavily biases the model toward abstract, language-basee strategies rather than region-level analysis (as shown in Takeaway I above), (2) since standard RI. provides rewards based solely on final correctness and general formatting, it amplifies behaviors that attain high rewards irrespective of how they actually maximize reward. Without actively biasing the initial policy and reinforcing visually grounded behaviors, RL naturally collapses onto shortcuts that maximize immediate rewards at the expense of richer visual reasoning. Thus, we hypothesize that encouraging explicit grounding biases the model's 's reasoning distributions toward exploring relevant visual regions, setting meaningful visual subgoals, verifying visual hypotheses, and effectively backtracking.
4 Visually Grounded Reinforcement Learning (ViGoRL)
The analysis in Section 3.1 shows that naive RL on small VLMs may degrade into ungrounded reasoning strategies, and that incentivizing grounding may improve visual reasoning behaviors that improve generalization. To incorporate explicit visual grounding into the reasoning process, we redefine each reasoning step as tuple: Th (sr, Mt)) where is textual thought and (xt, yt) anchors it to a specific image location. The full trajectory becomes T = [m.....,,y a]. This modifies the original factorization from Section 3:
(T [,9) = II 1=1
ma(me m.g. TTs(a (.g.ngr).
where each n, now includes both the reasoning step and its visual grounding. By introducing this grounding constraint, we explicitly guide the model to systematically reference specific image locations as evidence for its reasoning. This grounding incentivizes the model to iteratively explore and verify distinct visual regions, formulate and ground intermediate subgoals visually, and revisit prior regions when uncertainty or errors arise.
4.1 Our Approach for Grounded Reinforcement Learning
Building our findings in Section 3.1, we introduce comprehensive approach that directly addresses the ungrounded reasoning patterns in VLMs. We propose a two-stage pipeline to incorporate grounded reasoning detailed in the previous section: (1) warm-start supervised finetuning that biases the model toward generating structured reasoning chains with explicit spatial grounding, followed by (2) reinforcement learning that systematically refines these grounded behaviors. Finally, we extend our approach to multi-turn RL (Section 4.3.1), I), enabling fine-grained visual feedback at each reasoning step. This pipeline yields models that invest test-time compute into examining diverse image regions- precisely the behaviors our analysis showed were absent in standard VLM reasoning.
Grounded MCTS & Linearization
Warm Start
Grounded Reinforcement Learning
GAPO
(g)
Question Cole relative tedkly bear?
Supervised Fine-Tuning
witl my leath .19020
HRl eoe
Question (q): below bowls?
Chacking talase jes 200.
detevarte H40)
Answer (e"): Left
F
(1.g)
Answer (a"):
commers Ven oomawan
1
Figure Overview of the ViGoRL approach. (Left) We use MCTS with teacher model to generate reasoning chains grounded in specific image regions. (Middle) These reasoning trees are linearized and used for supervised fine-tuning (SFT) to train base model. (Right) We apply GRPO with an outcome-based reward to further refine the grounded reasoning.
4.2
Warm-Start Data Generation via MCTS
MCTS with Visual Grounding. We employ MCTS to generate grounded reasoning traces, where each node is a reasoning step nı, = (8t: y:)), anchoring thought &; to image coordinates (are, v:) (Figure 3, left panel). At each iteration: (1) Selection: nodes are traversed using UCB, prioritizing high-value, under-explored paths; (2) Expansion: the VLM samples new grounded steps (st, (F2, yt))
for unexplored nodes, each referencing a distinct image region; (3) Simulation: rollouts performed
5

---

by recursively generating visually grounded steps until a terminal answer is produced; (4) Backpropa- gation: judge scores terminal nodes, with rewards propagated up the path. This process ensures efficient exploration of promising image regions and reasoning steps. We provide additional details on our MCTS procedure in Appendix A5.5.
Why MCTS? Even large models (0wen..5-VL-7DD deployed with standard prompting explore only 2-3 regions, set few subgoals, and never backtrack (Table 5). Purely human-curated traces are costly to collect at scale, while linear rollouts cannot enforce the iterative exploration and corrective loops we desire. Empirically, distillation from such linear rollouts without MCTS leads to degraded generalization, performing worse on out-of-distribution spatial tasks after GRPO training (Table 4). In contrast, MCTS lets us systematically search the space of grounded reasoning steps, trading off exploration breadth and depth, and cheaply generate thousands of richly annotated paths that exhibit behaviors like wide exploration, through early branching to cover diverse image regions, and backtracking, by abandoning failing branches and revisiting alternatives.
Moreover, since our reasoning steps are already defined as (st, (zt, V+)) tuples, they map naturally onto MCTS nodes and transitions, making it both conceptually clean and computationally efficient.
Teacher-guided Search. We employ frozen, high-capacity teacher (Qwen2.5-VL-72) to expand each node in the MCTS tree. At node n, we prompt the teacher to either (a) generate a new grounded reasoning step s, with coordinate, or (b) emit candidate answer. We score leaf answers by correctness and backpropagate to guide tree expansion. From 1,500 prompts we derive ~30k high-quality reasoning traces-a a dataset orders of magnitude smaller than typical SFT corpora, but densely packed with exploration, verification, and backtracking behaviors.
Linearization for SFT. We then linearize selected root-to-leaf paths into two types of training examples: 1) Direct chains: successful trajectories leading to the correct answer with no detours, and 2) Corrected chains: trajectories where initial rollout fails, triggers a "wait, that seems off"" backtrack, and proceeds to the correct solution (Figure 3, left panel).
We denote the VLM finetuned on these MCTS chains as To, (Figure 3. middle). After fine-tuning, we commonly observe reasoning chains with dense visual subgoal setting, visual verification, broad region exploration, and visual backtracking, A representative trace can be seen in Appendix A9.2.
4.3 Reinforcement Learning with Opatially-Grounde Reasoning Steps
Although Teo imitates high-quality traces, does not reason optimally for new queries. We therefore apply Group Relative Policy Optimization (GRPO; 50) to directly maximize task reward while preserving fluency and grounding (Figure 3, right). More details can be found in Appendix A5.1.
Reward Design. We wish to incentivize reasoning traces that include explicit grounding coordinates, and reasoning traces that lead to the final answer. Our total reward is a weighted sum: R(T)= Amm + AukKTask where Tim encourages valid and interpretable output format, and Task captures task-specific correctness. Importantly, along with checking outputs are formatted with correct <think> and <answer> tags, we award +1 Timi only if all coordinate references are valid. We provide more details on task reward Appendix A5.2.
4.3.1 Multi-Turn Reinforcement Learning for Visual Feedback
Grounded reasoning chains generated by our MCTS procedure already push the model to look at many regions, but the visual encoder still processes the same globally resized image at every step. Fine-grained cues (small text, icons, object boundaries) are therefore blurred away, potentially limiting the benefit of additional reasoning if image region details cannot be perceived by the model. Inspired by how humans zoom in after selecting a candidate region, we let the model request a higher-resolutio crop, ot, after predicting coordinate. This "interactive microscope" supplies fresh evidence at detail level impossible to encode in the initial global view.
Multi-turn warm start: from single-turn chains to dialogs. To prepare the model for multi-turn rollouts, we convert single-step MCTS-derived reasoning traces from Section 4.2 into dialog-style traces. Given a linearized MCTS trace T = [(s), p.)....... (ST: pr): a], we convert it into a dialog:
1. At turn I, the model generates a textual thought 81+1 (tagged <think>.. </think>). The model then emits <tool_es stool_call> ("name": "crop", "arguments"" ("coordinate": p())
</tool_calls or the answer (tagged <answer>... ... </answer>), then the round terminates.

---

3. If a function call, the environment responds with or, with tag <observation> containing a w x w crop centered at p:. resized X r pixels, followed by </observation> and the loop repeats.
We fine-tune the base model on these multi-turn traces to initialize it for multi-step GRPO with visual feedback. We then apply GRPO, allowing the model to roll out full multi-turn dialogs, with crop feedback upon tool call outputs.
repeating coordinates, skipping tool use, or violating dialog structure. To mitigate these, we define a composite format reward: (1) Grammar reward "grammar: 1 if the dialog obeys strict tag automaton: <think>
Reward Design for Multi-Turn RL. Multi-turn settings introduce new failure modes:
+ </think> + <tool_call> + </tool_call> - <observation> - </observation> + repeat <answer> - </ </answer>. The dialog must end with complete </answer> and contain no malformed or out-of-order tags. (2) Diversity bonus rdiv: +0.2 for each sufficiently distinct coordinate in tool calls (2 10px from all previous), up to 4 times. Finally, this leads to the overall format reward:
Pimt
F "grammar + rdiv- Additional multi-turn RI. details can be found in Appendix A5.3.
5
Experiments
We evaluate our model, ViGoRI., on spatial reasoning and web grounding tasks, comparing against baseline and ablation variants to understand the contributions of grounding, MCTS-based warm-start supervision, and GRPO. Our results demonstrate significant gains in visual reasoning and web-based tasks when models are trained with our grounded reasoning training recipe. We investigate the following research questions:
i RQ1: How much does grounded reasoning help when evaluated on visual reasoning tasks?
RQ2: How important is each component of ViGoRL?
RQ3: What visual reasoning behaviors are amplified by grounded reasoning?
RQ4: Is the grounded reasoning accurate and interpretable?
use SAT-2 [48], sampling 32k training and Ik validation examples. The model is tasked with selecting the correct textual option, with randomized answer order to reduce position bias. For web grounding, we draw 12k (screenshot, referring expression, box) examples from OS-ATLAS [69] (4k each from mobile, web, and desktop), plus 1.5k warm start and 1.5k validation samples evenly split by domain. For web action prediction, use ICAL [49], dataset of 92 web navigation trajectories. We remove chain-of-thought and textual set-of-marks annotations focus on visual grounding. training on (image, instruction, action history) to predict the correct next action at each step. For visual search, we curate 1 1k question-answer pairs over small objects from Segment Anything [30], using GPT-4o given an object mask and scripted filtering to generate fine- fine-grained (image. question, choices, answer) tuples. Each question targets uniquely identifiable small object (<0..% of image) and tests visual discrimination (e.g., color, material) and relation questions between object pairs.
Training Datasets. For spatial reasoning,
5.1 RQ1: How much does grounded reasoning help on visual reasoning tasks?
Baselines. We compare against the following baselines:
We compare ViGoRL to baselines utilizing the same training data and base model (Qwen-2.5-VI.) used in our method, but differing in their training recipe: 1) SFT-direct: Supervised fine-tuning on our trajectory dataset using final answers only. 2) Vanilla GRPO: GRPO applied to the base model with standard rewards for <think> and <answer> formatting. and answer correctness, similar to previous work [39, 40J. Our model and all baselines decode with a temperature of 0.5 during evaluation.
1. Method Comparison:
2. General Proprietary and Open-Source Models: General-purposs vision- language models with closed-weights weights accessible through APIs [1] and open-weight models [37, 11, 12, 62, 5, 58].
VLM Tool-Using Pipelines: We compare against prompt-based models that explicitly decompose visual tasks into sequences of subgoals or intermediate steps [66, 22, 26, 87, 75, 65].
4. Web-Grounding Models: Models trained to specialize in web grounding tasks, through large- scale supervised finetuning and instruction tuning on curated web data [10, 24, 69, 36, 19, 45], reinforcement learning of chain of thought with outcome reward [40], or nauman-in-th-lloe action and chain of thought annotations collected from live webpages [49].
7

---

5.1.1 Visual Reasoning Evaluations.
Spatial Reasoning. We evaluate SAT-2 [48] validation (4,000 questions across 5 categories). BLINK [15] (depth ordering, multi-view reasoning. spatial relationships), and ROBOSPATIAL- configuration and compatibility split (228 questions real-world RGBD scenes) [53]. These benchmarks test generalization to novel environments, objects, and language configurations. Accuracy is measured via multiple-choice answer matching. We use max side length of 1 1260 pixels while keeping aspect ratio for evaluation.
GUI Understanding. For grounding evaluation, we ScreenSpot v2 [69, 10] (single-step localiza- tion), and ScreenSpot Pro [33] (high-resolution professional environments). To test small element grounding in low resolution, we additionally evaluate on ScreenSpot Pro with images downsampled to lower resolution of 1920x 1920, which we call ScreenSpot-Pyn Pro-L- (results Appendix Table A6). Performance is measured by checking if predicted coordinates fall within target element bounding [31] live web evaluation, using the visual-only configuration
boxes. We also test on where models receive sct-of-marks annotated webpage screenshots without HTML or text inputs. Task success is determined automatically by checking for specific criteria the agent trajectory (see Koh et al. [31] for details). We resize image to have maximum pixel size of 3600x3600.
Visual Search.
The visual search benchmark Bench tests fine-grained visual understanding using 191 high-resolution images from the SA-IB dataset (avg. (144x1182)) It includes 115 attribute recognition samples (e.g., color, material) and spatial relationship samples, evaluating models' ability to analyze detailed object properties and relative positions in complex scenes. We use resize image to have maximum pixel size 3600x3600.
5.1.2 Results
Table I: Performance (mean + 95% CI) on ScreenSpot and Visual WebArena. Multi-t = multi-turn RL with visual feedback.
Model
SereenSpot -V2
SereenSpot -Pro
VWA (Vision Only)
Table 2: Accuracy (mean # 95% CI) for spa- tial reasoning.
Proprietury Models
GPT-4o Claude Comp. Use
18.1
0.8
19.8 (wf
17.1
Model
SAT-2 Val
BLINK
RobaSpatial
Open source Models
Proprictary Models
Owen2-VL-78 Kimi-VL- Moni-VL-16B-Maun
1.6
2.9 (wl text)
76.2
92.8
34.5
GPT-4 Turbo
54.6
Claude Opus
44.
Web Grounding Models
Gemini Pro 1.0
45.2
SeeClick
55.1
1.1
CogAgenn-18B
Open Models
OS-Atlas-IB
71.9
3.7
LLaVA v1.6 34B
46.8
OS Allaslas:71 7B
84.1
18.9
instruciBLIP 13B
42,2
ShowL.T2R
7.7
Molmo
67.1
UGround-78
16.5
UGround-V1-7B
Comparison
UI-TARS-22
84.7
27.7
Qwen2.5VL-3B
46.1
44.4 (22.3)
(4..5)
UI-R1-3B
17.8
+ SFT direct
(8..))
46.4 (22.3)
62.3 (86.5)
ICAL-7B
8.2
Vanilla GRPO
(0..6)
(+2.))
69.7 (46.0)
ViGoRL-3b (Ours)
62.9 (+1.5)
48.5 (x2.3)
67.1 (16.1)
Method Comparison
Qwen2.5-VL-33
68.4
23.9
4,2 (+1.3)
Qwen2.5-VL-7B
(2.6
52.9 (22.3)
46.7 (19.5)
+ SFT direct
(0.6
(2201)
4.5 (21.4)
ViGoRL-7b (Ours)
67.5 (+1.5)
54.1 (42.3)
76.4 (67.5)
Vanilla GRPO
(2240)20)
(9.0()222)
4.8 (214)
ViGoRL-3h (Ours)
86.5 (+1.9)
31.1 (=2.3)
(+1.5)
Multi-t ViGoRL-3h (Ours)
86.1 (+1.9)
32.3 (+2.4)
Qwen2,5-VL-7B
73.6 (12.4)
29.0
(.1.5
ViGoRL-7b (Ours)
91.0 (21.6)
(321()23)
11.2 (42.1)
Grounded reasoning improves spatial accuracy. As shown in Table 2, on variety of spatial understanding benchmarks, ViGoRL significantly outperforms baselines without grounded reasoning. On SAT-2 test set, ViGoRL-3B achieves 62.9% accuracy- an improvement of +16.8 points over the base model and +12.9 points over Vanilla GRPO. Similar trends hold on the out of distribution benchmarks of RoboSpatial (67.1 1%) and BLINK (48.5%), with ViGoRL-3B outperforming SFT Direct by 2.3% and 4.8% on BLINK and RoboSpatial, respectively, and vanilla GRPO by 2.0% on BLINK. At the 7B scale, ViGoRL-7B further improves SAT-2 accuracy to 67.5% demonstrating the
8

---

method's scalability and effectiveness across model sizes. We observe the same trend in BLINK and RoboSpatial, with improved accuracies of 54.1% and 76.4%, respectively. Importantly. these gains come without sacrificing general visual-language capabilities, our method maintains performance on standard ofiit--distiibuttion VI.M benchmarks (Table A7).
Grounded reasoning helps localize complex web elements. We evaluate performance on vision- only web interfaces (Table 1), where models must resolve ambiguous UI instructions via grounded image understanding. ViGoRL-3B outperforms both the base model, direct answer SFT, and vanilla GRPO across all tasks. For instance, on ScreenSpot-Pe high resolution webpage images, accuracy improves 21.8 points over the base model, 6.1 points over SFT direct, and 2.1 points over vanilla GRPO. ViGoRL-7B achieves 91.0% on ScreenSpot- V2 and 33.1% on ScreenSpot-Pree outperforming open- open-source VLMs of comparable size, including Atlas and UGround variants, which finetune on order of magnitudes more GUT grounding examples (up to 13M training samples).
requiring grounding of small elements in
ViGoRL improves accuracy on live visual web evaluation. On Visual WebArena, which requires live interaction with web pages using only set-of-marks image inputs (no access HTML or underlying text), ViGoRL outperforms direct SFT and vanilla GRPO. Despite relying solely on images. ViGoRL surpasses the previous state- Shfate-offiee the-art for the same model size. ICAL [49], by eve% though ICAL has access to textual set-of-marks inputs derived from HTMI..
Table 3: Accuracy (mean +
95% CI) for visual search on V BeBnch..
Model Name
V Bench
Proprietary Models
Gemini- Pro
GPT-4V
$5.0
GPT-4o
66.0
Multi-turn RL with visual feedback improves visual search and small element detection. On V Bench,, ViGoRL-7B sig- nificantly outperforms both proprietary models and open-source tool-using pipelines. As shown in Table 3, our method reaches 86.4%, surpassing proprietary VLMs like GPT-4o (66.0% (6600) and Gemini-Pro (48.2 (48..%). and even advanced tool-using pipelines like
VLM Yool-Using Pipelines
VisProg
VisualChaGPT
37.6
MM-Reaet
41.4
SketcheadGGPT-o
80,3
IVM Enhanced GPT-4V
81.2
Open soume Models
LLaVA-1.5-7B LLLVA-1.6-13B SEAL
48.7
VisProg (41.4%) Sketchpad-Ce -GT-444 (80.3%).
61.8
74.8
Our model with multi-turn RI that incorporates zoomed-in visual feedback additionally shows significant improvements in small el- ement grounding tasks. It achieves 32.3% accuracy on ScreenSpot- Pro, 1.2 percentage point improvement over our best non-visual feedback model variant (31.1 %) (Table 1). In low-resolution en- vironments, the zooming capability delivers even more substantial relative gains, outperforming our best non-visual feedback method by 2.4% (Table A6). These gains illustrate that our approach enables more robust and precise visual reasoning through structured visual grounding and dynamic visual feedback interactions.
Qwen2.5 7B. VI. Multi-t ViGoRL-7B
(+3.0)
86.4 (+2.7)
Qwen2,5-3B-VL
74.2 (13.1)
ViGoRL-3B (Ours)
79.1 (43.0)
Multi-t ViGoRL-3B
81.2 (+2.8)
w/o diversity reward
78.0 (+3.0)
w/ bounding box outputs
81.2 (12.8)
5.2 RQ2: Ablation Studies
Table 4: Ablation results on SAT-2 Val and BLINK. Top-1 accuracy (+95% CI) with relative change. "Model never produced correct formatting. Gnd = Explicit grounding in the reasoning steps. SFT SFT Direct pretraining. Distill = warm-start teacher distillation.
Variant
GRPO
MCTS
Gnd
Distill
SAT-2
BLINK
Full (ours)
62,93 (+1.50)
48.50 (=2.33)
GRPO
58.83 (3)
44,97 (42.32)
MCTS
N/A"
NIA
Grounded
58.69 (-4
45,44 (+2.32) (-3.06%)
Distill w/o MCTS
63.28 (+1.49) (4).335)) (+0.35%)
46.18 (-..3) (-2.32%)
SFF direct pre-train
(21.4) (-0.33%%)
48.56 (a2.33) (+0.06%)
To understand which components drive performance, conduct targeted ablations (Table 4):
1. GRPO remains important. Removing GRPO reduces performance by 4.1 points on SAT-2 and
3.5 points on BLINK, confirming the importance of RL refinement.
2. MCTS-generated warm-start is essential. Without structured traces from MCTS, the model fails to emit valid outputs, underscoring the need for scaffolded learning signals.
3. Explicit grounding helps. Running our same method without introducing explicit grounding in the warm start data resulted a 3.4 point drop on BLINK, even with the same MCTS warm start and GRPO recipe.

---

4. Teacher distillation without MCTS preserves in-distribution performance but degrades out- of-distribution generalization. When using warm-start data from successful teacher linear rollouts (which contain grounded reasoning steps but no search), SAT-2 validation accuracy remains nearly unchanged (63.26% vs. 62.93%). However, performance on the more challenging cittof-disttibutioo BLINK benchmark drops significantly by 2.32%.
5. SFT direct pretraining adds little. Adding an additional SFT stage to directly predict the answer before warm start training and GRPO yields marginal gains (<1 point), reinforcing the idea that grounded thought processes drive improvement.
6. Pointing is sufficient for visual search. As shown in Table 3, we observe no difference in V*Bench accuracy when switching to training ViGoRL to output variable size bounding box crops as opposed to the fixed-size point cropping used in ViGoRI.. Fixed crops usually provide enough context and resolution for most cases, and ViGoRL. can handle larger regions through multiple overlapping crops that together capture necessary information. Adaptive cropping could be more efficient but is limited by imprecise bounding boxes and inconsistent aspect ratios.
5.3 RQ3: What visual reasoning behaviors are amplified by grounded reasoning?
Following previous work on behavioral coding
Table 5: Average visual behaviors per example.
in language models [17], we code VLM behav- ior when tasked with the SAT-2 spatial reason-
Regions
Grounded
Visually Verify
Model
Explored
Subgoals
Backtrack
Acc.
ing benchmark, which requires synthesizing evi-
Owenz. 5. VL-72B Shot)
dence across multiple image regions, in two sce-
Standard CoT
2.3
0.27
0.27
0.00
0.65
narios: (1) Zero-shot prompting with "think step-
Qwen2.5.VT. (Zero-Shot)
by-step" instructions without explicit ground- ing, and (2) RL-tuned models including vanillo GRPO, ViGoRL, and ablated version without explicit grounding incentives. We quantify rea- soning behaviors on 300 representative SAT-2 samples using GPT-4o evaluation, measuring visual regions explored, subgoal setting, verification, and backtracking. Additional study details can be found in Appendix A6.
Standard CuT
1.44
0.07
0.14
0.00
0.38
Qwen2.5- VLL-3B (RL-dd
Vanilla GRPO
1.8
0.00
0.17
0.00
0.48
Ours
3.5
1.1
0.39
0.47
0.64
who grounding
1.7
0.02
0.27
0.02
0.58
Findings. As shown in Table 5, explicit grounding substantially amplifies visual reasoning behaviors. ViGoRL explores more visual regions (3.5 vs. 1.44/1.8 1.8 in zero-shot/vanilla GRPO) and demonstrates dramatic increases in grounded subgoals (1.1 vs. 0.07, 15x higher) and visual verification (0.39 vs. 0.14, 3x higher). It uniquely develops visual backtracking behavior (0.47) absent in all baselines. These enhanced behaviors enable our 3B parameter model to achieve accuracy (0.64) comparable to the 72B model (0.65). The ablation study confirms these benefits stem specifically from explicit grounding incentives, as removing them causes substantial regression in all measured behaviors, particularly regions explored (1.7 vs. 3.5) and grounded subgoals (0.02 vs. 1.1).
5.4 RQ4: Human Evaluation Shows Accuracy and Helpfulness of Grounded Reasoning To assess our model's grounded reasoning traces, we conducted a human study evaluating whether pre- dicted coordinates (I) correctly referred to the in- tended image region, and (2) helped participants un-
Referent Aucuracy
derstand the associated reasoning step (details are shown in Appendix A7).
0.6
Relgbince
0.4
Findings. As shown in Figure 4, 72.8% of predic- tions were judged as accurately referring to the de- scribed region (95% CI: [66.8, 78.7]. N=20). On a 5-point Likert scale, participants rated the helpfulness of the highlighted region at 3.35 on average (95% CI: [3.03, 3.68), N=10). Helpfulness increased substan- tially when the prediction was correct (3.81; 95% CI: [3.51, 4.10J) and dropped when incorrect (2.26; 95%
0.0
Hopfuiness
pepliinss (Conect)
Helffnnss (Irommett
Accurscy
[Ovensl)
Figure 4: Human evaluation of grounded rea- soning. Participants judged the grounded pre- dictions as both accurate and helpful when correct.
Cl: [1.57, 2.94]).
These findings indicate that accurate spatial grounding meaningfully improves human interpretability and usefulness of the model's 's reasoning process, indicating that improvements in accuracy of reasoning step grounding can also improve human interpretability.
10

---

Acknowledgments. This material is based upon work supported by National Science Foundation grants GRF DGE1745016 & DGE2140739 (GS), ONR award N00014-23-1-246 AFOSR Grant FA9550-23-1-0257, and DARPA No. HR001 12490375 from the U.S. DARPA Friction for Account- ability in Conversational Transactions (FACT) program. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the United States Army, the National Science Foundation, or the United States Air Force.
This research project has benefitted from the Microsoft Accelerate Foundation Models Research (AFMR) grant program through which leading foundation models hosted by Microsoft Azure along with access to Azure credits were provided to conduct the research.
References
[1] Openai, gpt-4 technical report. arXiv preprint arxiv:2303.08771 2023.
[2] Arjun Akula, Varun Jampani, Soravit Changpinyo, and Song-Chul Zhu. Robust visual reasoning via language guided neural module networks. Advances in Neural Information Processing Systems, 34:1 1041-0141155 2021.
[3] Jean- Lean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35:
23716-23736, 2022.
[4] Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay. Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-reasonl: From physical common sense embodied reasoning. arXiv preprint arXiv:2503. 2025.
[5] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song. Kai Dang, Peng Wang. Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar Arv: 12002.1399... 13923, 2025.
[6] Dana H Ballard, Mary M Hayhoe, Polly K Pook, and Rajesh PN Rao. Deictic codes for the embodiment of cognition. Behavioral and brain sciences, 20(4):723-74) 742, 1997.
[7] Nicholas Budny, Kia Ghods, Declan Campbell, Raja Marjieh, Amogh Joshi, Sreejan Kumar, Jonathan D Cohen, Taylor W Webb, and Thomas L Griffiths. Visual serial processing deficits explain divergences in human and vlm reasoning. arXiv preprint arXiv:2509.25142 2025.
[8] Declan Campbell, Sunayana Rane, Tyler Giallanza, Nicolò De Sabbata, Kia Ghods, Amogh Joshi, Alexander Ku, Steven M Frankland, Thomas Griffiths, Jonathan D Cohen, et al. Understanding the limits of vision language models through the lens of the binding problem. Advances in Neurul Information Processing Systems, 37:1 (433-1114660) 2024.
[9] Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Bureu Karagol Ayan, Carlos Riquelme. Andreas Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. Pali: A jointly-scaled multilingual language-iman model, 2023. URL https::/ //aai///a.ii//////////a/eeeeaaa org/ wtwp/2009.0... .67994.
[10] Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401. 10935, 2024.
[11] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023. URL https:://///. wwwaaaii...gabbbaaaaaaaa 2305. 06500.
11

---

[12] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for ofte-of-the-art vision-language models. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 91-104, 2025.
[13] David Ding, Felix Hill, Adam Santoro, Malcolm Reynolds, and Matt Botvinick. Attention over learned object embeddings enables complex visual reasoning. Advances in neural information processing systems, 34:91 12-9124, 2021.
[14] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models, 2025. URL https://aaxxii http://aaxvv.oraaaaaaee wtwgg///////gge 2441.1443..
[15] Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang. Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. arXiv preprint arXiv:2404.12396 2024.
[16] Xingyu Fu, Minqian Liu, Zhengyuan Yang. John Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, and Cha Zhang. Refocus: Visual editing a chain of thought for structured image understanding, 2025. URL https / /twpiv.xvvvv/////////////a/a/ org/ aaaaabs///500 05452.
[17] Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable sepr-impgo reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.00 30000 2025.
[18] Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Utkarsh Tyagi, Oriol Nieto, Zeyu Jin, and Dinesh Manocha. Visual description grounding reduces hallucinations and boosts reasoning in Ivims. In The Thirleenth International Conference on Learning Representations.
[19] Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world humans do: Universal visual grounding for GUI agents. In The Thirteenth International Conference on Learning Representations, 2025. URL htttss//opeerrviit
[20] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama 3 herd of models. arXiv preprint 2077:2240.. 2024.
[21] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
[22] Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training, 2022. URL https: attps://arxiv> /twpiv..xv////////////.//./.aa org/ http//ass/////aaaa/e. 11559.
[23] Stevan Hamaad. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3): 335-346, 1990.
[24] Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14281-142999 2024.
[25] David Hoppe and Constantin A Rothkopf. Multi-step planning of eye movements in visual search. Scientific reports, 914):144 2019.
[26] Yushi Hu, Weijia Shi. Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. arXiv preprint arXiv:2406. 09403, 2024.
[27] Yushi Hu, Otilia Streteu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, and Ariel Fuxman. Visual program distillation: Distilling tools and programmatic reasoning into lisoun-language models, 2024. URL https: //arxiv. org/ /bs/ 2312. 03052.
12

---

[28] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed EI- El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey, al. Openai ol system card. arXiv preprint arXiv:2412. 16720, 2024.
[29] Chao Jia, Yinfei Yang. Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe Le, Yun- Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-langee lainu-la representation learning with noisy text supervision. In International Conference Machine Learning, pages 4904-4916. PMLR, 2021.
[30] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan- Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015-4022 4026, 2023.
[31] Jing Yu Koh Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po- Yu Huang. Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualweburena: Evaluating multimodal agents on realistic visual web tasks. arXiv preprint arXiv:2401.13649 2024.
[32] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023. URL htttps://arae wwwpxiii...rr..eooooooooo ab///2//22255 11916.
[33] Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma. Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use, 2025. URL https://1ikaaxxn22000 .iggttthhhb io wttp//apes//eeeee Sttpperrs//creestt_PPr. pdf. Preprint.
[34] Xinhao Li. Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-rl: Enhancing spatio-temporal perception via reinforce- ment tne-tuning, 2025. URL https: /tw.iv..xivvv//////////////aa orggobga//. /504. 25w.//694.........
[35] Zejun Li, Ruipu Luo, Jiwen Zhang, Minghui Qiu, Xuanjing Huang, and Zhongyu Wei. Vocot: Unleashing visually grounded multi-step reasoning in large multi-modal models, 2025. URL https://arxiv: org/ abs/2405 16919.
[36] Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou. Showui: One vision-language-action model for gui visual agent. arXiv preprint arXiv:2411 1.1111..14..... 2024.
[37] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. 2023.
[38] Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective, 2025. URL https: / //arxiv. wwwaiii.rrg/eooeeeeo..ooeeoooore abs /2503. 27703.2778........
[39] Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning, 2025. URL https://aaxxxi org/ abs/2503.
[40] Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, and Hongsheng Li. Ui-rl: Enhancing efficient action prediction of gui agents by reinforcement lcarning. arXiv preprint arXiv:2503.
2025.
[41] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL https: httpt:://aaat http::::///xxxv.vor// aaaaaaa//550 19393.
[42] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedhack. Advances Neural Information Processing Systems, 95:27730-27744, 2022.
13

---

[43] Pranav Putta, Edmund Mills, Naman Garg, Sumeet Motwani, Chelsea Finn, Divyansh Garg, and Rafael Rafailov. Agent q: Advanced reasoning and learning for autonomous agents, 2024. URL.
org abs/ 220//440..000 07199.
[44] Qi Ming Ding. Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong. Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, et al. Cogcom: A visual language model with chain-of siti--o--maipuaationn reasoning. arXiv preprint arXiv:2402.002236, 2024.
[45] Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, al. Ui-tars: Pioneering automated gui interaction with native agents. arXiv preprint arXiv:2501.12526, 2025.
[46] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 7748-8763. PMLR, 2021.
[47] Pooyan Rahmanzadchgervi. Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind: Failing to translate detailed visual features into words, 2025. URL https: //apxiv////xvv...// org/ abs/2407 06581.
[48] Arijit Ray, Jiafei Duan, Ellis Brown, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A. Plummer, Ranjay Krishna, Kuo-Hao Zeng, and Kate Saenko. Sat: Dynamic spatial aptitude training for multimodal language models, 2025. URL https: //ta::////xvvii org/ abs/ /412. 07755.
[49] Gabriel Herbert Sarch, Lawrence Jang, Michael J Tarr, William W Cohen, Kenneth Marino, and Katerina Fragkiadaki. Vlm agents generate their own memories: Distilling experience into embodied programs of thought. In The eiirth- Annual Conference on Neural Information Processing Systems, 2024.
[50] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://urxiv. hatpbb///422.330000
[51] Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-rl: A stable and generalizable r1-style large vision-language model, 2025. URL https://arxiv org// abs/2504. 07615.
[52] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang. Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu, Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409. 19256, 2024.
[53] Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. RoboSpatial: Teaching spatial understanding to 2D and 3D rision-languag models for robotics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. To appear.
[54] Didac Surís. Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. arXiv preprint wxXh:23033.88128 2023.
[55] Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv e-prints, pages arXiv-2503, 2025.
[56] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, and et. AL. Gemini 1.5: Unlocking multimodal understanding across millions of tokens 05530.
of context, 2024. URL https: : / /aaxivv..vv//////////a/////aaala org/
[57] Kimi Team, A Du, Gao, Xing, C Jiang, C Chen, C Li, C Xiao, Du, C Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms, 2025. URL https://arxiv, 0rg/abs//501.12559,
14

---

[58] Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, and et. Al. Kimi- VL technical report, 2025. URL https: org/abs abb.//544... 07491.
[59] Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio-linguistic compositionality. 2022. URL haapps///aaxxii wwwg/ogg////.ooooooeo.o.o.o. awwg//bb/220...... 03162.
[60] Anne M Treisman and Garry Gelade. A feature-integration theory of attention. Cognitive psychology, 12(1):97-136, 1980.
[61] Shimon Ullman. Visual routines. In Readings in computer vision, pages 298-328, Elsevier, 1987.
[62] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model's perception of the world at any resolution. arXiv preprint 240112149,.1111, 2024.
[63] Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang. Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. Ragen: Understanding self-evolution in Ilm agents via multi turn reinforcement learning, 2025. URL httph:ttt///aa org/ 2aabs/2500 2504. 20073.
[64] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of Shain-of-thouglee thought prompting elicits reasoning in large language models. Advances neural information processing systems, 35:24824-2438 24837, 2022.
[65] Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models, 2023. URL https://arxi:: org/ 2203. 22/.//3333......... 04671.
[66] Penghao Wu and Saining Xie. V*: Guided visual search as core mechanism in multimodal llms. arXiv preprint 13X1v:2312.14135, 2023.
[67] Tianhao Wu, Janice Lan, Weizhe Yuan, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. Thinking Ilms: General instruction following with thought generation, 2024. URL https:
//arxiv. //g//bb//44000
[68] Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui, and Furu Wei. Mind's ' eye of llms: Visualization-off tfuulizztion-of-thhuhht elicits spatial reasoning in large language models, 2024. URL https: /tte:///arzzz /arxiv. org/ /a////////// 2bbs244h /2404. 22604.03622.
[69] Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, et al. Os-atlas: A foundation action model for generalist gui agents. arXiv preprint arXiv:2410.232188 2024.
[70] xAI. Grok-1.5 vision preview. httpp::/z..ai/blogg grok- 1.0k-1.5v 2024. Accessed: 2025-05-
21.
[71] Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025. URL
org/abs/ /rg/11 /ttp///2......... 10440..
[72] Jihan Yang, Shusheng Yang. Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprint arXiv:2412.1417/ 2024.
[73] Scott Cheng-Hsin Yang, Mate Lengyel, and Daniel M Wolpert. Active sensing in the catego- rization of visual patterns. Elife, 5:e12215 2016.
[74] Scott Cheng-Hsin Yang, Daniel M Wolpert, and Máté Lengyel. Theoretical perspectives on active sensing. Current opinion in behavioral sciences, 11:100-108, 2016.
15

---

[75] Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. 2023.
[76] Huanjin Yao, Jiaxing Huang. Wenhao Wu, Jingyi Zhang, Yibo Wang. Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. Mulberry: Empowering milm with ol-like reasoning and reflection via collective monte carlo tree search, 2024. URL. https://asss /arxiv.aa//vvv/./a//// org/ absa abs/2412. 18319.
[77] Alfred L Yarbus. Eye movements and vision. Springer. 2013.
[78] Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in Ilms, 2025. hotLhttss:/aree org/ aalaaba//5// 2200/2202..0 03373.
[79] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, al. Dapo: An open-source Ilm reinforcement learning system at scale, 2025. URL. http::/aaxxiv. org/abs/2503. 14476.
[80]
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng. Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi, 2024. URL https: /ttp///aaavvv..eee/aeeeeeeeaaele wwwzziio.rrr///oo org/ abs/ 2211 16502.
[81] Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model?, 2025. URL https://aaaat http//xxvvvoor//e/ org h/tg//b///// hb/////ss./..../////// 13837.
[82] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning, 2022. URL https: : /arxiv...v www.ooog///////ee..oooooooe
[83] Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, and Sergey Levine. Fine-tuning large vision- language models decision-making agents via reinforcement learning, 2024. URL https: //tr:://////ii org/ abs/ 2405.10292.
[84] Xuan Zhang, Chao Du Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. Chain of preference optimization: Improving cfain-of-thought reasoning in 1lms. Advances in Neural Information Processing Systems, 37:333-355 2024.
[85] Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Mul- timodal chain-of-thought reasoning in language models, 2024. URL https: org/ abs/2302.0093 00923.
[86] Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, and Wenwu Zhu. Embodied-r: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning, 2025. URL https://arrae //ap://///xivv/// org/ abs /204. 12680.
[87] Jinliang Zheng, Jianxiong Li, Sijie Cheng, Yinan Zheng, Jiaming Li, Jihao Liu, Yu Liu, Jingjing Liu, and Xianyuan Zhan. Instruction-guided visual masking. arXiv preprint arXiv:2405. 19783, 2024.
[88] Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational 13372.
Linguistics. URL pttt:://axxiae org/
[89] Yaowei Zheng, Shenzhi Wang Junting Lu, Zhanghhi Feng, Dongdong Kuang, and Yuwen Xiong. Easyrl: An efficient, scalable, multi-modality rl training framework, ttttps//gitthnn tiggthh..... com/ hiyouga/EasyR1, 2025. 2025.
16

---

[90] Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu. Siyang Gao, and Yue Zhang. Image-of- thought prompting for visual reasoning refinement in multimodal large language models, 2024.
URI. httphtttss/aaii...oo www.aa//////eeoo.ee/..//ee.eeee/ htw.b///2455.888788 13872.
[91] Richard Zhuang*. Trung Vu*, Alex Dimakis, and Maheswaran Sathiamorrthy. Improving multi-tum tool use with reinforcement learning. https//www.bespokclaat wttp:/.....ooooooiiiiii./ multi-turn-towe wool-use-with-reeie wethhrrreiforemet--lemamie 2025. Accessed: 2025-04-17.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state central claim. These claims supported by empirical results across multiple domains, including human evaluations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The appendix explicitly discusses limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
This paper does not present new theoretical results or proofs; all contributions are empirical and algorithmic.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide detailed descriptions of our experimental setup, model variants, training procedures, and evaluation protocols in Sections 4-6 and Appendix, ensuring reproducibility. We also release anonymized code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
19

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We include anonymized links in the supplemental material with all code, trained models, and instructions to reproduce experiments, as well as generated datasets used in experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/nips. cctppblic/guiii httpe/gggiiiee/oddeuuuiieeee wwweesubmisiiooPoo...
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Section and Appendix provide complete details model configurations, data splits, MCTS parameters, training settings, and evaluation metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We report 95% confidence intervals for all performance metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We specify GPU configuration, and estimated per-run and total training costs in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //werri/////...e/eeeee cc cctppccccppcceeeeeccccccccccccet wwwehhiihhiieeeennnneeeeeeee
Answer: [Yes]
Justification: We adhere the NeurlPS Code of Ethies throughout the research. No ethical violations were encountered.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., - if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Section and Appendix discusses broader impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
21

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: To prevent misuse, we restrict downstream deployment via usage instructions in codebase, and impact statements in the paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite and respect the licenses for all datasets and models used (see refer- ences). All third-party assets are publicly licensed.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
22

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi wwwoee com/ con/dataset has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We release documented anonymized code, with well-documented readmes for training and data. We will release all code models upon publication.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/code/mmodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We conducted user study on Prolific and include full HTML instructions, compensation details, and example screenshots Appendix. All participants were paid significantly above minimum wage.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [Yes]
Justification: Risks were minimal and clearly disclosed to participants prior to consent (see Appendix).
Guidelines:
23

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
The core method involves using vision-language models (e.g.. Qwen2-VIL
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https://neur)) /ww.rrip..cc/oooeeeeeeeeeeeeeeee cwtco/////cooco/eeeecoooooo ccc//oofeerences//22////L. for what should or should not be described.
24

---

Conclusion and Discussion
Why does visual grounding help? Our findings suggest that spatially anchoring each reasoning step forces the model to engage in more structured, human-alige -aiinndd form of cognition. ViGoRL learns to iteratively reference, inspect, and verify content specific visual regions amplifying cognitive behaviors such subgoal formulation, visual verification, and backtracking.
This model architecture mirrors insights from cognitive science: humans rely on spatial attention and visual routines to decompose complex problems into manageable, perceptually grounded steps [61, 6]. Grounding serves not merely to reduce computational load (as human spatial attention often characterized), but to scaffold reasoning with external visual structure effectively using the content of the world as part of the thinking process [77, 23]. We observe similar benefits models: spatial grounding enables better generalization, especially in out-of-distribution settings, and improves interpretability by making intermediate steps physically referable.
Rather than treating grounding as a visualization tool or auxiliary supervision signal, our results argue for it as a central architectural algorithmic principle. By training models to reason with deictic reference pointing, zooming, verifying future systems may better reflect the iterative, grounded strategies that underlie human problem-solving. This opens up promising directions for building agents that not only reason effectively, but in ways that are queryable, adaptable, and aligned with perceptual experience.
A1 Appendix Overview
The structure of this Appendix is as follows:
Section A2 contains limitations and future work.
Section A3 contains broader impacts.
Section A4 contains additional discussion and connection cognitive science.
Section A5 contains additional methods details.
Section A6 contains additional details on the model behavioral analysis.
Section A7 contains additional details on the human evaluation.
Section A8 contains additional experimental results.
Section A9 contains example model outputs.
A2 Limitations and Future Work
While our approach demonstrates strong improvements in visual reasoning performance, several limitations remain and open avenues for future research:
Intermediate Reward. Our current reinforcement learning setup provides rewards only at the final answer level. In spatial reasoning tasks, this sometimes results in reward hacking, where the model receives positive reward despite generating partially incorrect or underspecified reasoning. Although our human evaluation confirms that the majority of grounding predictions are accurate and helpful, there remains measurable room for improvement. We hypothesize that introducing dense intermediate rewards- for both correct reasoning steps and accurate grounding could mitigate these issues and improve alignment between grounding, reasoning, and reward. We leave this exploration to future work.
Expanded Tool Use and Adaptive Control. While our model can point, crop, and zoom into visual regions, other forms of aug-enede perception (e-g., highlighting. region comparison, or 3D navigation) may further support compositional reasoning. Future work could explore mechanisms that learn to balance visual actions dynamically.
Learning When and How Much to Reason. We observe that the model often generates long reasoning chains, even for relatively simple questions. This inefficiency echoes known challenges in language-only chain-of-thougle models [64, 32], where reasoning is applied uniformly regardless of
25

---

task complexity. A promising direction for future work is to develop methods that help models learn when to reason and how much reasoning is necessary, adapting the depth of their chain-of of-though to the demands of the task.
Interpreting Attention Patterns. Although our behavioral and human studies demonstrate that explicit grounding amplifies structured visual behaviors and improves interpretability, the mechanisms through which grounding improves reasoning remain underexplored. Future work should examine how explicit grounding influences a model's internal attention dynamics and whether these patterns resemble human gaze behaviors. Further comparisons to human cognitive strategies- -scch visual routines [6 J. task-directed attention [77], and deictic planning [6]-could yield deeper insights into both model behavior and human cognition.
A3 Broader Impacts
Positive Societal Impacts. Grounded reasoning improves model transparency by producing inter- pretable chains of thought that reference specific image regions. This interpretability is especially valuable in high-stakes domains such as medical imaging, accessibility tools, assistive robotics, and scientific image analysis, where understanding the basis for model decisions is crucial. Additionally, our approach may enhance human-Al collaboration by making the model's 's internal decision process legible and actionable for human users. Because the model reasons over localized visual subproblems, can better align with human intuition, which may foster trust and oversight.
Potential Negative Impacts. Despite its benefits, the ability to produce detailed visual reason- ing traces could be misused for surveillance automated profiling. A grounded model that can isolate visual elements, track reasoning over time, and justify actions may inadvertently enable more fine-grained tracking of individuals or objects in sensitive contexts. Moreover, models that appear interpretable may be perceived as more reliable than they truly are; if the reasoning trace is superficially plausible but incorrect, users may over-trust the model's 's output. This risk is particularly relevant in domains where visual ambiguity or annotation bias affects ground truth.
Our approach also assumes the availability of visual data and may propagate amplify dataset biases, especially in domains where certain object types, environments, or affordances are overrepresented. Spatial grounding mechanisms may lock the model's attention onto biased or stereotyped regions of the image, reinforcing existing disparities unless care is taken in dataset construction and evaluation.
Mitigation Strategies. To mitigate these risks, future work could incorporate uncertainty estimation confidence calibration into the reasoning trace, explicitly marking when the model is uncertain about a grounding decision. It is also important to develop tools that allow users to audit the reasoning trace and provide corrections. In addition, efforts diversify training data and evaluate model behavior across demographic and situational axes are essential for fair deployment. Finally, deployment in sensitive settings should involve human-in in-the-loop oversight, particularly when grounded outputs inform consequential decisions.
A4 Additional Discussion
Our results show that explicitly grounding reasoning steps in spatial coordinates substantially im- proves performance across visual reasoning benchmarks. To better understand why, we draw from both the recent machine learning literature and grounding as a core mechanism in human and artificial reasoning.
cognitive science theories that point to
Grounding a Cognitive Scaffold. Classic studies in visual cognition suggest that humans reason about complex scenes by sequentially directing attention to spatially localized regions in service of goal-driven subproblems. Yarbus [77] demonstrated that eye movement patterns depend heavily on task demands, suggesting a deep link between overt visual attention and internal reasoning. Ullman [61] introduced the concept of visual routines, showing that reasoning over scene involves decomposable, spatially localized operations. Ballard et [6] formalized this via deictic codes, where visual fixations and gestures act as pointers binding abstract variables to perceptual content. These
26

---

mechanisms reduce cognitive load (as opposed to raw computational load), support compositionality. and facilitate subgoal execution by grounding symbolic reasoning in visual input.
Grounding Reduces Hallucination and Enhances Generalization. In machine learning, recent studies demonstrate that grounding in visual regions curbs hallucination and promotes generalization. Ghosh et al. [18] show that large VLMs hallucinate less and reason more accurately when first prompted to generate grounded visual descriptions. Qi et al. [44] propose a saaaii-o--maniulattie method where VLMs perform spatially localized actions (e.g., zoom, crop. verify), achieving stronger generalization and interpretability. These findings echo Treisman's 's seminal Feature Integration Theory [60], which argues that spatial attention is essential for integrating visual features, and Harnad's symbol grounding problem [23], which proposes that abstract reasoning is only meaningful when connected to perceptual representations.
Object-centric and Compositional Inductive Biases. Explicit grounding may also encourage models to adopt object-centric and modular representations. Ding et al. [13] show that attention over learned object embeddings improves structured visual reasoning. Akula et [2] demonstrate that
neural modules that condition on spatial cues improve the compositional generalization of the model. These results align with the cognitive perspective that humans naturally build scene representations around individual entities and their relations [6].
Overall, the convergence of cognitive science and ML literature suggests that explicit grounding does not merely reduce the search space, but acts as structural inductive bias that enhances compositionality, verification, and goal-directed reasoning, core ingredients for generalization in both human and machine learners.
A5
Additional Methodological Details
A5.1
Group Relative Policy Optimization (GRPO)
Group Relative Policy Optimization (GRPO) [50] stabilizes policy learning from long-form tra-
jectories by computing group-normalize advantages and applying clipped token-level PPO-style updates.
Specifically, for a group of trajectories O = {- conditioned on input x, each trajectory 7()) has a scalar reward (()) = ()) GRPO computes the centered advantage A = (()) R, where R Σ, (() is the group mean.
Let ((()) be the t-th token of trajectory ,00, GRPO minimizes the following clipped surrogate loss:
G
( A min p " 11 1--, A()) + 3 [L[TT||TTel)
LCRPO =
G
Fe( (1)
where
p(
is the importance weight, = 0.2 is the clipping parameter, and B is
mا((T ( 1
the KL penalty coefficient. This approach stabilizes learning in long- horizon, multimodal reasoning settings.
A5.2 RL Reward Functions
Task Rewards. We define reward functions specific to each benchmark:
Spatial Reasoning (SAT-2). A binary reward: Tink if the predicted answer matches the ground truth, and 0 otherwise.
Web Grounding (OS-Atlas), A binary reward: Tusk if the predicted coordinate lies inside the annotated bounding box.
Web Action Prediction (ICAL). A decomposed reward: Trask Ttype - Targ. where Ttype 0.5 if the predicted action type matches, and Fang = 0.5 if the predicted argument (e.g., DOM ID, string) matches.
27

---

A5.3 Multi-turn Reinforcement Learning Details
We first apply supervised fine-tuning (SFT) on multi-turn traces. These are derived from the same MCTS chains used to train the single-turn model, but reformatted into multi-turn training data by:
(I) For each node, formulating the text in a think block followed by taking the coordinate and formulating it into tool call. it is terminal node, think block is added with any remaining text followed by the predicted answer in answer tags.
(2) Cropping around the coordinate to obtain the feedback image, and appending this image to the training data as user turn the sample be used for SFT.
(3) Continuing (1) and (2) until terminal. We additionally include backtracking as described in Section 4.2.
For multi-turn scenarios, we apply GRPO over full dialog trajectories. Observation tokens are masked from the loss, ensuring gradients flow solely through the language model while retaining visual input to the encoder. additionally mask samples not ending in an EOS token [79].
Termination Enforcement. During RL rollouts, if a dialog reaches T max 5 turns without emitting an <answer> block, we append a soft prompt to the final assistant message:
<think> Please provide your response now </think>
This maintains structural fidelity and bounds rollout length.
KL collapse. While concurrent work observed occasional KL collapse in multi-turn RL when applied to base model with vision inputs [63, 91], we found that our initial warm start enabled stable training and allowed to maintain moderate KL coefficient (0.01).
Diversity bonus. We introduced the concept diversity turn bonus into our reward formulation in Section 4.3 of the main paper. In Figure AI, we show the response length over RL training with and without the diversity bonus. Without the diversity bonus, the model quickly converges to single turn outputs (green line), whereas the model avoids this and outputs >1 turn on average with the bonus reward (blue line).
Mean Response Length (all turns)
1200
1000
800
diversity bonus reward
wio diversity bonus reward
Step
20
40
60
80
100
120
Figure A1: Response length with and without turn bonus. Without the bonus, the model converges to always taking single turn (as also verified by examining model outputs), whereas the bonus enables the model to stabilize multi-turn.
A5.4 Training Implementation Details
General Setup. Training conducted on A100 GPUs with Qwen2.5-Vi models (3B and 7B). Supervised fine-tuning uses 3 epochs, while GRPO is applied for 500 rollout-update iterations. Evalu- ation uses decoding temperature of 0.5. We build on Llama-Faad Factory [88] SFT, and EasyR1 [89, 52]
for GRPO.
28

---

Training Hyperparameters Overview single- and multi-turn RL Tables A1-A3.
We provide all training hyperparameters used in SFT, and
Table Al: Supervised Fine- Tuning (SFT).
Table A2: GRPO Training.
Table A3: Multi-turn GRPO.
Hyperparameter
Value
Hyperparameter
Value
Hyperparameter
Value
Triining steps
500
Max prompt length
4096
Epochs
Learning rale
le.6
Max response length
4096
Learning rale
1e-6
Weight decay
0.01
(includes observations)
Weight decay
0.01
Warmup
0.05
Max geueration
Warmup ratio
0.03
Optimizer
Adama (b116)
Max turns
Batch
Group size
5
Crop resize
384x384
Gradient accumulation
KL coefficient
0.01
Crop size
100x100
Effective batch size
32
Clip ralio
0.28
Learning rale
le-6
Scheduler
Cosine
Gradient clipping
Max 1.0
KT. crefficient
0.01
Precision
b116
Rollout batch size
128
Weight decay
0.01
Flash attention
fa2
Gradient step batch
Warmup
0.05
Freeze vision tower
True
Rollout engine
vLLM
Clip ratio
0.28
sequence length
Max prompt length
4096
Gradient clipping
Max norm
Doopspeed config
ZoRO Stage 3
Max response length
2048
Rollout batch size
128
Top p
0.99
Gradient step batch size
64
Temperature
1.0
Training steps
500
Filter overlong
True
Group size
Freeze vision tower
Rollout engine
Mixed precision
bl16
Temperature
1.0
Top-p
0.99
Mixed precision
bf16
Freeze vision tower
Thue
A5.5 MCTS Implementation Details
To generate high-quality reasoning traces for fine-tuning, we use Monte Carlo Tree Search (MCTS) to explore possible sequences of reasoning steps over an image and question. Our MCTS procedure extends standard tree search with VLM the generative model for reasoning steps and judge to evaluate final answers, We use the same reward as described in Section A5.2.
Search Structure. Each node in the search tree corresponds to a reasoning step (a candidate <think> or <answer> block). The root node contains the input question. Nodes maintain the text of the current step, accumulated visited coordinates, running estimate of expected reward, and visit count.
Algorithm Phases. MCTS operates via the standard four-phase loop:
Selection: Starting at the root, the search follows a path through children using the UCB policy: Q cylog N/n, where Q is the average reward, N is the parent's visit count, n the child's, and c 2.0 is the exploration constant. The search terminates when node has unvisited children or reaches terminal state.
Expansion: At each expandable node, we generate up to three children using the VLM, each corresponding to a new thought or final answer. a generated step includes terminal <answer> block, it is marked as terminal. An example prompt for this step can be found in Listing Al.
For each new child, simulate reasoning steps using the VLM until final answer is reached or a rollout depth limit is exceeded. These simulated trajectories are not added to the tree. Each rollout receives scalar reward from the judge model comparing the predicted and true answers.
Rollout:
Backpropagation: Final 0/1 rewards are backpropagated up the tree along the visited path, updating the value estimate of each node using incremental mean and incrementing visit counts. We use the same task reward as described in Section A5.2.
We include our MCTS hyperparameters in Table A4.
29

---

A5.5.1
Linearization of Search Trees into Reasoning Chains
To convert MCTS- generated search trees into training data for supervised fine- fine-tuning, we developed structured linearization procedure that extracts diverse, grounded reasoning trajectories from the tree.
Tree Traversal and Chain Extraction. Each MCTS trace is stored as tree, with nodes repre- senting reasoning steps (<think>) or final answers (<answer>). From each trace, we recursively enumerate all root-to-lcaf paths that include at least one terminal node. For each terminal node, we extract:
Correct rollouts: Paths ending in high-reward (2 1.0) final answer.
Incorrect rollouts: Lower-reward paths used to generate synthetic backtracking examples.
both correct and incorrect rollouts exist at a node, we concatenate the incorrect trace with fixed backtracking phrase (e.g.. "Wait, this seems off. Let's try something else.") followed by the corrected reasoning path, forming complete trace with embedded revision behavior. All steps are wrapped in single <think> block, and the final answer in separate <answer> tag.
Token Cleanup and Deduplication. Each reasoning trace cleaned to remove residual XML markers (<think>, <answer>) before joining, then wrapped again after concatenation. Chains are deduplicated across samples to prevent redundancy in training.
MCTS accuracy.
As shown in Table A5, we observe improved top-1 accuracy using our MCTS
procedure.
Table A4: MCTS hyperparameters for Web Grounding.
Hyperparameter
Value
Model
Doeen2.5-VT.-22R--15
Simulations per inpur
20
depth
Rollouts per node
2
Childrea per expansion
3
Cpect
2.0
Sampling temperature
1.0
Top-p
1.0
Max tokens per node
512
Estimated time per sample
Parallel processes
A6 Behavioral Analysis Protocol
To systematically evaluate reasoning behaviors in VLMs, we implement a behavioral annotation pipeline that categorizes model-generated chain-of-thought (CoT) traces using GPT-4o evaluator. This procedure is based on Gandhi et al. [17] and enables fine-grained analysis of emergent reasoning strategies across different model variants and training regimes.
Sample Selection and Preprocessing. We run the same 300 reasoning traces (obtained by randomly selecting examples from the SAT-2 validation set) per model condition where the model's final answer was verified correct Sudge_score 1). From each trace, we isolate the contents of the <think> block.
Behavioral Categories. We define four behavioral dimensions of interest:
Visual Verification: Instances where the model confirms or checks a property of the visual scene
(e.g.. "Looking at the image, can confirm...")
Backtracking: Self-correction or re- re-interpretation of previously described visual element.
Subgoal Setting: Decomposition of the visual reasoning process into smaller steps across regions
(e.g., "First will check X, then Y...")
Visual Regions Explored: Count of distinct visual regions explicitly referenced in the reasoning
trace.
30

---

Annotation via GPT-4o. For each trace, we construct four behavior-specifee prompts and submit them to GPT-40 with temperature 0.0 and a max token limit of 256. Each prompt asks the model to identify and count instances of specific behavior, outputting numeric value between custom '<count>' tags. These counts are extracted and recorded as the behavioral profile for the trace. Our full prompts are displayed in Listing A2.
Comparison Across Conditions. We apply the above process across multiple model variants (e.g.. Our full model, Naive GRPO, Ablated grounded reasoning) and aggregate the behavior counts to compute the average number of reasoning behaviors per trace.
A7 Human Evaluation Setup
To assess the interpretabilityy and spatial accuracy of the model's grounded visual reasoning outputs, we conducted a structured human evaluation study using Prolific. The study was designed to evaluate whether the (x.y) coordinate output by the model accurately corresponded to the referenced region in the reasoning step, and whether this visual cue helped participants interpret the reasoning step. We obtained 80 samples randomly from the robospatial evaluation, this evaluation contained real images without visual marks on the image.
Data curation. We take 80 samples from model from the samples. To simplify the study for the participants and target step-level analysis, we filter sentences that require context from the entire reasoning trace using GPT4o. For each reasoning step and coordinate, we draw 100x 100 pixel blue circle on the image centered at the coordinate location, and display the reasoning step text.
Robospatial, and extract each sentence
Study Design. Each trial presented participants with a single image containing blue circle annotation, along with a natural language sentence that included the phrase "[shown with blue circle]" (which replaced the (x.y) coordinate) to denote the region being referenced. Participants answered two questions per image:
Accuracy (binary + unsure): Participants were asked whether the blue circle overlapped with the region described by the sentence. Options were:
Yes if any part of the described region was inside the blue circle.
No - if the region was entirely outside the circle.
Unsure - if the sentence was ambiguous the region could not be clearly judged. Re- sponses marked as "Unsure" were excluded from accuracy and clarity score calculations to avoid inflating deflating agreement metrics with ambiguous judgments.
2. Interpretability (Likert): Participants rated how much the blue circle helped them understand the sentence reference on 5-point Likert scale:
Strongly disagree
Disagree
Neutral
Agree
Strongly agree
Interface and Instructions. Participants began the study by entering a participant ID. A detailed instruction panel introduced the task goals, decision criteria, and included two illustrative examples with images. These examples demonstrated both "Yes" and "No" cases for spatial accuracy, and clarified how to interpret the clarity rating.
Participant Recruitment and Demographies. We recruited participants via the Prolific platform. A total of 20 participants completed the study, with all submissions manually reviewed and approved. The participant pool was demographically diverse: ages ranged from 71 (mean 39.4, SD 13.6), with balanced gender representation (1 female, 9 male). Participants were based in the United States or the United Kingdom, and all reported fluency in English. Participants completed the task in an average of 22 minutes, and were compensated at rate of S12/hr.
31

---

A8
Additional Experimental Results
MCTS accuracy OS-Atlas and SAT-2 We evaluate the accuracy of our MCTS procedure using Qwen2.5- - by measuring how often it reaches the correct answer, as determined by an oracle verifier. On 123 held-out samples from OS-Atlas (Table A5), MCTS achieves an accuracy of 82. 1, indicating strong search effectiveness when guided by a reliable verifier. On the SAT-2 benchmark, MCTS reaches 96% accuracy.
ScreenSpot-Pae Low Resolution Results. We provide results on SereenSpot-Pro- in Table A6. This variant of ScreenSpot-Prr Pro has downsampled images ScreenSpot-Pro-L.R the base Qwen2.5-VL-3B model achieves only 1.96% accuracy, while SFT improves performance to 16.89%. Incorporating GRPO further increases accuracy to 20.30%, and our full ViGoRL achieves 21 ..2%%. Notably, our multi-turn ViGoRL. reaches 23.72%, demonstrating that iterative visual feedback is especially beneficial in challenging perceptual settings.
resolution of 1920x 1920. On
Accuracy on Out-of-uisse Distrriiibttion Benchmarks. To ensure that our model does not lose foundational knowledge after our pipeline, we evaluate on some popular VQA benchmarks: MMMU [80], Real- WorldQA [70], and V [66]. On MMMU and RealWorldQA, we evaluate our model's performance when prompted without thinking, thus performing
aaples-to-apple. comparison to the base model. On V*, we prompt our model think to compare to the V* visual search paradigm. On all datasets, find that ViGoRI. either matches exceeds the performance of the base model.
Splitting versus combining training datasets. To quantify the effect of combining data, we conducted an experiment combining spatial and web datasets during both warm-start and RL phases. The results, shown Table A8, reveal:
Web grounding: Small decrease (-0.2% on ScreenSpot-Pro, -0.3% on ScreenSpot-V2)
Spatial reasoning: Improvement of +1.1% SAT-2
Both approaches significantly outperform vanilla GRPO (+12-15% SAT-2)
These findings indicate that combining data has small effects on relative performance gains from our method, with slight benefits for spatial reasoning the cost of web grounding performance.
Euclidean distance reward for grounding leads to lower accuracy. As shown in Table A9, we tested Euclidean distance reward, instead of binary reward, for grounding and found it significantly reduced performance: -2.2% on ScreenSpot- V2 and -0.7% on ScreenSpot-Pro (Table R3).
Web elements vary in size, so a distance- based reward unfairly penalizes valid clicks near the edges of large elements. For instance, clicking near the edge of an 800x60 navigation bar is functionally correct but would receive low euclidean reward.
Comparison to teacher Qwen2.5VI.-72B model We show zero-shot performance of our teacher model (Qwen2.5-VLL-27 VL--2BB in Table A A10 below. Despite using only 1K examples for distillation before RL, ViGoRL-7B outperforms Qwen2.5-VL-7229 by 6.0% SAT-2 while being 10x smaller, and closes the gap across most benchmarks.
Table A5: Held-out performance of Qwen2.5-VL-725 on 123 samples in OS-Atlas validation and 958 samples in SAT-2 validation. indicates MCTS Accuracy of the hold-out MCTS set used the training pipeline.
Model
Top-1 Accuracy
Top-3 Accuracy
MCTS Accuracy
-tlas SAT-2
42.3% 61.48%
54.5% 76.30%
82.1% 96.0%
Table A6: Accuracy (mean with 95% confidence intervals) on ScreenSpot-Pro-LE benchmark.
Model
ScreenSpot-Proo.e
Owee2.5-VL-3B
1.96% (20.68)
SFT direct
16.89% (#1 85)
Vanilla
(+2.24)
ViGoRL-3b (Ours)
21.32% (+2.28)
ViGoRI.3h (Multi-turn)
23.72% (+2,39)
32

---

Table A8: Accuracy (mean 95% CT) on ScreenSpot-Pro/VV and SAT-2.
Condition
ScreenSpat- V2/Pr/
SAT-2
Qwen2.5-3B-
23.9 (s1.)) 68.4 (12.6)
46. (s1.5)
Vanilla GRPO (seperate spatial web)
29.0 (222)) 84.4 (22.0)
50.0
ViGoRL-3B (seperate spotial web)
31.1 (#23)) 16.55 (11.)
62.9
VIGoRII-3D 3B (combined spatialeweh)
30.9 (+2.3) 86.2 (+2.0)
(41.5)
Table A9: Euclidean vs. binary reward ablation. Accuracy (mean + 95% CI) on ScreenSpot- V2 and ScreenSpot-Pro.
Condition
ScreenSpot-V2
ScreenSpot-Pro
Qwen2.5 533 VI.
68.44% (#2.6)
9.3% (+1.4)
VGGoRL-3BB w/ hinary bhox reward
86.5% (+1.9)
31.1% (+2.3)
ViGoRL-3B w/ cuclidean reward
84.3% (12.0)
30.4% (42.3)
Table A7: Accuracy (mean with 95% confidence intervals) Out-of-Distribution benchmarks. * indicates that the model was prompted to think.
Model
MMMU
RealWuridQA
V Bench
Qwen2.5-VLL 5---38 Base
47.44% (23 26)
55.65% (+3.52)
74.21 (6.22)
ViGoRL-3b (Ours trained on SAT-2)
46.44% (#3.26)
60.65% (43.46)
74.87% (#6.15y: 15y*
ViGoRI.-bb (Ours trained UGround)
47.56% (+3.26)
57.52% (+3.50)
(+6. (+6.16)* 16)*
A9 Example Model Outputs
A9.1 Failure Cases in Current VLMs
Despite high image-text alignment, models like Qwen2.5-VL-7B fail often produce abstract, shallow reasoning without examining many different image regions or describing content visual depth. which often leads to errors. A representative trace is below:
<think> From the image, the twotiered metal cart is positioned closer to the modern TV stand in blue. The cart is nearer to the TV stand, which is evident from its proximity and the angle at which it is seen. Therefore, the twotiered metal cart is closer the modem TV stand in blue. </think> <answer>twotiered metal carl</and /nswer>>
A9.2 Warm Start Reasoning Traces
Fine-tuming with MCTS-generatee generated warm traces leads to structured and grounded reasoning. Example
trace:
Table A10: Comparison to Qwen2.5-72B-V. Accuracy (mean 95% CI) across benchmarks.
Model
SAT-2
BLINK
RoboSpatial
ScreenSpot-V2 ScreenSpo-PPro
Qwen2.5
7BB 7LB
61.5% (=1.5)
59.5% (+2.3)
55.7% (26.5)
88.1 (+1.8)
34.2 (2.68)
ViGoRI.7B
67.5% (+1.5) 54.1% (+2.3)
76.4% (47.5)
91.0% (+1.6)
33.1% (#2.3)
33

---

Table A11: ViGoRL error analysis by type on SAT-2. Percent of failures by category.
Error Type
Percentage
Description
Excessive Reasoning
20%
The model overthinks the problem.
Incomplete Reasoning
10%
The model truncates its reasoning process.
Hallocination
10%
The model invents visual elements.
Challenging Visual QA
30%
The question or image presents inherently subtle cues, camsing reasoning erroes.
Base Perception
30%
The underlying model makes perceptual conceptual grounding errors.
<think>
I need locate Candice Georgiadis in the list. see the name Candice Georgiadis (330, 440). Now will check the 'Follow' button next to Next to Candice Georgiadis's 's name (330, 440), 1 see 'Follow' button at (440, 440). will verify if there are any other mentions of this name. should check another area to ensure there's no other 'Candice Georgiadis', There's s another sibling list and see the name Candice Georgiadis at (660, 770) but no 'Follow' button is associated with this name. Having verified the name "Candice Georgiadis' and the 'Follow' button, and ensuring there was no other mention of the name with follow button, am confident the desired location at the first identification (440, 440).
Wait, this seems off. Let's try something else.
I need to locate Candice Georgiadis in the list. see the name Candice Georgiadis at (330, 440). Now will check for the 'Follow' button next to it. Next to the name Candice Georgiadis at (330, 440), I see button at (365, 450) that looks like says 'Follow'. I should check another name on the page for similar formatting to confirm this is the correct "Follow' button. To ensure that this indeed the correct 'Follow' button, I will check the name John Doe and its corresponding button at (230, 210). This button has the same format as the one observed for Candice Georgiadis (365, 450). It reinforces that the button next Candice Georgiadis is the one need. Lastly, to eliminate all doubt, will examine different region of the image where the 'Follow' button Fred Smith at (500, 600) located. It aligns with the same design as the others, confirming the earlier observations. Therefore, can now finalize the answer for Candice Georgiadis's 's "Follow' button. </think>
A9.3 Additional Model Output Examples
We provide example outputs on RoboSpatial questions for two models: the vanilla GRPO model trained on SAT-2 (Figure A2), and our proposed ViGoRL trained SAT-2 (Figure A3).
A9.4 Failure cases
We categorizes SAT-2 errors in Table A11. 60% arise from perceptual or task-level challenges, while 40% reflect residual reasoning flaws (e.g., overthinking, truncation, hallucination). We believe scaling or denser rewards could address aspects of both, with further discussion in Section 6 and Appendix A2.
Example of overthinking ViGoRL for simple question.
Question: "Is there pot in the image?"
Image: Image of a stove with a large pot and pan on it.
Model response:
<think>
To determine if there's a pot, I'll check the regions where pots typically appear. At (300, 600), there's pot-like object with handle and a lid, suggesting it could be pot. To confirm the presence of pot, I'll check another region. At (700, 1000), there's no pot-like object present, indicating that the previous observation was accurate. Now. I'll verify there are any other potential pots the image. At (500. 800), there's no pot-like object present, which supports the previous findings. Next, let's examine the area at (400, 900) to ensure there's no hidden pot. There's 's still no pot-like object visible here reinforcing the previous observations. I'll now check the lower region (600, 1200) to confirm there are additional pots. At (600, 1200), no pot-like object is visible here, further supporting the absence of a pot in image. Lastly, I'll check upper region at (300, 200) to ensure there's no hidden pot. At (300, 200), no pot-like object is visible here, confirming the pot is at (300, 600) in the image. </think>
34

---

Listing Al: MCTS prompt for web grounding.
helpful assistant tasked systenatically reason through the problen step by step by checking and verifying relevant xebpage regions, , In the image:\nBacl reasoning atep must be enclosed within '<think>" taga and reference exactly one specific coordinate (x, yiiasthinkkaaatSigg ressoning step grounded paint) (x, Aa<//thikk>/naaan ready provide the final encloge it within "<anaver>" tage:\n<ansver: (xI, yf) </aoover>AnYour task is to help
grounding
page.
grounding reasoning steps specific (x, y) points
described ares/elenent/bbjeee based on description... An- Generate ONLY OKE reasening step OR the final ansver response. An- Regiona left, elenents like tree/bution, zones like mackroond/Toregruum should describe the region then evaluste it for its relevance to the task and to previoua An- exploring diverse regions, even if they seen less likely, to ensure compreheneive coverage before narroving do-n. Prioritize before deciding.\n- Aim for accurate, representative points in the described area/ comant/objjct..o
identify precise (x,y)
distinct, , non overlapping areas (0-8-r quadrants
top-
An- Each step
repeat
previous stepa. Begin
coverage
candidates
unclear, infer each step examining nultiple possible solutions before selecting final coordinate. \a- Format points as (x, y)
on lixely context or purpose Verity
35

---

apacific part element of the image Listing A2: Prompts used to evaluate visual reasoning behaviors in chain-of-thouge outputs.
Language generated snalyzing inage. The chain of reasoning output from the model is: (reasoning)
is ohain-of-reseouing
any visual varification ateps. viaual verification step is when the nodel confirms or checke sonething it sees in the see the object not cat, text confirus this visual aspect it correct", "I can verify this is indeed red", inage, can confirn...." regiona and implicit verifications. Count all instances where the model verifies information frem the image and provide the tage
Evaluata whather the chain reasoning
image. Examples
dog".
"Locking
explicit
image
</count>. say visual verification steps, please provide count of as <connt>0s/ounnt>
mlai--of-reasoiin
contain
Here is chein-of-reasont that Language Nodel generated while analyzing inage.
.. The chain reasoning output from the model is:
(reasoning)
Evaluate whether the chain -ww.sorrrrnnnngg contains any backtracking bshavior, where the model changes its interpretation or corrects itaelf. Examples include: "st first I it's actually Y", "I initially
thought I, but looking interpreted this as circle, but it's actually an oval", "On second thought.... *Let Count all instances shere the model revises its understanding and provide the count tags
carefully Actually, notice that...",
ny earlier observation...
not contain any backtracking behavior, please provide count of 0 <count c0u//
</count>.
reasoning
Hare is ch reasoonnng that Language Model generated while analyzing inage.
The chain of reasoning output from the model is:
(reasoning)
Evaluate whether the chain reasoning
any visual subgosl setting, where the madel breaks doun the image analysis into smaller steps or focuses on different part, then I'll leok at that object', *Let me check each element one by one", 'I need to this area", or any structured approach analyzing different of the inage.
parta of the inage sequence. Examples include:
identify
Count all instances where the model sets up plan or appreach for smelyzing the image betueen the <eount> </count>, If the ch reeasonnngg does not contain any visual subgoal setting, please provide a count of as <count >0</cout>>.
and provide the
Here is a chain-of reasoning that a Language Model generated while analyzing an image. The chain of reasoninng output from the model
(reasoning)
Count hox many distinct visual regions or elements the nodel explicitly mentions examining in the inage. Examples include: text at the top of the image", "The object in the center appears to be...". " *Looking any
dog in the corner', "There's
the left that the model is analyzing.
visual regions tags <count> </count>. If the chain roasoninng does not mention specific visual regions, please provide count of 0 ae <coutt>0</count>
mentioned
provide
36

---

Images
Question
Base Model (SFT Direct)
Naive GRPO Model
O. Can the abble fit above the bedt Auswert yes so. Arawer with the sesi of uplion..
Thoughts: From the
posttozed
table level with the bed, the surthce bed foot higher: Therefore, tablo ftvvv the hod.
Thoughts:
Producted Answer.
Ves
Predicted
coctus di in Answer with
Since the littes are next the litter box, the cnenus con plooed front of the lnter hox because se long spece for plants, and it can aucommodate the height from the ground the cactus. But the planss are taller, the cacous will not fin in front of the liser bot.
Thoughe:
Answer
Thoughs:
option.
Predicted Answer: No
Predicted Answer: Yes
Q. Can the pan ahove the Ridget Answer yes or no. Atswer with the text the option..
bonghte: the
frigee
and wider then the stowe, allowing the to he liftal above This that the lit above the frilge Thaefire, the
Thoughtt
Predicted Anowart No
Predicted Answer.
Figure A2: Example reasoning traces from vanilla GRPO the RoboSpatial dataset, which does GRPO directly on the base model without warm start.
Images
Question
ViGoRL
Thoughts: deternnee
fridge. need hright pan space fhinge. point (00), spacs.
pon
repeesents point poim frige
Can the pan showe the Imily/AAneeer you
top of pipe provent being placedi disectly againat point 500), whtch upper appears
thidge here. mapor
upper
fridge.
prociniity
spece above refrigeraoo, shows emppyy object
placed
fhidge, space at fridge's top would pan's placement provided present.
compact
renforces
Q. cook rice yes or option..
Thoughts: The Nook pot on the stove appears suiable for cooking noo (4 (412,876) The countertop scems cleen and fine of bunters, suggestng eas access for cooking rice. (74)
The stove looks functional and includes apprograte heating elements for cooking vanous diskes, icclding rice. (61 (05) he krichen the image also has reffigenntor that might stoce necesory cooking supplcen like noee (41 5366) No vattle cooking ingredients like diee peesent the image other than the bowl, (412.876)
Tredcsed-usseees Yes
Figure A3: Example reasoning traces from ViGoRL on the RoboSpatial dataset.
37


Reviews:
[{"review_id": "xg7iYzyu1W", "reviewer": "Reviewer_vmK2", "summary": "This paper introduces ViGoRL (Visually Grounded Reinforcement Learning), a method that trains vision-language models to perform visual reasoning by anchoring each reasoning step to specific spatial coordinates in images. The authors argue that current VLMs rely primarily on abstract, ungrounded reasoning rather than systematic visual region analysis. Their approach combines Monte Carlo Tree Search (MCTS) to generate high-quality grounded reasoning trajectories for supervised fine-tuning, followed by Group Relative Policy Optimization (GRPO) to reinforce correct grounded sequences. The method is evaluated on spatial reasoning benchmarks (SAT-2, BLINK, RoboSpatial) and web grounding tasks (ScreenSpot, VisualWebArena), showing consistent improvements over baselines. The paper also includes behavioral analysis demonstrating that grounding amplifies visual cognitive behaviors like region exploration, subgoal setting, and backtracking.", "questions": "1. How does ViGoRL perform on more open-ended visual reasoning tasks that don't have clear spatial structure (e.g., abstract visual reasoning, artistic interpretation)? Are there task categories where explicit grounding might be detrimental? For example, how's the performance on V* benchmark [a]?\n\n2. Why fixed crop size? Recent methods like CoF[b] and DeepEyes[c] adopt adaptive sizes for visual thinking. How does the proposed method compare to these methods? Are there any special concerns that hinder the authors from adaptive resolution?\n\n3. You mention reward hacking as a limitation. Have you experimented with intermediate rewards for reasoning step quality or grounding accuracy? How might this affect the training dynamics?\n\n[a] V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs\n\n[b] Chain-of-Focus: Adaptive Visual Search and Zooming for Multimodal Reasoning via RL\n\n[c] DeepEyes: Incentivizing \"Thinking with Images\" via Reinforcement Learning", "limitations": "Yes, the authors have adequately addressed limitations in Appendix A2, including reward hacking issues, computational considerations, and the need for intermediate rewards. They also provide thoughtful discussion of broader impacts in A3, covering both positive applications (medical imaging, accessibility) and potential negative uses (surveillance). The acknowledgment of dataset bias and the need for human oversight in sensitive applications demonstrates responsible consideration of societal impact.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n1. Novel and Well-Motivated Approach: The core insight that explicit spatial grounding can improve visual reasoning is compelling and well-supported by cognitive science literature. The connection to human visual attention patterns provides strong theoretical motivation.\n\n2. Rigorous Analysis of Visual Thinking: The paper provides an exceptionally thorough behavioral analysis of visual reasoning in VLMs, systematically quantifying cognitive behaviors such as region exploration, visual subgoal setting, verification, and backtracking. This rigorous examination of how models actually \"think\" visually offers valuable insights to the community about the internal mechanisms of visual reasoning and represents a significant methodological contribution beyond the core technical approach.\n\n3. Comprehensive Experimental Design: The paper includes thorough evaluations across multiple benchmarks, proper ablation studies, behavioral analysis, and human evaluation. The behavioral coding methodology following Gandhi et al. is particularly valuable for understanding what the model learns.\n\n4. Strong Empirical Results: Consistent improvements across all benchmarks are impressive, with particularly notable gains on SAT-2 (+12.9 points over vanilla GRPO) and competitive performance on web grounding tasks.\n\n## Weaknesses:\n\n1. Reward Design Limitations: The reliance on final-answer rewards may lead to reward hacking, as acknowledged by the authors. The format reward for valid coordinates is somewhat simplistic and may not capture the quality of grounding.\n\n2. Limited Analysis of Failure Cases: While the paper shows when grounding helps, there's insufficient analysis of when it might hurt or be unnecessary. Some tasks may not benefit from explicit spatial grounding.\n\n3. Generalization Questions: Most evaluations are on relatively structured tasks (spatial reasoning, GUI interaction). It's unclear how well the approach generalizes to more open-ended visual reasoning tasks.", "was_revised": true, "final_justification": "My concerns are addressed, and I choose to keep my positive rating.", "rebuttal": "Thank you for your thoughtful feedback. We were encouraged you found our work novel with compelling theoretical grounding in cognitive science, appreciated our \"exceptionally thorough behavioral analysis\" as a \"significant methodological contribution\", and recognized our comprehensive experimental design and \"impressive\" empirical results with consistent improvements across all benchmarks.\n\nWe have carefully addressed your concerns through additional experiments and analysis. To summarize: (1) we tested ViGoRL on V\\*Bench as suggested, finding it significantly outperforms strong baselines, (2) compared adaptive vs. fixed cropping and found no performance difference, with further discussion on each approach, (3) we provide further justification and discussion on intermediate reward and mitigation of reward hacking, and (4) we clarify when grounded reasoning may be unnecessary.\n\n**We present our answers in detail below and hope we’ve addressed your concerns. Please let us know if anything remains unclear. We would be happy to provide further clarification.**\n\n> **Reviewer comment:** Most evaluations are on relatively structured tasks (spatial reasoning, GUI interaction). How's the performance on V* benchmark?\n\nThank you for raising this point about testing the method on more open-ended visual reasoning tasks. **As suggested, we have now run ViGoRL on V\\*Bench**, which tests open-ended visual search including open-ended question answering about fine-grained image details.\n\nWe show the results in Table R1. Our key findings are:\n\n- ViGoRL-7B achieves 86.4%, **outperforming open-source reasoning models (as of May 1, 2025) and surpassing GPT-4o by 20.4%**.\n- Even our 3B model reaches 81.2%, matching sophisticated tool-using pipelines like IVM-Enhanced GPT-4V\n- Multi-turn ViGoRL improves performance over the single-turn variant by 2.1%, demonstrating the utility of multi-turn zoomed-in feedback in this visual search settings.\n\nThis strong performance on visual search reasoning tasks demonstrates that grounded reasoning generalizes beyond spatial and web domains.\n\n---\n\n### Table R1: Accuracy for visual search on V\\*Bench\n\n| **Model Name**                 | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| *Proprietary Models*           |                 |\n| Gemini-Pro                     | 48.2            |\n| GPT-4V                         | 55.0            |\n| GPT-4o                         | 66.0            |\n| *VLM Tool-Using Pipelines*     |                 |\n| VisProg                        | 41.4            |\n| VisualChatGPT                  | 37.6            |\n| MM-React                       | 41.4            |\n| Sketchpad-GPT-4o               | 80.3            |\n| IVM-Enhanced GPT-4V            | 81.2            |\n| *Open-source Models*           |                 |\n| LLaVA-1.5-7B                   | 48.7            |\n| LLaVA-1.6-13B                  | 61.8            |\n| SEAL                           | 74.8            |\n| Qwen2.5-7B-VL                  | 78.0 (±3.1)     |\n| **Multi-t ViGoRL-7B (Ours)** | **86.4 (±2.9)** |\n| Qwen2.5-3B-VL                  | 74.2 (±3.0)     |\n| Single-t ViGoRL-3B (Ours)    | 79.1 (±3.0)     |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.9)** |\n\nVisual search RL training data: We curate 11k question-answer pairs\nover small objects from Segment Anything, using GPT-4o given an object mask and scripted filtering to generate fine-grained ⟨image, question, choices, answer⟩ tuples.\n\n---\n\n> **Reviewer comment:** Why fixed crop size? \n\nThank you for this insightful question regarding adaptive cropping. We conducted an additional experiment where both MCTS warm-start and RL phases in ViGoRL use predicted bounding boxes (adaptive crops) instead of fixed-size point-based crops.\n\n**As shown in Table R2, we observe no difference in V\\*Bench accuracy between the adaptive bounding box and fixed-point cropping approaches.** We attribute this to two main reasons:\n\n1. **Sufficient context from fixed crops**: The default crop size is typically large enough to capture both the target region and its surrounding context at a high resolution for most V\\*Bench examples.\n\n2. **Multi-crop coverage of larger objects**: When the region of interest is too large for a single fixed crop, ViGoRL naturally samples multiple overlapping crops in the vicinity, analogous to human visual exploration through multiple fixations. A combination of these partial crops often suffices to infer the correct answer, especially when supported by a global view.\n\nWhile adaptive cropping could, in principle, be more efficient, we encountered practical limitations:\n\n* Bounding box predictions from Qwen are often imprecise.\n* Inconsistent aspect ratios from adaptive crops can introduce variability during training.\n\nWe agree this is a valuable design dimension and plan to further explore trade-offs between adaptive and fixed cropping in future work.\n\n---\n\n### Table R2: Additional multi-turn ablations\n\n| **Condition**                  | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| Qwen2.5-3B-VL                  | 74.2 (±3.0)     |\n| Single-t ViGoRL-3B (Ours)    | 79.1 (±3.0)     |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.8)** |\n| └─ w/o diversity reward        | 78.0 (±3.0)     |\n| └─ **w/ bounding box outputs**           | **81.2 (±2.8)**    |\n\n---\n\n> **Reviewer comment:** The reliance on final-answer rewards may lead to reward hacking, as acknowledged by the authors. Have you experimented with intermediate rewards for reasoning step quality or grounding accuracy? How might this affect the training dynamics?\n\nWe agree that relying solely on final-answer rewards introduces a risk of reward hacking, and we explicitly acknowledge this limitation. To mitigate this, we use a MCTS-based warm start, a KL penalty, and a structured format rewards that enforce correct tag usage and require valid grounding coordinates, acting as a minimal but effective constraint to maintain warm-start grounded outputs during training.\n\nTo assess the quality of intermediate reasoning, we conducted a human evaluation (Section 5.4), where **annotators judged reasoning steps to be both spatially accurate and helpful** for understanding model behavior, demonstrating the effectiveness of our minimal design.\n\nWe agree that incorporating more nuanced intermediate rewards (e.g., region relevance or semantic alignment) is a promising direction, but non-trivial: defining automated signals that robustly reflect grounding “quality” remains an open research challenge. These additional signals would require additional supervision such as human data (e.g., eye gaze) or specialized judge models. We plan to explore this in future work and will clarify this in the main paper.\n\n> **Reviewer comment:** While the paper shows when grounding helps, there's insufficient analysis of when it might hurt or be unnecessary. Some tasks may not benefit from explicit spatial grounding.\n\nWe appreciate the reviewer’s point. We agree that spatial grounding may not always be necessary, particularly for tasks dominated by global scene cues or simple object recognition. Prior work shows that vision-language models like GPT-4V can perform well on such tasks without explicit region-level reasoning \\[1]. \n\nHowever, a growing body of work highlights that grounding becomes helpful for tasks involving fine-grained spatial reference, ambiguity resolution, or multi-step visual reasoning. For instance:\n\n* Grounding improves spatial reasoning accuracy, especially when simple visual attention fails to resolve spatial relations \\[2].\n* Referring expression grounding benefits substantially from localized object anchoring \\[3].\n* Explicitly prompting for region references improves performance on spatial QA and tool-use tasks \\[4].\n* Prompting grounded intermediate steps boosts both interpretability and accuracy across visual tasks \\[5].\n\nWe provide an example below (Example R1) where our ViGoRL model does unnecessary grounding and verification for a recognition task. While not harming performance, this behavior may lead to excessive computation. We note that this overthinking occurs with existing reasoning models \\[6].\n\nWe will revise Section 6 to clarify when spatial grounding is particularly helpful, and add additional qualitative results in the appendix.\n\n\\[1] OpenAI, GPT-4V System Card, 2023.\n\n\\[2] Liu et al., What Makes Spatial Reasoning Hard for VLMs?, 2023.\n\n\\[3] Chen et al., RefCLIP: Referring with Class Name Aware Vision-Language Models, 2022.\n\n\\[4] Wu et al., V\\*Bench: Evaluating Tool-augmented VLMs, 2023.\n\n\\[5] Yang et al., Thinking in Pictures, 2024.\n\n\\[6] Chen, X. Do not think that much for 2+3=? On the overthinking of o1-like LLMs. 2025.\n\n---\n\n### Example R1: Overthinking\n> Question: “Is there a pot in the image?”\n\n> Image: \\<Image of stove with a large pot and pan on it>\n\n*Model response:* \\<think> To determine if there's a pot, I'll check the regions where pots typically appear. At (300, 600), there's a pot-like object with a handle and a lid, suggesting it could be a pot. To confirm the presence of a pot, I'll check another region. At (700, 1000), there's no pot-like object present, indicating that the previous observation was accurate. Now, I'll verify if there are any other potential pots in the image. At (500, 800), there's no pot-like object present, which supports the previous findings. Next, let's examine the area at (400, 900) to ensure there's no hidden pot. There's still no pot-like object visible here, reinforcing the previous observations. I'll now check the lower region at (600, 1200) to confirm there are no additional pots. At (600, 1200), no pot-like object is visible here, further supporting the absence of a pot in the image. Lastly, I'll check the upper region at (300, 200) to ensure there's no hidden pot. At (300, 200), no pot-like object is visible here, confirming the pot is at (300, 600) in the image. \\</think>"}, {"review_id": "iTDppaQnm5", "reviewer": "Reviewer_jnnD", "summary": "This paper introduces a method for training vision-language models (VLMs) to ground their reasoning steps in visual information. Previous approaches that directly apply reinforcement learning to base models yield biased and ungrounded models regarding visual information. The proposed approach first performs supervised fine-tuning (SFT) equipped with Monte Carlo Tree Search (MCTS), then applies reinforcement learning to scale. This enables VLMs to focus not only on abstract reasoning steps but also on visually-grounded thoughts. Experiments demonstrate performance gains achieved by the introduced grounding method.", "questions": "- Diversity bonus (+0.2): How was this value determined?\n- Computational Cost: How expensive is the introduced SFT component?\n\nMinors:\n- Table 2 is referenced before Table 1, which creates an awkward reading flow.", "limitations": "While limitations are discussed in the appendix, they should also be addressed in the main text for better visibility.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\nThe paper is well-motivated and clearly written, making it easy to follow and understand the core ideas. Each component of the technical contribution is introduced effectively by explaining its necessity and addressing specific problems, which significantly enhances readability. Overall, the work is technically solid and well-written. The community will benefit from the insights and results presented in this paper.\n\n### Weaknesses\n- Scope Clarification: The problem scope of the method is not clearly defined. While the introduced SFT/RL method enhances performance on target tasks, it may degrade performance on other tasks. A clarification of the method's scope would benefit future readers.\n- Limited Model Evaluation: Experiments consistently use Qwen-2.5-VL as the base model. The authors should justify this design choice and explain why other model options are not evaluated.\n- Failure Analysis: The paper demonstrates well how the proposed method works when successful, but lacks discussion of failure cases and scaling bottlenecks. Understanding when and why the method fails would help readers better grasp the paper's scope and limitations.", "was_revised": true, "final_justification": "The paper is well-motivated and clearly written. The technical aspect is solid, and experiments are extensively conducted. Thus, I will keep my positive rating.", "rebuttal": "Thank you for your thoughtful review and for recognizing our work as well-motivated, clearly written, and technically solid. We appreciate your positive assessment of our grounded reasoning approach and its benefits to the community.\n\nWe have carefully tried to address your concerns through additional experiments and analysis: (1) We clarify ViGoRL's scope, (2) We justify our choice of Qwen-2.5-VL, (3) We add comprehensive error analysis, (4) We provide ablation results on our diversity bonus, and (5) We detail computational costs of our SFT component.\n\n**We present our answers in detail below and hope we’ve addressed your concerns. Please let us know if anything remains unclear. We would be happy to provide further clarification.**\n\n> **Reviewer comment:** The problem scope of the method is not clearly defined. Training may degrade performance on other tasks.\n\nWe will clarify ViGoRL's scope in Section 3.1:\n“ViGoRL targets improved performance on specific visual reasoning domains (e.g., spatial understanding, web element grounding) by fine-tuning domain-relevant reasoning skills, while preserving performance on related tasks by leveraging transferable grounding and reasoning mechanisms.”\n\nTo further clarify and verify the scope:\n\n- Table R1 (also in Supplemental Table A7) shows ViGoRL maintains or improves performance on out-of-domain benchmarks (MMMU, RealWorldQA, V\\*Bench), suggesting that its grounded reasoning strategies are transferable and do not result in overfitting or negative transfer.\n\n- In Table R2, we also evaluate mixed-task training (spatial + web) and find only minor trade-offs, indicating that ViGoRL’s improvements are not narrowly scoped to a single domain. \n\n- As suggested by reviewer vmK2, we train and evaluate on the open-ended domain of V\\*Bench, finding ViGoRL significantly outperforms strong baselines in this additional domain (Table R3).\n\n---\n\n### Table R1: Accuracy on Out-of-Distribution (OOD) Benchmarks\n\n| **Model**                       | **MMMU**       | **RealWorldQA** | **V\\*Bench**     |\n| ------------------------------- | -------------- | --------------- | ---------------- |\n| Qwen2.5-VL-3B Base              | 47.4% (±3.2) | 55.6% (±3.5)  | 74.2% (±6.2)   |\n| **ViGoRL-3B (Ours, SAT-2)**   | 46.4% (±3.2) | 60.6% (±3.4)  | 74.8% (±6.1) |\n| **ViGoRL-3B (Ours, UGround)** | 47.5% (±3.2) | 57.5% (±3.5)  | 75.1% (±6.1) |\n\n---\n\n### Table R2: Training with combined vs. separate datasets\n\n| **Condition**             | **ScreenSpot-V2/Pro** | **SAT-2** |\n| ------------------------- | --------- | --------------------- |\n| Qwen2.5-3B-VL             |  9.3% (±1.4) / 68.4% (±2.6)       |   46.1% (±1.5)                    |\n| Vanilla GRPO (seperate spatial or web)             |  29.0% (±2.2)  / 84.4% (±2.0)        |    50.0% (±1.6)                    |\n| ViGoRL-3B (seperate spatial or web)     | 31.1% (±2.3) / 86.5% (±1.9)   |  62.9% (±1.5)                     |\n| ViGoRL-3B (combined spatial+web) | 30.9% (±2.3) / 86.2% (±2.0)  |    64.0 (±1.5)                   |\n\n---\n\n### Table R3: Accuracy for visual search on V\\*Bench\n\n| **Model Name**                 | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| *Proprietary Models*           |                 |\n| Gemini-Pro                     | 48.2            |\n| GPT-4V                         | 55.0            |\n| GPT-4o                         | 66.0            |\n| *VLM Tool-Using Pipelines*     |                 |\n| VisProg                        | 41.4            |\n| VisualChatGPT                  | 37.6            |\n| MM-React                       | 41.4            |\n| Sketchpad-GPT-4o               | 80.3            |\n| IVM-Enhanced GPT-4V            | 81.2            |\n| *Open-source Models*           |                 |\n| LLaVA-1.5-7B                   | 48.7            |\n| LLaVA-1.6-13B                  | 61.8            |\n| SEAL                           | 74.8            |\n| Qwen2.5-7B-VL                  | 78.0 (±3.1)     |\n| **Multi-t ViGoRL-7B (Ours)** | **86.4 (±2.9)** |\n| Qwen2.5-3B-VL                  | 74.2 (±3.0)     |\n| Single-t ViGoRL-3B (Ours)    | 79.1 (±3.0)     |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.9)** |\n\nVisual search training data details: We curate 11k question-answer pairs\nover small objects from Segment Anything, using GPT-4o given an object mask and scripted filtering to generate fine-grained ⟨image, question, choices, answer⟩ tuples.\n\n---\n\n> **Reviewer comment:** The authors should justify using Qwen-2.5-VL as a design choice.\n\nThank you for raising this important point about model selection. We chose Qwen2.5-VL as our primary base model based on both its technical and practical suitability for our setup. Qwen2.5-VL was one of the strongest open-source VLMs available at the time of our experiments, with strong visual grounding capabilities already incorporated during pretraining. It also comes in multiple sizes (3B, 7B, 72B), which enabled controlled scaling and distillation within a consistent architecture and tokenizer.\n\nTo address your concern about evaluating other models, we are currently running our pipeline with Kimi-VL using the EasyR1 framework, and these results will be included in the final release.\n\n---\n\n> **Reviewer comment:** The paper lacks discussion of failure cases.\n\nThank you for the suggestion. We now include Table R4, which categorizes SAT-2 errors. 60% arise from perceptual or task-level challenges, while 40% reflect residual reasoning flaws (e.g., overthinking, truncation, hallucination). We believe scaling or denser rewards could address aspects of both, with further discussion in Section 6 and limitations. We will include this analysis and additional qualitative failure cases in the revised manuscript.\n\n---\n\n### Table R4: ViGoRL Error Analysis by Type on SAT-2\n\n| **Error Type**        | **Percentage** | **Description**                                                                                                                                   |\n|-----------------------|----------------|---------------------------------------------------------------------------------------------------------------------------------------------------|\n| Excessive Reasoning   | 20%            | The model overthinks the problem.                   |\n| Incomplete Reasoning  | 10%            | The model truncates its reasoning process.                                            |\n| Hallucination         | 10%            | The model invents visual elements.                                          |\n| Challenging Visual QA | 30%            | The question or image presents inherently subtle cues, causing reasoning errors.     |\n| Base Perception       | 30%            | The underlying model makes perceptual or conceptual grounding errors.|\n\n#### Example R1: Overthinking\n\n> Question: “Is there a pot in the image?”\n\n> Image: \\<Image of stove with a large pot and pan on it>\n\n*Model response:* \n\\<think>\nTo determine if there's a pot, I'll check the regions where pots typically appear. At (300, 600), there's a pot-like object with a handle and a lid, suggesting it could be a pot. To confirm the presence of a pot, I'll check another region. At (700, 1000), there's no pot-like object present, indicating that the previous observation was accurate. Now, I'll verify if there are any other potential pots in the image. At (500, 800), there's no pot-like object present, which supports the previous findings. Next, let's examine the area at (400, 900) to ensure there's no hidden pot. There's still no pot-like object visible here, reinforcing the previous observations. I'll now check the lower region at (600, 1200) to confirm there are no additional pots. At (600, 1200), no pot-like object is visible here, further supporting the absence of a pot in the image. Lastly, I'll check the upper region at (300, 200) to ensure there's no hidden pot. At (300, 200), no pot-like object is visible here, confirming the pot is at (300, 600) in the image.\n\\</think>\n\n---\n\n> **Reviewer comment:** How was diversity bonus determined?\n\nThe diversity bonus is treated as a hyperparameter. We ablate the diversity bonus in Table R5. Removing the multi-turn diversity bonus reward leads to a decrease of 3.2% on V\\*bench. We found 0.2 to encourage turn exploration while not overpowering accuracy reward.\n\n---\n\n### Table R5: Multi-turn reward ablation\n\n| **Condition**                  | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.8)** |\n| └─ w/o diversity reward        | 78.0 (±3.0)     |\n\n---\n\n> **Reviewer comment:** How expensive is the introduced SFT component?\n\nThe introduced SFT stage, including MCTS-based data generation and supervised finetuning, incurred approximately 224 GPU-hours (192 for MCTS, 32 for SFT), while the RL stage required 576 GPU-hours. Thus, SFT accounts for ~39% of the RL computational cost. It is intentially lightweight (1.5k prompts) and provides structured, grounded reasoning behaviors that RL alone fails to induce (see Table 4 and Section 5.3).\n\nIncurring additional resources for SFT before RL is in-fact common among post-training recipes [1,2]. However, unlike prior warm-start approaches that rely on large-scale human prompt curatation or annotations [1,2], our warm start uses model+search to generate reasoning traces without human supervision.\n\n[1] DeepSeek-AI. (2025), DeepSeek-R1\n\n[2] Kimi Team (2025), Kimi-VL Technical Report\n\n> Table 2 is referenced before Table 1.\n\nThank you for the suggestions. We will revise the text to reference tables in the correct order for readability.\n\n> Limitations should be addressed in the main text.\n\nWe will move the key limitations from the appendix to the main text to ensure they are clearly visible to readers."}, {"review_id": "ZShhxIdmv6", "reviewer": "Reviewer_r46c", "summary": "This paper proposes a visually grounded reinforcement learning (ViGoRL) paradigm to enhance the visual reasoning capability of multimodal LLMs. Instead of solely generating textual reasoning thoughts, ViGoRL infers natural language CoT together with the corresponding spatial grounding location in the given image. This visual reasoning process mimics the cognitive behavior of humans and enriches the conventional CoT with spatially grounded cues. The proposed ViGoRL method improves the spatial reasoning and GUI understanding abilities of MLLMs. Experimental results on various benchmarks show the effectiveness of this proposed method.", "questions": "1. Why would directly combining data sources of spatial reasoning and web understanding for RL training benefit all tasks?\n2. What are the contributions and effectiveness of each proposed reward item?\n3. What is the motivation for jointly considering the visual and textual thoughts instead of solely adopting visual CoT for visual tasks?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. Grounding chain-of-thought reasoning into visual scenes is crucial and practical for multimodal reasoning tasks. It could improve the ability to handle tasks that require spatial understanding and capture the image regions that are relevant to the question.\n2. The proposed multi-trun reinforcement learning scheme with data generation pipeline is reasonable and aligns with the practical needs to address the challenges in traditional GRPO.\n3. The overall paper is easy to follow and well organized.\n\nWeaknesses\n1. In lines 263-270, the training data is curated from different sources, including spatial reasoning, web grounding, and web action prediction. However, the domain of spatial reasoning is distinct from the data of web understanding. Why would directly combining these data sources for RL training benefit all tasks? There seems to be no evidence or analysis in the current draft.\n2. In lines 245-252, the reward combines multiple terms and introduces several rewards, like the grammar reward and the diversity bonus. However, there is no detailed ablation study to analyze the effectiveness and contributions of each reward item. \n3. Previous work, like Argus (CVPR’25), also aims to adopt visual CoT to enhance the capability of MLLMs for visual-centric tasks. Compared with Argus (CVPR’25), the proposed ViGoRL jointly considers textual and visual CoTs. It would be better to clarify the motivation for jointly adopting visual and textual thoughts, not only applying visual CoT for the visual/spatial QA tasks.\n\nReference:\nArgus (CVPR'25): ARGUS: Vision-Centric Reasoning with Grounded Chain-of-Thought. CVPR 2025", "was_revised": true, "final_justification": "Thanks to the authors for your detailed response. My original concerns are well addressed. I will increase the score from 4 to 5.", "rebuttal": "Thank you for your thoughtful review and for recognizing our work's novel approach to grounding chain-of-thought reasoning in visual scenes, finding it \"crucial and practical for multimodal reasoning tasks.\" We appreciate your positive assessment of our multi-turn reinforcement learning scheme and data generation pipeline as reasonable and well-aligned with practical needs.\n\nWe have addressed your concerns through additional experiments and analysis: (1) We conducted ablations showing that removing textual reasoning causes a performance drop, demonstrating that visual and textual CoTs serve complementary roles, (2) We clarified that our training considers seperated datasets, with experiments showing minimal impact from dataset mixing, and (3) We performed comprehensive reward ablations revealing that each component contributes meaningfully to overall performance.\n\n**We present our answers in detail below and hope we’ve addressed your concerns. Please let us know if anything remains unclear. We would be happy to provide further clarification.**\n\n> **Reviewer comment:** Previous work, like Argus (CVPR’25), also aims to adopt visual CoT to enhance the capability of MLLMs for visual-centric tasks. What is the motivation for jointly considering the visual and textual thoughts instead of solely adopting visual CoT for visual tasks?\n\nThank you for the thoughtful question. To directly test the role of textual reasoning, we ran an ablation in which we removed all textual thoughts during both the SFT and RL stages, effectively reducing it to a visual CoT-style agent. **As shown in Figure R2 below, this visual-only CoT resulted in a performance drop of 1.2% on V∗Bench, indicating that grounding alone, while helpful, does not fully substitute for the explicit composition and referencing afforded by textual reasoning.**\n\nOur motivation for jointly using both visual and textual chains of thought stems from the observation that they serve complementary roles. Visual grounding and cropping provide perceptual feedback -- what the model “sees” --  while **textual reasoning supports binding and compositional inference over these grounded regions.** Much as in human cognition, this separation allows the model not only to explore regions but also to integrate and manipulate visual evidence in a structured way, yielding more interpretable and accurate reasoning trajectories.\n\nThat said, we agree with the reviewer’s broader point: we do not claim that explicit natural language reasoning is the only or necessary mechanism for visual region binding and manipulation. In the long term, latent reasoning may serve similar purposes with less reliance on textual supervision (e.g., \\[1]). Nonetheless, our approach offers a practical scaffold for structured supervision, demonstrates the value of grounded reasoning, and delivers immediate performance gains. We will clarify this distinction in the revised discussion and better articulate why language remains a practical and interpretable scaffold in our current formulation.\n    \n\\[1] Hao et al. (2024), Training Large Language Models to Reason in a Continuous Latent Space\n\n---\n\n### Table R2: Multi-turn ablations\n\n| **Condition**                  | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| Qwen2.5-3B-VL                  | 74.2 (±3.0)     |\n| Single-t ViGoRL-3B (Ours)    | 79.1 (±3.0)     |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.8)** |\n| └─ w/o diversity reward        | 78.0 (±3.0)     |\n| └─ w/o text thoughts           | 80.0 (±2.9)     |\n\n---\n\n> **Reviewer comment:** Why would directly combining data sources of spatial reasoning and web understanding for RL training benefit all tasks?\n\nThank you for this question about our training strategy. To clarify, we do not combine datasets during training and each model is trained on task-specific data. We will revise lines 263-270 to clarify this point.\n\nTo quantify the effect of combining data, we conducted an experiment combining spatial and web datasets during both warm-start and RL phases. The results, shown in Table R3, reveal:\n\n- Web grounding: Small decrease (-0.2% on ScreenSpot-Pro, -0.3% on ScreenSpot-V2)\n- Spatial reasoning: Improvement of +1.1% on SAT-2\n- Both approaches significantly outperform vanilla GRPO (+12-15% SAT-2)\n\nThese findings indicate that combining data has small effects on relative performance gains from our method, with slight benefits for spatial reasoning at the cost of web grounding performance.\n\n---\n\n### Table R3: Training with combined vs. separate datasets\n\n| **Condition**             | **ScreenSpot-V2/Pro** | **SAT-2** |\n| ------------------------- | --------- | --------------------- |\n| Qwen2.5-3B-VL             |  9.3% (±1.4) / 68.4% (±2.6)       |   46.1% (±1.5)                    |\n| Vanilla GRPO (seperate spatial or web)             |  29.0% (±2.2)  / 84.4% (±2.0)        |    50.0% (±1.6)                    |\n| ViGoRL-3B (seperate spatial or web)     | 31.1% (±2.3) / 86.5% (±1.9)   |  62.9% (±1.5)                     |\n| ViGoRL-3B (combined spatial+web) | 30.9% (±2.3) / 86.2% (±2.0)  |    64.0 (±1.5)                   |\n\n---\n\n> **Reviewer comment:** What are the contributions and effectiveness of each proposed reward item?\n\nWe thank the reviewer for this valuable suggestion. We have ablated all components of our reward formulations in Tables R4 and R5 below.\n\nSpecifically:\n- Removing the multi-turn diversity bonus reward leads to a decrease of 3.2% on V\\*bench. Appendix Figure A1 shows that removing this term leads to turn collapse.\n- Removing the format reward leads to a decrease of 0.6% on SAT-2.\n- Removing the accuracy reward leads to a decrease of 2.3% on SAT-2.\n\nWe will include these ablations in Section 5.2 of the revised manuscript.\n\n---\n\n### Table R4: Multi-turn reward ablation\n\n| **Condition**                  | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| Qwen2.5-3B-VL                  | 74.2 (±3.0)     |\n| Single-t ViGoRL-3B (Ours)    | 79.1 (±3.0)     |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.8)** |\n| └─ w/o diversity reward        | 78.0 (±3.0)     |\n\n---\n\n### Table R5: Additional reward ablations\n\n| **Condition**          | **SAT-2**    |\n| ---------------------- | ------------ |\n| ViGoRL-3B (Ours)     | 62.9% (±1.5) |\n| └─ w/o format reward   |    62.3% ± (±1.5)          |\n| └─ w/o accuracy reward |      60.6% ± (±1.5)        |"}, {"review_id": "hDliysz37u", "reviewer": "Reviewer_Dj1B", "summary": "Summary\n\n1.\tThe paper introduces ViGoRL, a vision-language model designed to enhance visual reasoning capabilities through reinforcement learning (RL).\n\n2.\tThe motivation is to compose reasoning step explicitly anchored in the image coordinate, i.e. n = <s, (x,y)>. It can encourage the model to do more targeted and systematic cross-reference between text and image.\n\n3.\tIt proposes a two-stage pipeline including warm-start supervised finetuning to produce structured reasoning steps with MCTS data generation, and reinforcement learning to refine grounding response with multi-turn visual signal.\n\n4.\tViGoRL has been assessed across a range of visual reasoning benchmarks, including SAT-2 and BLINK for spatial reasoning tasks, VBench for web-based action prediction, and ScreenSpot and VisualWebArena for web-based grounding challenges. The model demonstrates superior performance compared to conventional supervised fine-tuning methods and RL baselines that lack explicit grounding components.", "questions": "Comment\n\n1.\tSince a teacher model (Qwen2.5-VL-72B) is used to distill grounded reasoning traces, it is required to provide to the zero-shot performance of the teacher model in Table I, Table II and Table III. \n\n2.\tAn ablation study in terms of accuracy on reward design [line220-225] should be provided.\n\n3.\tThe contribution of multi-turn visual feedback is unclear. In Table I, the enhancement of small element grounding (ScreenSpot-Pro) is not obvious and the degradation of Screenspot-V2 need to be further elaborated.\n\n4.\tLine 800, for the reward of Web Grounding, it is curious about why don’t use the euclidean distance between the predicted coordinates and center of bounding box.\n\n5.\tThe crop size for multi-turn GRPO is fixed to 100x100. However, some objects such as the pendant light in Figure 1 is not with a regular shape. How to deal with such problem?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strength\n\n•\tAttaching grounding signal during reasoning stages is of novelty.\n\n•\tPresentation of this work is easy to follow.\n\n•\tContribution of this work for visual grounding field is significant. \n\n\nWeakness \n\n•\tSOTAs performance is not demonstrated in this work.\n\n•\tOut-of-distribution generalization should be further improved.", "was_revised": true, "final_justification": "Most of my concerns have been addressed. This work is demonstrated as a SOTA work. It is helpful in the multimodal domain.", "rebuttal": "Thank you for your thoughtful review. We were pleased that you recognized our work's novelty in attaching grounding signals during reasoning stages, appreciated our clear presentation, and found our contribution to the visual grounding field significant.\n\nWe have addressed concerns through additional experiments and analysis. To summarize: (1) we conducted reward ablations, (2) we validated the contribution of multi-turn visual feedback on V\\*Bench, (3) we clarify our state-of-the-art performance, (4) we clarify our OOD generalization, (5) we empirically validated our binary reward design over Euclidean distance, (6) compared adaptive vs. fixed cropping with experiments, and (7) we added teacher model comparisons.\n\n**We present our answers in detail below and hope we’ve addressed your concerns. Please let us know if anything remains unclear. We would be happy to provide further clarification.**\n\n> **Reviewer comment:** Reward ablation study should be provided.\n\nWe thank the reviewer for this valuable suggestion. We have ablated all components of our reward formulations in Tables R1 and R2 below.\n\nSpecifically:\n- Removing the multi-turn diversity bonus reward leads to a decrease of 3.2% on V\\*bench. Appendix Figure A1 shows that removing this term leads to turn collapse.\n- Removing the format reward leads to a decrease of 0.6% on SAT-2.\n- Removing the accuracy reward leads to a decrease of 2.3% on SAT-2.\n\nWe will include these ablations in Section 5.2 of the revised manuscript.\n\n---\n\n### Table R1: Multi-turn ablations\n\n| **Condition**                  | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.8)** |\n| └─ w/o diversity reward        | 78.0 (±3.0)     |\n| └─ w/o text thoughts           | 80.0 (±2.9)     |\n| └─ **w/ bounding box outputs**           | **81.2 (±2.8)**    |\n\n---\n\n### Table R2: Additional reward ablations\n\n| **Condition**          | **SAT-2**    |\n| ---------------------- | ------------ |\n| ViGoRL-3B (Ours)     | 62.9% (±1.5) |\n| └─ w/o format reward   |    62.3% ± (±1.5)          |\n| └─ w/o accuracy reward |      60.6% ± (±1.5)        |\n\n---\n\n> **Reviewer comment:** The crop size for multi-turn GRPO is fixed. How to deal with irregular shapes?\n\nWe conducted additional experiments where both MCTS warm-start and RL phases in ViGoRL use predicted bounding boxes (adaptive crops) instead of fixed-size point-based crops.\n\n**As shown in Table R1 above, we observe no difference in V\\*Bench accuracy between the adaptive bounding box and fixed-point cropping approaches.** We attribute this to two main reasons:\n\n1. **Sufficient context from fixed crops:** The default crop size is typically large enough to capture both the target region and its surrounding context at a high resolution for most V\\*Bench examples.\n\n2. **Multi-crop coverage of larger objects:** When the region of interest is too large for a single fixed crop, ViGoRL samples multiple overlapping crops in the vicinity, analogous to human multiple fixations, which suffices to infer the correct answer.\n\nWhile adaptive cropping could, in principle, be more efficient, we encountered practical limitations:\n\n* Bounding box predictions from Qwen are often imprecise.\n* Inconsistent aspect ratios from adaptive crops can introduce training variability.\n\nWe agree this is a valuable design dimension and plan to further explore trade-offs in future work.\n\n> **Reviewer comment:** Why not use the euclidean distance between the predicted coordinates and center of bounding box?\n\nWe thank the reviewer for the suggestion. **We tested the Euclidean distance reward and found it significantly reduced performance: -2.2% on ScreenSpot-V2 and -0.7% on ScreenSpot-Pro (Table R3).** We will include this in the revision.\n\nWeb elements vary in size, so a distance-based reward unfairly penalizes valid clicks near the edges of large elements. For instance, clicking near the edge of an 800×60 navigation bar is functionally correct but would receive a low euclidean reward.\n\n---\n\n### Table R3: Euclidean vs. binary reward ablation\n\n| **Condition**                     | **ScreenSpot-V2** | **ScreenSpot-Pro** |\n| --------------------------------- | ----------------- | ----------------- |\n| Qwen2.5-3B-VL                     | 68.4% (±2.6)               | 9.3% (±1.4)               |\n| ViGoRL-3B w/ binary bbox reward | 86.5% (±1.9)   | 31.1% (±2.3)   |\n| ViGoRL-3B w/ euclidean reward   | 84.3% (±2.0)          | 30.4% (±2.3)          |\n\n---\n\n> **Reviewer comment:** The contribution of multi-turn visual feedback is unclear.\n\nWe find that multi-turn visual feedback is especially valuable when the model cannot perceive fine-grained image details in the initial input resolution. ScreenSpot-V2 has large, visible elements, so feedback offers no gain (–0.4%), while ScreenSpot-Pro features smaller, high-resolution targets, where feedback improves performance (+1.2%).\n\n**To validate this claim further, we evaluated ViGoRL on V\\*Bench in Table R4, which requires identifying small details in ultra high-resolution images. Multi-turn ViGoRL outperforms the single-turn variant by 2.1%**, due to better disambiguating details via the zoomed-in crops. This reinforces that visual feedback is crucial when the global image tokenization blurs away fine-grained cues.\n\n---\n\n### Table R4: Accuracy for visual search on V\\*Bench\n\n| **Model Name**                 | **V\\*Bench**    |\n| ------------------------------ | --------------- |\n| *Proprietary Models*           |                 |\n| Gemini-Pro                     | 48.2            |\n| GPT-4V                         | 55.0            |\n| GPT-4o                         | 66.0            |\n| *VLM Tool-Using Pipelines*     |                 |\n| VisProg                        | 41.4            |\n| VisualChatGPT                  | 37.6            |\n| MM-React                       | 41.4            |\n| Sketchpad-GPT-4o               | 80.3            |\n| IVM-Enhanced GPT-4V            | 81.2            |\n| *Open-source Models*           |                 |\n| LLaVA-1.5-7B                   | 48.7            |\n| LLaVA-1.6-13B                  | 61.8            |\n| SEAL                           | 74.8            |\n| Qwen2.5-7B-VL                  | 78.0 (±3.1)     |\n| **Multi-t ViGoRL-7B (Ours)** | **86.4 (±2.9)** |\n| Qwen2.5-3B-VL                  | 74.2 (±3.0)     |\n| Single-t ViGoRL-3B (Ours)    | 79.1 (±3.0)     |\n| **Multi-t ViGoRL-3B (Ours)** | **81.2 (±2.9)** |\n\nVisual search training data: We curate 11k question-answer pairs\nover small objects from Segment Anything, using GPT-4o given an object mask and scripted filtering to generate fine-grained ⟨image, question, choices, answer⟩ tuples.\n\n---\n\n> **Reviewer comment:** SOTAs performance is not demonstrated in this work.\n\nTo the best of our knowledge, ViGoRL matches or exceeds prior SOTA results across several key benchmarks, as outlined below. If we have overlooked a relevant baseline, we would welcome including it in additional comparisons.\n\nThat said, our core contribution extends beyond outperforming existing models: we demonstrate that grounded reasoning provides consistent gains over standard SFT and vanilla (ungrounded) GRPO across diverse reasoning domains, supported by extensive ablations and behavioral analyses.\n\n* **VisualWebArena:** Our 7B ViGoRL model outperforms the previous 7B SOTA (ICAL) by +3.0%, without relying on human reasoning annotations or HTML inputs, which ICAL requires.\n* **V\\*Bench:** As shown in Table R4 above, ViGoRL establishes SOTA among 7B-scale reasoning models (as of May 1, 2025).\n* **Web Grounding & Spatial Reasoning:** ViGoRL approaches the performance of leading commercial models such as Kimi-VL, while being fully open-source. We release all code, models, and data in the submission supplemental.\n\n> **Reviewer comment:** OOD generalization should be further improved.\n\nViGoRL achieves strong OOD generalization compared to SFT and vanilla GRPO baselines. That said, we will add more comparisons or specific experiments if the reviewer has suggestions.\n\nWe further validated OOD generalization on benchmarks with minimal training overlap. **Table R5 below (also in Supplemental Table A7) shows ViGoRL maintains or improves performance on OOD benchmarks (MMMU, RealWorldQA, V\\*Bench), addressing concerns about overfitting or negative transfer.** We discuss potential extensions for even stronger OOD performance in Section 6 and limitations.\n\n---\n\n### Table R5: Accuracy on Out-of-Distribution (OOD) Benchmarks\n\n| **Model**                       | **MMMU**       | **RealWorldQA** | **V\\*Bench**     |\n| ------------------------------- | -------------- | --------------- | ---------------- |\n| Qwen2.5-VL-3B Base              | 47.4% (±3.26) | 55.6% (±3.52)  | 74.2% (±6.22)   |\n| **ViGoRL-3B (Ours, SAT-2)**   | 46.4% (±3.26) | 60.6% (±3.46)  | 74.8% (±6.15) |\n| **ViGoRL-3B (Ours, UGround)** | 47.5% (±3.26) | 57.5% (±3.50)  | 75.1% (±6.16) |\n\n---\n\n> **Reviewer comment:** It is required to provide to the zero-shot performance of the teacher model.\n\nWe have added the zero-shot performance of our teacher model (Qwen2.5-VL-72B) in the revised manuscript, with results provided in Table R6 below. Despite using only 1k examples for distillation before RL, ViGoRL-7B outperforms Qwen2.5-VL-72B by 6.0% on SAT-2 while being 10× smaller, and closes the gap across most benchmarks.\n\n---\n\n### Table R6: Comparison to Qwen2.5-72B-VL\n\n| **Model**      | **SAT-2** | **BLINK** | **RoboSpatial** | **ScreenSpot-V2** | **ScreenSpot-Pro** |\n| -------------- | --------- | --------- | --------- | ----------------- | ------------------ |\n| Qwen2.5-VL-72B | 61.5% (±1.5)  | 59.5% (±2.3)  |  55.7% (±6.5)  | 88.1 (±1.8)          | 34.2 (2.6±)           |\n| ViGoRL-7B      | 67.5% (±1.5)  | 54.1% (±2.3)  | 76.4% (±7.5)  | 91.0% (±1.6)          | 33.1% (±2.3)            |"}]

Author rebuttal:

Meta-review: The paper introduces ViGoRL, a novel reinforcement learning framework that enhances visual reasoning in vision-language models by explicitly grounding each reasoning step in spatial image coordinates. Its core claim is that such grounding improves performance on spatial and web-based reasoning tasks by mimicking human visual attention. Strengths include a rigorous behavioral analysis, strong empirical results across multiple benchmarks (e.g., SAT-2, V*Bench), and a well-motivated methodology inspired by cognitive science. Weaknesses initially included concerns about reward design, generalization to open-ended tasks, and limited failure analysis.

During the rebuttal period, reviewers raised several key points: vmK2 questioned generalization (addressed via V*Bench results showing +20.4% over GPT-4o); jnnD requested scope clarification and failure analysis (added error categorization and OOD results); r46c sought reward ablations and multi-modal justification (provided component-wise reward contributions and textual vs. visual CoT comparisons); Dj1B queried reward design and adaptive cropping (showed binary reward outperforms Euclidean distance and fixed vs. adaptive cropping parity). All reviewers acknowledged their concerns were addressed, with two increasing scores post-rebuttal.

The ratings of this submission are 5,4,5,5, suggesting a clear recommendation of acceptance. Taking all reviews into consideration, I recommend acceptance of this paper.
Decision: Accept (poster)