No Labels, No Problem: Training Visual Reasoners with Multimodal Verifiers
Authors: Damiano Marsili
Georgia Gkioxari
Venue: ICLR 2026 Poster (2026)
Keywords: VLM
spatial reasoning
LLM
RL
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: An annotation-free training framework that uses multi-modal verifiers to improve reasoning and grounding for spatial reasoning.

Abstract: Visual reasoning is challenging, requiring both precise object grounding and understanding complex spatial relationships. Existing methods fall into two camps: language-only chain-of-thought approaches, which demand large-scale (image, query, answer) supervision, and program-synthesis approaches which use pre-trained models and avoid training, but suffer from flawed logic and erroneous grounding. We propose an annotation-free training framework that improves both reasoning and grounding. Our framework uses AI-powered verifiers: an LLM verifier refines LLM reasoning via reinforcement learning, while a VLM verifier strengthens visual grounding through automated hard-negative mining, eliminating the need for ground truth labels. This design combines the strengths of modern AI systems: advanced language-only reasoning models for decomposing spatial queries into simpler subtasks, and strong vision specialist models improved via performant VLM critics. We evaluate our approach across diverse spatial reasoning tasks, and show that our method improves visual reasoning and surpasses open-source and proprietary models, while with our improved visual grounding model we further outperform recent text-only visual reasoning methods. Project webpage: https://glab-caltech.github.io/valor/

No LABELS, No PROBLEM: TRAINING VISUAL REA- SONERS WITH MULTIMODAL VERIFIERS
Damiano Marsili, Georgia Gkixoari California Institute of Technology
ABSTRACT
Visual reasoning is challenging, requiring both precise object grounding and un- derstanding complex spatial relationships. Existing methods fall into two camps: language-only ohain-of-thought approaches, which demand large-scale (image, query, answer) supervision, and program-synthesit approaches which use pre- trained models and avoid training, but suffer from flawed logic and erroneous grounding. We propose an annotation-free training framework that improves both reasoning and grounding. Our framework uses Al-powered verifiers: verifier refines LLM reasoning via reinforcement learning, while a VLM verifier strengthens visual grounding through automated hard-negative mining, eliminat- ing the need for ground truth labels. This design combines the strengths of modern Al systems: advanced language-only reasoning models for decomposing spatial queries into simpler subtasks, and strong vision specialist models improved via performant VLM critics. We evaluate our approach across diverse spatial rea- soning tasks, and show that our method improves visual reasoning and surpasses open-source and proprictary models, while with our improved visual grounding model we further outperform recent text-only visual reasoning methods. Project webpage: ttass//llab-caltech wwa.tech..ituub.io/.. github.io/vaaoor
LLM
INTRODUCTION
Visual reasoning is key skill for artificial intelligence: to understand and act in the world, sys- tems must not only identify objects in images but also reason about their spatial relationships and attributes. For example, answering the query in Fig. requires grounding objects (fireplace, coffee table, sofa), inferring 3D size from 2D cues, and combining attributes to produce the final answer.
Visual reasoning methods fall into two categories. The first integrates grounding with language rea- soning, where lingon-language models (VLMs) generate chain-of-though explanations in text (Fan et al., 2025; Sarch et al., 2025; OpenAI, 2025b). These methods can handle simple spatial rela- tions, but suffer from weak visual understanding and logical errors. For example, in Fig. 1, GPT-5- Thinking ignores real-world 3D object sizes and considers only pixel-wise dimensions, incorrectly concluding the coffee table is six times shorter than the sofa. These methods are also data-hungry, requiring extensive supervision. Another line of work sion specialists (Surís et al., 2023; Gupta & Kembhavi, 2023; Marsili et al., 2025), but relies on proprictary LLMs and pre-trained specialists that are poorly aligned with spatial reasoning.
LLMS for program synthesis with vi-
In this work, we tackle visual reasoning via tool use through a scalable, annotation-free training framework that jointly tunes reasoning LLMs and vision tools for spatial understanding. Drawing inspiration from reinforcement learning with verifiable rewards for mathematical reasoning (Jaech al., 2024; Guo al., 2025; Team et al., 2025), we design verifiers form structured reward models which capture correct logical decomposition, tool use and syntax. We show this reward model acts as a strong learning guide for spatial reasoning in the absence of ground truth labels. We name approach VALOR as it integrates Verifiers for Annotation Fig. shows, VALOR accurately invokes visual grounding tools, converts 2D to 3D mcasurements by integrating object depth, and accurately combines the measurements to produce the right answer.
LOgic and Reasoning. As
Accurate visual grounding is critical for spatial reasoning, as localization errors propagate through later steps. Vision specialists are pre- mre-trained on web data that diverge from spatial reasoning domains like robotics and manipulatable objects. Fine-tuning on target domains is onerous due to annotation cost. To address this, extend the verifier-based principle to the task of visual grounding. A

---

Query: What the ratio of the height of the lireplace to the combined height of the coffee table and the sofa to the right of the coffee table?
GPT-5-Thinking
VALOR (OURS)
eetere dee Hap citertie becers Elr efacct
Firelenn anard cil 2113
wnict eerterts cffiee 1ols whe gen stalls
celter sties the firekes Mil..
Answer: 0.58
Answer: 0.95
Visual reasoning relies accurate reasoning and visual grounding. To tackle the task, propose an annotation- training paradigm, called VALOR, that learns to decompose the task and invoke tools by leveraging multimodal verifiers, without the need of ground truth supervision.
Figure I:
VLM-based verifier refines the outputs of grounding model and refined predictions are recycled as training data. This loop strengthens grounding for spatial reasoning, building on ideas from bootstrapping and hard-negative mining face detection (Sung, 1996; Sung & Poggio, 1998) and object recognition (Rosenberg et al., 2005; Shrivastava et al., 2016; Radosavovic et al., 2018).
We evaluate VALOR on broad range of spatial reasoning benchmarks with natural images and show that our verifier-powered annotation-free training framework improves reasoning when com- pared to both open-source L.MMs and larger proprietary models. Additionally, we show that through reasoning and tool use, VALOR outperforms VLMs that either predict answers directly or are specif- ically tuned for visual grounding. Through a scaling analysis, we show that verifier-powe prifier-ppoweed training signal has upward trend in both reasoning and execution accuracy, offering scalable alternative to collecting expensive ground truth labels for visual tasks.
2 RELATED WORK
Tool Use in VLMs. While VLMs (Anthropic, 2024; OpenAT, 2025a; Team et al., 2023) demonstrate strong performance on visual captioning, they struggle with complex spatial reasoning tasks (Rah- manzadehgervi al., 2024; Ray et al., 2024; Thrush et al., 2022; Wu & Xie, 2024; Tong et al., 2024). Visual programming approaches tackle spatial reasoning by generating interpretable, exe- cutable programs that invoke vision specialists (Gupta & Kembhavi, 2023; Suris al., 2023; Chen et al., 2024; Marsili et al., 2025), but suffer from faulty program logic and poor execution they rely on pre-trained models not well-suited for the task. VALOR also generates programs and invokes vision specialist tools, but aligns models to the task by tuning them with multimodal verifiers.
Reinforcement Learning for Reasoning, Reinforcement learning (RL) has enhanced LLM rea- soning in domains with verifiable rewards, such as mathematics and programming (Jaech et al., 2024; Guo et al., 2025; Team et al., 2025), using tailored algorithms (Yu et al., 2025; Liu et al., 2025b). While RL for text-based reasoning is well studied (Wang et al., 2024a; Cui et al., 2025; Lambert et al., 2024), visual reasoning remains undercxplored. Recent works extend RL to visual tasks (Sarch et al., 2025; Fan et al., 2025; Dong et al., 2025; OpenAI, 2025b) by interleaving visual grounding with text- reasoning. However, our experiments show these methods struggle with grounding and reasoning, while requiring ground truth labels to train. We show VALOR outperforms them in both reasoning and grounding, with the largest gains in 3D spatial understanding.
Model Distillation. Methods in knowledge distillation typically trains student model to imitate teacher's ' output distributions (Zelikman et al., 2022; Li et al., 2024; Muennighoff et al., 2025) or internal representations (Hu et al., 2024; Xu et al., 2025; Sarch al., 2024), offering dense target- level supervision. In contrast, our work uses pretrained models as verifiers that provide feedback over intermediate outputs. providing reinforcement-basee optimization rather than teacher imitation.
Training with LLM Verifiers. LLM verifiers provide unavailable, with self-verification methods SCoRe (Zweiger et al., 2025), SQLM (Chen al.,
when ground truth labels
2

---

Quary: object washing rightmost chair9
LLM
RL
LLM
Verifier
Pytnon Execution
Answer: Righlmos"
chair
VLM Verifier
SFT
Visual
Grounding
Model
Dep
Estimation
Training
Execution
Figure 2: Method overview. Our method, VALOR, tackles visual reasoning across a broad range of spatial tasks, in 2D and 3D. During training, LLM verifiers used to improve reasoning via RL while VLM verifiers serve as critics to tune vision grounding models via SFT.
2025b), and RISE (Qu et al., 2024) generating synthetic data for RL training. SEAL (Zweiger al., 2025) applies this idea to supervised fine-tuning. SWiRL (Goldie et al., 2025) extends this approach by using high-capacity verifiers to improve low-capacity models through multi-step RL, while other methods (Liu et al., 2025a; Chen et al., 2025a; Zhao et al., 2025) leverage verifiers at inference time.
Training with VLM verifiers. Recent work integrates VLMs into vision model training to guide training. SemiVL VL (Hoyer et al., 2024) injects VLM guidance to semantic segmentation. In object detection, PB-OVD (Gao et al., 2022) uses VLMs to generate pseudo-labels from image-caption pairs. Others propose filtering: VLM-PL (Kim et al., 2024) and AIDE (Liang al., 2024) use VLMs to curate candidate labels for training, and MarvelOVD (Wang et al., 2024b) uses the detector to verify VLM outputs. While prior work uses VLM refinement, approach unifies reasoning and vision tuning via L.LM and VLM verifiers, respectively, with focus on spatial reasoning.
TRAINING VISUAL. REASONERS WITH VERIFIERS
We tackle spatial reasoning from images by combining LLM-ppwered reasoning with specialized tool use. VALOR employs LLM generate plans and executable programs and invokes vision specialists for execution. Both the reasoning and the vision grounding model are tuned for the task a label-free training paradigm. This is achieved by leveraging multimodal verifiers that critique model outputs. Their feedback serves as learning signal to improve both components, the LLM responsible for logic and the vision specialists responsible for grounding. Fig. shows our approach.
Plan & Code Generation. Given a query, the LLM generates natural language plan followed by corresponding program in Python. Available to the LLM the APIs of three function calls: GD_DETECT, returns the bounding box of all object instances specified by the noun description c.g., GD_DETECT(*CO ("CCR"),
DEPTH, returns the depth of pixel the image DEPTH(IMAGE, X, Y)
VQA, returns object's attribute (e.g., color) from the input image crop around the object e.g.,
VQA(IMAGE.CROM) CROP, "WHAT IS THE COLOR OF THE OBJECT IN THE IMAGE?")
The API specifications are provided in the prompt to the LLM (see Appendix A.10). Plans and programs are delimited by <plan></plan>>>>> and <answer></ answer> tags, respectively.
Vision Specialists. VALOR employs three vision specialist models: GroundingDINO (Liu et al., 2023b) for object localization (GD_DETECT), MoGe2 (Wang et al., 2025) for pointwise metric depth estimation (DEPTH), and GPT-5-mini (OpenAI, 2025a) for VQA with an image cropped around an object bounding box (VQA). Implementation details are in Appendix A.2.
3.1 IMPROVING REASONING WITH LLM VERIFIERS
Visual reasoning requires decomposing tasks into subtasks, correctly invoking APIs, and applying perceptual principles (e.g., distant objects appear smaller). However, ofate-of-the-an LLMs struggle across all three axes: they fail to break down queries into coherent steps, misuse tools, and neglect basic perceptual principles. Fig. 3(a) shows program generated by a pre-trained LLM, Qwen3- 8B (Yang et al., 2025): it misses the spatial object relationship (shelf "below" sink) and considers
3

---

Query: the sink 0.5m tall, how wide the shelf below it?
Query: there
Step
bicycles and helmets image?
Qwen3-88
Qwen3-88
Spotial Roward
Reward
Pormal
20 JAOO
Step 2 Parccrop objec check
(eet be on the
Aftibute Reword
Systos
Baward
St. deicch socce Wanaer)
Coopp
teggt
Step S. Ded..pppoone ppicsiion
logic Reward Feniin
Adherance
Reward
thehiggh
(a) LLM verifier
(b) VLM verifier
Figure 3: (a) LLM verifiers reward semantic correctness by evaluating logical correctness, object attribute and spatial relationship consideration, and if the code adheres to the predicted plan. Python interpreters check format adherence and syntax. (b) VLM verifiers refine visual grounding over- predictions through three stages, generating pseudo-labels from spatial reasoning queries.
wrong object dimensions (sink width instead of height). To refine reasoning, we introduce verifiers that provide feedback on the generated programs. We design reward model specialized for spatial reasoning and use it to fine-tune the L.LM to improve both program decomposition and tool usage.
Setup. Given query 9. our base LLM Te. parametrized by 9, generates natural language plan p and corresponding Python code e with access to predefined tools via their API A: (p, c) = (o; We aim to find a model, To-, that optimizes program correctness, API usage and coordination.
Reward Model. We design a reward model which verifies whether model outputs - plan & code correct. Our reward decomposes program quality into six binary components, each targeting specific aspects of spatial reasoning: (1) Format Fint (p, c) = 1 if output follows bllaa><<ppla><ansswe> <><////we>>>>> answer> template, O otherwise; (2) Syntax I'n (c) 1 if code executes without Python errors, 0 otherwise; (3) Logic () = 1 if verifier considers the plan reasonable and coherent for query q. 0 otherwise; (4) Attribute Tat (a (q. p) = 1 if all object properties (height, color, etc.) are correctly identified from the query, O otherwise: (5) Spatial Tsp (g.p) = 1 if plan addresses all spatial relationships in query. otherwise; and (6) Adherence rad (p, c) = if
the code faithfully implements the plan without deviations, 0 otherwise. The final reward is:
R(q, P, c) = Timt (p.c) [>en (c) + >oog Nlg
Mogg(..) (g.P) + Aae Fau (u,(g,)) p) + Asp Tap(y, + Aad aad (p, c)
The format reward Timt acts as hard constraint and is applied as a multiplier, while the weighted sum of the remaining rewards evaluates content quality. All r E {0, 1] and Σ. A = 1.0.
Format and syntax rewards use deterministic python interpreters (where tool calls are replaced with dummy functions), while reward components (3)-(6) require semantic evaluation, and thus employ a frozen, pre-trained LLM as the verifier model with suskispecific prompts that output binary deci- sions. Fig. 3(a) shows the output of our reward model: the verifier highlights the errors in spatial, attribute and logic decomposition and the correct components in format, syntax and adherence. We highlight the role of each reward head and provide further details in Appendix A.6.2.
Optimization. We optimize our LLM no using GRPO (Guo et al., 2025). GRPO maximizes ex- pected advantages while maintaining policy stability by preventing drift from pre-trained base policy. We provide further details in Appendix 1.
pairs, {[[ 9;), without requiring ground- truth answers, using the reward model as the sole learning signal. This design enables training to extend beyond the small labeled datasets available for spatial reasoning to arbitrary image corpora. We sample real-world images from SA-IB (Kirillov et al., 2023) and prompt Gemini-2.5-Flaan (Co- manici et al., 2025) to generate five queries per image, and uniformly select one. This process pairs unlabeled images to spatial reasoning queries. An example generated query includes "How many animals are visibly seated within the beige vehicle?".. More details and examples of generated queries are in Appendix A.4. To ensure strong coverage of 3D spatial reasoning, we additionally include (image, query) samples from MAN13D-BENCE (Brazil et al., 2023), omitting the answers.
Generating training data. We train on
4

---

In practice, found training on few hundred queries is optimal for GRPO. Our final dataset contains 800 (image, query) pairs: 400 generated from SA-IB via our query generation engine and 400 from OMNI3D- We show the impact of varying training data in experiments.
Error Analysis. As verifiers provide the learning signal in VALOR, it is essential to assess their reliability. Thus, we manually annotate rewards for 100 model outputs and treat these as ground truth. Gemini-2.5-
the LLM verifier in VALOR agrees with these annotations in 87% of cases, with most mismatches due Gemini under-rewarding. Two open-source verifiers perform worse, matching ground truth only 15% (Qwen3-8B) and 7% (Llama-3.22 2-1 IB-Instruct) of the time. This gap highlights the need for high-capacity verifiers and supports our choice of Gemini-2.5-Flash.
Implementation. We use Qwen3-8B as our base LLM Tg. pre-trained for language-only tasks (Yang al., 2025). We train 8 A 110 GPUs for 4 epochs with batch size of 64, group size G 5, and learning rate of 10-6. We use verl (Sheng et al., 2025) for training. All hyper-parametees are in Appendix A.6. We denote this trained model VALOR-RL in all experiments.
3.2 IMPROVING VISUAL GROUNDING WITH VLM VERIFIERS
In addition to logic, visual reasoning relies on accurate grounding. While math reasoning relies on error-free tools like calculators, spatial reasoning depends imperfect visual grounding mod- els such as object detectors. A major failure mode, also observed Marsili et al. (2025), is in- correct grounding of target objects. This is not surprising visual grounding is present in all queries, from counting to complex 3D reasoning. Modern detectors like GroundingDDINO (Liu et al., 2023b), trained web data, are error-prone and struggle to generalize beyond their train- ing domains. Fig. 3(b) shows GroundingDINO's output for "bicycle" and "helmet" from the input query. While it correctly finds no bicycles, it misclassifies baseball caps as helmets. Fine-tuning with domain-specific labels can mitigate these issues, but collecting such annotations is labor intensive.
We propose an alternative: improving visual grounding through VLM verifiers. Vision specialists cast predictions, VLM verifiers evaluate them, and the feedback augments their training set. This approach requires no manual annotations and scales across domains without additional labels.
VLM verifiers as data labelers. We use VLM verifiers to generate cnnntt-annotations for object detection that better capture the object distribution and target domain for spatial reasoning. We rely on image query pairs { } For each query qj. our LLM reasoning model generates plan and code. (py. ey). From code cj. we parse all grounding queries e.g., GD DETEEC and execute them with pre-trained detector. To ensure high recall, we lower the detector's confi- dence threshold. This leads to overprediction, which we validate with frozen VLM in three steps: (1) coarse filtering removes invalid detections given the image and boxes overlaid, (2) per-crop ver- ification validates remaining detections on cropped regions, and (3) de de-duplication eliminates dupli- cates. Confirmed detections form the new training set. Per-stage outputs are shown in Fig. 3(b). See more examples in Fig. 11 in Appendix A.4.2.
Generating training data. Our approach requires no ground truth object annotations. Instead we use spatial reasoning queries without answers to gather detection signals through VLM verifiers. Since few images come paired with reasoning queries, we scale training through (image, query) pairs generated as in $3.1. We generate thousands of (image, query) pairs, which result in thousands of object annotations, verified by VLMs. Leveraging VLM verifiers as pseudo-labeke allows us to scale object detection training annotation-free, going beyond specialized spatial reasoning datasets to large, unconstrained image collections.
We augment the generated (image, query) pairs with pairs from small, existing training sets of spatial reasoning datasets, to improve generality and coverage of pseudo-labels, In total, our training set consists of 7,373 images and yields a total of 30,826 bounding box annotations spanning broad range of object categories. Examples of the generated data are provided in Appendix A.4.
Error Analysis. We assess our three-stage VLM verification pipeline by measuring label precision on 100 random pipeline samples. For each stage, we compute precision from manually identified true positives (TP), false positives (FPP) and false negatives (FN) using P +TPFFF+ PN Precision increases every stage: 0.45 after coarse filtering, 0.50 after per-crop verification, and 0.75 after
1P
5

---

OMNI3D- RENCH
ROBOSPATIAL
BLINK
VSR
REALWORLD
GQA
TALLY QA
COUNTBENCI
QA
QA
Pmoprietry Models
GPT-4o
38,0
56.6
64.2
67.4
54.5
58.0
49.9
67.6
04-mini
35.6
6188
66.5
68.5
65.9
65.0
47,9
66.8
Gemini-2.0-Flas
37,0
57.0
51.7
58.5
47.3
52.1
43.6
62.1
Gemini-2.5-
37.1
68.7
61.5
68.5
62.2
65.2
48,9
65.6
Claudo-3.5-HHan
43.6
55.7
64.6
51.5
54.4
50.1
65.2
Open Source Models
Llama-3,22 3,2-1IB
32.0
58.3
54.3
58.5
46.9
39.9
47.7
66.2
Gemma-3 12B
24.4
57.4
57.9
48.9
Qwen3-8B
37.5
60.5
63.9
68.2
53.3
57.4
50.1
68.6
VALOR-RL (ours)
43.9
61.8
67.3
20.3
53.5
57.6
49.5
67.6
VALOR (ours)
44.0
69.5
69.2
75.6
$7.3
64.4
51.0
75.9
Table I: VALOR LLMS with tool use. Language-only models generate Python programs with
access to the same API of vision specialist models as VALOR. We highlight the best and second best proprietary model and the best and second best open- model.
de-duplication. Our final stage of de-duplication yields the largest improvements (+-0.25) this is by design, our pipeline hinges on the detector over wer-preeictiing boxes, which are subsequently pruned by the verifier. The pipeline achieves final precision of 75% without any human annotations.
Implementation. We use our generated data to fine-tune DroundingDINO (Liu et al., 2023b). We use the GroundingDINO-1 variant with a Swin Transformer (Liu al., 2021) vision backbone and BERT (Devlin et al., 2019) language encoder. We keep the vision and language encoder frozen and train the rest of the model on A100 GPUs for epochs with a batch size of 16 and learning rate of 10-6, Details are in Appendix A.6. We show the impact of training data size in our experiments.
4 EXPERIMENTS
We experiment on a wide range of visual reasoning tasks that test VALOR's reasoning and grounding capabilities. Our goals are threefold: (I) to measure verifier-driven RL improves task decompo- sition and tool use over strong LLM baselines; (2) to assess if multimodal verifiers improve visual grounding without annotations; and (3) to compare our approach against state-of-the aef-the-an art alternatives, including supervised RL-tuned VLMs, program synthesis methods, and direct-answer answer models.
Evaluation Benchmarks. We evaluate VALOR on diverse real-world visual reasoning tasks cover- ing different aspects of spatial understanding: OMNI3D-BENCE (Marsili et al., 2025) for 3D spatial understanding: TALLYQA (Acharya et al., 2019) and ENCHQA (Beyer et al., 2024) for counting: GQA (Hudson & Manning, 2019), REALWORLD (xAI, 2024), and VSR (Liu al., 2023a) for 2D spatial relationships and image understanding. We include relevant single-image tasks from ROBOSPATIAL. (Song et al., 2025) (compatibility, configuration) and BLINK (Fu al., 2024) (counting, spatial relationships). For DMNI3D-BENNCE we reserve 100 queries for zero-shot evaluation and use the remaining 400 queries, without answers, to train our LLM with RL ($3.1), We report exact-match accuracy for yes/no, multiple choice, and integer response queries, and MRA for floating point response queries as in Marsili et (2025). Additional details are Appendix A.3.
Baselines. We evaluate against four families of methods:
(1) LLMs with tools; We prompt proprietary and open-source LLMs (language-only) with query to generate plans and programs using our vision-specialist APIs. LLMs output natural language plans in <plan> tags and programs in <answer> tags. We execute these programs and measure accuracy. This comparison highlights the gain of augmenting LLM reasoning with verifiers.
(2) Visually grounded, RL-tuned VLMss: GRIT (Fan al., 2025) and ViGoRL (Sarch et al., 2025) finetune VL.Ms for visual grounding tasks but rely on ground truth labels for training. This compar- ison evaluates our label-free approach against RL-tuned methods with labels.
(3) Visual program synthesis methods: ViperGPT (Surís al., 2023), VisProg (Gupta & Kembhavi, 2023), and VADAR (Marsili et al., 2025). These comparisons contrast our proposed annotation- training framework with competing training free program synthesis methods.
(4) Direct-ansee answer VLMs: Proprietary and open-source VLMs directly predict answers from (image, query) pairs. This comparison contrasts VL.M prediction with reasoning and tool paradigm.

---

(o)) Query: the combined width of the two sinks 1.5m, how tall is the bathtub in meters?
[b] Queery mony onjects heightt collee stack other achieve object height as trepiace?
VALOR (OURS)
Qwwnn--og -8
VALOR (OURS)
Qwwnn--Bl 5-88
[c) Query: How many cars are in the opposite lone that are still driving? Options: (1.2.3)
(d)) Query: If the black table were pushed directly against the sofa, would the sofa visible?
VALOR (OURS)
Qwen3
VALOR (OURS)
o4-min minl
[e) Query: the armchair 1.5m wide, how tal the combined cabinet right next to it in meters?
[t)] Query: the sofa 0 4m tall, how wice is the stoal in front of the piano in meters?
VAIOR (OURS)
VIGoRL
VALOR [OURS)
GRIT
DI
Outputs for VALOR, LLMs with tools, and RL-tuned VLMs. For each example, show the image, query, and model output. We recommend zooming in to read the outputs.
Figure 4:
4.1 VALOR vs. LLMS WITH TOOL USE
Table I compares VALOR against proprietary and open-source LLMs. All methods receive query and generate plans and programs with their respective language-only LLM and access to identical vision-specialist APIs as our method. Since the baselines rely solely on language-on) LLMs, there no risk of data leakage between models and our evaluation benchmarks. All baselines, apart from the last row (VALOR), use the same visual grounding modules. We summarize our findings below:
VALOR-RL vs open-source LLMs:
The impact of verifier-imprrvedd reasoning. Among open- --mmma-3-12B, Qwen3-8B), Qwen3-8B performs best. In turn,
source LLMs (LLama-3.2-11) VALOR-RI, our RL training approach on Qwen3-8B, shows gains over the base model: +6.4% on MANI3D-BBNCEE (43.9% vs. 37. 37.5%), +3.4% on BL.INK (67.3% vs. 63.9%), +2.1% on VSR (70.3% vs. 68.2%). Both use the same specialists, isolating contributions from training with the LLM verifier. On counting tasks TALLYQA and COUNTBENCHQA (e.g.. "How many people are there?") and simpler sets like GQA (e.g.. "Who is wearing the dress?"), reasoning is less critical, and VALOR-RI RL matches Qwen3-8B. 8B. Fig. 4(a)-(c) compares programs from VALOR and Qwen3- 8B. Qwen3-8 3BB misses key details- - ignoring two distinct sinks in (a), omitting the doubled height of the coffee table in (b), and overlooking the "still driving" constraint in (c) which VALOR captures. Example (C) also shows VALOR's 's superior tool use, framing VQA as yes/no rather than open-ended.
The impact of verifier-improved visual grounding. Table compares our full method, VALOR, that improves visual grounding via multimodal verifiers. VALOR and VALOR-RL share programs, isolating contributions from our improved visual grounding model
VALOR vs VALOR-RL:
7

---

BASE MODEL
OMNE3D- BUNCH
ROBOSPATIAL
BLINK
VSR
REALWORLD QA
GQA
TALLY QA
COUNTBENCH
QA
RL-Tuned VLMs
GRIT
Qwen2.5- -V--3BB
27.3
58.8
70.3
77.6
55.6
75.0
46.4
68.6
VIGoRL
Qwen2.5-VL-7B
68.4
79.1
52.8
49.3
Ours
VALOR-RI.
Qwen3-8B
43.9
61.8
67.3
70.3
53.5
49.5
67.6
VALOR
Qwen3-88
44.0
69.5
69.2
75.6
57.3
51.0
Table 2: VALOR vs RL-tuned VLMs. GRIT (Fan et al., 2025) and ViGoRL (Sarch et al., 2025)
rely on labeled data and ground in text, whereas VALOR trains without labels and leverages tools. VALOR outperforms them on reasoning-heas -eavy spatial tasks (OMN13D-BENCH ROBOSPATIAL).
Query: S bag fight of the chair? Answer yes or.
Query: is the bowl behind the chair? Query: is the montor tront of the cup?
Query: How many posters are there in the image?
Qvery: How many biue floots ore there
Query: the ciacciu bae From liscre?
VALOR (ours)
VIGool
VALOR
VALOR RL
Figure 5: Visual grounding in VALOR, Figure 6: VALOR VALOR-RL. Multimodal GRIT and ViGoRL.
verifiers improve grounding
($3.2). VALOR yields strong gains, particularly grounding COUNTBENCHQA, +7.7% on ROBOSPATIAL, and +5.3% on VSR. Improvements on OMNI3D- BENCH smaller, as complex queries make reasoning the main challenge for smaller LLMs as we show in Fig. 8 in Appendix A.1.1. Fig. 6 shows our verifier-trained object detector in VALOR correcting the overpredictions made by the original GroundingDINO in VALOR-RL. Notably, im- proving visual grounding for spatial reasoning does not harm general object detection; our training slightly boosts performance on the COCO (Lin al., 2014) validation set: 48.4% to 48. 48.7% mAP. To further explore the impact of perifier-improsee grounding, we include comparisons of the baseline LLMs with the improved detector in Appendix A.7. We illustrate failure cases in Appendix A.1.2.
grounding-focused focused benchmarks: +8.3%
VALOR-RL vs proprietary LLMs. Although not our primary focus, we compare against propri- etary LLMs with access the same specialists (top of Table 1). These LLMs are far larger or were distilled from larger models compared to open-source variants; with few details about them known. VALOR-RL, with much smaller LLM, performs favorably, e.g., VSR (70.3% vs 68.5% from 04-mini and Gemini-2.5-Flaas and DMNI3D-BENC (43.9% vs 43.6% from Claude-3.5-Haiik)
Fig. 4(d) compares our method to o4-mini, the most favorable proprietary LLM,, on a query requiring reasoning over size-based visibility if a table were positioned in front of the sofa. 04-mini computes the overlap between the 2D bounding boxes and separately their depth ordering, essentially assessing current visibility. In contrast, VALOR incorporates depth into 2D measurements to estimate 3D object widths to correctly determine visibility under the given hypothesis.
4.2 VALOR VS. RL-TUNED VLMS
We compare VALOR to RL-tuned VLMs GRIT (Fan et al., 2025) and ViGoRL (Sarch et al., 2025). Unlike VALOR, these models (i) reason in text without tools and (ii) require labeled data while ours is label-free. GRIT trains TALLYQA and VSR, while ViGoRI. trains on the synthetic SAT2 dataset (Ray et al., 2024); both use Qwen2.5-Ve as their base VLM. Little known about Qwen2.5- VL's training data or safeguards against overlap with multimodal spatial reasoning benchmarks, so leakage is a concern for these basclines. This is not the case for VALOR, which uses language-only LLMs trained language-only tasks.
Table 2 shows VALOR outperforms GRIT and ViGoRL on OMNI3D OMNI3D-BENCH BENCH (44.0% vs 27.3% from GRIT) and ROBOSPATIAL (69.5% vs 64.9% from ViGoRL). Notably, VALOR outperforms GRIT on TALLYQA (51.0% vs 46, 4%) despite GRIT training on its train set and our method being trained without labels. Fig. 4(e)-(f) shows reasining outputs from GRIT and ViGORL that reveal they ignore
8

---

grounding measurements: GRIT uses prior knowledge of stools in (e), ViGoRL estimates the cabinet height from nearby objects in (f). In contrast, VALOR correctly uses grounding information to scale object dimensions per the query.
Tool use yields explicit grounding. key difference of these approaches lies their visual ground- ing. GRIT predicts bounding boxes and ViGoRL points of interest through text generation which steer model predictions. In contrast, VALOR explicitly invokes specialized visual grounding models and uses the grounding output compute object dimensions and attributes. This difference creates notable discrepancies in grounding ability, shown in Fig. 5. In the top example, GRIT omits query- relevant objects like the chair. In the bottom example, omits the bottom-right poster. ViGoRL's point predictions are not aligned with target objects. In contrast, VALOR detects all relevant objects and grounds its reasoning accordingly.
4.3 VALOR VS. PROGRAM S SYNTHESIS METHODS
We compare visual program synthesis methods, Vis- Prog (Gupta & Kembhavi, 2023), ViperGPT (Surís et al., 2023), and VADAR (Marsili et al., 2025), which produce executable programs for spatial reasoning. These meth- ods use proprietary LLMs,, such as GPT-4o, with sophis- ticated prompting to generate programs. VALOR tunes much smaller Qwen3-8B base model. All methods are annotation-fres but VALOR trains for reasoning and vi- sual grounding with the help of verifiers.
BASE MODEL
OMNI3D.
GQA
BENCH
VisProg
GPT-4o
17.8
46.9
ViperGPP
GPT:3.5
25.5
42.0
VADAR
GPT-4o
46.1
VALOR-RL
Qwen3-8B -BB
43.9
55.0
VALOR
Qwen3-88
44.0
63.0
Table 3: VALOR vs Program Synthesis.
and GQA. We evaluate on OMNI3D-BENCH'S 100 held out queries and the GQA subset from VADAR (Marsili et al., 2025). VALOR outperforms all methods on OMN33D BENCH (44.0% vs 38.9% from VADAR) and GQA (63.0% vs 46.9% from VisProg), despite using a smaller open-source LLM than GPT-based baselines.
Table 3 shows results on OMNI3D-BENTE
4.4 VALOR VS. VLMS
We compare VALOR proprietary and open-source VLMs that predict answers di- rectly from (image, query) pairs. As these are trained on proprietary datasets, data leak- age a concern; thus, we evaluate on bench- marks released after their release. Table 4 reports results on OMNI3D-BENCHI, Ro- BOSPATIAL, and COUNTBENCHQA. Al- though the benchmarks predate the model release, we include comparisons GPT-5- mini in Appendix A.5.
OMN13D- BENCH
ROBOSPATIAL
COUNTBENCH
QA
Pmprrcttry Models GPT-4n
35.0
$7.9
84.7
Gemini-2.0-Flass
34.6
60.9
88.6
Claude-3.--Halke
32,5
59.6
77.6
Open-Source Models
Llama3
22.7
39,9
71.5
Qwen2.5 VB.
20.7
60.1
82.5
VALOR-RI.
43.9
61.8
67.6
VALOR
44.0
69,5
75.9
Table 4: VALOR Direct Prediction VLMs.
We observe that VALOR outperforms larger proprietary models on complex reasoning benchmarks DMNI3D-BENCE (44.0% vs 35.0% from GPT-40) and ROBOSPATIAL (69.5% vs 60.9% from Gemini-2.0-Flaan demonstrating the effectiveness of approach. These improvements are larger when comparing to open-source models: VALOR achieves +21.3% 13%% from Llama3.2 on OMNI3D- BENCH (44.0% vs 22.7%). On COUNTBENCHQA, VALOR trails. This suggests that VLMs may be stronger bases for visual grounding than current state-of-fi of-the-art models like GroundingDINO.
4.5 IMPACT OF TRAINING DATA SIZE
Training Samples
OMN13D- BENCH
Our proposed annotation- free training framework scales inher- ently. We analyze how the number of training samples impacts both reasoning and visual grounding capabilities in VALOR.
Qwen3-8B VALOR-
37.5
40.0
VALOR-RL
160
39.2
VALOR-RL
400
40.8
Impact of training set size reasoning. Table 5 shows VALOR-RO RL performance on DMNI3D-BENCEE our most chal- lenging spatial reasoning benchmark, training set size scales.
VALOR-RL
800
43.9
Table 5: Effect of training set size for reasoning.

---

We tune Qwen3-8B with N (40, 160, 400, 800) queries (half from OMNI3D-BENCH, half gen- erated), and also report results for the base model. The VALOR-RL variant reported in experiments trains on 800 samples. Specialist models are fixed across all variants. We find VALOR surpasses the base Qwen3-8B model with as few as 40 samples (40.0% vs 37.5%), and continues improving with more data (43.9% with 800 samples vs 40.8% with 400 samples). This demonstrates strong data scalability.
Impact of training set size visual grounding. Fig. shows the impact of training data on verifier-trained visual grounding. We tune GroundingDINO with M {5.6k, 8.1k, 17.4k, 30.8k) pseudo- annotations and plot accuracy on Ro- BOSPATIAL, VSR, and COUNTBENCHQA benchmarks which heavily rely on visual grounding. All variants execute identical programs, performance differences arise solely from grounding. The upward trend training data increases demonstrates the scalability of our verifier-guide training.
Accuracy (%)
72
88
%
Benchmarks RaboSpatial
64
VSR
CountBenchgA
3000
10000
15000
20000
25000
30000
Training Pseudo-labels
Figure 7: Effect of training data for visual grounding.
4.6 SUPERVISED PINE-TUNING VS. REINFORCHMENT LEARNING
Although not the primary focus of our work, we ablate the post-training algorithm by comparing our verifier-based RL training ($ 3.1) with supervised fine-tuning (SFT) in Table 6. Our method optimizes a sparse verifier reward via reinforcement, whereas SFT provides dense supervi- sion over full model outputs. To keep provenance of out- puts and verifier signal consistent, we construct an SFT dataset by sampling VALOR training outputs with veri- fier reward 2 0.7, ensuring both methods learn from the same verifier-filtrred program traces.
OMNL3D- BENCH
ROBOSPATIAL
COUNTBENCH
VALOR
44.0
69.5
75.9
Table SFT vs RL. We tune Qwen3- 8B on high-reward samples with SFT and compare to our RL-based VALOR.
We fine-tune Qwen3-8B these samples using SFT and evaluate the resulting model with the improved visual grounding module for a direct comparison to VALOR. SFT consistently underper- forms RL reasoning-heate benchmarks such OMNI3D-BENCH (38.3% 44.0%) and Ro- BOSPATIAL (64.5% vs 69.5%), indicating that GRPO more effectively improves model reasoning. On the counting benchmark COUNTBENCHQA, both approaches perform similarly. We include all evaluation results in Table 9 in Appendix A.8.
5 CONCLUSION & FUTURE WORK
We introduce VALOR, an annotation-free training paradigm for visual reasoning that leverages mul- timodal verifiers to improve LLM reasoning and visual grounding. A key insight driving our ap- proach is that stronger VLMs/LLMs are often more reliable as verifiers than generators, motivating a verifier-based strategy for improving visual reasoning. We evaluate VALOR on visual reasoning tasks spanning 2D and 3D image understanding to object counting, highlighting success and failures through an extensive quantitative analysis. Further failure analysis Appendix A.I, Fig. 8 and 9. We release code and models to facilitate future work. Some promising directions are below:
We show how multimodal verifiers can pseudo-annotate images and improve grounding. Integrat- ing them into RL training is an extension of this paradigm.
Harvesting hard negatives with VLM verifiers improved visual grounding models. Following similar principles for reasoning through guided query generation is a promising direction.
VLMS are strong grounders, with Gemini-2.0-F achieving 88. 88.6% on COUNTBENCHQU via direct prediction. Adapting them grounding tools could further benefit methods like VALOR.
10

---

ACKNOWLEDGMENTS
We thank Aadarsh Sahoo, Ilona Demler, and Ziqi Ma for their feedback on the project. The project funded by Meta through the LLM evaluation research grant and partly through Caltech's CAST program. We also thank Google's Gemma Academic program for granting us API credits for their LLMs.
REFERENCES
Manoj Acharya, Kushal Kafle, and Christopher Kanan. questions. In AAAI, 2019.
Tallyqa:
Answering complex counting
Anthropic. Claude, 2024. URL httpp::/ww..nttrrott// wwtp:ppp.....cccooooaooee clan/c
Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Koppula, Fangyu Liu, Adam Grycner, Alexey Grit- senko, Neil Houlsby, Manoj Kumar, Keran Rong. Julian Eisenschlos, Rishabh Kabra, Matthias Bauer, Matko Bošnjak, Xi Chen, Matthias Minderer, Paul Voigtlaender, loana Bica, Ivana Bal- azevic, Joan Puigcerver, Pinelopi Papalampidi, Olivier Henaff, Xi Xiong, Radu Soricut, Jeremiah Harmsen, and Xiaohua Zhai. PaliGemma: A versatile 3B VLM for transfer. arXiv preprint arXiv:2407.07726, 2024.
Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3D: A large benchmark and model for 3D object detection in the wild. In CVPR, 2023.
Jiefeng Chen, Jie Ren, Xinyun Chen, Chengrun Yang, Ruoxi Sun, Jinsung Yoon, and Sercan Ö Arik. Sets: Leveraging self-verification and self-correction for improved test-time scaling. arXiv preprint arXiv:2501. 2025a.
Lili Chen, Mihir Prabhudesai, Katerina Fragkiadaki. Hao Liu, and Deepak Pathak. Self-questioning language models. arXiv preprint arXiv:2508.03668 2025b.
Zhenfang Chen, Rui Sun, Wenjun Liu, Yining Hong, and Chuang Gan. Genome: generative neuro- symbolic visual reasoning by growing and reusing modules. In ICLR, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities, arXiv preprint arXiv:2507.06261, 2025.
Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456. 2025.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In ACL, 2019.
Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. In CVPR, 2025.
Yue Fan, Xuchai He, Diji Yang, Kaizhi Zheng, Ching-Chen Kuo, Yuting Zheng, Sravana Jyothi Narayanaraju, Xinze Guan, and Xin Eric Wang. Grit: Teaching milms to think with images. In NeurlPS, 2025.
Xingyu Fu, Yushi Hu, Bangzheng Li. Yu Feng, Haoyu Wang. Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. ECCV, 2024.
Mingfei Gao, Chen Xing, Juan Carlos Niebles, Junnan Li, Ran Xu, Wenhao Liu. and Caiming Xiong. Open vocabulary object detection with pseudo bounding-hox labels. In ECCV, 2022.
11

---

Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D Manning. Synthetie data generation & multi-step rl for reasoning & tool use. arXiv preprint arXiv:2504.04736 2025.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. CVPR, 2023.
Lukas Hoyer, David Joseph Tan, Muhammad Ferjad Naeem, Luc Van Gool, and Federico Tombari. Semivl: semi-supervised semantic segmentation with vision-language guidance. In ECCV, 2024.
Yushi Hu, Otilia Stretcu, Chun- Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ran- jay Krishna, and Ariel Fuxman. Visual program distillation: Distilling tools and programmatic reasoning into vision-language models. In CVPR, 2024.
Drew A Hudson and Christopher Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, 2019.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.1672)) 2024.
Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Car- ion. Mdetr-modulated detection for end-to-end multi-modal understanding. In ICCV, 2021.
Junsu Kim, Yunhoe Ku, Jihyeon Kim, Junuk Cha, and Seungryul Baek. VIm-pl: Advanced pseudo labeling approach for class incremental object detection via vision-language model. In CVPRW, 2024.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan- Wan-Yen Lo, et al. Segment anything. In ICCV, 2023.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15122 2024.
Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jianwei Yang. Chunyuan Li, Yiwu Zhong. Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre- training. In CVPR, 2022.
Zejun Li, Ruipu Luo, Jiwen Zhang, Minghui Qiu, Xuanjing Huang, and Zhongyu Wei. Vocot: Unleashing visually grounded multi-step reasoning in large multi-modal models. arXiv preprint arXiv:2405. 16919, 2024.
Mingfu Liang, Jong-Chyi Su, Samuel Schulter, Sparsh Garg, Shiyu Zhao, Ying Wu, and Manmohan Chandraker. Aide: An automatic data engine for object detection in autonomous driving. In CVPR, 2024.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.
Fangyu Liu, Guy Edward Toh Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 2023a.
Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can lb LLM surpass 405b L.M?? rethinking compute-optimal test-time sealing. In ICLRW, 2025a.
Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint Xxiv:2303.05499 2023b.
12

---

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer. Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10012-10022, 2021.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang. Chao Du, Wee Sun Lee, and Min Lin. Understanding rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503.2078: 2025b.
Damiano Marsili, Rohun Agrawal, Yisong Yue, and Georgia Gkioxari. Visual agentic ai for spatial reasoning with dynamic api. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19446-19455, 2025.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.1939; 2025.
OpenAl. gpt-5, 2025a. URL https::/p/// //pp:/ai//p/....////// wttiinaaa..cmmaaammmmmmmmmeeeeee http:///add///ttttrooooonnngggg.
OpenAL.
04-mini, 2025b.
URL
introducing-o3- -000------aa-----ii-----
Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching language model agents how self-improve. In NeurlPS, 2024.
Ilija Radosavovic, Piotr Dollár, Ross Girshick, Georgia Gkioxari, and Kaiming He. Data distillation: Towards omni-supervised learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4119-4128 2018.
Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. Vision language models are blind: Failing to translate detailed visual features into words. arXiv preprint 1,6558, 2024.
Arijit Ray, Jiafei Duan, Ellis Brown, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, et al. Sat Dynamic spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024.
Chuck Rosenberg, Martial Hebert, and Henry Schneiderman. Semi-supervised self-training of object detection models. In WACV-W, 2005.
Gabriel Sarch, Lawrence Jang, Michael Tarr, William W Cohen, Kenneth Marino, and Katerina Fragkiadaki. Vlm agents generate their own memories: Distilling experience into embodied programs of thought. In NeurIPS. 2024.
Gabriel Sarch, Snigdha Saha, Naitik Khandelwal, Ayush Jain, Michael Tarr, Aviral Kumar, and Katerina Fragkiadaki. Grounded reinforcement learning for visual reasoning. In NeurIPS, 2025.
Christoph Schuhmann, Robert Kaczmarczyk, Aran Komatsuzaki, Aarush Katta, Richard Vencu, Romain Beaumont, Jenia Jitsev, Theo Coombes, and Clayton Mullis. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. In NeurlPS- W, 2021.
Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. ICCV, 2019.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, 2025.
Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 761-769, 2016.
Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. Ro- boSpatial: Teaching spatial understanding to 2D and 3D vision-language models for robotics. In CVPR, 2025.
13

---

K-K Sung and Tomaso Poggio. Example-bar based learning for riew-based human face detection. IEEE Transactions on pattern analysis and machine intelligence, 20(1))39-51) 1998.
Kah-Kay Sung. Learning and example selection for object and pattern detection. MIT, 1996.
Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. In ICCV, 2023.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint drXiv:2312.11805, 2023.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501.12599, 2025.
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio-linguistic composi- tionality. In CVPR, 2022.
Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang. Adithya Iyer, Xichen Pan, et al. Cambrian-l: A fully open, vision-centric exploration of multimodal 1lms. NewrIPS, 2024.
Huaijie Wang, Shibo Hao, Hanze Dong, Shenao Zhang, Yilin Bao, Ziran Yang, and Yi Wu. Offline reinforcement learning for Ilm multi-step reasoning. arXiv preprint 0rXiv:2412.16145 2024a.
Kuo Wang. Lechao Cheng, Weikai Chen, Pingping Zhang, Liang Lin, Fan Zhou, and Guanbin Li. Marvelovd: Marrying object recognition and vision-language models for robust open-vocabulary object detection. In ECCV, 2024b.
Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang- Moge-2: Accurate monocular geometry with metric scale and sharp details. arXiv preprint arXiv:2507.02546 2025.
Penghao Wu and Saining Xie. V Guided visual search core mechanism in multimodal Ilms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 13084 1004--3005 13094, 2024.
xAI. Real world qa. 2024. URL https: /x/////x. www.iiiia.... http///ws/grool1.1/
Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. In ICCV, 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source II reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. NeurIPS, 2022.
Eric Zhao, Pranjal Awasthi, and Sreenivas Gollapudi. Sample, scrutinize and scale: Effective inference-tine search by scaling verification. In ICML, 2025.
Wei Li Zuwei Long. Open grounding dinorthe third party implementation of the paper grounding dino, 2023. URL htt.t:ttt////c...mm / thub. com/ 1 1oggzwl 997 /ptp:///nnnornndii...e innino....o.....
Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, and Pulkit Agrawal. Self- adapting language models. arXiv preprint arXiv:2506. 10943, 2025.
14

---

A
APPENDIX
CONTENTS
Appendix
A.1
Failure Cases
A.1.1
Reasoning Failure Cases
A.1.2
Grounding Failure Cases
A.2
Additional Vision Specialist Details
A.3
Additional Evaluation Data Details
A.4
Generated Training Data
A.4.1
Generating Spatial Reasoning Queries for Improved Reasoning.
A.4.2
Generating Data for Improved Visual Grounding
A.5
VALOR GPT-5-mini
A.6
Additional Training Details
A.6.1
GRPO.
A.6.2
Reward Design.
A.6.3
LLM Reasoning Training.
A.6.4
GroundingDINO Fine-tuning.
8
A.7
Additional Evaluations of Verifier-Improved Visual Grounding
A.8
Supervised Fine- Tine-Tnnig vs Reinforcement Learning
A.9
Hyperparameters
10
A.10 Prompts
10
A.1
FAILURE CASES
In this section, we show error cases of VALOR. We include errors in reasoning in $A.1.1, 111, along with a brief explanation on why the program is incorrect. $A.1.2 we highlight errors in grounding.
A.1.1 REASONING FAILURE CASES
We show reasoning failure cases in Fig. 8. In the query on the left, we find VALOR oversimplifies the challenging hypothetical posed the query by ignoring the direction of travel specified for the chalice, and performing comparison of depth over the wrong objects. In the second query the right, VALOR incorrectly conflates depth with distance, while this query requires considering distance in 3D (the utensil holder is directly below the drawer and directly to the right of the edge of the counter). Additionally, VALOR omits comparisons with the drawers. Despite improved performance, the challenging hypotheticals in OMNI3 remain difficult for VALOR.
A.1.2 GROUNDING FAILURE CASES
We show grounding failure cases VALOR in Fig. 9. We find that while VALOR improves ground- ing, under predictions remain: in the first example VALOR finds 5 toilets opposed to 6 and only of the basins. In the second example, VALOR does not successfully ground all the windows of the bus. Lastly, some duplicate predictions remain, as shown in the duplicate detection of the glass.

---

Query: the silver chalice were to fall the dircction of the polaa dol chair, would it hit the cream- chair hitting the foor?
VALOR (OURS)
Query: Is the ufensil holder closer to the edge of the black counter or the brown
VALOR (OURS)
drower above them?
Expeoofiie hwoodhoooeee criiing the chellcok netwern olgeci
conscotion compoiing toown dibir
cholce
Figure 8: Reasoning failure cases of VALOR challenging hypothetical examples from OMNI3D- BENCI. For each query, we show the image and generated program, as well as an explanation of incorrectness.
Query: mony of the basins?
Query: many windows front and not the sides of the bus?
Query: How many glosses empty?
Figure 9: Grounding failure cases of VALOR. For each query we show the original image and the outputs of the verifier-tuned grounding model.
A.2 ADDITIONAL VISION SPECIALIST DETAILS
Our vision specialist API consists of object detector, segmentation model, depth estimation model, and a VQA model. Fig 16 shows the full API docstring. We provide additional details of each specialist model below.
Object Detection. We take GroundingDINO-T (Liu et al., 2023b) model pretrained on Ob- jects365 (Shao al., 2019). GoldG (Kamath et al., 2021), and Cap4M (I.i al., 2022) object detector. We specify our API that the prompt to the object detector can either be comma-separated nouns, e.g., "fireplace, coffee table, sofa", nouns accompanied by appearance attribute, e.g., "black coffee table". We additionally specify that spatial relationship attributes should not be in- cluded in the prompt, as we found the object detector unable consider them appropriately.
Depth Estimation. For depth estimation we use the vit-l-norma. checkpoint from MoGe2 (Wang et al., 2025). Our depth module takes as input object bounding box and returns the estimated depth at the center of the box.
The primary purpose of the VQA module is to answer queries about the appearance of an object specified by an input bounding box (e.g. "What color is this object?"). We take a crop of the image around the input bounding box and pass the cropped image to GPT-5-mini (OpenAI, 2025a)
VQA.
alongside the query. We also allow broader questions in the event of an early exit due to failed detections. specified by a bounding box of None (see Fig. 17). In this case, the entire image is passed to GPT-5-mini.
A.3 ADDITIONAL EVALUATION DATA DETAILS
We provide additional details on the datasets used for evaluation below. We report exact-match accuracy for all yes/no, multiple choice, and integer response queries, and MRA for floating point response queries following Marsili et al. (2025). Additional details the evaluation datasets found in Appendix A.3. To account for synonyms in the open-ended answers of GQA, we use GPT- 5-mini (OpenAI, 2025a) as judge for that benchmark. The prompt GPT-5-mini is in Figure 20.
be
2

---

Omni3D-Bendh Bendh is VQA dataset focused on 3D grounding and reasoning (Marsili et al., 2025). The dataset includes 500 challenging (image, question, answer) tuples of diverse real-world scenes sourced from Omni3D (Brazil et al., 2023). Examples include "If the 3D height of the two-seat sofa is 0.50 meters, what is the 3D) height of the dining table in meters? ? and "Which object is closer to the fireplace: the sofa or the white coffee table?". We keep the first 100 samples in OMNI3D- BENCH for evaluation and use the remaining 400 samples for training.
TA is VQA dataset that exclusively tests counting (Acharya al., 2019). Examples include "How many people are in this picture?" and "How many sheep do you see?". We evaluate on a subset of 491 questions provided in GRIT (Fan et al., 2025) which are uniformly sampled to ensure an equal distribution of ground truth object counts from 0 to 9.
GQA is popular visual reasoning dataset where queries generated using scene graphs (Hudson & Manning, 2019). Queries in GQA primarily pertain to 2D spatial relationships and the visual appearance of objects. Examples include "Who is dressed in yellow?" and "What is the name of the animal to the left of the bookcase?", We evaluate on a subset of 509 questions proposed in GRIT (Fan al., 2025), which has been manually filtered to remove ambiguous answers.
RoboSpatial is a large-scale VQA dataset of indoor and tabletop environments for spatial reasoning in roboties (Song al., 2025). Queries in ROBOS are procedurally generated from 3D scans. Examples include "Can the chair be placed in front of the table?" and "Is the mug to the left of the laptop?". " We evaluate on the relevant compat ibiliiiiitttt litty and con configuration tasks.
VSR is a spatial reasoning dataset composed of images from COCO (Lin et al., 2014) annotated with spatial relationships between two objects in the image (Liu et al., 2023a). Queries formatted as statements which model must evaluate as True or False. Examples include "The potted plant at the right side of the bench" " and "The cow is ahead of the person".
BLINK is a benchmark of 14 visual reasoning challenges that can are trivial for humans but chal- lenging for VLMs (Fu et al., 2024). The dataset is composed of single-image queries and multi- image queries. We restrict our evaluation to single-image queries and evaluate on the val split of the relevant count ing and spatial relationship tasks.
CountBenchQA is a counting benchmark released with the PaliGemma model (Beyer et al., 2024). The dataset consists of manually annotated counting queries with images sourced from the LAION- 400M dataset (Schuhmann et al., 2021). Examples include "How many light bulbs are there in the image? and "How many food containers are there in the image?". We evaluate on the publicly available 491 image split.
WoaldAA is dataset of 765 VQA queries sourced from diverse real-world scenes (xAI, 2024). The benchmark tests general spatial understanding with an emphasis on real-world outdoor scenes. Examples include "Is there a bike lane my right? Please answer directly with single word or number." and "How many pedestrians are there within 50 meters from us? A. 3 B. 5 C. 7 or more". We evaluate on the full test split.
A.4 GENERATED TRAINING DATA
Our approach involves training both base LLM for logic and program decomposition as well as a visual grounding model. In both cases, we leverage our method's lack of reliance on ground truth labels to generate supplementary synthetic data used for training. In this section we provide additional details on this process.
A.4.1 GENERATING SPATIAL REASONING QUERIES FOR IMPROVED REASONING.
Since our method requires only spatial reasoning queries without labels, we can scale training be- yond small labeled datasets by sampling images from SA-IB (Kirillov et al., 2023) and prompting Gemini-2.5-FFie to inspect each image for suitability (e.g., multiple objects for grounding). For suitable images, we prompt it to generate five queries per image and uniformly select one. While don't 'tn'tuu use these images to train our LLM, seeding query generation with reference images produces more diverse spatial reasoning queries and ensures referenced objects are in-distribution. Similarly,
3

---

sampling from five predictions per image yields greater diversity than generating single queries di- rectly. We show example queries with their reference images in Figure 10 and include the generation prompt in Figure 19. We use default hyperparameters for Gemini
Reference imge
Generated Quary
Reference Image
Ger Query
Reference Imoge
Generated Query
the height of of the curdoor bor the opprodemato height of one of the sou dre potted plonts. Ind metent
Is the uo mat on the floor postioned entirely to the loft the cllarr
Are the trees locoted teehndd bont the buiiting with vonoden
the red stiping on brack oar 1005 many meter wibe the fie on the front fight wheel
Ihe hoight of the concaale pilia unib, woruld heightt if the blant noll upper than zumbh
How many lorge, block lompoossis are visible in the Image
Figure 10: Example spatial reasoning queries generated from reference images.
A.4.2 GENERATING DATA FOR IMPROVED VISUAL GROUNDING
We use spatial reasoning queries to generate detection signals through VLM verifiers to capture the distribution of object types and target domains pertinent spatial reasoning. We source (im- age.query) pairs from spatial reasoning benchmarks TALLYQA, GQA, VSR and OMNI3D-BENCH, and supplement these with queries generated as described by $A.4.1. W
From these limage.query: pairs, we use our language-basee based reasoning model to produce a plan and code. We subsequently extract all visual grounding queries in the program (GD DETECT) and execute them using GroundingDIINO model with both BOX THRESHOLD and TEXT. THRESHOLD lowered to 0.2 to ensure higher recall. Lowering the thresholds leads to consistent overprediction of bounding boxes, which we then filter using three step process using GPT-5-mini (OpenAI, 2025a) as our VLM verifier. We use default generation hyperparameters for GPT-5-mini. All prompts used for verification are in Fig, 21. We detail verification step below:
1. Coarse filtering: We overlay all predicted boxes on the original image and pass both the annotated and original images to GPT-5-mini. Each box is numbered and labeled, with JSON of predictions provided boxes whose content doesn't 't match the predicted label or that are poorly fitted. This coarse filtering removes significant portion of incorrect boxes.
the model. GPT-5-mini prompted remove
2. Per-crop object check:
For each remaining box, we crop the image around the box and upscale it so the longest side is 640px while preserving aspect ratio. We pass each crop with its predicted label to GPT-5-mini and ask whether the primary object matches the label. We discard boxes that GPT-5-mini flags as incorrect.
3. Deduplication: As a final step, we repeat the process from Step with all remaining boxes. GPT-5-mini is prompted to focus duplicates, these likely weren't 't filtered in the second stage due to being semantically correct. We remove boxes flagged as duplicates and keep the remaining boxes as our final pseudo-labels.
We show additional samples from our VI.M-bseed verifier in Figure 11. For each sample, we show the original image, the bounding boxes predicted by the GroundingIIINO model with reduced confi- dence threshold and the final verifier outputs. We overlay all bounding box predictions in red along with the predicted label. The low-threshold predictions are the input our verification pipeline (step above). We observe that despite a highly cluttered input image, our three-stage verification pipeline is effective removing semantically incorrect and duplicate boxes.
Verifier Failures. In Figure 12, we visualize failure cases for our VLM verifier. For each sam- ple, we show the original image, the boxes predicted by the GroundingDINO model with reduced confidence threshold and the final verifier outputs. From Figure 12, we observe that our verifier occasionally fails when two semantically identical objects overlap significantly. This is highlighted in the discarded boxes on the right side the truck in the top-left example, the crowd of people in the bottom left example, and the two men on the motorcycle in the bottom right. On the top left, the
4

---

Original Image
Low-Ihreshaid predictions
Verilior Output
Original Image
Low-Ihreshold predictions
Verilier Output
Figure 11: Additional examples of VLM-based verification of visual grounding model outputs. For each sample we show the original image. the predicted bounding boxes by the low-th wwwthhrehoold model, and the final pseudo-annotations produced by the VLM verifier.
Original Image
Low-threshald predictions
Verfler Outpul
Original
Low-threshold predictions
Verifer Culput
Image
Failure cases of our VL.M- verifier. For each sample we show the original image, the predicted bounding boxes by the low-threshold model, and the final aseudo-annotationn produced by the VLM verifier. Our verifier occasionally fails when two similar objects overlap significantly.
Figure 12:
small bicycle bounding boxes discarded by the per-crop semantic check, likely due to being too small to accurately resolve. Verification in these settings remains a challenge.
A.5 VALOR VS GPT-5-MINT
We report direct prediction accuracy of GPT-5-mini in Table 7. We note that GPT-5-mini was released after all of the benchmarks, so cannot rule out data leakage, as it is possible these benchmarks were included in its training corpus. Additionally, to explore how object grounding affects performance for GPT-5-mini, we report an additional variant GPT-5-mini (grounded) where the model is prompted to explicitly predict bounding boxes for the relevant objects in the query prior to final response.
Despite its scale and capabilities, VALOR outperforms GPT-5-mini on MAAN3D--BECCEE our most reasoning-intent intensive benchmark. Notably, VALOR built on a much smaller 8B language-only model - achieves strong results across all benchmarks, which underscores the effectiveness of our verifier-guided training framework. The grounded variant of GPT-5-mini performs similarly to the most tasks but degrades several reasoning-intenste benchmarks, improving only on the counting datasets (TALLYQA. COUNTBENCHQA), This suggests that while explicit grounding can help VLMs in object-centric counting scenarios, visually grounded multi-step reasoning without tool-use remains challenging even for large proprietary VLMs.
direcc-prediction version
Figure 13 additionally compares the visual grounding produced by VALOR with that of GPT-5- mini, the VLM we use as a verifier. Although GPT-5-mini is highly effective at evaluating object detections, observe that it often struggles when generating bounding boxes for spatial queries.
5

---

OMNI3D- BENCH
RODOSPATIAL
BLINK
VSR
REALWORLD
GQA
TALLY
COUNTBUNCH
QA
GPT-S-mini
40.9
76.3
81.0
84.4
62.2
81.9
55.2
84.3
GPT---mmini (grounded)
35.5
76.2
82.4
57.4
84.7
VALOR
44,0
69.5
69.2
75.6
57.3
64.4
51.0
75.9
Table 7: VALOR GPT-5-mini direct prediction. For the (grounded) variant, GPT-5-mini is prompted to predict bounding boxes for all relevant objects prior to responding.
As shown in the figure, GPT-5-mini frequently outputs misaligned or overly large boxes, failing to localize key objects reliably. In contrast, VALOR produces accurate and well-localized detections. These examples underscore an important distinction between verification and generation abilities: VLM can provide reliable binary judgments about correctness even when its own direct grounding predictions are imperfect.
Query: the rightmast sofa were placed fireplace. would the fireplace still be visile??
Query: rightmost sofa were placed the fireplace. would the fireplace stil be visible?
VALOR
GPT-5-mini
VALOR
GPT-S-mini
Figure 13: Visual grounding comparison of VALOR GPT-5-mini. Despite serving as powerful VLM verifier. GPT-5-mini struggles to accurately ground important objects, often predicting poorly placed and overly large boxes.
A.6 ADDITIONAL TRAINING DETAILS
In this section we provide additional details on VALOR's training. Specifically, $A.6.2 includes further detail on our reward models for LLM reasoning, $A.6.3 provides details on RL for LLM reasoning training and $A.6.4 6.4 includes details on verifier-based visual grounding model training.
A.6.1 GRPO.
We provide an overview of our optimization algorithm, Group Relative Policy Optimization (GRPO; Guo et al. (2025)) below.
Setup. Given a query g. our base LLM parametrized by 0, generates a natural language plan and corresponding Python code with access to predefined tools via their API A: (p. c) = (g AA) We aim to find a model, Tg-, that optimizes program correctness, API usage and coordination.
Optimization. We optimize our LLM тe using GRPO; Guo (2025). For each query g, sample G responses O ((g.p(0, c and compute centralized advantages:
A R(g, (0), (0)
1 G
E Mgg, (00)
The GRPO objective maximizes expected advantages while maintaining policy stability:
G Σ min ((0) clip s(0) 1,,, ) - -3KL [XL[[Tell Tree)
LGRPO (0)
(3)
where (6) (t (0)) (g) is the importance ratio, 0.2 is the clipping parameter and 0.01 (p()) [(0gg
the KL penalty coefficient. This formulation encourages improved program synthesis quality while preventing drift from the pre-trained base policy nee--

---

A.6.2 REWARD DESIGN.
We design reward model powered by LLMs which verify if predicted programs are correct. Given a query q, our policy LLM is tasked with predicting a natural language plan p and Python program Our final reward function for these outputs is given as:
R(q, p) = Time (p, c) [Aen Tan (c) + Atog Flog (g.P) P) + Aatt Tat (a p) + Asp Tap p) + And Taa c)
We describe each reward function in further detail below. For all LLM-based rewards (logic, at- tribute, spatial, and adherence), use frozen Gemini-2.5-Flash LLM as our verifier. All prompts to LLM verifiers can be found in Figure 18.
Format Reward (Fime): We employ format reward Pimit to ensure the LLM predicts parsable answers in the expected <plan></ p an> <arr-aaaww>>//sswweee either of the tags missing, the output inside them is empty, the reward returns 0. Otherwise, the reward returns 1. This reward acts as a hard constraint and ensures the model is only rewarded for properly formatted responses.
Given an LLM output,
Syntax Reward (Ten): Our syntax reward (c) checks whether the predicted program с exccutes without Python errors. As we evaluating syntax and not execution accuracy, we replace all calls to vision specialist models with dummy functions to lower runtime. The reward is for all programs that execute without error and O otherwise. Since we found pre-trained LLMs to be proficient at writing code, we lower the weight of this reward to Asn = 0.1.
Logic Reward (riog): We use an LLM verifier to act as critic over the logical correctness of the predicted plan. Importantly, the critic LLM is only given the query and the plan, not the code, as we found it more adept at judging the logical correctness of a plan in natural language as opposed to program. The critic LLM is prompted to return binary decision of 1 if the plan is reasonable and sequentially coherent for the query 0 otherwise. We found logical correctness be the largest source of eтror in pre- pre-trained LLMs, leading us to raise the weight of this reward to Alog = 0.3.
Attribute Reward (Fan): We task an LLM verifier with ensuring all object properties (height, color, etc.) mentioned in the query are correctly identified in the plan and the plan details how to compute them correctly. We found this reward particularly important on OMNI3D- dimensions are computed in a 3D-aware manner. We weigh this reward Autt = 0.2.
to ensure object
Spatial Reward (rsp):: We found the omission of spatial relationships highlighted in the query be a significant source of eиror among pre-trained LLMs (e.g. treating "leftmost sofa" as "sofa'), ) Thus, we invoke an LLM verifier to ensure all spatial relationships detailed in the query are explicitly and correctly addressed in the plan. We set Aup = 0.2.
Adherence Reward (raa): As the other semantic rewards (logic, attribute, and spatial) encourage detailed and correct plans, we employ an adherence reward Tad to ensure the predicted code imple- ments the plan faithfully. The reward is computed by an LLM verifier that is given the plan and predicted code and prompted to return 1 if the code matches the plan exactly and O otherwise. We set Xad 0.2.
Reward Model Analysis. Each of our reward models targets distinct aspect of spatial reasoning, with its own evaluation criteria. Figure 14 illustrates the role of each reward head evaluated by an LLM verifier spatial, attribute, logic, and code by showing representative training outputs that failed to achieve full reward. By focusing on examples that were (correctly) penalized by a particular head, we highlight the types of errors that would otherwise go unaddressed in the absence of that head.
In the Spatial Reward example, the model fails to verify which potted plant is closest, an omission which the reward model detects that would otherwise lead to an incorrect answer when multiple plants are present. The Attribute Reward example demonstrates that the model does not check whether the candles are on, which is correctly detected by the reward model. In the Logic Reward panel, the reward model detects discrepancy between the VQA prompt invoked in the plan and the original query. Finally, the Code Reward identifies that the generated program does not compare the largest table, even though the plan explicitly says to.
7

---

Spatial Reward (r= 0)
Attribute Reword [r 0]
Logic Reward (r= 0]
Code Reward 0] Plan
hove makeen gor to the closest potlcc di plan- how hv
Quary: How mony candles are on
Quaary Which device, seen or the the imagge dislaysi batttryy percentage o13
ge!
Plan
Plan
Plan
proope
wieut tw
(tuntet auge aits propp morl
LoteJso.
"candla" detece all
"preeiin" to sirect all dovicee.
ne ceuide ls feand, vaa
Laaiee estans by with alm.
stare this stoee in tanelaatoerr
Uhe utrrre with
end returt
rinc wose firsl saown
the one on the left aide the Lmuge.
dovired fird
Code
stad dhiding witers
coluuatie sreeings dilarce the pitted
centies (hagght of candles
trust the nambrr of detected
that Jeft s-ceerd
teuading tea''s contr scoere is less
bes pat deit the (icaest)) Hell the buyaes the buter
Fitel stower.
itore the resale
variable
the derice's bounding box and prospe cextect":
call with
Explonation: The plan does not
cheek which of the dieeectedt
ckan
Explonation: The plan does porlied piont, The spattolreward corecctl' lotwh Iik incorvel, waurring
candies are on. This flogged by
teplonation: The pitn recometll mii
the Alfibull beowad.
Dxplenation: The program doen nol
renums the ottibure "on"
oorserrage when the qucty cato
Pal giil
ictte
compuleed eecoruclly...
typi dedee The logs Reword
dauutho or8 tcdae Cods Rlwwrrd
000 10 omiil
Figure 14: Analysis of reward models. For each reward type evaluated by an LLM verifier spatiall, attribute, logic, and code- - we show representative model output from training that failed to receive the full reward. Each panel presents the input provided to the reward model, along with description of the specific error the reward model correctly identified. We highlight sources of error in red. We recommend zooming to read model outputs.
A.6.3 LLM REASONING TRAINING
Training Data. We train our base LLM with 800 spatial reasoning queries, consisting of 400 gen- erated queries (5A.4.1) and the last 400 queries from OMN13D-BENCH. We reserve the first 100 queries from OMNI3D- D-BENCH for evaluation.
Implementation. We optimize a Owen3-8B model using GRPO (Guo et al., 2025), building on the verl repository (Sheng et al., 2025). We use the system prompt shown in Fig. 15 and the reward de- scribed in $A.6.2. We set enable to predicting the plan in <plan></plan /aan>>/plan>>> tags while reducing output length. We train on 8 A100 GPUs for epochs with an batch size of 64, a micro batch size per-gpu of and group size of 5. We detail all hyperparameters in Table 10.
ng False in our tokenizer as we found it equivalent
A.6.4 GROUNDINGDINO FINE-TUNING
Training Data. For training we use data generated by our VLM-verificattio pipeline described in $A.4.2. We source images from the training sets of spatial reasoning benchmarks TALLYQA, GQA, VSR and DMNI3D-BENCH as well as queries generated as described in $A..411. Our training dataset consists of 30,826 seudo-annotations sourced from 7,373 images. Of these images: 124 are from AMNI3D-BENCCE 2,312 are from GQA, 1,655 are from TALLY AA, 1718 are from VSR, and 1,564 are generated from SA-1B (Kirillov et al., 2023). In our experiments, we evaluate how our verifier-based training affects GroundingDDINO's performance on the validation set of COCO (Lin et al., 2014). Despite training without labels, to ensure this is a fair evaluation we discard images from TALLYQA and VSR that are sourced from the validation set of COCO.
Implementation. We fine-tune GIONndingDINO using the Open-GroundingDIIe reposi- tory (Zuwei Long, 2023). We use the Groundingo et al., 2021) vision backbone and BERT (Devlin et al., 2019) language encoder. We keep the vision and language encoder frozen and train the rest of the model. We train 4 A100 GPUs for 7 epochs with a batch size of 16 and learning rate of 10-6 6 As the number of images from our source datasets not uniformly distributed, dataset with equal probability. We use a step learning rate scheduler that decreases the learning rate to 10-7 the start of epoch 4. We detail all hyperparameters in Table 11.
variant with a Swin Transformer (Liu
implement a balanced batch sampler that samples from each source
8

---

OMNE3D BENCH
ROROSPATIAL
BLINK
VSR
REALWORLD
GQA
TALLY
COUNTBENCH
QA
QA
QA
Direct Comparisons
Gomma-3-128 (besc GD)
34.0
57.4
47.3
46.0
48.9
67.8
Gemma-3-122 (tuned GD)
25.2
57.0
53.9
60.9
44.7
52.1
49.5
72.7
Qwen3-8R
50.1
Qwen3-SB (runed GD)
37,9
66.7
68.8
72.4
56.2
63.9
50.1
74.1
VALOR-RL (buse GD)
43.9
61.8
67.3
70.3
53.5
57.6
49.5
67.6
VALOR (tuned GD)
44,0
69.5
69.2
75.6
57.3
64,4
51.0
75,9
Table 8:
Additional evaluations of verifier-improved visual grounding. We report the perfor-
mance of baseline LLMs with the base detector (base GD) and the fine-tuned detector (tuned GD).
OMNI3D- BENCH
RoBoSPATIAL
BLINK
VSR
REALWORLD
GQA
TALLY
COUNTBENCH
QA
QA
QA
SFT
38.3
64.5
67.2
68.2
56.3
60.5
49.5
74.5
VALOR
44.0
69.5
69.2
75.6
57.3
64.4
51.0
75,9
Table 9: SFT vs RL. We tune Qwen3-8B 8B high-reward samples with SFT and compare to RL-based VALOR. Both models use the verifier-improved grounding module.
A.7 ADDITIONAL EVALUATIONS OF VERIFIER-IMPROVE VISUAL GROUNDING
To additionally illustrate the impact of the verifier-improved Grounding (GD; Liu al. (2023b)) module other models, we report the performance of Gemma-3-12B and Qwen3-8B equipped with the fine-tuned detector (tuned GD) in Table 8. These results are directly comparable to the variants reported in Table 1 the only difference is the grounding module. We include the baselines with the pre-traine trained detector (base GD), VALOR-RI (that also uses the base GD) and the full VALOR (that uses the tuned GD) for completeness.
We observe that fine-tum sene-tunigg the visual grounding module improves performance across datasets for both open-source base LLMs (base tuned GD). At the same time, comparing VALOR against Qwen3 with the tuned detector shows that particularly on benchmarks where reasoning is the core challenge (e.g., (eni3D-BBncch) improving visual grounding alone is insufficient; the verifier- guided reasoning improvements introduced by our method in Section 3. are important.
A.8 SUPERVISED FINE- TUNING VS REINFORCEMENT LEARNING
We include evaluation results for the supervised fine-tuning vs reinforcement learning abla- tion ($ 4.6) in Table 9. We find that VALOR outperforms SFT on reasoning-intensive bench- marks BRANID--BENCHE ROBOSPATIAL), but peforms similarly on counting tasks (TALLYQA and COUNTBENCHQA).

---

A.9 HYPERPARAMETERS
We include all hyperparameters used for both LLM reasoning training and GroundingDINO fine- tuning in the tables below. For inference, no-think prompting of Qwen3-8B (Yang et al., 2025).
the recommended sampling parameters for
Hyperparameter
Value
Hyperparameter
Value
Batch size
64
Batch size
16
Group size
5
Optimizer
A
Max prompt length
6000
Weight decay
0.0001
Max response length
2000
Epochs
4
Learning rate
le-6
Learning rate
le-6
Optimizer
AdamW W
Scheduler
StepLR
Weight decay
0.01
Step schedule
[4]
KI. coefficient
0.01
Gamma
0.1
Clip ratio
0.2
Freeze vision backbonc
True
Epochs
Freeze language encoder
True
Rollout engine
vLLM
Vision backbonc
swin T.n.T_2224_11
Temperature
1.0
Language encoder
bert-base-uncased Max norm 0.1
Top-p
Gradient clipping
Gradient clipping
Max norm 1.0
Table 11: Hyperparameters for GroundingDINO fine-tuning
Table 10: Hyperparameters for RL
training for L.LM reasoning.
A.10 PROMPTS
You are on expert at solving spatial reasoning queries. You will ie given a query and an API of nothode you can call.
You promuce stop stop plan
<pian></plan> tags and then pythor program exccutes the
plan inside annserro</nswerr tags.
You collowing guidelines:
produce 2. Assume each question is asking for 30 measurements, HOT pixei measerements, You must multiply 23 neasurements ay depth to get pseudo-3D tor comparieene.
plan inside <plen></plan> tage.
leave all ansvers decinals, some thing. Beicht refers to dimension 1e. Depth reters to the distance of an object zrom the cameza.
Ansvers should NEVER rounced ceiling's
asked ratios.
Beight degth
aot
object, namoly
5. Be very speciic in about which tools to call and now to call then, steps should be very specific.
called einai_ansver name output that.
xill parsing for
that vaciable, so you
7. You mast ensure that the final curput stop matchas the specified prebian response zype, For example, if the question gives some options, the last stop nust specity that must return one the the options. Above, selow, front zelationships should handled dapth.
benesth relationehips should hanaled coordinates. Behind and
Here is your API:
Bere are sone examplea of how you night solve problen: (ICL Examples]
can assuna that variable ina_pth already been dafined. overzida You wSI put a text-based plan inside <plan></pian> teqs, and a program inside <unovar></anxxer> tags. You MUST put your tinal answer inside python variable naned tina-_ your pyshon progran (2 will be extracting variable)
anowars, keep outpets as
when you
rounded).
Figure 15: Program Generation System Prompt. The API can be found in Fig. 16 and the ICI. examples in Fig. 17
10

---

The List is NOT def depth (Ane_pta) bbex):
Eacinates depti
specified by bounding
Paramelery
inc_pth
Path input inage Eile-
bbox 11st int, int, insl
Object bounding box in form (82, yi, x2, y2].
Recurns
Fluat
daptà objeat speaified bounding box.
def de_det ping prompt):
nee
juct detection an inage and relued Lhe poss processed bounding boxes.
Not necessarily in the same order as the proopt.
Paramelwss
ing : atr
Path to the impul inage file.
prorge
object detected that describes the object's appearance. can be nean, e.ghr "ficuplace, cotfee table, aota", or accompanied by an appearance alti ibuus, e.g.r
Natural language de
"the black ooffee table", tehe pink pi pool, pess spatial selationahip abcribues, wighr "the rightnost snfa" wrong Uney donf
relate to the object's appeorance.
Becurns
Hist [dict]
A List winew wath slement is a diet ("Sbox"; [xl, yl, x2, y2l, *label%; causs), n.*
necessarily order the pronpt.
det vga (ing pth, pth, bbox, prorpt):
inswers query
abjsct specitied by bounding (abex). query should solate
agpearance of the obgect and should be simple, eig. "What color the object",
Should la not be used for netric val uss about objects le.g. objest height, depth, widtht. image event of Can with bbow lion questions questiors. broacer,
this
Farametors
ing_pth
Path to the input inage tile.
bbox list/int, int, int, insl None
bounding
abject used questions
visusl query in x2. image early exit.
proupt str
The questica to ask the GPT nedel. Xeep questions simple.
Returns
str
the quexy.
Figure 16: API for vision specialist modules.
11

---

Question 1: I2 the 30 heicht of the two-seat scra ia 0,50 nateze, what is the 30 height of the dining table in mecers?
Bolution 1; <olan>
ad_detect
inage pxcmpt "Evo-soat aota, dining to get detections both objects.
2. Identilfy which delect ion corsesponés to the sofa (label containg *sofa") and whach to the dining table ( label contains "Lable").
either abject La found, finel_anserr and return.
actire quezy with boox
this answer in
Por each cbject:
a. Extract its hounding box [Mi, yt, xz, y2].
Compute
height in
ly2
e. the
depth to gez scalar depth valca
e. Multiply y the 2b height by the depch to get the preudo-30 30 height.
5. let sufa, prendo be the pawudo 30 height of the sofe. We given that ita seal 30 beight is 0.50 metecw.
scale
0.50 sofa_
Huleiply the dining table's a poeudn- height by this seale factor get its real height, meters.
7, Stoze the swaalt in the varsable final ansses withoa. rounding. c/plan>s/anawes
Step
dining
delections giLiletet (inguphh, "Lao seat sefa, diring Lablet)
sofa_det None
Lable_det for det in detections:
label det det [[bl=")).
iE "soi=" in label:
"table" label:
Lable_iee det
sofa_det
table_det
tina Linalannee anever vga ling None, "Iz the 30 neight or the two-sest ecta is 0.56 neters, what is the 30 height of the dining table in necers?*)
sweurn
+ Step Compote pseudo-30 besght of the sofa axl, syl, 1x2, sy2 = sofa_det ["ibux*!
betah_eeight_22 flost ley? sofa_depth denth pth, sota_pocudo sott_height__d sota_depth
("shox*]]
Step 3: Conpute pseado-SD So height of the dining table tyl, tx2, ty2 sable_det("abba table height 2d float ity? -yll
table deeehh = (ingh (ing.pth, table_det [et [*bbox*]] table_pseudo table_heightt_25
t Scop 4: Convert to rezl-world metors using the knovE, sata height scale = 0.50 / sota_pseudo
Final
table psendo scsle
Osestiion 2: What is closer to the canera: the tv or the sofs? Solution 2:<plen>>
de_detect detect
image, storing bounding bax.
if cithar abject is not found, store the other obtect in finsi_ and retura.
For each dotected object (IV and scta), call dopta with the object rounding box to got the deptas for each object
set__eepth).
Compare tV_doyth dopth cv_dopth sofa_dapth, fiasl "iv", fincl_answer "sota",
</pl siplann-<ansves>
Stcpp Detect and
detections qd_detect (ing_pth, "cy, sota*)
sota_det: None
detecticns:
label cot["laboi"].iowe it "sota" in label:
*ty" det
tv_det det
Stop Check detections
cv_det None:
finai_anszer *sotar seturn
aofa_det:
final_answez seturn
ty_det ["boon1 abox sofa_del (""iba""""""be"""
Step 2: quezy dopths
srgglegt desth (ingpihy Uv.bbuxt dota_dopth dopth sata_
Step Compase deptia (smalles clozec)
cy_dapth < soti_dopsh:
Finel_
<law:
12

---

ainal_anszez "scta"
6/answer>
Ooestion 3: What is the number placenate divided by the numcer of plants?
Solut ion 3: splan
Use giLdelect ingueth with pronpt "placenal" to detecl all placetalss
ing. pth pronpt "plant" wither object la not found, ass vag the entire query with boox None, store this answer in anavees and seluen.
plante.
Count nunser detected placenate (length placenst detections listl.
planca (lnngh of plant Compute the ratio: nurber st placenaas divided by cumber_ot_plance
list]-
7, Store the result in the variable Finel answer,
enplann
Scep
piacenats
pl placenat_ deta gel detect Hing pth, Hing "placenst")
Jetect all plante
gd. Hing pth, "plant")
t Step 3: Check detections
if lentplacenet det =* 0 'entplant, dets) sinal_anexer vga (ing_pth,, Nono, seturn
namber of placonate divided by rumber plantot")
Count placenats
sun placenate (plecemmt_seeer
Step Count plants innaanttt_eee dete)
t Step 6: Conpute the ratio
tinel_answes ansver man placerats / run_plance </answer>
Ocestion 4: Whica cbject closer to the caners: the washing mechine or the rightnos: chair? Optionss washing nachine, rightnost chair] Solution
ad_detact ing_sth with prompt "vashine nachine" datest washing
Use qd_detect on ing_sth with prompt "chair" to dctect all chairs.
It either object is not tound, store the otaer object in tinel_ansver and return,
detaction, maximum conter as the rightnest chair.
20
x-coordinata
select detectica with
5, For zoth the wesning machine and the rigatnost chair, call depth with the obgect bounding box to get depths for eash abjact ien_depth, chais_dapthi.
wi_deyth chair_depth: the smaller valus indisstes closer object. wn_dapth chaie_dapth, aet ainee_ "eashing nachine") chaiz*.
final_answer "rightnost
</plan><ensve>>
szep Detect
nachine
wiLdeto qd_dotect (inq_sth, "vashing machine")
# S:ep Detect all chairs
qd_detect (imq_pth, "chals")
$ Scop 3: Check dotections
if ien (xm]dets) ==
*righinoal chazs*
if icn (chair_detts 0: Final_answer *washing nachins"
Biep 4: Identity the sightnoat chais by nax centez def centor_x(dot]
yl, y2 del ["bbox]]
securn 1xl x2)
Eightmost_chhie.. widdet Mi_dats10]
fcheir__dtss key=center_x)
i Suep 5: Query deptla for both objject wnndaeth desth (lng_eth, de_det ( ("bbox"]) dheth degthh (ing.pth, rightnost
("bbox"])
i Scep 4 T: Compare deptha (analler closes)
if wn_depth i chair deepthh final_ansaws *washing maching"
else:
Final aanseen "rightnost aheir*
</answe>>
Ocext ion S: Bow many people are wearingg blue whirts? Opc Fanss 1,,1,3
Aolution fon St splans
Use detect
with pronpt "persont detect all the people.
via 3. fritialiss counter to 0.
person
query
this Lanswee
celuzn.
Por each detected, sell with the person's bounding box and the promps His this wearing
shire?*
If the reaponse From the uga cell "yes", increnent: the counter.
13

---

5. EI the countes not one of the options, ask vga the entire quezy with bbox as None, stoze this answes An Final Lanser and return, Staze
in the variable
agplann-ensswes
+ Scep 1: "eteett the people
peopla_dets giLdeeeec, (ing.pth, "person")
S:ep 2: check detect lons
if
=- 0;
tinal anever ling None, "How people weaning blue shirts?")
Step 3: Initialize counter counter
Scep For person in peopla_det:::
pereon("bbox"
uqe ting person box, this person veering shirt?")
12 "yes" in counter +-
Stop
it counter not in 10,1,3,213
final aneser viga (ing pth, None, "Hlow many people are wearing blue shirts?")
Step cinel_anever counter <answer>
Figure 17: ICL Examples. We pass four in- in-context examples demonstrating API usage and 3D spatial reasoning.
14

---

sre are an agent tasked with varifyina a plan to selve a viscal reasening euery seing pro-detiaed 12I. You do not have to check cact. stop of the plan indivimacily but would like you quezy, and if the aceps
whether final You can assuna thst all the steps will execute correctly, but you should ensure logically consi tha plan details what do if delection fails. It is fine to use vga for nstric oe broad quest iona sny IF it is as fail safe for Called detectioon If the stepa are all logically consiatent, and the final answez correctly anavees it quezy you
correctlyy adiressea
ocherxise give reburn the other ob ject ane nos found.
of hinary choice quarias, it fine
Query: 13
21an:
Available API:
(1
ALL quecies can be solved but requiem detailed plana that invoke the Lools de su. IF plan shorleuts and predict cinal anaver directly clains quary unaolvable, ignare everyshing and If you see degenerate text le-gx randon words or a different language), ignare everyshing and give a score of C.
of
21ease ONLY put your final acore and Xo OTHER TEXT inside <ansser></anguez> taga. You put your final soore
Kenpver>s/ansver) tage.
Jub is tasked with veri verifylng fylnng plan to solve visual reasoning query that all of the apatial selalionshaps sentioned in query are being correctly addressed in the plan.
_SPATTAT REMARD, PROMPT ene are
asing pre defined API, Your
Specificall lyy for any spat ial relat innship nent the plan should detail exeotlly How so bandle it. Spe Spacial ial selai ionshigs should be handled esplicilly (e.g. compasing bounding bowss) and NOT the detection 25 relationshipo all spat celation in the query are handled correctly ly and explicitly, give soore 3, otheruise give a scoze of D.
module, He define left/right ght /above
image.
Overy: >lan:
[I
Availlable Lable APT:
(I
plan ignores epatial relationehips or does not atterpt to ansver the query, ignore everything and give score of D, you see degenerete text le-g. rar rardon words or difrrnent language), igrore everything and give a store of D.
Please ONLY put put your final
tinal and OTHER TEXT inside <onever></ansver> tags. draide <onoverss/ansver> tags.
tasked with veri plan solve vi viaull reasoning query using a pre-detiaes API, Your job is to caees tast all attriastes (height, width, length, depth) in the query are neing corputed correct ly. The steps for doing so should d be detailed in the plan,
ATTRTRITR ATTRANUTT_ARAAAT RESARO PRINPT eanyou
All val uee should be comptted as paeuda- 30 lale, where 20 meaxuretents are nult ipl fed by degth, IF all otaervise give score
values computed correctly, Qvery:
Plan:
[1
Available APT:
ALL atcributes require a tool call to be resoived, It plan directly sets an object attribute or does not atterge to ansver the query, igrore everything and give a score of 0. If you see degenerate text (e-g. rardon score et
words diffcront language),
everything and OTHER TEXT ineide <onevers</ansves> tags.
2lease ONLY put your You MUSI put your final score inside <ansve>></anover> taqs.
score
_CODE_ PROMET "erood aze an agenz tasked varitying that a pyshon preeran natohas a pro-vritton plan solve a visual rezsoning query using a pro-detined API,
ADL proolems can be sclved by using the tools, There is NEVER a lack ot intormation, it the code says the logic),
ansver cannot predicted, or circetly predicts the solutisa (vitheat using REGARDLESS of what the plan eays, Otherwise, your ONLY iob to check that the pyshan progran correctly acheres to the pre-written plan, and that all toole in the pre-decined API are ueed appropriately in the pzogran.
The
the exact proqzan correstly adharea all of the steps in the plan, give score of otharvise dive score D. Plan:
just needs
in autcomes, It the
Code:
Available API:
[1 Thore NEVER Lack of informaticn. sbe solution (witheut using tools logie), ienore evezyching above give scoce o. you aus degenarate test (o.g. randon words or a different laseuagel, iqnoze evarything and cive a seare of o. 21case cutput zn explanation and zhen put your riazl score and no OISER TEXT inside canower></snewer> tags. You put your final
codo says the
prodicted. divectly predicts
inside /navees</anavee> caga.
Figure 18: Reward Model Prompts. The prompts passed to Gemini 2.5-Flash, which is prompted to return a binary score for each criteria.
15

---

QUERY_ CENERATION PROMPT
will provide you with inage depicting multiple objects in a 20 30 scenc. Flease generate excetly rive language questions that requics chailenging
reasening.
The questions can invelve:
Hulciple ressoning slaps, othezs,
idencity object by actrisutes, locate it via spatial relations relative
count.
spat Lerms, auth as left of, front of, behind, or above/below, including degih Quant or comparative seasoning, such as counting ab abjects, corpacing sizea, or infezring hypothetioal masurrmentts IIc that sbject is in tall how tall is another?l. The hypothotical massrrenents can be mrmellstic.
No external world knowladee anavers must inCorable solely from inage.
5. Xll nunerio putputs should id be grounded with Fygotheti ral values you should not twet abso lute remeutements only zelativs mesureeents.
Hust ansverable
single short
number (a.g.2, yes, blue,
Othe: commenis:
Ground all questions the scene given in the imace.
that question
wets looking image- queet on should
solvable without the inage.
3. Hake the five questions of varying difriouity. They do not all need to be vary hard, creative!
sample queations,
HO? lerplaces, they
you the difficulty and
expected:
1. "What is the in of sofas to coffee tables? Araver as decinel."
2. "IE the 3D widch of the teluviate is 200 melurs, what is the 30 hiight the chair?"
pot large vehiele to raght to the of the vehicles near the strewt?
colors?"
"Bow nany sata are theze??
Sefore Ling the ions,
you inspect the image deternine if suitable types
of questions you are vriting, It the inage in not suitable tor a spatial reasoning question, please ONLY wrice SKTP in one cansver></ansver> tag and nothing else.
akipping the image, only outpat question, should pai in of tags (one each quest unless you are skipping, then there is only gne,
you
the <ensvers</aneve> tags,
Sere the images
e*e
Figure 19: Prompt for generating spatial reasoning queries using Gemini-2..5 2.5-Flash.
16

---

GFT_5_ PROMP?
You are respensibic for vezitying anzxers. I Nil1 çive you predicted ansveE and the ground truth ERONSE. check
ically sinilas. they somantically zoughly they are not. Please put your anaver inside (anmerts/aammeee Lags.
should retorn
Queation: 13
Ground Truth:
Psedicbed Ansame:
Figure 20: GPT-5-mini synonym equivalence prompt. use GPT-5-mini as a judge for GQA benchmark to account for synonyms in the open-ended answers of the benchmark.
17

---

SEARER_FILIERRIIG... PROMPT
You are an Object-Detaction Varisice. You
inage Itruta)
ANXCTATED innge (sane image with munberee boxes labels]. The munbte bertesponds to the box that
INSIDe of.
numberad 1ist detactions Your Lask:
labai, (x1,y,,22,22)
Decide which detect
ions ions to KEEP and which to DROP.
3eturn CNLY the JSCM requized by the schana; ("Kaop_irdiceesttt---l "drop_indd:ossst..... "notes":I...I]
DeCinitions (judge againal
CLEAN intege);
labeled ob jects label naccheaz box reasonably tight Jsone background or a snall part of another ooject visible in the borders of the box is DK], of: wrong_ 1ottu_laal----tttee coontentt don't natch the label bad, box_ box---frr ton iigh mostly backgrourd, contains the entirety of mulsiple Instances, or chops
Cosrect: the dominant fect
box an iastance
Incarseal: drep for
cleas obtest.
off the object. Indexing:
Indioes b-based
Fu partitioned keep drep.
repeats.
Aweg_indioes should socted by inpus urdes.
Notes (shozt, par decision):
"4 wrong. lasel siel Ulabelecat, object=dog!"
"5 bod
off left side)*
Never propose new boxes or relabel. When uncertaia, favor precision (drop).
Return ONLY the dBoN object no proze. r*e
PER_CROP PRP_CCPPPPPPPPP
enegou ase single box verifies.
Trgut:
ChOPPED inage.
Gne predicted lstel
Geal:
Decide whether the astain visible object in the clearly natcher the labelo. strict]:
Set keepetrue only if the labeled object the raderirant subject is sollearly visiblexs, and recugnizeblyy. that class, The corsect ubjectt should doninate the Lnage. tiny incidentel. Minor background time; object should image abject is vixible
acceptable it percent
should
up
image.
Set keepetaice for any of the tolinnigg:
Wrong class: the visible object is diffrntnt cstegory, a substantial part the cannot be contidently veritied. Unclear central coject grop includes several objects and it'ss not clear which one natches the label. Multiple objects: grop includes signifieant portions/all of several objects of the correct class, not just
o'ear object of the labeled class ix present.
Iruncations crop
Outputs
Rctorn ONLY JSoN natching the calier's scheme, c.g.: ("Keep"; /eee/ "reason": "very brict justitication"). Ne extra text narkdown.
DEOUPT ECATION PROMPT
You FINMIFFASS sejesttditocticn verifier.
receive:
CLEAN inage itruth)
AMNOTATED image (same inage with numbered boxes lsbels). The number ccrsesponds to the box that it is
INSIDE OF.
cumberad 1ist detactions (index, label, y1,,
Goal:
Do last sanity eweep on the remaining detections. Remove any veincorrect boxes and resolve #sresidual dup:icates=--tthh special attention nested (' ('bub-object') !
that vary close other
What to KEEP (oocrect):
The box contzins one main, doninan: object.
Label matches
zeasonably object class.
without chopping the obtest.
I= is screptable if small parz of ansther object is visible in the bordars of the box -- but it should not take nore than 20 pereent or the box.
-=xton__labl++---oaaaaaa don's natch label, or
obfect that clase.
**caplicatee----antttt sotection reters to the cane physical instance (rules below!.
Deslicate rules comp//copaai class):
instance MY They islly overlap and doninant objees of both boxes
detections duplisatos
same.
2) one box 11ss almost eatirely insids the ether (rastad/sub sub-obttess case).
Their conters and extents are almost the came (ccarly coiacident), even is overlap isn't huge.
3esolving duplicates:
Keep **exactly oneer per duplisate groUp) drop the rest.
Protez the nox that best covers the rull object with the least extra background,
If the tightez bex chops the ob jeet, keep the largez.
s:ill Indwxing:
the carlier
Indices are 1-based and must be futly partitioned into keep or drep, No onissiona or repeats. kesp_indices should follow iapu: order.
(shozt, per decisian:
*7 duplicata_ot Inested; covera
abject!
"12 duplicato_of 9 (asarly coincidenti kept tigater 12)*
*4 wrong_laael "Idwil/iisplaoe jbj
"5 box (teo
nostly backesound)"
Outputs
Retusn ONLY the JSCN sequiced by the schena: ("hoep_iddiwwaattt.... shoppinddisssssff.. "noteats....1..
18

---

No prese or naskdown. FRB
Figure 21: VLM Verifier Prompts. We use GPT-5-mini as our VLM verifier three-step verifi- cation process.
19


Reviews:
[{"review_id": "y9mIiX3oIz", "reviewer": "Reviewer_rpw5", "summary": "The paper proposes a novel visual reasoning training paradigm, VALOR, which enables models to improve both reasoning and visual grounding capabilities without any manual annotations. By allowing the model to tackle visual reasoning tasks through tool invocation, VALOR introduces a two-stage training approach: It first refines the reasoning capability of the base LLM via RL with a Python interpreter and an LLM verifier, which strengthens its ability to plan and generate executable code. Then it fine-tunes the visual grounding module (serving as a tool) using a VLM verifier that generates pseudo-labels. Experiments demonstrate that VALOR surpasses both open-source and proprietary models across a broad range of spatial reasoning benchmarks.", "questions": "- The paper shows that both reasoning and grounding performance increase with the number of training samples (Table 5, Fig. 7).\nHave the authors investigated how far this scaling trend continues? Does the performance saturate beyond the current dataset size, or does it continue improving at larger scales?\n\n- The paper mentions using GPT-5-mini as the VLM verifier. Could the authors report its standalone performance on the same benchmarks to better understand the verifier's contribution?\n\n- It would be interesting to explore whether the model could serve as its own verifier for self-training, rather than relying on an external verifier.\n\n- Following Weakness 1, I am interested in understanding the isolated impact of post-training the visual grounding module. Compared to VALOR-RL, this stage appears to contribute the larger performance gain, so disentangling its effect would be insightful.\n\nI would be happy to raise my score if the authors could address these questions and clarify the above points.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is clearly written and well structured.\n\n- It introduces an innovative training framework that enables the model to jointly improve both itself and the tools it invokes, entirely without human supervision.\n\n- The ablation studies are convincing, showing that verifier-based RL enhances reasoning logic, while verifier-based pseudo-labeling improves visual grounding, with cumulative performance gains when both are combined.", "weaknesses": "- The visual grounding module in VALOR is further fine-tuned using verifier-generated pseudo-labels, while all baselines still rely on the frozen pre-trained detector (Table 1). This makes the comparison with baselines partially unfair, since the tools they are allowed to invoke are in fact not properly aligned.\n\n- The paper lacks a quantitative analysis of verifier errors, which would help assess the reliability of verifier supervision.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YHdvyRHhOr", "reviewer": "Reviewer_3T5W", "summary": "This paper introduces VALOR, a verifier-guided, tool-using Visual Language Model (VLM) designed to improve spatial and reasoning-heavy visual understanding tasks without human-annotated labels. The method integrates a language-only model (Qwen3-8B) with structured reinforcement learning through Group Relative Policy Optimization (GRPO) and a multimodal verifier that evaluates generated reasoning programs along six dimensions: logic, spatial, attribute, syntax, adherence, and format. The system further enhances visual grounding by retraining GroundingDINO on verifier-filtered pseudo-labels, improving object detection and spatial relation accuracy. The authors demonstrate strong results across diverse benchmarks such as OMNI3D-BENCH, VSR, and ROBOSPATIAL, outperforming both proprietary and open-source VLMs. The paper provides clean ablations for the reasoning-RL and grounding-improvement stages, showing monotonic performance gains with increasing verifier data. Overall, VALOR presents a unified, label-free approach that aligns structured reasoning, verifier feedback, and improved grounding to achieve significant gains in compositional and spatial visual reasoning.", "questions": "The following are the list of questions:\n\n1. Verifier Reward: Can the authors provide an ablation isolating each reward head (logic, spatial, attribute, syntax, adherence) to show which components most affect reasoning performance?\n\n2. Verifier Pipeline: Could the authors quantify how each stage of the pseudo-labeling pipeline (coarse filter, per-crop check, deduplication) impacts label precision and downstream accuracy?\n\n3. RL Baseline: Can the authors compare GRPO with simpler baselines such as PPO or supervised fine-tuning on top-rewarded samples to confirm that GRPO itself drives the improvement?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The following are the strenghts of the paper:\n\n**Originality:** The paper presents an integration of verifier-guided reinforcement learning with explicit tool use for visual reasoning, eliminating the need for labeled supervision. The structured multi-head verifier and verifier-filtered pseudo-label pipeline seem to be novel and effective extensions of prior VLM paradigms.\n\n**Technical Quality:** The paper is empirically strong. The experimental design cleanly isolates the effects of reasoning-level RL and grounding-level retraining, supported by consistent scaling studies. The improvements are large and reproducible across benchmarks.\n\n**Clarity:** The paper is clearly written and logically structured. The pipeline is illustrated with detailed figures and algorithmic descriptions that make both the reasoning and grounding stages transparent. \n\n**Significance:** VALOR overcomes a key limitation of current VLMs, weakness in spatial and multi-step reasoning, without extra manual annotation. It generalizes across domains and offers a scalable, verifier-driven way to enhance foundation models, making it practical and influential.", "weaknesses": "The following are the weaknesses of the paper:\n\n- The paper introduces a rich multi-head verifier reward but does not ablate the contribution of each component. Since logic, spatial, attribute, syntax, and adherence rewards are argued to fix distinct reasoning errors, omitting a head-wise ablation leaves uncertainty about which rewards are essential versus redundant.\n\n- The three-stage verifier pipeline for generating pseudo-labels in grounding (coarse filter, per-crop verification, deduplication) is central to the claimed improvement, yet there is no quantitative breakdown of each stage’s impact on precision, recall, or downstream accuracy. This weakens the causal link between the pipeline design and the observed performance gains.\n\n- The paper relies on Group Relative Policy Optimization (GRPO) but does not compare it against simpler alternatives such as PPO or supervised fine-tuning on top-rewarded samples. Without this comparison, it is unclear whether the advantage comes from the RL algorithm itself or from the verifier-generated signal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aX9OyfeWfb", "reviewer": "Reviewer_DLBC", "summary": "The paper addresses the challenging problem of query-based visual reasoning, which requires both accurate object grounding and understanding of complex spatial relationships. The authors propose an annotation-free training framework that integrates two AI-powered verifiers, an LLM verifier that refines reasoning via reinforcement learning, and a VLM verifier that strengthens grounding through hard-negative mining. The approach is evaluated on a variety of spatial reasoning tasks, where it demonstrates superior reasoning and grounding performance compared to both open-source and proprietary baselines.", "questions": "1. When prompting the LLMs and VLMs, which generation strategy did you use, e.g. greedy generation? For proprietary models, did you use default parameters, or did you adjust reasoning settings (e.g., reasoning effort in GPT-5-mini)?\n2. In Table 3, you compare results on only two benchmarks. Is there a specific reason for this selection?\n3. In Figure 1, you mention \"GPT-5-Thinking\", presumably the model option from the OpenAI web app? It might be good to clarify which model these terms refer to (as the API only has gpt-5 and gpt-5-mini, etc.)\n4. Line 318: \"framing VQA as yes/no rather than open-ended\", is this referring to example (c) or to something else? I didn't understand the yes/no framing.\n5. I like the various examples given in the paper; however, Figure 4 requires quite some zooming. I think it would improve the paper if the number of examples were reduced or the way the small texts are presented were improved.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is well written and clearly structured. It effectively presents both the high-level motivation and the technical components of the method, with helpful examples that aid comprehension.   \n- It tackles an important and timely problem in visual reasoning, particularly the issue of grounding, which remains a key bottleneck for reliable program-based reasoning systems.\n- The annotation-free design is compelling and addresses an important challenge in scaling visual reasoning systems.", "weaknesses": "- My only concern lies in the VLM verifier quality. Since the proposed method uses the VLM to generate the training data for grounding, the VLM may itself produce imperfect outputs. Do you think the fine-tuned model can ever outperform the VLM that labeled the data? Did you compare VALOR against GPT-5-mini at some point, as in Table 4? It might be helpful to discuss this topic more explicitly in the paper. For example, should the long-term strategy be to continually use the strongest available VLMs for data generation, or are there other potential directions?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zBSUzcaY6h", "reviewer": "Reviewer_5PmE", "summary": "This work applies multimodal-based verifiers in form of tool-use in order to improve the visual reasoning capabilities of a base model. These verifiers are used to finetune the systems program and visual grounding modules. The final approach is evaluated on VQA tasks and compared to monolithic as well as program synthesis based models.", "questions": "It seems VALOR-RL performs worse than VALOR in nearly all evaluations. Whats the intuition behind this? Is this just an error in the training set generation?\n\nVALOR vs proprietary models: I'm a little confused what the benefit is of comparing to different propietary models, couldn't VALOR be applied on top of these as well? \n\nWhy were different dataset subsets used across the different evaluations?\n\nWhat is the difference between evaluations in section 4.4 and 4.1?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The experimental results across a diverse set of datasets appear promising.", "weaknesses": "General Comments:\nI find the paper's overall narrative and key contributions difficult to follow. Below, I outline specific areas that require clarification.\n\nConceptual Clarity:\n\t1.\tMethod characterization: The authors describe their approach as \"label-free finetuning via pretrained model verifiers.\" Could the authors clarify how this fundamentally differs from knowledge distillation from the verifiers into the base model(s)? A clearer distinction would help readers understand the novelty of the approach.\n\t2.\tCode execution mechanism: Section/Figure 2 shows images being passed to an executor, but the technical details of how Python code is actually executed over the image remain unclear. Specifically, what representation is used for this? The current presentation makes this process appear overly simplistic or unclear.\n\t3.\tVision specialist integration: The paper would benefit from a clearer explanation of how vision specialist models are integrated into the overall system architecture.\n\nPresentation Issues:\nFigure readability: Most figures contain text boxes that are too small to read, severely limiting their utility. If the information in these figures is essential for understanding the authors' arguments and results, they must be made readable. Currently, I cannot verify the claims or follow the clarifications the authors make in reference to these figures.\n\nExperimental Design:\n\t1.\tEvaluation objectives: For several experiments, the specific goal and relevance to the main claims are unclear. I recommend adding brief statements at the beginning of each evaluation section explicitly stating what research question is being addressed.\n\t2.\tVALOR vs. VALOR (RL): The distinction between these two variants is not clearly explained in the text. Please provide a concise description of the key differences.\n\t3.\tVerifier and base model specification: Critical information about which models serve as verifiers versus base models appears to be missing (or I may have overlooked it). This should be clearly indicated in the results tables alongside the VALOR specifications, as the choice of verifier likely has substantial impact on performance.\n\nCritical Concern - Experimental Comparisons:\nI find the experimental comparisons difficult to interpret due to potentially confounded variables. For example, when comparing VALOR with Qwen3-8B:\n\t•\tWas Qwen3-8B also finetuned in any way?\n\t•\tWas Qwen3-8B used as a verifier in VALOR's training?\n\t•\tThe approaches appear fundamentally different, with VALOR receiving substantially more information, possibly larger models as verifiers and additional training. Doesn't this alone explain the performance differences?\nWithout clarity on these points, it's difficult to assess whether the comparisons are fair or what specific conclusions can be drawn.\n\nRecommendations:\n\t1.\tAblation study on verifiers: Given the central role of verifiers in the method, an ablation study examining how different verifiers affect performance would be valuable. This would help isolate the contribution of the VALOR framework from the contribution of powerful verifiers.\n\t2.\tClarify contributions: Each evaluation section would benefit from a concluding paragraph that: (a) summarizes the key findings, (b) relates them back to the paper's main claims, and (c) explains their significance for the overall contribution.\n\t3.\tSharpen claims: Throughout the paper, the goals and contributions could be stated more crisply and specifically, making it easier for readers to follow the logical thread from motivation through methods to conclusions.\n\nOverall Assessment:\nDue to the issues outlined above, I find it challenging to fully evaluate the validity of the authors' claims. Addressing these concerns would significantly strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)