Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning
Authors: Shengguang Wu
Xiaohan Wang
Yuhui Zhang
Hao Zhu
Serena Yeung-Levy
Venue: ICLR 2026 Poster (2026)
Keywords: visual programming
spatial reasoning
tool abstraction
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Spatial reasoning in 3D scenes requires precise geometric calculations that challenge vision-language models. Visual programming addresses this by decomposing problems into steps calling specialized tools, yet existing methods rely on either fixed toolsets or speculative tool induction before solving problems, resulting in suboptimal programs and poor utilization of induced tools. We present Transductive Visual Programming (TVP), a novel framework that builds new tools from its own experience rather than speculation. TVP first solves problems using basic tools while accumulating experiential solutions into an Example Library, then abstracts recurring patterns from these programs into reusable higher-level tools for an evolving Tool Library. This allows TVP to tackle new problems with increasingly powerful tools learned from experience. On Omni3D-Bench, TVP achieves state-of-the-art performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11%. Our transductively learned tools are used 5x more frequently as core program dependency than inductively created ones, demonstrating more effective tool discovery and reuse. The evolved tools also show strong generalization to unseen spatial tasks, achieving superior performance on benchmarks from SpatialScore-Hard collection without any testset-specific modification. Our work establishes experience-driven transductive tool creation as a powerful paradigm for building self-evolving visual programming agents that effectively tackle challenging spatial reasoning tasks. We release our code at https://transductive-visualprogram.github.io/.

Published as a conference paper at ICLR 2026
TRANSDUCTIVE VISUAL PROGRAMMING: EVOLVING TOOL LIBRARIES FROM EXPERIENCE FOR SPATIAL REASONING
Shengguang Wu, Xiaohan Wang, Yuhui Zhang, Hao Zhu, Serena Yeung-Levy Stanford University
ABSTRACT
Spatial reasoning in 3D scenes requires precise geometric calculations that chal- lenge vision-language models. Visual programming addresses this by decompos- ing problems into steps calling specialized tools, yet existing methods rely either fixed toolsets or speculative tool induction before solving problems, result- ing in suboptimal programs and poor utilization of induced tools. We present Transductive Visual Programming (TVP), novel framework that builds new tools from its own experience rather than speculation. TVP first solves problems using basic tools while accumulating experiential solutions into an Example Li- brary, then abstracts recurring patterns from these programs into reusable higher- level tools for an evolving Tool Library. This allows TVP to tackle new problems with increasingly powerful tools learned from experience. On Omni3D- TVP achieves state-of-the-ard performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11%. Our transductively learned tools are used 5x more frequently as core program dependency than inductively created ones, demonstrating more effective tool discovery and reuse. The evolved tools also show strong generalization rior performance on benchmarks from SpatialScore-Haut collection without any testset-specific modification. Our work establishes experience-driven transductive tool creation as a powerful paradigm for building self-evolving visual program- ming agents that effectively tackle challenging spatial reasoning tasks. We release our code at
unseen spatial tasks, achieving supe-
ive vissullsllee program..p..mmmmmmmoommmm github, io/
INTRODUCTION
Transductive Visual Programming (TVP)
How mony objects of the same height as the theignnnost cabinet table would noed to make sttrectuc as sofa
Library
Program
access to
GT: 1.875
get, 21. sbj siset!
depth1l
touresth dastliangge telall
bocesth,, tittigh get_24 [i_bb__szee sizei lmgg,,
deptept
Programming (VisProg, ViperGPT, VADAR)
Library."
Program
Lrre estimate_34 Instancoe counti
depte depthiliogs, sefa-al)
new abstracted
inagmimegs.
loell dentn!
sel Luitt, helth 22 sizeeee solult))
concrate program experience
bewe.prenste catunet tablee.
get_20,00 stee()
seigt eera_
Example Library
bask flsenieennn sotio pla.eettn.n--aaieet targer, pr proppte. (Mart", "te"1.
"baighe"
voell ob3()
_aatth deptht umge,
cojert_ojjett__PPPo ()
sizelannge.
count colored asjects()
tu M0,hhig anighe riLeeess De.hegg
</>
</>
</>
tanget_ennnnieen stone.
higher-eeel tools
Figure 1: (a) Prior methods operate in open-loop manner: tools created without experience from solving problems. (b) TVP maintains both an Example Library of experiential solutions and Tool Library of abstracted functions, forming closed-loop system where tools are created from proven solution experience, and are then reused to guide future problem-solving.
Reasoning about 3D spatial relations in real-world scenes requires precise geometric calculations beyond visual perception alone. Despite the progress of pretrained vision- lisoun-language models (VLMs) on visual question answering, precise 3D spatial reasoning with real-world dimensions remains challenging even for leading models (Lee et al., 2025) Marsili et al. 2025). Consider the query in Fig. determining how many cabinets would equal the combined height of a sofa and TV.

---

Published as a conference paper at ICLR 2026
The answer requires object localization, depth-aware height estimation for multiple objects, and computing the precise ratio of 1.875- level of precision that VLMs struggle to achieve.
This limitation motivates visual programming. compositional approach that decomposes complex visual reasoning into discrete computational steps. These steps call special- ized tools (e.g., depth estimators, object detectors, geometric functions) and combine their outputs through programmatic logic, reaching the precision that monolithic VLMs lack. The effectiveness of visual programming system depends critically on how its tool repertoire is developed. An ideal approach would mirror how human programmers develop ex- pertise: solving concrete problems with basic tools, recog- nizing recurring patterns, then abstracting them into reusable higher-level functions that make future programs simpler and more reliable.
100
94.2%
Initial Usage
Created Tool Only Usage
80
63.7%
60
Existing work deviates from this ideal by skipping invert- ing the experiencetto-abstracee
36.3%
cycle. Earlier approaches skip this cycle entirely, relying on fixed, predefined tool-sets that cannot adapt to new tasks (Gupta & Kembhavi [023] Surís et al. 2023). More recent work inverts the order: cre- ating new tools through cnduction-speculatiinn
20
5.8%
about useful functions based solely on problem descriptions--befre solv- ing them (Marsili et al. 2025). This inversion creates an open- loop system (Fig. panel a): tools flow in one direction from creation to application, with no problem-solving experience to guide abstraction. Tool development requires closed-loop process (Fig. I panel b) where experience with concrete pro- grammatic solutions guides abstraction, ensuring that new tools address proven needs. Without this experiential grounding, speculatively induced tools often seem helpful in theory but fail to simplify problems in practice, as evidenced by severe under-utilization: 94.2% of programs still rely on basic predefined tools despite having access to newly created functions (Fig. 2
VADAR
(ours)
Figure 2: Tool usage distribution: transductive (TVP) inductive (VADAR) abstraction.
We propose Transductive Visual Programming (TVP), which realizes this experience-driven ideal through closed-loop process enabled by dual libraries. An Example Library stores successful program solutions as experiential memory, while a Tool Library contains available functions, start- ing from basic vision tools and expanding with abstracted tools. When solving visual reason- ing problem, T'P retrieves similar examples from the Example Library as few-shot demonstra- tions, then generates program candidates using functions from the Tool Library. High-quality pro- grams are stored in the Example Library, while similar programmatic solutions are clustered and abstracted into new parameterized functions for the Tool Library. This creates a circular program- tool-program cycle: program solutions inform tool development, and these tools then enable better future programs (Fig. I] panel b).
On Moni3D-BBnncle (Marsili et al. 2025) for 3D spatial reasoning, TVP achieves state-of-the-art performance, outperforming GPT-4o by 22% and the previous best visual programming system by 11% (3.1). This advantage is most pronounced on the hardest batch of problems, where TVP demonstrates most drastic improvement as its dual libraries grow (Figs. 6 and 77. Com- pared to inductive tool creation, TVP's learned tools used 5x more frequently as core program dependency-36.333 programs require only abstracted tool calls (Fi.. 2). Programs using TVP's abstractions achieve both greater accuracy (+3.4%) and lower complexity (66% reduction in cy- clomatic complexity) (Fig. 5). The dual libraries that TVP evolves also transfer to unseen spatial reasoning tasks. With the libraries built by running zero-shot performance on novel queries from other spatial benchmarks including 3DSR-Bench (Ma) al. 2024). SpatialSense (Yang et al., 2019), and VG-Bench (Wu al. 2025), outperforming all baselines without any testset-specific modification (Tab. 2 Fig. 9).
Omni3D-Bench, TVP delivers superior
In summary, grounding tool creation in problem-solving experience produces highly-reusable ab- stractions that enable simpler, more precise programs and transfer well across tasks. TVP's trans- ductive paradigm points toward programming agents that continuously improve through accumu- lated experience. building increasingly sophisticated capabilities from basic operations to complex reasoning functions, analogous how human programmers develop expertise.
2

---

Published as a conference paper at ICLR 2026
2 TRANSDUCTIVE VISUAL PROGRAMMING
Transductive Visual Programming (TVP) implements the closed-loop paradigm from Fig. bb) via Iwo interconnected libraries: an Example Library & that accumulates program solutions as expe- rience, and a Tool Library T that maintains functions abstracted from these programs. The dual- libraries enable the circular program-tool- propogrra cycle: solving problems generates experience, experience guides tool creation, and newly created tools improve future problem-solving
Given a dataset D = {L q))r= of images and spatial questions. TVP alternates between two continuous phases. The first phase solves questions using available tools while accumulating high- quality solutions E; the second phase mines patterns from to abstract new tools for T. As more problems are solved, both libraries Pro-eexpanding experience and increasingly sophisti- cated tools enable simpler, more precise programs. The complete workflow formalized Alg. [] with key sub-processes detailed in Algs. to how the phases interact.
illustrates this dual-library architecture and
Phase Experience Accumulation
Phase II: Transductive Tool Abstraction add validated higher-level tool
judge
multiple candidates
TOOL
clustering common program solutions Sovie
validation
legit
compute. compute.3Lretioon M.retto compute the ratic of tho 30
urower
LacO
compute retiic
calculate objects size
[quansiont)
abyert menurmmenr
"coact
quility
O
solution
teighte)
Eeprogg
chatrisidis get (lo (itnnci thatr
charn Jepen depthccrage. chenr?
EXAMPLE
colculate
object cstagory
[ind_closest.0093
finde antco
eoe. trice snelysi soding ogic
eue. sourdhress
old
the add high-quality experience
question, program
question
questorr
solutor
distance comero:
Fros graro 11.t
closest to
zmpge
tool usage
Figure 3: TVP's dual- architecture. (Phase D) Problem-solving, and experience accumulation: For each query, TVP retrieves similar examples from the Example Library and generates programs using the current Tool Library; high-quality solutions join the Example Library. (Phase II) Tool abstraction: Accumulated examples are clustered, and common patterns abstracted into new tools, which, if passed validation, are added to the Tool Library for future use.
2.1 SYSTEM INITIALIZATION
TVP begins with an empty Example Library + 0, and a Tool Library T initialized with basic vision tools (inherited from Marsili al. 2025): object localization (loc) and bounding box detec- tion (get_2d.co j_si__si1_ se) with GroundingDINO (Liu et al.) [2024;; depth estimation (depth) via UniDepth (Piccinelli et al. 2024] ; visual question answering with GPT-4o (vqa); and bounding box overlap verification (same object). The predefined tools provide atomic operations that compose into more complex geometric reasoning patterns.
2.2 PHASE I:
AND EXPERIENCE ACCUMULATION
For each visual reasoning question, TVP generates programs using its current libraries. This pro- cess involves four steps: retrieving similar past solutions, generating candidate programs, executing them, and evaluating their quality. High-quality solutions enter the Example Library, building con- crete problem-solving experience.
Problem Solving Workflow. (1) Example retrieval: Given a question q: TVP retrieves up to kmax previously solved queries from & with text embedding similarity above threshold Tsims as sufficiently similar questions may well require similar solution logic. Retrieved examples serve as in-context demonstrations for the program generator. If no examples meet the similarity threshold, TVP gen- crates programs without demonstrations. (2) Program generation: The program generator receives the question qi the O-Kimx retrieved examples with their solutions, and the current Tool Library T (tool signatures and docstrings). It samples m candidate programs, because visual reasoning problems often have multiple valid approaches, and sampling increases the chance of discovering high-quality solution. (3) Program execution: TVP executes each candidate program p, in Python with access the image 7; and the full Tool Library T implementations. Execution produces both
3

---

Published as a conference paper at ICLR 2026
final answer and complete trace capturing all intermediate computations. Programs that fail to execute or produce no result are filtered out, leaving only valid candidates for quality evaluation. (4) Quality judgment: A VLM judge assesses each candidate program's 's quality. The judge has access to the program implementation, execution trace and the produced answer, evaluating them against the question and image visual evidence. More details on the judge criteria are discussed in (B.3) The highest-scoring candidate becomes the final solution; if its score exceeds the quality threshold Tq. it enters the Example Library E
Experience A Accumulation and Update. Across T iterations over the entire dataset, newly added programs can replace prior solutions for the same question in & if they achieve higher quality scores. This replacement happens naturally as TVP abstracts more powerful tools that enable simpler, more efficient programs for previously solved questions (discussed in $24).. The accumulated high-quality solutions in & form the concrete experience from which tools are abstracted.
2.3 PIIAS II: TRANSDUCTIVE TOoL ABSTRACTION
At regular intervals (na questions processed), TVP analyzes the Example Library to identify re- curring solution patterns and abstract them into reusable tools. This abstraction is transductive: it converts concrete program solutions directly into parameterized functions, so that every tool is grounded in actual problem-solving experience. The abstraction process involves clustering similar solved problems, creating parameterized functions that capture shared logic, and validating these functions before adding them T.
Pattern Mining Through Clustering. TVP first clusters all queries in the current & by embedding similarity. For clusters where similarity exceeds Tsim and size exceeds Teluster an LLM analyzes the cluster's program solutions to assess abstraction potential. This analysis evaluates whether the cluster shows recurring computational patterns- such as repeated ratio calculations or relative camera distance (Fig. 3) that warrant parameterization. More details on the criteria for abstraction potential are discussed in $B.3 Clusters scoring above Tpotential trigger tool creation.
Creating Parameterized Tools. A tool abstractor receives all questions, program solutions, and execution results from the cluster, along with the current Tool Library T. creates a parameterized function capturing the cluster's s shared computational pattern. As shown in Fig. 3 clusters repeatedly calculating 3D size ratios lead TVP to create compute_dddecccccc aatttttioo....ooooooooooo while clusters finding objects closest camera yield find.closest-oh j. These parameterized tools replace multi-step pro- gramming logic with single function calls, thus simplifying solutions to similar future problems.
Rigorous Tool Validation. Before entering TVP's toolbox T, each abstracted tool must pass valida- tion to ensure it executes correctly and maintains solution quality. Validation proceeds in two stages (detailed in Alg. 3) execution validation and correctness validation. In Stage 1, TVP rewrites each program in the cluster to use the new tool, then runs the rewritten version. All examples must execute successfully (100% success rate)-any failure immediately rejects the proposed tool. In Stage 2, when rewritten programs produce different results from the original, a VLM judge evalu- ates whether the new result is equally valid or superior given the visual evidence. Such divergence is common for geometric calculations involving floating- point arithmetic, where multiple answers may be equally valid within numeric precision. The tool is accepted only if overall correctness (con- sidering both identical and validate--divergent results) exceeds Teomect- Successfully validated tools join T, and the cluster examples in & are rewritten to use the new tool.
2.4 CLOSING THE LOOP: TOOLS IMPROVE PROBLEM-SOLVIEE
With new tools added to T, future problems in Phase gain access increasingly powerful ab- stract functions. The program generator can thus produce solutions that are simpler (lower code complexity) and more precise (better accuracy) (evidenced in Fig. and discussed in $3.1p. While these cleaner, more accurate programs accumulate in E, subsequent abstraction (Phase II) work from higher-quality patterns, enabling even more sophisticated tool creation. As such, this program-tool- program cycle continuously improves TVP's dubl--ibr..i.s.
2.5 TOOL LIBRARY MAINTENANCE
4

---

Published as a conference paper at ICLR 2026
As TVP processes questions and performs ab-
Tool Library Maintenance &
straction at regular intervals, functionally similar tools
may emerge from different clusters. Therefore, ev-
ery nd questions, TVP identifies and merges simi- lar tools (detailed in Alg. A and illustrated in Fig. 4p. For instance, tools like compute.3d.rat sappute.dd.groyy srrouussizzeee ratio merge into a more general
question, question, solution solution
question, question. solution solution
.rd.rt io and
calculate objects' 3D height ratlo
calculate 3D size ratio of groups of objects combined
that han- dles both pairwise and group ratio calculations. Merged tools undergo the same two-stage validation ($2.3) against all examples that used any of the original tools, ensuring no functionality is lost. This maintenance keeps the Tool Library concise and organized, making it casier for the program generator to select appropriate tools.
clusters of similar common solutions
both already abstracted into tools
HOOH
compute__dd Catttutt__ddrrtiiOC
conpute. 3d group.. size_ratio()
1 "*"coroube rabio of two 30 jbjecct
en """orppue of coebined 30 stzes
ratio
#tosunsments (irddths or heights))
betoeen tep groups
Through repeated merging, tools evolve into increasingly general abstractions that cover broader functionality, traced in Fig. and discussed in [3.3) Fig. I0(a) shows concrete program updates via these evolved tools that be- come progressively simpler across iterations.
objects
update
merge
Tool
validation compute_objects.oo
both
computt the ratio conbined
sizes
objects"" "*a
replace the duplicate tools
the merged, more general tool
Figure 4: Tool Library maintenance via merging ($2.5). Functionally simi- lar tools emerge from different clusters; TVP merges them into a more general abstraction covering both use cases.
3 EXPERIMENTS
We test TVP 3D spatial reasoning. domain requiring geometric precision that significantly challenge mono- lithic VLMs ($T). Our experiments demonstrate three key results: (1) TVP achieves superior performance through transductive tool creation-with especially strong advantages on hard questions: (2) TVP's dual li- braries continuously evolve to produce increasingly efficient and accurate programs; (3) the learned dual libraries generalize strongly unseen spatial reasoning benchmarks.
3.1
3D SPATIAL REASONING PERFORMANCE
challenging test set of 501 non- templated spatial queries on real-world scenes requiring 3D geometric reasoning. The benchmark includes Yes/No, Multiple Choice, and Counting questions (evaluated by exact-match accuracy af- ter minor string normalization); and Floating point calculations evaluated via Mean Relative Accu-
Setup. We evaluate on Omni3D-Bench (Marsili et al. 2025),
racy (Yang al. 2025ap: MRA = [C| Eeec [10=9 < 1 with C {0.55, 0.55, 0.95),
and 10%) accuracy for predictions within 10% tolerance. Overall accuracy counts rect answers across all question types, using Float(+:10%) for floating-point questions. We com- pare against three baseline categories: generic VLMs (GPT-4o, LLaVA-One Visio-O---nisinnn-B--hClll Li et al. 2024 Dwwen--VL-7BB-Instteee
Wang et al. [202al] Molmo-7B-D Deitke et al. 2025), spatial- finetuned VLMs (SpaceMantis Jiang et al. [2024] SpatialBot-3B Cai et al. (025). and prior vi- sual programming systems (VisProg Gupta & Kembhavi 2023, ViperGPT Suris et al. 2023 VADAR Marsili et al. 2025). We run TVP for 7 3 iterations using GPT-4o for program gen- eration and quality judgment, with tool abstraction and merging at every step (n.
= 1).
Additional implementation details are in $B
TVP achieves state-of-the-art through experience-groundee tool creation. TVP scores 33.3% overall accuracy on Omni3D-Bench (Tab. 1). outperforming the previous best visual programming system VADAR (29.9%) by +11% and GPT-4o (27.2% by +22%. Notably, monolithic VLMs handle perception-heavy tasks reasonably well (GPT-40 reaches 65.3% on yes/no questions) but struggle with exact 3D measurements requiring multi-step geometric reasoning and arithmetic operations (8.2% on float calculations). Even opatial-finetumi models like SpaceMantis show no improvement over generic VL.Ms, again reflecting the necessity of compositional approach for pre- cise spatial reasoning.
Finetuned following SpatialVI.M Chen et al. 2024
5

---

Published as a conference paper at ICLR 2026
Table 1: Performance on Omni3D-Bench. TVP achieves state-of-the-art through transductive tool learning. Best bold, second underlined. "Results from VADAR paper.
Accuracy by Question Type (%)
Method
Overall (%)
Yes/No
Multiple Choice
Counting
MRA
Float (# (1%)
Generic VLMs
GPT-4o
65.3
60.5
18.6
26.7
8.2
27.2
Swen2-VL--7B..
58.7
33.7
12.9
21.5
10.0
21.8
LLaVA-OV-7B-Chr
60.0
27.9
22.9
26.8
23.0
Molmo-7B-D
16.7
41.9
18.6
28.4
8.9
21.6
Spatial-Finetuned Finetuned VLMs
SpaceMantis
53.3
30.2
21.4
18.2
SpatialBot-3B
60.0
30.2
0.0
17.7
8.5
18.8
Visual Programming
VisProg"
54.7
25.9
2.9
0.9
ViperGPT"
56.0
42.4
20.0
15.4
VADAR
56.0"
57.6"
21.7"
35.5"
15.9
29.9
TVP (ours)
60.0
61.6
24.3
36.5
19.3
33.3
TVP w/o Tool Lib
60.0
61.6
21.4
35.5
17.0
31.7
Cyclomatic Complexity Comparison
Transition API to
TVP Performance API
3.0
3,30
2.0
3.18
2.88
Average Median
+3.4%
33.8%
35
34.45
Eaalc with Mew APS
36 2%
2.5
Accuracy (5)
25.30
2.0
2.0
Cyclomatic
2.5
10
0.5
VADAR
TVP-tert
TVP-Bar2
TVP-terd
TVPeter <) performance with/without new tools
1yA-ter
program complexity
b) Improvement by switching to new tools
Figure 5: TVP's closed-loop learning produces three measurable benefits. (a) Program complex- ity decreases as higher-level tools replace multi-step patterns. (b) Programs gain +3.4% accuracy when switching to abstracted tools. (c) Performance with abstracted tools improves +388 across iterations as TVP learns more capable tools and better examples.
TVP enables simpler, more accurate, and continuously improving programs. Fig. reveals three facets of how transductive tool abstraction improves programs. First, program complexity steadily decreases (panel a), with median McCabe's 's Cyclomatic Complexity Number (CCN) (Mc-
Cabe. 1976) ($B.2) dropping from 3.0 to 1.0 across iterations. This reduction occurs because ab- stracted tools replace multi-step programming patterns with single parameterized function calls. Second, programs gain +3.4% % accuracy when switching to abstracted tools (panel b). This tool creation: each abstracted function en- capsulates recurring high-quality solutions and passes rigorous validation, thus reducing potential error rates compared reimplementation with basic tools. Third, programs utilizing abstracted tools improve significantly across iterations (panel с) from 22. 22.4% accuracy in iteration 1 to 31.0% in iteration 3, a +388 relative main-while programs using only basic tools maintain stable performance. This improvement reflects the closed-loop refinement process ($2.4): later iterations benefit from (1) more sophisticated tools abstracted, and (2) higher-quality examples demonstrating optimal tool usage patterns.
improvement stems from TVP's experience-groundee
Tool abstraction facilitates progressive self gelf-mprrovement, especially on hard problems. To iso- late the Tool Library's contribution beyond in-context learning from examples, run TVP with

---

Published as a conference paper at ICLR 2026
Easy (1-3)
Medium (4-6)
Hard (7-10)
TVP(Full) TVP(Example Lib. Only)
4+4. 6..%%
30.9
33.1
$4.7
(tsample-Li--Onl)) TVP (Pall)
Delte (%
Accurscy (N)
28.5
4+2.0%
25.4
26.7
a+0.4%
4+0.4%
4+0.8%
22.4
22.2
&r 0.0.
12 12.3%
455%
-6
Medium 6)
Hard 17-10)
Questton Complexity
Figure 7: Performance delta between TVP (Full) and TVP (Example-Lib--nlly across iterations for each question complexity level. TVP full system (with tive tool creation) shows most significant gains on the hardest batch of questions.
Figure 6: Performance comparison across question complexity levels on Omni3D- Bench. The complexity scores are rated with criteria defined $B.4
only the Example Library active ("w/o Tool Lib" in Tab. 1b, providing retrieved examples as few-shot demonstrations but restricting the program generator the basic initial tools from $2.1] Notably, the Example Library alone maintains competitive performance (31.7% overall, outperforming all prior baselines). This strong performance demonstrates the quality our accumulated experi- ence, enabled by TVP's example library admission design.
The full TVP system with active tool creation shows more significant improvement across iterations (31.3% 31.9% + 33.3%) while the Exanple-Library-Oni variant maintains static (31.7% + 31.5% + 31.5%). The progressive improvement stems from the closed-loop design in Fig. hb): abstracted tools encapsulate past experience and enable better future programs, which become better examples, from which better tools can be abstracted. Fig. 5jb) also indicates this effect: when programs switch from basic tools newly created abstractions, they achieve +3..4% accuracy improvement. Without tool creation in the loop, this self-improving cycle is weakened.
Furthermore, find that the tool abstraction contributes most value on hard problems. We use GPT-5 to rate the difficulty of the spatial reasoming questions on a 1.0-10.0 scale (details in ŞB.4), then divide questions into three groups: Easy (1-3), Medium (4-6), and Hard (7-10), Fig. shows accuracy across methods for different complexity levels. TVP (Full) delivers the best per- formance on both Easy and Hard batches. For easy questions, thoroughly validated created tools avoid potential reimplementation errors, leading to more stable performance. For harder questions, created tools provide simpler solution steps that eliminate complicated logic, thus easing the pro- gram reasoning. Fig. 7 reveals the evolution of the benefits brought by active tool abstraction across iterations, as we compare the the performance delta between TVP (Full) and TVP (Example- Lib-Only) for each complexity level. On the hardest batch, TVP (Full) shows the most signifi- cant improvement trajectory, starting at -4.5% relative to Scample-Lib-Onie in iteration 1, but ultimately surpassing by +6.7% in iteration 3. This demonstrates that our created tools-beyond just in-context they encapsulate past experience of complicated code logic into simple function calls. Examples are illustrated in Fig. 10 where newly created tools serve as convenient single-step solutions for otherwise complex spatial reasoning calculations.
effeesiiiietllllly reduce the reasoning workload for most challenging questions,
TVP's dual libraries evolve steadily through closed-loop interaction. Fig. 12
visualizes how
TVP's dual libraries evolve through three iterations on Omni3D-Bench, tracing the closed-loop program-tool. mogram-tooll-rogram cycle introduced in Fig. ljb.. The Fxample Library grows steadily from 0 to 304 high-quality solutions as TVP processes questions, accumulating concrete problem-solving experience that grounds tool abstraction. The Tool Library expands with controlled maintenance:
while 61 tools are created total, only 11 remain active after periodic merging ($2.)). This selective retention ensures manageable Tool Library capturing genuinely reusable abstractions with minimal redundancy, making it easier for the program generator to select appropriate tools.
TVP is robust to backbone LLM choice, showing a clear scaling trend with model sizes.

---

Published as a conference paper at ICLR 2026
We further investigate TVP's robustness to open-source smaller LLMs, represented by the (Qwen2.5-Coder.
32
VADAR (GPT4o)
Instruct (Hui et al. generator, spanning from 7B to 32B parameters. Specific configurations are in (B.2 Fig. 8 presents the scaling behav- ior, where TVP exhibits a clear performance improvement with increasing model capacity. Notably, using an open- source 32B model, TVP achieves performance close to our UPT-44-backer variant (30.7% vs. 31.3%), and surpasses the previous best baseline VADAR (29.9%) despite its more capable GPT-4o backbone. This result underscores that TVP does not rely on proprietary-specinf optimal LLMs, and strong performance can be achieved with more accessible open-source alternatives. The consistent scaling trend also validates TVP's architecture as model-agnostic, and suggests significant future potential of transductive tool creation, as foundation models with enhanced capabilities become available.
2024) family the backbone program
29.9
Porformance Overall
27.7
27.2
27
26
Qwen-148
Backbone Model Size
Figure 8: TVP performance scales consistently with backbone model capacity.
Additional empirical analyses on TVP's robustness to random data ordering are provided C..1
3.2 GENERALIZING To UNSEEN SPATIAL REASONING QUERIES
Generalization to unseen tasks provides a critical test of our transductive paradigm: tools abstracted from experience on one benchmark should capture fundamental spatial reasoning patterns that trans- fer to new questions. We investigate whether TVP's dual libraries, built only Omni3D-Beme transfer to unseen spatial reasoning queries without any modification.
Setup. We evaluate on SpatialScore-Hard collection (Wu et al. 2025). drawing 256 samples from 3DSR-Bench (Ma et al.) 2024). SpatialSense (Yang al. 2019), and VG-Bench (Wu et al. (025) that ask single-image questions without visual bounding box hints (matching Omni3D-Bench 's data structure). These queries span four spatial reasoning categories (Fig. 9): object properties, object localization, depth and distance estimation, and 3D positional relations. Yes/no and multiple-choiee questions are measured with accuracy, and numeric calculations with accuracy within + 100% toler- ance. We compare TVP's zero-shot transfer against the same VLM baselines from $3.1 and VADAR as the representative visual programming system, noting that VADAR creates new tools specifically for these test sets while TVP uses its Omni3D-Bench libraries zero-shot.
Table 2: Results on benchmarks from sampled OpatialScore-Hard collection. TVP generalizes zero- shot with only libraries built from Omni3D-Bench. Best bold, second underlined.
59.2
3D Relation
40.8
43.8
Depth
Method
3DSR-B. SpatialSense VG-B. Overall
20.3
Generic VLMs
73.5
GPT-4o
52.1
46.5
20.3
42.6
LLaVA-OV
12.4
9.9
9.4
10.9
Localization
33.3
Owen2-VL-77
49.6
32.4
7.8
34.4
Molmo-7B-D
41.3
54.9
12.5
37.9
32.8
Doatial-Fimetuncee VLMs
Object Properties
16.4
GPE-4o
SpaceMantis SpatialBot-3B
37.2
19.7
7.8
25.0
GVP-Generaliaing
20.7
62.0
6.2
28.5
20
60
Accuracy
Visual Programming
VADAR
24.8
40.8
39.1
32.8
Figure 9; TVP's libraries transfer well on SpatialScore-Haree particularly for
TVPGenerllte Generallte
52.9
59.2
43.8
52.3
3D spatial and depth/distance reasoning.
8

---

Published as a conference paper at ICLR 2026
Transductively created tools generalize with strong zero-shot performance. As shown in Tab. 2 TVP achieves 52.3% overall accuracy, substantially outperforming baseline VLMs and VADAR (32.8%). While VADAR's 's inductive approach creates specific new tools by analyzing these test set questions, TVP'ss experience-grounee transfer. The category-wis breakdown in Fig. 9 further reveals TVP's particular strength challenging spatial reasoning consistent with $3.1 59.2% on 3D positional relations and 43.8% on depth and distance estimation.
tools still lead to more effective solutions even in zero-shot
Strong transfer occurs because our transductive tool creation inherently produces general func- tions through three mechanisms. First, tools are abstracted from clusters of similar problems (Alg. (). forcing abstractions to capture shared computational logic rather than query-specific de- tails. Second, rigorous validation (Alg- 3) ensures the abstraction generalizes correctly within its cluster before entering the Tool Library. Third, tool maintenance (Alg. 4 merges functionally simi- lar abstractions, leading to more general functions validated on the union of their source examples. Together, these mechanisms ensure tools encode fundamental reasoning patterns validated on di- verse examples, thus enabling effective generalization to queries of different phrasings and contexts.
3.3 QUALITATIVE ANALYSIS OF TOOL UTILIZATION AND EVOLUTION
Transductively abstracted tools apply to diverse problems. Fig. 10 illustrates TVP's hierarchi- cally evolving tools that simplify programs and apply diverse problems, both within Omni3D- Bench and in zero-shot transfer.
Panel (a) demonstrates how program solutions simplify tools evolve (complementing the concep- tual illustration in Fig. II.. The same spatial query compuicn the combined 3D height of a TV and TV stand given the sofa's reference height-is solved using progressively more general tools: from basic .compute e e.3dd_...... _d__bbd_b jecpeeectiiieee handling multiple dimensions, to the most general iestimate330 _eesseee om. ference that aggregates multiple target objects in single call. This evolution reduces program complexity while maintaining correctness.
calls, to the unified
Panel (b) shows compute 3d di dimenstin on match count handling diverse dimension- matching calculations within Omni3DD-eecch-
determining how many small stools fú in an armchair's 's volume to calculating how many rightmost stools must stack reach chair's 's height. Such versatility arises from our transductive tool creation ($2.3) that abstract general underlying pattern from a cluster of examples while parameterizing specific objects, dimensions and selectors.
Panel (c) demonstrates tool transfer across datasets: .find.largest.by. wyw.aargges....ddmeet.......e..e wee.yddderrrrireeeeeeeeeeeeeeeee ab- stracted from Omni3D-Bench, is directly applied to EDSR-Bence queries comparing train and street light elevations. Successful transfer occurs because the tool captures fundamental reasoning logic ("compare multiple objects specified 3D metric and identify the largest") that generalizes across object categories and scene types, regardless of domain specifics.
Tool hierarchies emerge through iterative generalization. TVP produces increasingly sophis- ticated tool hierarchies through iterations of abstraction and library maintenance. Fig. 11 traces the evolution of a representative tool hierarchy. Starting from compute.3d_heigtte that computes ratios of summed 3D heights between object groups, TVP generalizes this pat- tern via dimension parameter, creating compute_grouse wtt.ppp.ddiimmsiio......... dles both height and width calculations uniformly. As more examples accumulate, subse- quent abstraction cycles reveal patterns requiring selective object filtering, leading to the merged compute objects si ze. ratio with conditional filtering capabilities-an even more capable tool that subsumes the earlier versions. Fig. 10ja) shows concrete program updates with an evolving hierarchy of tools: the same spatial query is solved using progressively more general tools, with multi-step solutions collapsing into a convenient single-step function call. The quantitative impact of these tool hierarchies is also clear: programs using increasingly evolved tools achieve both higher accuracy and lower complexity (Fig.
that han-
This continuous refinement demonstrates that TVP doesn't ' just solve problems but progressively learns more sophisticated reasoning capabilities from basic operations to increasingly powerful ab- stractions, mirroring how human programmers develop expertise.

---

Published as a conference paper at ICLR 2026
4 RELATED WORK
4.1 SPATIAL REASONING
Spatial reasoning requires precise understanding of real-world 3D relationships beyond pixel-level image representations (Kamath et al.)
Majumdar al. 2024 Fu et al. 2024 Tong et al. Cai et al., 2025 Zhang et al. 2024 Yang et al. 2025b), which remains challenging for
2023
2024
ithic VLMs. Even with specialized spatial finetuning (Chen et al. 2024 Cheng et al. Cai et al., (025), these models still struggle with diverse 3D reasoning queries (Lee et al, Marsili et al. - with our findings in $3.T These limitations motivate visual programming approaches that decompose visual tasks into steps calling expert vision tools. Vis- Prog (Gupta Kembhavi (023) uses a domain-specific language for composing vision specialists, while ViperGPT (Surís et al. (023) generates Python code to call vision APIs; both are limited to static predefined tools. VADAR (Marsili et al. 2025) introduces dynamic tool set but relies on pure induction speculating about potentially useful functions solely from question texts before solving any problems. This leads to under-utilizatie of created tools in practice, as shown Fig. 2
2024
2025
TVP takes a fundamentally different approach to visual programming by learning tools transduc- tively through experience: it first solves problems with basic tools, accumulates experiential solu- tions, and only then abstracts recurring patterns from these proven solutions into new functions.
4.2 TOOL USE AND ABSTRACTION
Agentic systems calling specialized tools have demonstrated strong performance across web navi- gation (Zheng et al. 2025 Wang et al. 2025), robotic control (Liang et al. (023), graphics gen- eration (Hu et al., 2024 Sun et al. (025), ) game exploration (Wang et al. 2023), and biomedical experiments (Jin et al. 2025). In 3D visual tasks, Yuaan et al. (2024) proposes view-dependent and -independent modules in visual programs for zero-shot open-vocabulary grounding, Mi et al.) (2025) introduces "code as spatial relation encoders" optimized through test suites before deploy- ment. Kamali & Kordjamshidi (2025) uses symbolic logic operators to enhance the expressiveness of visual programming. Beyond applying existing tools, recent work studies automatic tool creation for self-evolving agents (Cai et al. 2023 Wang al. 2024bb Yuan al., 2023) Qian et al., 2023
Jin et al., 2025). LII.O (Grand et a 2023) abstracts tools by compressing programs into symbolic X-expressions. Alita (Qiu al. 2025) creates specialized model context protocols connecting web search with tool generation and execution. Skillweaver (Zheng et al. (025) identifies novel skills from web tasks through a simple-to himple-tocooplee co-compeex curriculum. ASI (Wang et al. (0225) shares our insight that tool abstractions should emerge from experience, proposing new functions from individual ac- tion trajectories web environments.
TVP evolves its toolbox through a dual-library architecture: the Example Library accumulates con- crete programs across many problems, and the Tool Library abstracts reusable functions from clus- ters of these examples. Their closed-loop interaction ensures that newly created tools, grounded in diverse experiences, will in turn facilitate solving a broad range of future problems.
5 CONCLUSION
We introduce Transductive Visual Programming (TVP), a framework that learns to build tools from concrete problem-solving experience, mirroring how human programmers develop expertise from practice. Through a program-tool-progranl cycle-where solving problems generates experience, experience guides tool creation, and new tools improve future solutions- TVV continuously evolves its own visual programming capabilities. We find three measurable benefits: programs simplify as recurring patterns compress into tools, accuracy increases with higher-level tool abstractions, and performance grows across iterations as tool usage improves. TVP achieves state-of-the-art on the challenging 3D spatial reasoning benchmark Omni3D-Bench. and the evolved tools transfer to unseen spatial fasksnevidence that transductive learning captures fundamental reasoning pat- terns beyond dataset-specifi solutions. Our transductive paradigm offers a general architecture for self-evolving agents that develop increasingly sophisticated capabilities through accumulated experience from basic operations to complex reasoning functions across any domain requiring compositional problem-solving.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. Large language models as tool makers, arXiv preprint arXiv:2305. 2023.
Wenxiao Cai, Iaroslav Ponomarenko, Jianhao Yuan, Xiaoqi Li, Wankou Yang, Hao Dong. and Bo Zhao. Spatialbot: Precise spatial understanding with vision language models. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp- 9490-9498. IEEE, 2025.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14455-14465, 2024.
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision- langualanguage models. Advances in Neural Information Processing Systems, 37:1 37:135062-135092 2024.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for sate-of-the--ee vision-language models. Proceedings of the Com- puter Vision and Pattern Recognition Conference, pp. 91- 2025.
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference Computer Vision, pp. 148-166. Springer, 2024.
Gabriel Grand, Lionel Wong. Maddy Bowers, Theo X Olausson, Muxin Liu, Joshua B Tenenbaum, and Jacob Andreas. Lilo: Learning interpretable libraries by compressing and documenting code. arXiv preprint ar arXiv:2310.19791, 2023.
Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14953-14962, 2023.
Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf. Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. Scenecraft: An Ilm agent for synthesizing 3d scenes as blender code. In Forty-first International Conference on Machine Learning. 2024.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang. Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. 12 186, 2024.
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv preprint arXiv:2405.01483, 2024.
Ruofan Jin, Zaixi Zhang, Mengdi Wang, and Le Cong. Stella: Self-evolving Ilm agent for biomedical research. arXiv preprint wwXiv22507.02004 2025.
Danial Kamali and Parisa Kordjamshidi. Neptune: A neuro-pythonic framework for tunable com- positional reasoning on vision-language. arXiv preprint arXiv:2509.25757 2025.
Amita Kamath, Jack Hessel, and Kei-Wei Chang. What's "up" with vision- lision-lnguuaee models? inves- tigating their struggle with spatial reasoning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9161-9175, 2023.
Phillip Lee, Jihyeon Je, Chanho Park, Mikaela Angelina Uy, Leonidas Guibas, and Minhyuk Sung. Perspective-aware reasoning in vision-language models via mental imagery simulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9241-9251, 2025.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408. 03326, 2024.
11

---

Published as a conference paper at ICLR 2026
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International conference on robotics and automation (ICRA), pp. 9493-99999 90000 IEEE, 2023.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang. Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pp. 38-55. Springer, 2024.
Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Jieneng Chen, Celso M de Melo, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825, 2024.
Arjun Majumdar, Anurag Ajay, Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff, Sneha Silwal, Paul Mevay, Oleksandr Maksymets, Sergio Arnaud, et al. Openeqa: Embodied question answering in the era of foundation models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16488 16498, 2024.
Damiano Marsili, Rohun Agrawal, Yisong Yue, and Georgia Gkioxari. Visual agentic ai for spatial reasoning with dynamic api. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19446-19455, 2025.
T.J. McCabe. complexity measure. IEEE Transactions on Software Engineering, SE-2(4):308- 320, 1976. doi: 10.1 1976.2338777
Boyu Mi, Hanqing Wang, Tai Wang, Yilun Chen, and Jiangmiao Pang. Language-to-space program- ming for training-free 3d visual grounding. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 3844-3864, 2025.
Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10106 10116, 2024.
Cheng Qian, Chi Han, Yi Fung, Yujia Qin, Zhiyuan Liu, and Heng Ji. Creator: Tool creation for disentangling abstract and concrete reasoning of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 6922-6939, 2023.
Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, et al. Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution. arXiv preprint arXiv:2505.20288 2025.
Yun Sun, Shengguang Wu, Christian Jacobsen, Thomas Yim, Haoming Zou, Alex Zook, Shangru Li, Yu-Hsin Chou, Ethem Can, Xunlei Wu, et al. 3d-generalist: Self-improving vision- language-action models for crafting 3d worlds. arXiv preprint 25X.::2577006888 2025.
Didac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11888- 1898, 2023.
Peter Tong. Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri Iyer, Sai Charitha Akula, Shusheng Yang. Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-I: I: A fully open, vision-centric exploration of multimodal Ilms. Advances in Neural Information Processing Systems, 1773110-77666666 2024.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305 16291, 2023.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, al. Qwen2-vl: Enhancing languae model's 's perception of the world at any resolution. arXiv preprint arXiv: /2/9/, 2024a.
Zhiruo Wang, Daniel Fried, and Graham Neubig. Trove: Inducing verifiable and efficient toolboxes for solving programmatic tasks. arXiv preprint arXiv:2401. 12869 2024b.
12

---

Published as a conference paper at ICLR 2026
Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, and Daniel Fried. Inducing programmatic skills for agentic tasks. arXiv preprint arXiv:2504.06821, 2025.
Haoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang, Yanfeng Wang, and Weidi Xie. Spatialscore: Towards comprehensive evaluation for spatial intelligence. arXiv preprint arXiv:2505. 17012. 2025.
Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. C-pack: Packed resources for general chinese embeddings, arXiv preprint arXiv:2309.07597, 2023.
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 10632-10643, 2025a.
Kaiyu Yang, Olga Russakovsky, and Jia Deng. Spatialsense: An adversarially crowdsourced bench- mark for spatial relation recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2051-2060 2019.
Shusheng Yang, Jihan Yang. Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng. Yifan Xu, Muhan Wang. et al. Cambrian-s: Towards spatial supersensing in video. arXiv preprint arXiv:25/1.04611 2025b.
Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R Fung, Hao Peng, and Heng Ji. Craft: Customizing Ilms by creating and retrieving from specialized toolsets. arXiv preprint SiKi::2309.142288 2023.
Zhihao Yuan, Jinke Ren, Chun-Mei Feng, Hengshuang Zhao, Shuguang Cui, and Zhen Li. Vi- sual programming for zero-shot open-vocabulary 3d visual grounding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 20623-20633, 2024.
Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities. arXiv preprint arXiv:2410. 17385, 2024.
Boyuan Zheng, Michael Y Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, et al. Skillweaver: Web agents can self-improve by discovering and honing skills. arXiv preprint arXiv:2504.07079, 2025.
A THE USE OF LARGE LANGUAGE MODELS (LLMs)
In this paper, used LLMs to polish the manuscript's 's presentation, including grammar checking, light editing, and styling of figures and tables.
B IMPLEMENTATION DETAILS
B. VADAR REPRODUCING CONFIGURATIONS
We directly utilized VADAR Marsili al. 2025''s 's official codehase and adhered to the official
hyperparameter settings throughout, including; random batches of 15 questions for API proposal, GroundinDDINO-SSinn-O (Liu al. 2024) for object detection and UniDepth-v2-ViTSS4 (Pic- cinelli et al. 2024) for depth estimation- the exact same tools used in our TVP implementation.
B.2 TVP CONFIGURATIONS
We run the TVP pipeline on Omni3D-Bench ($3.1) with the following configurations:
For the main pipeline (Alg D. we process all N 501 questions from the entire dataset over T 3 iterations. During each iteration, we generate m = program candidates per question and retrieve kmax 3 similar examples from the example library E using EEGg-LargeEn--a J..-v...eemebeddingg embeddings (Xiao) ctt al. (023) with an embedding similarity threshold Faim 0.8. Programs are accepted into the ex- ample library only if their quality score exceeds Tq 8.5 10-point scale. The tool abstraction
13

---

Published as a conference paper at ICLR 2026
the oo height the sofs 0.50 metirr, what is the combined oo height of the tv and the stand meters?
Which object has the largest 30 height the lefomost black armchalr or the fireplace
Owni Be-ncch
Iargert snject 7ind, langest awtric)
imugeningge.
caloperacm) Poafiie
"4t replacer).
acda- Theogh"
13pes taiget
staue teighte Jompud sejject slaat
Firal recalte laraest abject
ixtervnce
unlectonetin
ixterecce
Consider the realworld 30 locations of the
Jnight Hiige iepastnng, reforence Scigi, 396.33 target sitettan-firrr"
1arper
Which object has higher location? (A)
train (8) street lights
305R- Bench
Fuaal resdte
CreDr SESsEi Tectoage "CHEr"
belght
lask
"Street U1itt tock
roforenet actonton (iatt, target
seter first".
steatt)
higher ctfecte
snage-ompe.
(rrnae" ssgons the tackt)
relacice-kone kone,
Frast realte to a) wight staw teigit
program updates hierarchical evolution tools
reuse across datasets
of
many rightmost
small stool an object volume the left-most armchair?
you have stack reach the same height
the left most chair?
Final, re compets, clweasaannerddd...coooooo poandt
tual rass Loe _omputt cooeesdan parco, posst
referie prrecte wett stan?",
reference www.ppreeeeeenn segterastst""
Beigety
actselleight
referance jelactoreeriee riihhemntt""""""".....
pelectaree Lofteees""
Agg ragnne sargete Falee
agprepete ferpets-fateee
b) reuse across diverse tasks
Figure 10: Transductively created tools enable hierarchical evolution and diverse reuse within and across benchmarks. (a) Programs simplify as tools evolve: the same spatial reasoning query is solved using progressively more general tools. (b) A single learned tool handles diverse dimension- matching problems within Omni3D- (c) Tools transfer to unseen benchmarks without modi- fication, solving structurally similar problems in different contexts.
filter
def _coepute abjjctts jiizes_sizee______eee_e (as_ssiieeeaaeeeeeeeeeeeeeeeeeee Size_ratio((rage,
nunerator prospts,
dimension
def _compute__ppp www.groouppp.p. 3d dimension letgel nurerator: prompts,
nuneratoo axis.
denominator axis,
def wccoppttte_aaccccc 3d, height. hw__heighh_ratttiinneee ratiolinage,
surerator: dimension.
nunerator rilTer questtons,
nunerator, prorpts,
denovinator, prompts,
demenimator filter questiom]:
denoninator proepts):
denoxinator: dimension):
"""oopput the ratio of combined sizes between two groups of objects image WITh conditionsl rstrortin
"ccospute the ratio sunned (30 heights
compute the ratie of combined rest world abjects
one set of objects to another set
dimensions between
Figure 11: Hierarchical tool evolution through closed-loop refinement. TVP progressively abstracts more general tools through clustering ($2.)) and merging (5.5) eachh evolution subsuming the previous tool's functionalities.
process (Alg. triggered continuously after every n, step (effectively at every step). We cluster examples using a similarity threshold of Tsi = 0.8 and require a minimum cluster size of Teluster examples. Clusters with an abstraction potential score above Tpotential 9.0 are consid- ered for tool creation. The validation process (Alg. requires minimum execution success rate of 100% and an overall correctness rate of at least Teomect = 0.85 (85%), where identical results are counted as correct and divergent results must be validated by the correctness validator (LLMconec).. Both abstraction and program rewriting allow up to Amax = and Rigwrite = retry attempts respec- tively. Tool deduplication (Alg. 4 is also performed after every na step. Tools considered duplicates when their similarity exceeds 0.95. The merge process allows up to Rmerge = 2 retry attempts to create unified tool that passes validation.
Throughout our experiments, we maintain a uniform random seed of 42 across all pipeline com- ponents, governing aspects such as datapoint order (discussed more in C....
14

---

Published as a conference paper at ICLR 2026
In our main experiments Tab.1
employ GPT-4o the backbone program generator (LLMprog).
and as the VLM-based quality judge (LLMjudge) & correctness validator (LLMooneed). We use the reasoning model 04-mini for clustering (LLMeluster), abstraction (LLMabstract); deduplication (Ld.M merging (LL.Merre)). and program rewriting tasks. In our ablations on the scaling be- havior (Fig. 8), we switch to Qwwen25--Coder-nnstnn 1/11 (Hui al., (2024) for the backbone program generation, and run TVP for I" = iteration. Results discussed in $3.1 demonstrate TVP's robustness to the backbone LLM, as well the clear scaling trend with model capacity.
Unless required by specific reasoning models like o4-mini (temperature = 1.0), LLM temperatures are set to 0.0 for deterministic tasks (quality judgment and correctness validation), ensuring rigor- ous assessment; and 1.0 for more creative tasks (program generation, abstraction, and rewriting), increasing the likelihood of finding better solutions.
In Fig. 5(a), we use McCabe's Cyclomatic Complexity Number (CCN) (McCabe 1976) as the program complexity measure, computed via the Lizard library following the practice in Yuan al. 2023
Pipeline Evolution: Growing
Example Library
End Itar Examples: 200 Active Toolsc
End Iter
End Iter
250
Total Tools Created
Tool Abstraction
Examples: 271
Active Tools:
Examples: 304
Active Tools: 11
Tool Librany Size (Total)
2200 2000
Tool Merge
Iteration Boundary
150
Statistics:
Example
100
Active Tools:
Examples: 304
-40
Tools Created: 61
50
20
200
1000
1200
1400
Pipeline Stops (Questions Processed on 0mni3D)
Figure 12: Evolution trajectory of TVP's dual libraries. The Example Library grows steadily 304 solutions while the Tool Library expands atategically-creaiinn 61 tools total but maintaining 11 active abstractions through periodic merging.
B.3 CRITERIA TVP's JUDGE COMPONENTS
The program quality judge ( 2.22 gates the admission to TVP's Example Library through evalua- tion across five comprehensive dimensions (as shown Prompt []: (1) 3D spatial understanding. following Marsilli et al. (2025)'s official implementation for 3D concepts and definitions; (2) answer correctness with visual verification against the provided image: (3) appropriate program tool usage; (4) code quality including readability and efficiency; and (5) robustness to edge cases. These dimen- sions align with the critical requirements of both spatial reasoning and programming. The reliability of our quality judge is empirically validated in Tab. 1 where enabling only the Example- in TVP already outperforms baselines. This demonstrates accurate admission high-quality solutions in our Example Library that provide strong in-context examples.
The criteria for tool abstraction potential can be found in Prompt 2 which analyzes a group of program solutions clustered via question embeddings (embedding similarity is the first step of clus- tering, refer to $2.3). The abstraction potential focuses on general code abstraction requirements: (1) common computational patterns; (2) logical flow; (3) generalization capability; and (4) parame- terization potential. We allow this flexibility in tool abstraction to enable more diverse exploration of higher-level tools, while still ensuring new tools' quality through the rigorous validation against all examples in the cluster before Tool Library admission
B.4 COMPLEXITY RATING OF 3D SPATIAL REASONING QUESTIONS
In both our complexity groupedd evaluation illustrated in Figs. and 7). and the curriculum-ordent TVP run discussed in C1.I
the question complexity scores rated via GPT-5 ("high" reason- ing effort) with the prompt given in Prompt The complexity rating evaluates questions along five axes, considering egg., 3D understanding; single-/multi-objeet relationships and multi-step reason-
15

---

Published as a conference paper at ICLR 2026
Pipeline Evolution Comparison: Growing Libraries TVP with Different Datatpoint Ordering
100
350
400
Curriculum Order Tools
Random Order Examples
Examples: 355 Activa Tools:
Per
Examples: 394 Activa Tools:
Rer
Examples: 406
Acthve Tools:
Her
(otal Created)
200
Rendom Order Tools
Bar
Rar
Tiar
Examples: 200 Active loolsi
Examples Active Wolsi 11
Examples: 204
Example Library
Active Toglo
200
150
Curricluu
Examplle 406
60
100
Tools Creatod: 51
Exammples 306
Tools Creatod: 91
Pipeline Steps
Figure 14: Pipeline evolution comparison between ouriculuumorderee (as defined in $C.1 and rander- (as our main experiments $3.1) datapoint processing. While curriculum ordering enables faster initial accumulation of examples and tools, random ordering ultimately creates more diverse tools through broader exploration of the problem space.
ing: cognitive and computational load. Based on these scores on the scale of 1.0-10.0, we partition questions into three complexity buckets: Easy (1-3), Medium (4-6), and Hard (7-10).
C ADDITIONAL EMPIRICAL ANALYSES AND DISCUSSION
C.I TVP's RESILIENCE RANDOM DATAPOINT ORDERING
TVP is designed operate the fly without any dataset-specifie priors, unlike previous methods such as Skillweaver (Zheng et al. 2025) that depends on human-defined curriculum for structured progression (see also $4.2). To validate our prior-free design choice, we compare the original TVP with random ordering, as used in our main experiments ($3.1) to curriculum ordering based on easy-to-hard progression through question complexity scores (details in $B.4.
Despite the intuition that starting experience with simpler problems, then grad- ually attempting harder problems seems natural fit, we show in Fig. 13 that the overall performance is mostly on-par (both outperforming baselines), with the randomly-orderes TVP gradually getting better than the curriculum-ordered variant. To understand this result. Fig. 14]revals the evolution dynam- ics under both ordering strategies. The curriculum prior intro- duces two notable early-stage effects. First, it leads to earlier example accumulation: datapoints with similar complexity clustered together facilitate more frequent example retrieval,
33.3
31.9
32
Overall Accuracy
31.7
resulting in 355 accumulated examples versus 200 with ran- dom ordering the end of iteration 1. Second, promotes
29.9
earlier tool creation, as similar and simpler examples form cligible clusters sooner, yielding 11 active tools compared with random ordering after the first iteration.
28
TVP Randdmmdddrrr
TVP-CurricukmOrder
27.2
VADAR
GPT40
However, random ordering proves more beneficial for sus- tained library growth. By encountering datapoints of vary- ing complexities and patterns throughout processing. TVP ex- plores more diverse solution space. Although initial accumu- lation may be slower, this diversity enables continued progres- sion as both libraries capture richer patterns. By completion, random ordering generates total tools compared 51 with curriculum ordering, demonstrating the value of diverse explo- ration over structured progression.
Iter1
Iter2 Iteration
Iter3
Figure 13: Performance compari- son between TVP runs with ran- dom vs. curriculum-be data- point ordering.
The above analysis speaks for TVP's design choice of resilience to random datapoint ordering that enables truly on-the-fly operation without requiring any dataset-specific priors. The diverse explo- ration inherent random ordering also fosters greater variety in accumulated experiences, leading
16

---

Published as a conference paper at ICLR 2026
to comprehensive tool creation that better covers the problem space. This mirrors human learning that benefits from exposure to varied challenges rather than strictly structured curricula.
C.2 COMPUTATIONAL COST AND EFFICIENCY
We detail computational requirements for running TVP below:
Cost structure and runtime. TVP operates in distinct stages with different cost profiles:
(1) Building TVP's 's dual libraries from scratch involves processing the test set on the fly (Omni3D- Bench in $.11, analogous training model. This stage requires approximately S80 per iteration with our GPT-4o + 04-mini configuration, equivalent to about $0.16 16 per question per iteration.
(2) Applying TVP's built libraries solve questions (SpatialScore-Hard collection in $32) has minimal cost, usually equivalent to a single GPT-4o call per query.
GPUs are strictly optional in both stages. When used, the system requires under 4GB VRAM only to store the basic vision tools (GroundingDINO (Liu et al., 2024) and UniDepth (Piccinelli et al,, CPUs. The runtime for building TVPP's dual libraries (analogous to
2024)), which can also training) stands at approximately 7 hours per iteration with our current implementation that executes programs sequentially.
Efficiency optimizations. We implement several strategies to improve TVP''s cost-efficiency when building dual libraries:
(1) Early exit in tool validation: Abstracted tools must achieve 100% execution success and 85% correctness on their validation cluster per our current configurations (4B.2) For cluster of e.g. examples, validation exits early-thor thut avoiding unnecessary computation when any one example fails execution (100% requirement not met); or when two fail the correctness check (5/7 71%, drops below 85% pass rate)
(2) Easy resumability: We maintain comprehensive state checkpoints, supporting TVP's pause and resume at any point.
(3) Embedding bank: Since question embeddings remain unchanged, we keep a persistent storage of embedding vectors that enables simple lookup when retrieving 2$2.) clustering examples ($2.3)
(4) Parallel program generation and quality judge: We generate program candidates in parallel and batch the quality judging for all valid candidates to reduce run-time.
D PROMPT TEMPLATES
Prompt 1: Quality Judge
You are an expert jadge evaluating the quality of progzan that solves 3D spatial reascning preblem with (funct task program specific crilwrie assign quality 10.0.
rating
$4 TASK OVERVIER
quastion
#:6 Program Evaluate "python
azdgsan__odde
Execut iun
**Statusive excc_atacua: auscesa/Cailoge
ssPinal knsaerive isa ansper_tex:
T<oull Taedzx Loula
present
$16 Exwoul lon Ransspace (A11 Variablus) nenexpace text
EVALIATION PRAMEMORK
#4 Visual Rvidence Firificctinn for
You provided with
the scene, Use this visuel evidence throughout evelustion Verity progran's approach sligne what's
image
ieste the anexer's ressonableness based on visuel proportions
17

---

Published as a conference paper at ICLR 2026
Check if spacial rolationalipa are correctly inserpieced
Confien intermediate results match the visual scene Encure all vorification consicars 30 content, not jest 20 positioning
Nanaspace Mnalysis considaraticna
Whon reviexing the secutioo namaspact, specifically check tor:
Correct abjact Ldentification (right object selected fron multisle candidates;
bounding
Eapected inteemeiiale iuno
Appropriate ob ject Ziltering
Correct depth-azsed 30 convarcions
Misnatched bounding
Unexpectad intermediate calsulation results
objects that should have betn filtered but werca't
Spatial
Deriniticns
.*Cor befinitlones*
+CCoozinnate aysten:es (wsdth, height, length) (s, Y, axia
.>bapth:** Distance fren camera (smalier depti: closez to caneza) sise/diat .+30 neasuzementa:e Sisa/diatance
space (image zeal world
++20 siza formala:se '30 size size desth"
.+20 distanoe Eurmulasee Buclidean distance belweea object center cuerdinatess 1(xx) x2) y2r
0.5*
..20 distance cornelaiee "ip_cistance (2D_dlotancovve] (dapthi 22)
A-Distanoo Lo Cettraaee Simply the object'ss depil, value
<aKey
20 sizes are in pixel apace. converl Lo sixe, multiply by depth
cb jecttt with sane 20 dimensions but different depths have different 30 aizes
30 distance requires the Pythegorean formula conbining 20 di dianae and depth di fferenre defined abouw aot "le "/ compoard descriptiona must locate objecta than cilter the
courdinates
'loc()" function
desired condition
All db jecta satialying condit.on must be checked, not just the firal
Maltiple objects
property Hyposhee het sbject counts (m.g., if table has legs*) require oount lin uctuel objects image
require proper tie-breaking
BATENG CRETERIA 61.0 10.0 Scale!
Spatiel Understandingg Propesly convecta between and measurenenta
correctly handles 30 size/eist ance cal celculation
Correctly center coordinates for dixtance esloulat ions and "a LeEtrost//s rrightmest detwsminnt ions Interprets spatial relationships correctiy (e.g., "Largest" JD,
Anewer is visual ly verifiable and reasonable
I+ 2, A+Crreattnes
Visusl Veriticationes image
problen correctly Aligns with visual evidence Intermeiaate results are consiscent with visible scene Spatial relationehiga match visual reality
image
#f 3, ToToo Usage Etticiency-
dees appropriate tools for the cask Leverages higher-level "learnes" reimplenent inenting existing Notes Basic tools are acceptable when no hegher-level tools Fit
scitable
** AxCode itylity. Well-structured
clear variable
Folloxs tool usage patterns correctly Erficient without uneceesaryy operations
Incluses helpfui comnents
** 5. Robusttness
Ydge Ceresa Hardles maitiple
Propezly filters located objects for propertics rather than using complex "1oc1)" queries with property tie-breaking!
Marages empty
appropriately
Kanages contzincr relatimatisps (e.g., *in", *ca") properly
Incluses appropriate etror cheacing
REQUIRED OUTFUE
You MUST provide your responce exactly this tornst:
<reasonings
[peraited explenation covering:
Row visual evidence oupparts/cattrasien the progran's logic
vaaknesses idantified
Analysis spatial reazzning appreach
Evaiostion at intermediate execution results Missed ospoztualties to use available tools all criteris]
Overall assessment </reasoning>
Where WINBER is decinal between 1.0 and 10.0.
AEPENDIX: Available Tools Refercacs
had tool_counta total: nan_basis_raole basic tun_erested_ooll learnad
18

---

Include nany blooks needed, lollowed by Published as a conference paper at ICLR 2026
#f Basic Tocls (Gavel 6) cool__ignator,, toc__docstring
Leazned (Level 1+]
cool__lgnature, tocl_docsszing
Prompt 2: Abstraction Potential Analysis
You are an expert anslyzing visual reasoning progrems to identity connon patterne shat could be abstractes into reussble functions,
Your Ta5k
Analyze nun_clostor, _xannies visual reasening examples co: examples
corputational Group then into clusters on thared tunotionality 2. Rate each alustar's abstraction potential (3-10 scale)
Exarples Anslyze
examples (question, program soluticn!
# Clustoring Criteria
Ideatity based
--Comonn compatational potternsee c.G.. finding Largest/emallest, counting
*iimilar operations sequencers a.0., locate filter +> compute -> compare
+*barred locic structuro.e 2.g., iteration patterns, comparison icgic
**bbstractable tunctionsitty..
paraneterizod reusable function
Evaluction Requirements
+16 For Each Cluster Provide:
+*comonn pattorn** explanation
Baaametszzs that vary belween esamples
Aistraction 3otential satinger (0-10) based ea:
vel1 Faranelus variability coverage
gencrolizes
Clazity of the abstraction sinilas tacks rating
leusebiliity
ssaasoninggs
#* Critical rainss
Each exanple must belong to exaetly ONE cluster oz be nacked as unclusceredrs
sinilarities
only create cluster strong shared patterns
et Respense Format
Drovide your maaysis
optional unclustered block:
dexangle dds foomna seperated liss of example The] s/examplll id. que Leans (Dwacrigt igt fon of the comman computationa pailsrn] c/pat
sparametezg> [List of pazamoteza that vary between exanplea] exanelea|s/paranee s/parametrrr (Batiig from 0-100///bbtt sowasonings Explanation for ing how palleza sould abatracted] s/ssasuning </al usters
ior_potentiai:
Redditional
blocka needeć...]
cunel uxteredo
dexangle le Idaa Ieomna separated list of example IDs Chat don't Fil c/ustess] why exergles
creasonings (Fpp s/uncluste ustereds
</ressnning
Eowry exeegle apgear lyactll
group.
Prompt 3: Question Complexity Rating
You new an expeet wvelueting complexity of spat ressaning queat ione. assiggn complexity score (1.0 10.0 acale) to single quastion based on its inherent spatial reasoning diffieulty.
QUESTION
Queasiinss qusstion
AAnoves Typessa answer types floet /toet//nttegerr/nlliiiii choice/sto.
RVALUATTON PRAMFMORK
Spatial Beasening Bequicemetise
Roes the quest fon require understand and and ing of (pixel/i apacel vs. 30 [reel-vorld; measurenents?
Doex: it involue depth underssandle and distarcw from cenera concegti??
zequire
coleolacions underetardirg
depths
sizes?
19

---

Published as a conference paper at ICLR 2026
calorlations (combining distance depth differences)? Docs it zequire converting between massrrenent spaces?
Doea invoive
**Boatial Belationship Complexityre
many objects lavoivad
spatial reasening?
Types of relstionships:
Simple propecty identification (ccloz, naterial, count)
comporison, relative position] Complex apalial relationhaps (e.g.r
relaticaships (distanes,
right boss it zequize malti-stap reasoning with intermediate conclusions?
behiad Y1)
Comparative (udmments ve. absolute measurements
CCCaniiiiee
Number et constraints or conditicas that must be simultancossly satistied ueed Lo identily and distinguish between multiple candidate objecta ("12 Y Handling iple of objects wish potentially anbiguous descript Containe: selatlenship (ob jecta tin or "on" other objecta)
Hypothetioal conditional
than.
sefalcalaiion Quantitacivo Corpexxitys Simele ficatini count ing va. numerical Ratio, propostion, or porcenzage calculations Multiple nessrrment Distance size conputationa Precision zoquirenence
iaona
formulaa
4:# 5. v>Anaver Type Indicatoravs
**yes/no (binary) often simpler verification
complex depending beingg Fied sxmuttipl cheice jais upt.ona) z.a Bequirea discrininat lun anong bounded opcioms ssmnariaal ifloat/iat) axe often recuizes precise caloulationa and measusementa stringras May require Ident! Fination and oxtegorization
COMPLEXITY
Conside: the full ageclium of complextt::
Aloowr undzaa Simple, direct quest ions requiring mininal apat iel reasoni ng
Single wbje propecty ident.
Raeid
Single yes/so verificstion
criteris
atti odle rangezas Moderste spatial reasoning and cslorlation
Siaw distance compariaans becseen paira of objects
Single colculatione
Chject ident i ificatioon
miltiple constraints
Basic 20 to 50 cnnversions
Complex multi step spatial reasoning
Multiple ab ject congar isons
munerous comstraiat
complex celculations involving combines ressuremente
hepot lexl ressoning with condit ronal calcalations
Spatial colationsaips invoiving many obtects inte:dependen::ies
Ratios or combined or derived quancities
Aesign
scale
10.0 based question's pesition this complexity together.
REQUIRED OUTPUT FORMAT
your sesgonse
this
<ecomeex. Xx/rrooeex///ore>>>
<roasoning>
[Decailed explenation covering:
Which eva cat dimension one concribute nost to complexity Specitic azpects that increase oz decrcase dirficulty Xey spatial </reasoning>
lenges question)
and judanent appsopriately on complexity spectrum.
decimal
E COMPLETE TVP ALGORITHM
20

---

Published as a conference paper at ICLR 2026
Algorithm Transductive Visual Programming (TVP) Pipeline
Input: Dataset D = (14, (images, questions)
1: Initialize: Example Library & + 0, Tool Library 7 + (predefined tools)
2: Parameters: quality threshold Ty. abstraction interval Ra. deduplication interval nd
3: for iteration f to do
4: n4 6-0
processed question counter
5: for each question qi D do
6:
Retrieve similar examples
7:
Exim +- RetrieveSimilar(( qi kimax 3)
excluding qi itself
8:
Generate program candidates
9:
5-- 0
for j = to m do
> m candidates per question b in-context learning with Exim
11:
Pj f- LLM prog (4i, Esim, T)
12:
- {pj}
13:
end for
14:
Execute and filter none results
15:
Cauce < /
16:
for each py E C do
17:
resultz. namespace, Execute(p)- 4ú, result, * None À -error then
18:
Cance Cuuce result,, namespace,) end if
20:
end for
22:
if Csuce + 9 then
23:
# Judge quality for each (pj, result;, namespace,) E Csnce do
24:
25:
quality, +- LLMjudgo (Yi, Pj. namespace 44) end for
1-10 scale, criteria in
B.3
26:
27:
# Select best and update example library p², quality", namespace" +- arg max, quality,
28:
29:
quality" 2 Tg then e Example(q; quality" namespace") if Be' E & with e' then
30:
31:
b update existing entry in &
32:
if qualiity" e' qualiity or (quality" e' quallty and different tools) then
33:
+ (E [
end if
35:
else
36:
E Eu
add new entry to
37:
end if
38:
end end if
39:
40:
Rg +- ng 1
41:
# Abstraction interval
42:
if ny mod n., 0 then
43:
T,E +- AbstractTools(E, T)
2.3
Alg.
44:
end if
Deduplication interval if n. mod n, = 0 then T.E +- Deduplicate TooiatteeoooooTTee
46:
2.5 see Alg
48:
end if end for
49:
50:
end for
51: return E,
21

---

end for Published as a conference paper at ICLR 2026
Algorithm AbstractTools Tool Abstraction from Example Clusters
Input: Example Library ε, Tool Library
Output: T with new tools, & with updated records of abstracted entries
1: Parameters: similarity threshold Tsim = 0.8, cluster size threshold Teluster
4, abstraction
potential threshold Tpetential 9.0
2: Filter eligible examples
3: Eeligible < fe & e.status * "abbstacted"))
4: # Initial cluster by similarity
5: G ChusterBySimillrttyy... (Edligiityyyiiilleee Tsim)
6: for each cluster G E G with |G] 2 Teluster do
7:
Analyze cluster for abstraction potential and common pattern
8: pattern, potential +- LLM (G)
9:
if potential 2 Tpotential then
abstraction potential threshold, criteria in BB.
10:
# Create tool with retry
11:
for retry = 1 to Rmax do
12:
t +- LLMaastracc pattern, T) # Validate tool
13:
14:
val VolidateTooll G,
Alg.
15:
val.passed then
T TU
17:
# Update examples with new tool
for each with successful rewrite do
19:
e.program + val.rewrittenfe (e]
e.status "aabstrccted"
mark as already abstracted
21:
e.tools_used <- {+}
22:
end for break
23:
24:
end
25:
26:
end
27:
end for
28: return T,E
22

---

Published as a conference paper at ICLR 2026
Algorithm ValidateTooll Two-Stage Tool Validation
Input: Tool f, Examples G, Tool Library T Output: Validation result with rewritten programs
1: Parameters: correctness threshold
Teomect 0.85
2: # Stage 1: Execution validation
3: rewrites < 0
4: for each example
do
5: for retry = 1 to Rrewiie do
6: p' < --RevriePPorrane--pogrrr t)
7: result', namespacee &- Execute(pf e.image, T U (t))
8:
if iesult' + None error then rewrites[e] (p', result', namespace') break end if
9:
10:
11:
12:
end for
13: if é rewrites then
100% execution success required, otherwise early exit
14:
return (passed False, errors : failuue)
end if
16: end for
Stage Correctness validation for divergent results
18: correct + 0, divergent +- 0
19: for each e E G with successful rewrite do
20:
if rewrites rewrites|ee.resuu reeultt & e.result then
21:
divergent + divergent +
22:
verdict < LLM (ewrriiieeeeeeeeissseeee.elll e.image)
23:
verdict "CORRECT" then
24:
correct <- correct
25: end
26: end if
27: end for
overall.correct (IG) divergent + correct)/ G|
29: if correcl 2 Teorreet then
85% minimum correctness, B.2
30: return (passed rewrites rewrites)
31: else
return (passed: False, emrors correctness_ failures)
33: end if
23

---

Published as a conference paper at ICLR 2026
Algorithm DeduplicateTools Merge Similar Tools
Input: Tool Library T, Example Library
Output: T with merged tools, & with updated records of entries using affected tools
1: Filter eligible tools
2: Teligible +- (t. T I.level 0 -..deprecated)
3: # Find duplicate tools and merge strategies
4: MM M++LLN <LLMacdup (Taligile)
functional similarity 0.95, BB.2
5: for each merge group (M, strategy M) E M do
6: Get examples using these tools
7: Em t fe 3: 3t E M. E e.tools _sedd
8:
for retry = Rmerge do
9:
mmerged +- LLM (M, strategy as) val < Validate Tool (tmergd Em, T) if val.passed then
10:
D sec Alg.
11:
12:
T+TU (fmerged) for each E M do t.deprecated <- True
13:
14:
I.reason "Merged into merrged" end for
16:
Update examples
18:
for each E Ess with successful rewrite do
Update e with merged tool
20:
end for break end if
21:
22:
23:
end for
24:
end for
25:
return T
24


Reviews:
[{"review_id": "MDcLogP2YV", "reviewer": "Reviewer_uXe8", "summary": "This paper presents Transductive Visual Programming (TVP), an innovative framework that enables visual language models to evolve by learning reusable tools from their own problem-solving experience.\n\nThe method’s dual-library closed-loop design is conceptually clear, technically comprehensive, and contributes a structured approach to self-improving reasoning in LLM-based systems.\n\nWhile the framework is well-presented and methodologically sound, key evaluation details (e.g., scoring criteria, abstraction prompts, complexity metrics, and computational cost) remain under-specified.\n\nEmpirically, the performance gains over ICL baselines are modest, and the claimed generalization largely reflects in-domain transfer rather than genuine out-of-distribution generalization.\n\nOverall, TVP is a promising and well-presented system paper: its transductive framework and dual-library design are conceptually valuable and clearly executed. Despite under-specified evaluation details and modest gains, I lean weak accept, contingent on clarifications.", "questions": "Evaluation standards for Example and Tool Libraries\n\n+ Could the authors clarify the evaluation criteria for Example Library admission and Tool Library abstraction? Specifically, what dimensions does the VLM judge consider when scoring examples (e.g., logical correctness, semantic relevance, visual consistency)?\n+ How was the threshold τq = 8.5 chosen—through tuning, validation, or heuristic selection? Similarly, for the Tool Library, how is abstraction potential measured, what prompts are used, and on what basis was τpotential = 9.0 determined?\n+ If both evaluations rely solely on LLM-as-judge scoring, how do the authors control for potential bias or inconsistency across runs?\n\nEmpirical significance and complexity metric\n\n+ The improvement over the ICL baseline is relatively small (+1.6 pp) and only appears after three iterations. Could the authors provide more evidence that the observed gains stem from true tool abstraction rather than ICL-style template learning?\n+ Also, please clarify how program complexity is computed (e.g., McCabe complexity, function calls, or another proxy metric), and explain why its reduction should indicate better abstraction quality.\n\nComputational efficiency\n\n+ Since performance exceeds the ICL baseline only at iteration 3, what is the computational cost of running multiple iterations?\n+ Please report runtime, token usage, or cost per iteration, and discuss whether the modest gain justifies the overall expense or scales to larger datasets.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Innovative and Well-Structured Framework\n\n+ The paper introduces Transductive Visual Programming (TVP), a novel and conceptually original framework that enables a model to iteratively learn reusable tools from its own problem-solving experience. Its dual-library closed-loop design (Example–Tool Library) is systematic and complete, effectively realizing a self-improving learning cycle.\n\nStrong presentation quality\n+ The paper is clearly written, logically organized, and well-illustrated with informative figures and detailed algorithms, making the methodology easy to follow.", "weaknesses": "Lack of Transparency in Evaluation Mechanisms\n\nThe evaluation procedures governing both the Example Library and the Tool Library are under-specified, which raises concerns about reproducibility and interpretability. Specifically:\n+ Unclear criteria for Example Library admission.\nAlthough the paper states that a VLM judge scores each generated program and admits examples whose quality exceeds a threshold of τq = 8.5, it does not define the concrete scoring dimensions—such as logical correctness, semantic relevance, visual consistency, or execution success. The basis for selecting τq (e.g., validation tuning versus heuristic choice) is also not explained. Moreover, no analysis is provided regarding how scores vary across task types or problem complexity, leaving the quality control process for examples largely opaque.\n+ Opaque evaluation of tool abstraction potential.\nThe paper introduces an LLM-based cluster analyzer that outputs a textual “pattern” and a numeric “potential” score, using τpotential = 9.0 as a threshold for initiating tool abstraction. However, the work omits any description of how this score is computed, the intended scale, or the prompts used to elicit it. The rationale behind the chosen threshold is also absent. If this potential measure relies solely on LLM-as-judge scoring, it is likely susceptible to style bias and semantic drift, undermining objectivity.\n\nLimited Empirical Gains and Undefined Complexity Metric\n\nWhile the paper claims that TVP achieves improved performance and reduced program complexity through iterative transductive learning, the empirical evidence supporting these claims appears limited and partially confounded.\n+ Marginal performance improvement over ICL baselines.\nThe main results show that the Example Library-only configuration (essentially an ICL baseline) already achieves 31.7% overall accuracy, while the full TVP framework after three iterations reaches 33.3%. This modest +1.6% gain raises doubts about whether the improvement truly stems from the proposed abstraction mechanism, or instead from the growing in-context example set providing stronger template guidance to the LLM. The observed benefits may therefore largely reflect in-context pattern imitation rather than genuine tool learning. Moreover, performance surpasses the ICL baseline only at iteration 3, when computational cost and LLM usage are substantially higher—yet the paper offers no analysis of cost-effectiveness or scaling trade-offs.\n+ Undefined program complexity metric.\nThe paper reports that “program cyclomatic complexity decreases from 3.0 to 1.0,” using this as evidence that TVP learns simpler, higher-level abstractions. However, no formal definition or computation method for this complexity measure is provided. It is unclear whether this refers to classical McCabe complexity, the number of function calls, or another proxy metric. Without such clarification, the claimed reduction in complexity cannot be meaningfully interpreted or independently verified.\n\nHigh Computational Cost and Unanalyzed Efficiency\n\n+ TVP surpasses the ICL-only baseline only at iteration 3, implying that multiple costly iterations are required for modest gains (+1.6 pp). Yet the paper provides no analysis of runtime, token usage, or cost. Given that each iteration repeatedly invokes GPT-4o for program generation, judging, abstraction, and validation, the overall expense is likely high. Without quantitative efficiency reporting, it is unclear whether the observed improvement justifies the computational overhead or scales beyond small benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q8Nkeyg1ke", "reviewer": "Reviewer_JN1p", "summary": "This paper proposes Transductive Visual Programming (TVP), a framework for visual reasoning that dynamically creates and refines its own library of tools. The core idea is to learn from problem-solving experience. TVP maintains a \"dual-library\" system:\n\n- An Example Library that stores (question, program, solution) tuples for high-quality solutions it has found.\n\n- A Tool Library that contains callable functions (tools).\n\nWhen faced with a new query, TVP retrieves similar examples from the Example Library to use as in-context demonstrations for generating a solution program. Critically, TVP periodically analyzes its Example Library, clusters similar solutions, and \"transductively abstracts\" recurring programming patterns into new, higher-level tools, which are then added to the Tool Library. This allows the system to evolve from basic tools to more complex, specialized, and reusable functions. The paper shows that this approach achieves SOTA on the Omni3D-Bench for 3D spatial reasoning and that the learned tools generalize well to unseen spatial benchmarks.", "questions": "- Could the authors please comment on the computational cost of the TVP framework? Specifically, how many total LLM/VLM calls (and of what type, e.g., GPT-4o vs 4o-mini) are required, on average, to process a single query (including the amortized cost of library maintenance)? This \"meta-cost\" seems like a major factor in its practical utility.\n\n- The paper's related work cites other tool-creation work (e.g., Skillweaver, ASI) which also learn from \"trajectories\" or \"experience.\" Could you elaborate on the key differences between TVP's \"transductive abstraction\" and the skill-discovery methods used in these agentic works?\n\n- How robust is the TVP framework to the choice of its \"meta-LLMs\"? The system relies heavily on GPT-4o for crucial steps like quality judgment and program generation. If a weaker, open-source model (e.g., Llama3-8B) were used for these orchestration tasks, would the system still be able to successfully identify, abstract, and validate high-quality tools?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The core idea of \"transductive abstraction\" from a library of successful solutions is elegant and well-motivated. It ensures that created tools are practically useful and grounded in experience, which is a clear advantage over VADAR's more speculative, question-based induction (as clearly shown in Fig 2).\n\n- The zero-shot generalization results on the SpatialScore-Hard collection (Table 2, Fig 5) are a key strength. Showing that tools learned only on Omni3D-Bench are effective on completely different datasets (3DSR-Bench, SpatialSense, VG-Bench) is a powerful demonstration that the system is learning robust and reusable reasoning patterns.\n\n- The paper includes a strong set of analyses, such as the reduction in program cyclomatic complexity (Fig 4a), the performance boost from using new tools (Fig 4c), and the visualization of the library evolution over time (Fig 6).", "weaknesses": "- The TVP framework itself is extremely complex and computationally expensive. For each query, it makes multiple LLM/VLM calls (retrieve, generate m programs, execute m programs, judge m programs). It then has a heavy, periodic maintenance loop that involves more LLM calls for clustering, abstraction, validation, and merging. This \"meta-cost\" of running the TVP framework is not discussed but seems prohibitively high, likely many times more expensive than just running a baseline model.\n\n- As mentioned, the field of tool-use and tool-creation is moving very fast. This paper proposes a new way to make tools. While this is a good contribution, it's an improvement on an existing line of work (VisProg -> ViperGPT -> VADAR -> TVP). It's not clear that this is a fundamentally new direction for the field, especially when compared to orthogonal approaches like differentiable soft-logic (e.g., NePTune).\n\n- The entire framework (generation, judgment, abstraction, merging) is orchestrated by powerful, closed-source models (GPT-4o and 4o-mini). This makes the system dependent on SOTA models and raises questions about its robustness. Would the framework collapse if these \"meta-LLMs\" were replaced with less capable open-source models? The quality of the \"judge\" and \"abstractor\" seems critical.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wym7WJGgHM", "reviewer": "Reviewer_fsyr", "summary": "This paper introduces Transductive Visual Programming (TVP), a novel framework that enables visual reasoning systems to evolve reusable tool libraries from problem-solving experience. TVP adopts a transductive approach: it first solves problems using basic vision tools, then abstracts recurring solution patterns into higher-level functions grounded in actual use. The architecture maintains a dual-library design: an Example Library storing verified program solutions and a Tool Library storing learned abstractions. Through iterative cycles of example accumulation, clustering, abstraction, validation, and merging, TVP progressively refines its tools and produces more efficient, accurate programs. On Omni3D-Bench, TVP achieves clear performance gains, surpassing GPT-4o and VADAR. The evolved tools exhibit strong zero-shot generalization to unseen spatial reasoning benchmarks, demonstrating robust transferability across domains.", "questions": "1. Does TVP’s method also improve performance on conventional 2D visual reasoning tasks?  \n2. How sensitive is TVP to the choice of backbone LLM? Would similar gains be observed when replacing GPT-4o with smaller or open-source models, and could the authors provide scaling trends or partial ablations in this direction?  \n3. Could the authors provide a quantitative estimate of TVP’s computational and memory cost per iteration, and clarify whether any optimizations were applied to make the system practically deployable?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- **Conceptual originality:** The paper introduces transductive tool evolution, which learns abstractions from experience rather than induction before use. This represents a genuine conceptual advance in visual programming and aligns well with human-like skill acquisition.  \n- **Technical soundness:** The dual-library architecture and full algorithmic specification (program generation, clustering, abstraction, validation, and merging) are rigorous and clearly grounded. The validation mechanism ensures newly learned tools remain correct and reusable.  \n- **Strong empirical results:** TVP achieves clear and consistent gains over prior visual programming systems and even large VLMs, particularly on complex 3D spatial reasoning and zero-shot transfer tasks.", "weaknesses": "- **Limited scope of evaluation:** While visual programming was originally designed for 2D visual reasoning and perception tasks, this paper evaluates TVP only on 3D spatial reasoning. It remains unclear whether the proposed transductive abstraction also benefits conventional 2D visual reasoning benchmarks (e.g., MME, MMMU).  \n- **Heavy dependence on large proprietary models:** TVP’s components rely heavily on GPT-4o and its mini variants. It remains unclear how performance scales with smaller or open-source models, which may limit reproducibility and accessibility.  \n- **Computational overhead:** The pipeline includes iterative clustering, abstraction, validation, and merging — likely computationally expensive. The paper reports no quantitative analysis of time or resource cost, which would be important for assessing practicality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TrpbXAAytY", "reviewer": "Reviewer_44Xv", "summary": "This paper introduces an advanced visual programming method for 3D spatial understanding. The method starts with basic tools and automatically generates, stores, and optimizes useful tools (functions) while observing and solving problems. The reported quantitative results on 3D spatial reasoning benchmarks surpass all baselines, demonstrating the powerful potential of the method.", "questions": "See the weaknesses section. If I have any misunderstandings, I would greatly appreciate the authors' clarification.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- The paper is well-written. Figures 1 and 3 clearly present the motivation and method.\n\n- The self-evolving pipeline is carefully designed. The composition of spatial functions is novel in this area.\n\n- The quantitative results on the sampled SpatialScore-Hard are promising.\n\n- The case study and evolution curve effectively demonstrate the impact of evolution", "weaknesses": "Visual programming for spatial reasoning has also been used in 3D vision tasks, such as 3D visual grounding [1, 2], and [2] also employs a self-evolving process for spatial reasoning. The authors may consider discussing these works in Section 4.1.\n\n- In lines 159–161, the LLM explores $m$ programs, but only one final answer is provided. Is the final answer selected by the VLMs?\n\n- In line 169, there is mention of a quality threshold for functions, but the metric for quality is not provided.\n\n- In Figure 4(c), why is the performance with new APIs lower?\n\n- There is no analysis of the additional token costs for evolution. Is the cost worthwhile for the observed improvement in overall performance (3.4% in absolute terms, approximately 10% in relative terms)?\n\n- In line 165, does the \"final answer\" refer to the ground truth answer?\n\n- There is no analysis of the accuracy of the VLM judge.\n\n- Does the ordering of the Omni3D-Bench data used for codebase construction affect the codebase? Would altering the data order drastically impact the performance of the resulting code?\n\nReferences:\n\n[1] Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding, in CVPR, 2024.\n\n[2] Language-to-Space Programming for Training-Free 3D Visual Grounding, in EMNLP, 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)