Spatially Guided Training for Vision-Language-Action Model
Authors: Jinhui Ye
Fangjing Wang
Ning Gao
Junqiu Yu
Zhu Yangkun
Bin Wang
Jinyu Zhang
Weiyang Jin
Yanwei Fu
Feng Zheng
Yilun Chen
Jiangmiao Pang
Venue: ICLR 2026 Poster (2026)
Keywords: Spatial Priors
Robot Manipulation
Instruction Following
Primary area: foundation or frontier models, including LLMs

Abstract: Large vision–language models (VLMs) excel at multimodal understanding but fall short when extended to embodied tasks, where instructions must be transformed into low-level motor actions. We introduce SP-VLA, a dual-system **V**ision–**L**anguage–**A**ction framework that leverages **S**patial **P**riors as a bridge between linguistic instructions and embodiment-specific control. 
introduce SP-VLA aligns action learning with spatial priors through two stages: (i) spatial grounding pre-training, which equips the VLM with transferable priors via scalable point, box, and trajectory prediction from both web-scale and robot-specific data, and (ii) spatially guided action post-training, which encourages the model to produce richer spatial priors to guide action generation via spatial prompting.
This design preserves spatial grounding during policy learning and promotes consistent optimization across spatial and action objectives. Empirically, introduce SP-VLA achieves substantial improvements over vanilla VLA, with performance increasing from $66.1{\rightarrow}84.6$ on Google Robot and from $54.7{\rightarrow}73.2$ on WidowX, establishing new state-of-the-art results on SimplerEnv. It also demonstrates stronger generalization to unseen objects and paraphrased instructions, as well as robustness to long-horizon perturbations in real-world settings. These results highlight scalable spatially guided training as a promising direction for robust, generalizable robot learning. We will release code, data, and model checkpoints to support future research.
See more visualization results at the anonymous page: https://sp-vla-anonymous.vercel.app

Published as a conference paper at ICLR 2026
ST4VLA: SPATIALLY GUIDED TRAINING FOR VISION- LANGUAGE-ACTION MODEL
Jinhui Ye Fangjing Wang ,ing Gao' Junqiu Yu!.4, Yangkun Zhu', Bin Wang' Jinyu Zhang Weiyang Jin', Yanwei Fu', Feng Zheng?, Yilun Chen't, Jiangmiao Pang'
'Shanghai Al Laboratory The Hong Kong University of Science and Technology "Southern University of Science and Technology FFudan University
ABSTRACT
Large vision-language models (VLMs) excel multimodal understanding but fall short when extended to embodied tasks, where instructions must be trans- formed into low-level motor actions. We introduce ST4VLA, a dual-system Assion-aanguageAAee Action framework that leverages Spatial Guided Training to align action learning with spatial priors in VLMs. ST4VLA includes two stages: (i) spatial grounding pre-training, which equips the VLM with transferable pri- ors via scalable point, box, and trajectory prediction from both web-scale and robot-specific data, and (ii) spatially guided action post-training, which encourages the model to produce richer spatial priors to guide action generation via spatial prompting. This design preserves spatial grounding during policy learning and promotes consistent optimization across spatial and action objectives. Empirically, ST4VLA achieves substantial improvements
vanilla VLA, with performance increasing from 66.1 1 to 84.6 on Google Robot and from 54.7 to 73.2 on WidowX Robot, establishing new state-of-the-art results on SimplerEnv. It also demonstrates stronger generalization to unseen objects and paraphrased instructions, as well as robustness long- -orizon perturbations in real-world settings. These results highlight scalable spatially guided training as promising direction for robust, generalizable robot learning. Source code, data and models are released at https: //1 W'internrobor cs. github. io/ htt//ineervlll a-mml .gg.tuu....... 1o
INTRODUCTION
Large multimodal foundation models et al. (2024b;; Chen et al. (2024); Bai et al. (2025b); Ye et al. (2025a); Radford et al. 22021) Zhai et al. (2023), Liu et al. (2025b) have demonstrated remarkable generalization capabilities by learning from web-scale vision-language data. However, critical gap remains when transferring these capabilities to the physical domain, because robots must not only understand what an instruction means but also determine where and how to act in the 3D world. This gap is fundamental, as real-world robotic tasks must align textual instruction with embodiment- specific motor actions. However, textual instruction is sparse, whereas real-world actions demand continuous, embodied interactions. Yet, such text-to-action pairs are inherently scarce in standard VLM training data.
Core spatial priors, such as object recognition, affordance grounding, visual trajectory reasoning, and relative localization, provide transferable and generalizable knowledge for robotic manipulation. Once these spatial priors are established, embodiment-speeiffic learning can focus on concrete control strategies (c.g., manipulator joints, end-effector trajectories, humanoid locomotion, mobile navigation). Such a division clarifies the role of spatial priors as general-purpose foundations while leaving embodiment-spper abstract linguistic instruction and grounded physical execution.
details to downstream adaptation, thereby bridging the gap between
Prior work has approached this challenge through hierarchical robotic systems Huang et al. 2023
2024a); Liu et al. (2024); Huang et al. (2024b); : Qi et al. (2025); Cao et al. (002)) y Yuan et al. 2024
"quual contribution
Corresponding author

---

Published as a conference paper at ICLR 2026
Modid Unde
Pakel Grounding
Task Moreing
Synthels Moriccatioon buta
Ar rage dindo bul
Dala
Grounding
Praning
Since Crean Bock
fedecder
Plaving
Model
VLM Planner
DiT Actor
hspu
Sparial Flansing Frompt
Acton
Groundap
Gruuddrg
Deployment
Figure I: ST4VLA integrates spatial priors into the vision- Marton--nngagg-aaae training pipeline. Given a task instruction, the VLM planner produces latent plans through explicit spatial prompting, which then effectively guides the action expert to generate control signals.
which explicitly encode spatial priors using foundation models Fang et al. (0023); ) Kirillov et al. (2023); Oquab al. (2023). However, these methods often rely on rule-based task decomposition and manually designed planning heuristics. The rigid separation between symbolic task structures and low-level motor control makes difficult to scale automatically to complex and diverse tasks, and par- ticularly limits the potential for end-to-end policy learning. In contrast, recent data-driven VLAS Kim
al. [024]; Brohhn et al. (202)): Black et al. (2024); ; Shi et al. (025) y Al (2024); [ee et al. (2025) leverage pretrained vision- language models and large-scale teleoperation datasets Collaboration et al. (2023); Khazatsky et al. (2024); Bu et al. 2025at; Wu et al. (2024) ; star VLA Contributors (2025) to
directly learn robot control. While these approaches remove the need for manual task heuristics, they tend to overfit low-level motor patterns and thus fail to fully exploit spatial priors during execution. Our empirical analysis in Figure further confirms this limitation: naive fine-tuning VLM to VLA joint training with spatial data yield weak alignment between spatial perception and action-learning objectives, which undermines spatial grounding during policy learning.
To address the fundamental gap between multimodal understanding and embodied control, we propose ST4VLA, dual-system vision-language-actinn framework that explicitly integrates spatial priors into robot control through spatial guided training. Unlike prior approaches that either rely on rule-based task decomposition or overfit low-level motor patterns, ST4VLA strategically separates where and what to act from how to act, ensuring reliable and generalizable manipulation. At its core, ST4VLA introduces a two-stage training pipeline. 1) spatial grounding pre-training, the VLM planner acquires transferable spatial priors (point, box, trajectory) by unifying web-scale multimodal grounding data with robot-specific datasets, thereby equipping the model with affordance grounding, localization, and trajectory reasoning. 2) spatially guided action post-training, the action expert conditioned on these spatial priors through lightweight spatial prompting, aligning optimization between perception and control while preserving the VLM's grounding capacity.
This spatially guided training recipe offers three key benefits. First, it preserves spatial grounding during policy learning, thereby avoiding the collapse observed in naive co- co-training. Second, it aligns the optimization dynamics of multimodal perception and action objectives, resulting in more stable and robust learning. Third, it enhances generalization unseen objects, novel instructions, and
2

---

Published as a conference paper at ICLR 2026
long-horizon tasks in real-world settings. Empirically, ST4VLA achieves ofte-of-the-ar results SimplerEnv benchmarks, improves large-scale simulation tasks by over 6% on average, and reaches 92% success real-world long-horizon manipulation under distribution shifts. These findings highlight spatially guided training as a scalable and reliable paradigm for generalist robot learning.
This work makes the following contributions:
We observe that directly fine-tuning a VLM with an action expert as a VLA model leads to a collapse of spatial priors, and that naive co-training with spatial data introduces gradient conflicts between spatial grounding and action objectives. In contrast, simple spatial prompting effectively mitigates these issues (Section 3.T).
We propose ST4VLA, a spatially guided training framework that explicitly aligns action optimiza- tion with spatial grounding objectives, preserving perception while enabling robust control.
We present comprehensive evaluation of ST4VLA, establishing leading performance on large- scale simulation and real-robot experiments benchmark. ST4VLA substantially improves general- ization to unseen objects, novel instructions, and oitor--distribution environments, outperforming strong baselines such as no Black et al. (0024) and GROOT Bjorck et al. (2025).
METHODS
We propose ST4VLA, a spatially guided training framework that bridges spatial understanding with embodied control through novel two-stage training recipe 2.2 As shown in Figure 2 our approach decouples the acquisition of spatial priors from embodiment-specific control, enabling robust instruction following in diverse and complex scenes.
Muttmoodl understanding
Sparial Grounding (box point/ trace)
Sub lask Parniing
Collect snoces
Latent planning
Actions
Condfored/
VLM
VLM Planner
DiT Actor
State (apt)
+
grounding prompt
Spalial prompt
Actions
[bos]
[ooln]]
oliject aeedded
muli-odd
reol&
[boce]
robot
ebservation
Stage
Spatial Grounding Pre-training
Stage
Spattally Guided Action Post-training
Figure 2: Overview of ST4VLA. ST4VLA adopts spatially guided two-stage training pipeline. Stage (spatial grounding pre-training): the VI.M is trained on large-scale multisource multimodal spatial grounding data to learn embodimeen-agnnosicc spatial priors. Stage 2 (spatially guided action post-training): the VLM Planner, functioning as a slow but reliable System reasoner, generates latent planning tokens via spatial prompting the condition to the action expert (instantiated as DiT Actor) to execute as a fast System embodiment-speciff controller.
2.1 MODEL ARCHITECTURE
Dual-system, Dual-superviiion We propose new a dual-system, end-to-end VLA framework based on Qwen2.5-VI., which can foster alignment between the optimization dynamics of the spatial grounding objective and the action policy objective. The framework builds on dual-system architecture: System 2 (the VLM planner) employs as a multimodal encoder to capture spatial and semantic priors, while System I (the Action Expert) adopts compact diffusion transformer (DiT) Peebles & Xie (2023) and a DINOv2 visual encoder Oquab et al. (2023) for embodiment- specific control.
Specifically, the VLM planner is aligned with a broad range of spatial grounding data, both real and synthetic, covering tasks such object detection, affordance recognition, and visual trajectory
3

---

Published as a conference paper at ICLR 2026
planning. In parallel, the Action Expert is trained on robot demonstration data, enabling it to specialize these priors into embodiment-spee -peeciif motor commands. This supervissni strategy establishes a cohesive link between high-level semantic perception and low-level motion control, which is essential for robust instruction following in both simulation and real-world settings.
To connect the VLM Planner with the action expert, adopt lightweight querying transformer (8.7 MB) conditioned on the latent spatial grounding embeddings produced by the VLM Planner. The querying transformer stabilizes expert learning and inference by mapping variable-length input tokens into a fixed set of learnable query tokens. It is implemented as a k-layer cross-attention module, where the query tokens selectively attend intermediate layers of the VLM (e.g. attends only to the final layer).
Latent grounding via spatial prompting. To explicitly activate the spatial perception capability learned during spatial grounding pre-training, we employ spatial prompting during post-action training stage. For instance, in general object manipulation tasks, we append simple prompts such as "Figure out how execute it, then locate the key object needed" after the task instruction. The extracted feature embeddings provide the planner with explicit spatial cues that facilitate more reliable grounding. Motivated by prior studies Driess et al. (2025;; Zhou et al. y [Bjorck et al. (2025) showing that direct gradient flow between action and VLM modules may distort multimodal knowledge, introduce a gradient decay factor within the querying transformer. This attenuates the gradients propagated from the Action Expert back to the VIM (e.g.. by a factor of 0.5), thereby preserving the Planner's ' semantic reasoning ability while still enabling effective joint optimization.
2.2 TRAINING RECIPE
To leverage spatial priors for stronger embodiment--peecffic control in diverse scenarios, ST4VLA adopts spatially guided two-stage training pipeline:
Stage 1: Spatial grounding pre-training. The objective of the first stage (see Figure 2 to establish foundational alignment between generic visuo-linguistic understanding and the specific spatial reasoning demands of robotics, thereby priming the model for the subsequent co-adaptation of grounding and action objectives. To this end, strategically combine large-scale internet vision- language grounding corpora (e.g.. RefCOCO YeuCoCooYuet et al. (216)). LLaVA-One Vision Li et al. (2024a) with targeted robot-specific datasets (e.g., RoboReflt Lu et al. 2023) AO AO et al. (2025b), and ST4VLA Data). This combination ensures that the VLM's spatial priors are not only grounded in broad visual concepts but are also directly relevant to robotic tasks such as bounding-box detection, affordance recognition, and trajectory prediction. By reformatting all robotic data into unified QA structure consistent with web-scale pre- traiiiinn we enable the VLM to develop a spatially- awware representation space under standard supervised fine-tuning framework, which serves as synergistic foundation for joint optimization with the action policy.
Stage 2: Spatially guided action post-training. This stage focuses on learning gaabodiment-speciffe control while maintaining and refining the spatial priors acquired in Stage Beyond co-training with spatial grounding data, where the VLM backbone is updated via next-token prediction on aname- pairs, we further introduce spatial prompting for action data to enhance alignment between semantic reasoning and motion generation. For action sequences, task instruction with a spatial prompt that clicits the VLM's internal reasoning about scene geometry; for example, the instruction "store all toys into the toy box" is extended to "Identify all relevant toys and their spatial relationships to the container."
augment the standard
EXPERIMENTS
We conduct comprehensive experiments evaluate whether aligning the optimization dynamics of multimodal grounding and action policy objectives enables robust robot manipulation. First, we perform a preliminary study to explore the alignment between spatial grounding and action learning during training (Section 3.1.. Next, we assess performance on public simulated benchmarks to establish competitive baselines (Section (.22). We then evaluate large-scale instruction-foll followiig pick-and-place in simulation and real-world to test generalization (Section 3.3 and Section 3.4 ) Finally, we examine real-robot performance on both short-horizon and long-horizon tasks to validate practical deployment capabilities (Section 3.5
4

---

Published as a conference paper at ICLR 2026
3.1 PRELIMINARY: PERCEPTION-ACTION CO-OPTIMIZATION
To systematically investigate whether spatial grounding capabilities influence the manipulation performance of VLAs, we track the co-optimization of spatial perception and manipulation success during training. Furthermore, inspired by Raghu et al. (2017); Fang al. (2024), we quantify the alignment between the two objectives using similarity between gradient matrices. We compare three distinct training strategies using the OXE dataset for action data and curated set of spatial grounding datasets for multimodal co-training:
Vanilla VLA: direct fine-tuning of a pre-trained VL.M on manipulation data only.
Vanilla Co-training VLA: joint optimization on both spatial grounding data and action data.
Spatially Guided Training VLA (ST4VLA): incorporates spatially pretrained and spatial prompt- ing during pretraining and co-training with multimodal data during post-training.
Empirical experiment analysis. Figure 3 (a) and Figure (b) illustrate the interaction between manipulation success (WidowX) and perception performance (on RefCOCO-g) across training steps. Vanilla VLA shows rapid spatial perception degradation, with RefCOCO-g performance dropping to near-random levels by 20k steps, indicating that action-only optimization disrupts spatial representations. Vanilla co-training partially preserves perception but exhibits unstable oscillations in both metrics. Our Spatially Guided approach achieves the best balance: it maintains 70% of original RefCOGO-g performance while reaching 60% WidowX success just 20k steps.
These trends are further substantiated by the comprehensive benchmark results in Table Compared the vanilla co-training baseline, our ST4VI.A achieves superior robotic manipulation performance (84.6% VM / 75.9% VA Google Robot and 73.2% on WidowX) while simultancously preserving stronger multimodal perception and spatial grounding capabilities across all evaluated tasks.
SHIYLA
Vonill Co-trilingg
Windowe
projection Mastrix Simnita diy
St0VLA
Vendi VIA
Vonilo Covnoiiing VIA
S10VA
Training Steps 00)
Training Steps (k)
Training Steps (K)
(6) Percaplion Paformance
[b) Manipulation Parformance
(c) PrjectiioonSSe Similarity
Figure 3: Ablation study the effect of auxiliary spatial prompting during co-training. From left to right: (a) perception performance (loU@0.5 on RefCOCO-g) (b) manipulation performance (Average Success Rate on WidowX); (c) shows the gradient similarity of the spatial grounding and action policy objectives, when taking vanilla co-training or the proposed spatially prompting co-training.
Table 1: Study of VLA training strategies and their effects on multi-modal understanding, spatial grounding, and robot manipulation performance.
Multi-mudal Understanding
Sputial Groanding
Robotic Manipulation
Models
MME
MMV
TestVQA
OCR
POPE
Acc
COCO Caption
BLEUROUGE.
Refenea-g leU0.5
Whereiplace point- Ace
Refit-testit Acc0.5
Google Robot VMVAA
WidowX
VM
Vanilla VLA
66. 1/63.5
54.7
Vanilla
1106
104/15.11
21.4
66.7
70.2/66.5
+Sputially Guided
1374
23.0
13.0/3.7
78.8/70.0
+Sptialll Pretrained
1411
13.013.4
25.5
84.6/75.9
Gradient dynamics analysis. We introduce Projection-Spac Similarity (PSS) Raghu et al. (201) a method to quantify the alignment between the optimization dynamics of the multimodal grounding objective and the action policy objective. The core idea is to compare the gradients induced by each objective shared set of model parameters. Higher PSS values indicate better subspace alignment
5

---

Published as a conference paper at ICLR 2026
between the two optimization processes, validating that action policy optimization coherently builds upon spatial representations. Further methodological details are provided in Appendix Section
As shown in FigureB.o (e), vanilla co-training of action data with spatial data yields PSS of only 0.25, indicating significant misaligmment between the gradient subspaces. In contrast, our spatially guided training approach increases the PSS to 0.42, demonstrating substantially improved optimization consistency. This enhanced alignment correlates with better preservation of spatial perception capabilities and faster convergence in manipulation tasks.
3.2 EXPERIMENTS ON PUBLIC BENCHMARK
We evaluate ST4VLA on the SimplerEnv simulation suite to assess its robustness to visual appearance shifts in Istruction-following tasks. SimplerEny includes both WidowX and Google Robot platforms, short-horizon atomic tasks, and controlled variations in lighting, color, surface texture, and camera pose. We report results on three task sets: Google Robot-VM (visual matching under viewpoint and lighting changes), Google Robot-VA VA (visual aggregation with varying textures and colors), and WidowX-VA VM (cross-robot generalization). We further evaluate ST4VLA on the LIBERO simulation suite, detailed in Appendix Section B.2
Table 2: Result comparisons of robotic manipulation on SimplerEnv (Google-Robol benchmark. The underlined scores indicate the best results excluding ST4VLA. Numbers are officially reported; otherwise, we reimplement and mark such entries with
Google Robot
Pick Coke Can
Move
Open/Close Drawer
Open Top Drawer Place Apple
Models
TraiTain
Avg
Brohan
2022
85.7
44.2
73.0
6.5
52.4
RT-
|XX Colaborationn cral.
2023
56.7
31.7
59.7
21.3
42.4
RT-2-X Broban et al 2025
78.7
77.9
25.0
3.7
46.3
OpenVLA et 2024
18.0
63.0
0.0
34.3
Visual Matching
CogACT
20246
91.3
85.0
71.8
50.9
74.8
Spatial Qu al 2025
X
86.0
77.9
57.4
75.1
To Black all M0
72.7
65.3
38.3
58.8
No
2025
75.3
67.5
42.9
61.9
GROOT NI.5 Bjorck 2025
51.7
$4.0
27.8
7.4
35.2
Magma Yang él al. 20255
83.7
65.4
56.0
6.4
52,9
Vanilla VI.A
90.0
69.8
$2.5
52.2
66.1
Vanilla Co-training VLA
91.3
75.1
55.0
59.4
70.2
ST4VLA
97.3
98.0
65.3
77.8
84.6
RT-1 Brohan et al. 2022
89.8
50.0
32.3
2.6
43.7
RT-1
2023
49.0
32.3
29.4
10.1
30.2
Bimhha
82.3
35.3
OpeaVLA(Kın Kin et
2014
60.8
67.7
28.8
0.0
39.3
Variant Aggregation
CogACT a. 20240
89.6
80.8
28.3
46.6
61.3
SpatialVLA 2025
88.0
82.5
41.8
70.7
Black
75.2
63.7
25.6
54.8
TA-T Pertenn 2025
77.6
68.2
31.3
59.0
GROOT NI.3 Bjorey 2025
69.3
68.7
35.8
4.0
44.5
Magma Yang 20123
68.8
$3.4
18.5
$1.6
Vunilla VLA
92.3
80.3
50.1
31.4
63.5
Vanilla Co-training VI.A
82.6
73.5
62.4
47.5
66.55
ST4VLA
95.6
74.5
68.0
65.3
75.9
Baselines. We compare state-of-the-art open VLA systems, including mo Black
(2024).
GROOT Bjorck et al. 2025), OpenVLA Kim et al. 2024), CogACT al. (2024c), and etc. We also include a Vanilla VLA built on Owen2. 5- - InstInstrct with DiT action expert. When available, we use official reported numbers; otherwise, we reimplement and mark such entries with *. keep training data, observation spaces, and action type aligned with the most popular setups al. (224cc ensure fair comparison.
Result Analysis. The main experimental results are presented Table 2 and Table 3 Compared with prior state-of-the-art models, it attains a 5.9% gain in Google Robot Visual Matching, 5.3% gain in Visual Aggregation, and a 9.8% gain on the WidowX benchmark. These results highlight the strong competitiveness of ST4VLA within the community. Compared to the Vanilla VLA based ST4VLA achieves substantial improvements: a 14.6% increase in Google Robot Visual Matching and 12.4% increase in Visual Aggregation, along with 17.0%
on Qwen

---

Published as a conference paper at ICLR 2026
Table 3: Result comparisons of robotic manipulation on SimplerEnv (WidowX) benchmark. The
underlined scores indicate the best results, excluding our results.
WidowX Robot
Models
Co-Train
Put Spoon Towel
Put Carrx Plate
Stack Green Block Yellow Block
Put Eggnlant Yellow Basket
Avg
KF-1-X (Boohan al. 2022
K
0.0
4.2
0.0
0.0
1.1
Octo- Base CC10 Modell
2024
15.8
12.5
0.0
41.7
17.5
Octo Seee-Smaliua Octo Movel Lean ct
2024
41.7
0.0
56.7
26.7
Visual Matching
CogACT
Open VLA Kim ai 2024
4.2
0.0
0.0
12.5
4.2
SpatialVT>x Dir mi 2025
16.7
25.0
29.2
100.0
42.7
Black
29.1
16.6
62.5
27.1
mc-FAS1 PF75cc 2025
29.1
21.9
10.8
66.6
48.3
GROOT NI 3 [3jorci
2025
75.3
$4.1
$7.0
61.3
61.9
Magma Yang ot
Vanilla VI.A
56.6
63.3
27.0
71.8
54.7
Vanilla Co-trainig VIA
70.3
68.4
20.5
85.2
61.1
ST4VLA
80.2
79.2
35.4
98.0
73.2
improvement on the WidowX benchmark. These results demonstrate the effectiveness of our spatially guided pre-training and action post-training strategies.
3.3 EVALUATION IN SIMULATED LARGE-SCALE PICK-AND-PLACE
Simulation Setup. Existing benchmarks, such SimplerEnv and LIBERO, are limited in both scale and diversity, which restricts their capacity to evaluate instruction following manipulation in cluttered and varied environments. To address these limitations, we construct large-scale simulation benchmark in Isaac-Sim by GenManip Gao et al. (2025). comprising 200 pick-and-place tasks, each involving distinct manipulated objects. With the inclusion of background elements, the benchmark encompasses more than 3,000 objects and containers. Each task was executed once through the data generation pipeline to ensure executability. Furthermore. for each the 200 tasks, we additionally collected trajectories with identical object sets but randomized layouts, which were used for tost-training. Four evaluation tracks in-distribution, unseen objects, new backgrounds, and unseen instructions established to assess the model's 's multidimensional generalization in pick-and-place tasks. Additional details about the evaluation are provided in Appendix Section D.2
75
48
72
75
Succes Rote
distibution
Unseen object
New background
Average
GRODT N1.6
STAVIA
Figure 4: Success rate (%) across different generalization settings on 200 simulated instruction- following pick-and-place tasks.
Results. Since both baseline methods, To Black al. (2024) and GROOT N1.5 | al. (2025), underwent extensive pretraining on large corpora of action data, we ensured a fair comparison by post-training model large-scale dataset of 244K pick-and-place demonstrations simulations generated in Isaac-Sim following stage 2 paradigm. As illustrated in Figure { ST4VLA consistently achieves state-of-the-ar performance across all four evaluation tracks. Our approach outperforms both and GROOT N1.5, highlighting its robust generalization in vision and language, as well as its effectiveness in multi-task learning under spatial guidance.
3.4 EVALUATION IN REAL-WORLD CLUTTERED-SCEES PICK-AND-PLACE
We use the Franka Research robot to evaluate the generalization performance of our model and baselines on the real-world aa--and-pacc tasks. The robot is instructed sort specified objects into designated containers based on natural language commands. We collect IK pick-and- place trajectories involving 23 objects and 5 containers, which used for post-training. Unlike the
7

---

Published as a conference paper at ICLR 2026
200 simulated tasks, the post-training leverages both large-scale simulation data and real-word worldworld trajectories. Details of the weal-world robot setup and additional experimental configurations are provided in Appendix Section D.3
Table 4: Comparison of results on real-world generalization of pick-and-plane tasks. Success rates (%) are reported. Abbreviations: In dist.: in-distribution; New inst.: new instance; Similar dist.: similar distractors; New bg.: new backgrounds; Unseen obj. pos.: unseen object position; Unseen obj.
orient.: unseen object orientation; By attr.: by attribute; By spatial: by spatial relation.
object
obj.
Unseen obj.
Unseen instruction
Models
dist.
Avg.
New inst.
Similar dist.
New bg.
orient.
By altr.
By spatial
no Black
GROUT NL. Bjock al. 2025
78
46
40
47
20
40
59
53
48
ST4VLA
62
63
73
61
65
As shown in Table 4 beyond evaluating model performance across multiple tasks in the in-distribution setting, we further assess generalization along four challenging dimensions: unseen objects, unseen object poses and orientations, and novel instructions. ST4VLA outperforms all baselines across real-world test settings. Even under highly challenging conditions, such as interference from visually similar distractors, novel object instances, and paraphrased instructions, ST4VLA achieves strong results through spatial pretraining and spatially guided post-training. These findings demonstrate the model's robust visual and linguistic generalization in cockannd- tasks. Furthermore, in evaluations involving unseen object poses and orientations, our approach significantly surpasses the baselines mo and GROOT N1.5, benefiting from the diverse grasp positions and trajectories introduced by co- co-training on large-scale simulation data.
3.5
EVALUATION IN LONG-HORIZON MANIPULATION
Sorting Hemms Info Drowers.
Making Sandwiches
Desklop Sorinn.
20
Tark replorving
Phydcal vinference
Tork replareing
dillbcttoon
Physical Winderecce
Tork replonning
OREON MIS
Figure 5: Demonstration and results long-horizon instruction-followin manipulation tasks.
A key strength of our dual-system framework is its ability to leverage the high-level planner System 2 to decompose complex, reasoning-hen -havy tasks into sequences atomic actions, which are then ro- bustly executed by the low-level controller System 1. To evaluate this capability, we design tasks such as desktop sorting, drawer organization, sandwich making, requiring multi-step planning, progress monitoring, and dynamic adaptation. We collect 22 hours of telcoperated demonstrations, segment trajectories into subtasks, and train ST4VLA jointly on task decomposition, subtask identification and action prediction. Performance is evaluated under three settings: In-distribution, physical interference and task replanning. Results in Figure S show that ST4VLA consistently surpasses GROOT N1.5 and mo, reliably grounding high-level goals into executable steps, adapting to disturbances, and dynamically revising plans with minimal degradation, demonstrating strong resilience in dynamic, real-world environments. Additional details on the long-horizon task setup and evaluation settings are provided in Appendix Section D.4
4 RELATED WORK
Hierarchical robot system. Bridging high-level instructions with low-level actions is a central chal- lenge in embodied AI, often addressed by introducing intermediate representations (IRs) ranging from
8

---

Published as a conference paper at ICLR 2026
symbolic structures to learned embeddings Xie al. (0019). Inspired by Chain-of- reasoning,
many works train vision- laagonalagggeeecco action (VLA) models to first output textual plans, improving
interpretability and long- mongzoorzoon performance Zawalsii al. (2024). Beyond text, IRs have taken the form of perceptual cues (e.g.. bounding boxes Griffin 2023). ). grasp points [Ten Pas & Platt (2017) 7).
or dense features Laskin et al. (2020); [air et al. (2022) persistent 3D scene graphs for grounding
plans Rana et al. (2023). and action-centric affordances specifying end-effector poses Nasiriany al. 2024). Recent work further generates spatial localizers directly usable by controllers Huang et al. (025a); Gu et al. (2023); Li et al. (025c). or leverages large foundation models that unity planning with affordance prediction Team et al. (202)); [uo et al. (2025). Specialized models such as RoboRefer Zhou et al. (2025a) target fine-grained spatial grounding with reinforcement learning.
LLaRA Li et al. (2024d) similarly adapts VLMM to robotic control via instruction-styl data. In contrast, our method unifies these directions by modeling latent spatial guidance jointly with action learning, enabling end-to-end optimization from real-world feedback.
Embodied reasoning and planning in VLA. Several recent VLA approaches leverage large-scale multimodal co-training to improve generalization. RT-2 Brohan et al. (2023), ChatVLA Zhou et al. (202cc), and GR-2/3 Cheang al. 2024 2025) combine internet viston language data with robot trajectories, while InstructVLA Yang et al. (2025b) and Tu.s Intelligence et al. (2025) further incorporate instructional signals, sometimes with spatial annotations such bounding boxes, to enhance language-action alignment. Parallel efforts explore explicit reasoning: ECOT Zawalski et al. (2024) generates textual plans, RT-H|Belkhale et al./ (222)) introduces action language for hierarchical control, Instruct VLt Yang et al. (2025b) jointly optimizes reasoning and action, One Lin al. 2025) alternates between "thinking" and execution, RAD Clark al. (2025) distills reasoning from human videos, and graph-based IRs IRs/Hung et al. (2025b) support spatial reasoning. Beyond textual reasoning, recent works have incorporated visual foresight and structural planning: CoT- VLA Zhao et al. (2025) generates future video frames as a visual chain-of-thoun tfthouh,, Chain-of- Action Zhang et al. (2025) and LBP Liu et al. (2025a) apply backward goal-based planning, while ATM Wen et al. (2023) extracts control signals from unlabeled videos via point-trajecten prediction and LLARVA Niu et al. (2024) leverages visual-trace representations to align vision and action. While these methods expand semanties and interpretability, they often treat multimodal data as generic supervision, overlook explicit spatial grounding, and rely on costly generative reasoning. In contrast, our approach strategically emphasizes spatial grounding data and introduces a spatially guided co-training scheme with gradient alignment, coupled with lightweight post-training phase that unlocks intrinsic reasoning in VLMs without requiring explicit outputs.
Generalist robot policy. Recent progress in general-purpose robotics follows three main paradigms. Monolithic VLA models directly map multimodal inputs to tokenized actions with a single net- work Brohan et al. (2023); Kim et al. (2024): Lee et al. (2025). Unified architectures decouple high-level cognition from lewelevel control, enabling modularity and interpretability Black et al. (2024); Li et al. [024cc 2025a); Zheng et al. (2024); Intelligence et (2025)) Song et al. (2025); Zhou et al. (2025b); Yang et al. (2025b); Shukor et al. (hhann et al. (2025). World models instead learn predictive environment dynamics to plan in latent space, offering strong foresight but at higher computational cost Ye et al. (2025b); Bjorck et al. | (2025); Li et al. (2025b); Cen et al. (2025);
Liao et al. (025); Tian al. (2024) Bu et al. (2025b); Wang et al. (2025) y Ly et al. (2025). Similar to ours, Magma Yang et al. (2025a) also adopts spatial pre-training, though it does not explicitly leverage
spatial prompting to guide action generation. Our approach extends unified VLA architectures with
dual-system design that boosts adaptability for real-world tasks.
5 DISCUSSION AND CONCLUSION
In this work, we presented ST4VLA, a unified laatoon-lanuag-aattiioo framework that leverages spatial grounding priors to bridge high-level multimodal reasoning with low-level robotic execution. By combining large-scale multimodal pre-training with spatially guided post-training, our model effectively transfers perceptual and reasoning skills into embodied control, achieving strong general- ization to unseen objects, instructions, and environments. Extensive evaluations across simulation and real-world settings demonstrate that ST4VLA surpasses existing VLA models and specialized systems in instruction following, long-horizon manipulation. and multimodal grounding, highlighting spatial reasoning as unifying substrate for scalable and reliable generalist robots.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research is supported by Shanghai Artificial Intelligence Laboratory, and is funded in part by the National Key R&D Program of China (2022ZD01 (0201).. Jinhui Ye partly supported by the Research Grants Council under the Areas of Excellence scheme grant AoE/E-601/22-R, Prof. Yanwei Fu partly supported by Shanghai Municipal Science and Technology Major Project (0055H2DZX0025602), This paper systematically analyzes the impact of spatial training on (VLA) models and extends the manipulation spatial data for generalist purposes within the Intern framework [Contributors] (2025). We would like to extend our sincere gratitude to all contributors for their work on Intern LL--MI Team and the Intern Robotics Team which encompasses data collection, model development, simulation, benchmarking, real-robot deployment, and open-source efforts: Xinyi i Chen, Jiaya Jia, Hao Li, Yao Mu, Yu Qiæo, Yang Tian, Bolun Wang, Hanqing Wang, Tai Wang, Ziqin Wang, Xueyuan Wei, Chao Wu, Shuai Yang, Jia Zeng, Jingjing Zhang, Shi Zhang, Bowen 2hou (listed in alphabetical order by last name).
Vision-Language
REFERENCES
Figure AL Helix, 2024. URL
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang. Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang. Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang. Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, Junyang Lin, and Qwen2.5-vl technical report. arXiv preprint ar Xiv: 13923, 2025a.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv 13923, 2025b.
Suneel Belkhale, Tianli Ding, Ted Xiao, Pierre Sermanet, Quon Vuong, Jonathan Tompson, Yevgen Chebotar, Debidatta Dwibedi, and Dorsa Sadigh. Rt-h: Action hierarchies using language. arXiv preprint arXiv:2403.0188 02233 2024.
Johan Bjorek, Fernando Castañeda, Nikita Cherniadey, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr0Ot nl: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503. 14734, 2025.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. \pig: A Asioonlanguuge-ace flow model for general robot control. arXiv preprint arXiv:2410.24164 2024.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.0683 2022.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.158// 2023.
Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He. Xuan Hu, Xu Huang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.0660 066699, 2025a.
Qingwen Bu, Yanting Yang. Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Univla: Learning to act anywhere with task-centric latent actions. arXiv preprint arXivv2505.06111. 061111, 2025b.
Mingyu Cao, BAAI RoboBrain Team, et al. Robobrain 2.0 technical report. Technical report, Beijing Academy of Artificial Intelligence (BAAT), 2025. arXiv preprint DoXiv:2507002022
10

---

Published as a conference paper at ICLR 2026
Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo. Fan Wang, et al. Worldvla: Towards autoregressive action world model. arXiv preprint arXiv:2506.2153, 2025.
Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative rideo-language-action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.061 2024.
Chilam Cheang. Sijin Chen, Zhongren Cui, Yingdong Hu, Liqun Huang. Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Xiao Ma, et al. Gr-3 technical report. arXiv preprint arXiv:2507.15493, 2025.
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing. Muyan Zhong, Qinglong Zhang. Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24185-24198, 2024.
Jaden Clark, Suvir Mirchandani, Dorsa Sadigh, and Suneel Belkhale. Action-free reasoning for policy generalization. arXiv preprint arXiv:2502.03729, 2025.
Open X-Embodiment Collaboration, Abby O' O'Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acom Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang. Andrey Kolobov, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerinn Beomjoon Kim, Bernhard Schölkopf, Blake Wulfe, Brian Ichter, Cewu Lu, Charles Xu, Charlotte Lc, Chelsea Finn, Chen Wang. Chenfeng Xu, Cheng Chi, Chenguang Huang, Christine Chan, Christopher Agia, Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Driess, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Foster, Fangchen Liu, Federico Ceola, Fei Xia, Feiyu Zhao, Felipe Vieira Frujeri, Freek Stulp, Gaoyue Zhou, Gaurav Sukhatme, Gautam Salhotra, Ge Yan, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guangwen Yang, Guanzhi Wang, Hao Su, Hao-Shu Fang, Haochen Shi, Henghui Bao, Heni Ben Amor, Henrik I Christensen, Hiroki Furuta, Homanga Bharadhwaj. Homer Walke. Hongjie Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jaehyung Kim, Jaimyn Drake, Jan Peters, Jan Schneider, Jasmine Hsu, Jay Vakil, Jeannette Bohg, Jeffrey Bingham, Jeffrey Wu, Jensen Gao, Jiaheng Hu, Jiajun Wu, Jialin Wu, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jic Tan, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jingyun Yang, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi Salvador, Joseph J. Lim, Junhyek Han, Kaiyuan Wang. Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go, Keerthana Gopalakrishnan, Ken Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Lin, Kevin Zhang, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh, Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti, Lawrence Yunliang Chen, Lerrel Pinto, Li Fei-Fei, Liam Tan, Linxi "Jim" Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka, Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang, Mingyu Ding, Minho Heo, Mohan Kumar Srirmma, Mohit Sharma, Moo Jin Kim, Naoaki Kanazawa, Nicklas Hansen, Nicolas Heess, Nikhil J Joshi, Niko Suenderhauf, Ning Liu, Norman Di Palo, Nur Muhammad Mahi Shafulllh, Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R Sanketi, Patrick "Tree" Miller, Patrick Yin, Paul Wohlhart, Peng Xu, Peter David Fagan, Peter Mitrano, Pierre Sermanet. Pieter Abbeel, Priya Sundaresan. Qiuyu Chen, Quan Vuong, Rafacl Rafailov, Ran Tian, Ria Doshi, Roberto Mart Mn-Mart in, Rohan Baijal, Rosario Scalise, Rose Hendrix, Roy Lin, Runjia Qian, Ruohan Zhang, Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante, Sean Kirmani, Sergey Levine, Shan Lin, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham Sonawani, Shubham Tulsiani, Shuran Song, Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola. Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suncel Belkhale, Sungjae Park, Suraj Nair, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony 7. Zhao, Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vikash Kumar, Vincent
11

---

Published as a conference paper at ICLR 2026
Vanhoucke, Wei Zhan, Wenxuan Zhou, Wolfram Burgard, Xi Chen, Xiangyu Chen, Xiaolong Wang, Xinghao Zhu, Xinyang Geng, Xiyuan Liu, Xu Liangwei, Xuanlin Li, Yansong Pang, Yao Lu, Yecheng Jason Ma. Yejin Kim, Yevgen Chebotar, Yifan Zhou, Yifeng Zhu, Yilin Wu, Ying Xu, Yixuan Wang, Yonatan Bisk, Yongqiang Dou, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yue Cao, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang. Yunfan Jiang. Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma. Zhuo Xu, Zichen Jeff Cui, Zichen Zhang, Zipeng Fu, and Zipeng Lin. Open X-Embodiment: Robotic learning datasets and RT-X models. petpss:/arxiin org/ hwtgg/bb///331 23w. 08864 2023.
IntemVLA-Mi Contributors. Intervlaaml:: A spatially guided ussion-languag-- framework for generalist robot policy. arXiv preprint arXiv:2510. 13778, 2025.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang. Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison- Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Borchardt, Dirk Groeneveld, Jen Dumas, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif. Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi. Molmo and pixmo: Open weights and open data for thaeeoffthhe-art multimodal models. arXiv preprint arXiv:2409.17146. 2024.
Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, et al. Knowledge insulating vision- Assionlanguuge- action models: Train fast, run fast, generalize better. arXiv preprint arXiv:2505.23700 2025.
Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains. IEEE Transactions on Robotics, 2023.
Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. arXiv preprint arXiv:2410.02355, 2024.
Ning Gao, Yilun Chen, Shuai Yang, Xinyi Chen, Yang Tian, Hao Li, Haifeng Huang, Hanqing Wang, Tai Wang, and Jiangmiao Pang. Genmanip: Lim-driven simulation for generalizable instroction-followinng manipulation. In CVPR, 2025.
Brent Griffin. Mobile robot manipulation using pure object detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pp. 561-571,
Jiayuan Gu, Sean Kirmani, Paul Wohlhart, Yao Lu, Montserrat Gonzalez Arenas, Kanishka Rao, Wenhao Yu, Chuyuan Fu, Keerthana Gopalakrishnan, Zhuo Xu, et al. Rt-trajectory: Robotic task generalization via hindsight trajectory sketches. arXiv preprint arXiv:2311.0197 2023.
Haifeng Huang, Xinyi Chen, Yilun Chen, Hao Li, Xiaoshen Han, Zehan Wang, Tai Wang, Jiangmiao Pang, and Zhou Zhao. Roboground: Robotic manipulation with grounded vision-language priors. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22540-22550, 2025a.
Haoxu Huang, Fanqi Lin, Yingdong Hu, Shengjie Wang, and Yang Gao. Copa: General robotic manipulation through spatial constraints of parts with foundation models. arXiv preprint arXiv:2403.08224, 2024a.
Helong Huang, Min Cen, Kai Tan, Xingyue Quan, Guowei Huang, and Hong Zhang. Graphcot-vla: A 3d spatial-aware reasoning lision-languuge-acction model for robotic manipulation with ambiguous instructions. arXiv preprint arXiv:2508.00 07650, 2025b.
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Voxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05973. 2023.
12

---

Published as a conference paper at ICLR 2026
Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal rea- soning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652 2024b.
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. plo.s: vision- vision-language-attiin model with open-world generalization. arXiv preprint arXiv:2504. 16054, 2025.
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna. Sudcep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint ar arXiv:2433. 2at.:2443..999.. 12945, 2024.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision- lision-anggaage-cctionn model. arXiv preprint arxi::2406.0922 092406660996..... 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-tanguage-action models: Optimizing speed and success. arXiv preprint arxiv:2502. /9645, 2025.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02664, 2023.
Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning. In International conference on machine learning, pp. 5639-5650. PMLR, 2020.
Jason Lee, Jiafei Duan, Haoquan Fang, Yuquan Deng, Shuo Liu, Boyang Li, Bohan Fang, Jicyu Zhang, Yi Ru Wang, Sangho Lee, et al. Molmoact: Action reasoning models that reason in space. arXiv preprint arXiv:2508.07911 2025.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprins arXiv:2408.00 033326 2024a.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0333 2024b.
Hao Li, Shuai Yang, Yilun Chen, Yang Tian, Xiaoda Yang, Xinyi Chen, Hanqing Wang, Tai Wang, Feng Zhao, Dahua Lin, et al. Cronusvla: Transferring latent motion across time for multi-frame prediction in manipulation. arXiv preprint arXiv:2506. 19816, 2025a.
Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision- lssionllaguugge-cctoon model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411. 2024c.
Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified video action model. arXiv preprint wwXiv:2503.00200 2025b.
Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, et al. Llara: Supercharging robot learning data for vasion-language policy. arXiv preprint 2iw:Xi::446.20000 2024d.
Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Raymond Yu, Caelan Reed Garrett, Fabio Ramos, Dieter Fox, Anqi Li, et al. Hamster: Hierarchical action models for open-world robot manipulation. arXiv preprint arXiv:2502.05485. 2025c.
Hongzhuo Liang, Xiaojian Ma, Shuang Li, Michael Gömer, Song Tang, Bin Fang, Fuchun Sun, and Jianwei Zhang. Pointnetgpd: Detecting grasp configurations from point sets. In 2019 International Conference Robotics and Automation (ICRA), pp. 3629-3635. IEFF, 2019.
13

---

Published as a conference paper at ICLR 2026
Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, Shengcong Chen, Yuxin Jiang, Yue Hu, Jingbin Cai, Si Liu, Jianlan Luo, et al. Genie envisioner: A unified world foundation platform for robotic manipulation. arXiv preprint arXiv:2508.05633, 2025.
Fanqi Lin, Ruiqian Nai, Yingdong Hu, Jiacheng You, Junming Zhao, and Yang Gao. Onetwovla: A unified iasion-laguaae-acttn model with adaptive reasoning. arXiv preprint arXiv:2505.11917, 2025.
Dongxiu Liu, Haoyi Niu, Zhihao Wang, Jinliang Zheng, Yinan Zheng, Zhonghong Ou, Jianming Hu, Jianxiong Li, and Xianyuan Zhan. Efficient robotic policy learning via latent space backward planning. arXiv preprint arXiv:2505. 06861, 2025a.
Fan Liu, Jindong Han, Tengfei Lyu, Weijia Zhang, Zhe-Rui Yang, Lu Dai, Cancheng Liu, and Hao Liu. Foundation models for scientific discovery: From paradigm enhancement to paradigm transition. Authorea Preprints, 2025b.
Fangchen Liu, Kuan Fang, Pieter Abbeel, and Sergey Levine. Moka: Open-vocabulary robotic manipulation through mark-based visual prompting. In First Workshop on Vision-L.anguage Models for Navigation and Manipulation ICRA 2024, 2024.
Yuhao Lu, Yixuan Fan, Beixing Deng, Fangfu Liu, Yali Li, and Shengjin Wang. VI-grasp: 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes. In 2023 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 976-983, IEEE, 2023.
Gen Luo, Ganlin Yang. Ziyang Gong, Guanzhou Chen, Haonan Duan, Erfei Cui, Ronglei Tong, Zhi Hou, Tianyi Zhang, Zhe Chen, et al. Visual embodied brain: Let multimodal large language models see, think, and control in spaces. arXiv preprint 2xX.:25566000233.. 2025.
Qi Lv. Weijie Kong, Hao Li, Jia Zeng, Zherui Qiu, Delin Qu, Haoming Song, Qizhi Chen, Xiang Deng, Michael Yu Wang, Liqiang Nic, and Jiangmiao Pang. Fl: A vision-language--ction model bridging understanding and generation actions. 2025. URL httpp:://axxvv.orgg////o//////
05099069570
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo. Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2 10470, 2021.
Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L. Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 11 11-20, 2016.
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203. /2601, 2022.
Soroush Nasiriany, Sean Kirmani, Tianli Ding, Laura Smith, Yuke Zhu, Danny Driess, Dorsa Sadigh, and Ted Xiao. Rt-affordance: Affordances are versatile intermediate representations for robot manipulation. arXiv preprini N.Xi::2411.02704. 2024.
Dantong Niu, Yuvan Sharma, Giscard Biamby, Jerome Quenum, Yutong Bai, Baifeng Shi, Trevor Darrell, and Roei Herzig. Llarva: Vision-action instruction tuning enhances robot learning, arXiv preprint arXiv:2406.. 11815, 2024.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Jocy Hejna, Charles Xu, Jianlan Luo, Tobias Kreiman, You Liang Tan, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, 2024.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafranicc, Vasil Khalidov, Pierre Fermandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinoy2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07/999 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEFE/CVF international conference on computer vision, pp. 4195-4205. 2023.
14

---

Published as a conference paper at ICLR 2026
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision ssion-language-action models. arXiv preprint arX1v:2501.09747, 2025.
Zekun Qi. Wenyao Zhang, Yufei Ding, Runpei Dong, Xinqiang Yu, Jingwen Li, Lingyun Xu, Baoyu Li, Xialin He. Guofan Fan, Jiazhao Zhang, Jiawei He. Jiayuan Gu, Xin Jin, Kaisheng Ma, Zhizheng Zhang, He Wang, and Li Yi. Sofar: Language- - orientation bridges spatial reasoning and object manipulation. CoRR, abs/2502. 13143, 2025. doi: 10.48550ARXIV2550213333 URL
htttps://do .ww.oooorrg.ooo wtt.o/////.../////////
48550/ aw./0v/arxvv..m 2502. 13143
Delin Qu, Haoming Song. Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, Jia Yuan Gu, Bin Zhao, Dong Wang. et al. Spatialvla: Exploring spatial representations for disual-language action model. arXiv preprint arXiv:2501.15833 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PMLR, 2021.
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. Sveca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Advances in neural information processing systems, 30, 2017.
Krishan Rana, Jesse Haviland, Sourav Garg, Jad Abou-Chakra, Ian Reid, and Niko Suenderhauf. Sayplan: Grounding large language models using 3d scene graphs for scalable robot task planning. arXiv preprint 0rXiv:2307.06113 2023.
Lucy Xiaoyang Shi, Brian Ichter, Michael Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang. Niccolo Fusai, et al. Hi robot: Open-ended instruction following with hierarchical vision-language-aciin models. arXiv preprint arXiv:2502. 19417, 2025.
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zoui- tine, Michel Aractingi. Caroline Pascal, Martino Russi, Andres Marafioti, al. Smolvla: A vision- language-action model for affordable and efficient robotics. arXiv preprini arXiv:2506.09 844, 2025.
Shweta Singh, Aayan Yadav, Jitesh Jain, Humphrey Shi, Justin Johnson, and Karan Desai. Bench- marking object detectors with coco: A new path forward. In European Conference on Computer Vision, pp. 279-295. Springer, 2024.
Haoming Song, Delin Qu, Yuanqi Yao, Qizhi Chen, Lv, Yiwen Tang, Modi Shi, Guanghui Ren, Maoqing Yao, Bin Zhao, et al. Hume: Introducing system-2 thinking in visual-language-action model. arXiv preprint arXiv:2505.2244
2025.
starVLA Contributors. Starvla: A lego-like codebase for vision-languuag-accionn model developing. GitHub repository, 2025. URL htttt::tts:://gttttt
com/ starVLA/staaraa....
BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang, Enshen Zhou, Yi Han, et al. Robobrain 2.0 technical report. arXiv preprint arXiv:2507.02022 2025.
Andreas Ten Pas and Robert Platt. Using geometry to detect grasp poses in 3d point clouds. In Robotics Research: Volume /, pp. 307-324. Springer, 2017.
Yang Tian, Sizhe Yang, Jia Zeng, Ping Wang, Dahua Lin, Hao Dong, and Jiangmiao Pang. Pre- dictive inverse dynamics models are scalable learners for robotic manipulation. arXiv preprint 1rXiv:2412.15109, 2024.
Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang. Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-sceing project v2: Towards general relation comprehension of the open world. In European Conference on Computer Vision, pp. 471-490. Springer, 2024.
15

---

Published as a conference paper at ICLR 2026
Yuqi Wang. Xinghang Li, Wenxuan Wang, Junbo Zhang, Yingyan Li, Yuntao Chen, Xinlong Wang, and Zhaoxiang Zhang. Unified Dision-language-attinn model. arXiv preprint arXiv:2506.19850 2025.
Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning, arXiv preprint arXiv:2401.00025 2023.
Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. Robomind: Benchmark on eubtiieembo intelligence normative data for robot manipulation. arXiv preprint ar arXiv:2412. 2024.
Bin Xiao, Haiping Wu, Weijian Xu, Xiyang Dai, Houdong Hu, Yumao Lu, Michael Zeng, Ce Liu, and Lu Yuan. Florence-2: Advancing unified representation for variety of vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 2024.
Yaqi Xie, Ziwei Xu, Mohan S Kankanhalli, Kuldeep S Meel, and Harold Soh. Embedding symbolic knowledge into deep networks. Advances in neural information processing systems, 32, 2019.
Rongtao Xu, Jian Zhang, Minghao Guo, Youpeng Wen, Haoting Yang, Min Lin, Jianzheng Huang, Zhe Li, Kaidong Zhang, Liqiong Wang. Yuxuan Kuang, Meng Cao, Feng Zheng, and Xiaodan Liang. A0: An affordance-aware hierarchical model for general robotic manipulation, 2025a. URL
bltps::/arxxer httpos//aaxx..orrrr/////22 12636
Rongtao Xu, Jian Zhang. Minghao Guo, Youpeng Wen, Haoting Yang, Min Lin, Jianzheng Huang, Zhe Li, Kaidong Zhang, Liqiong Wang, et al. A0: An affordance-aware hierarchical model for general robotic manipulation. arXiv preprint arXiv:2504. 0ee::2504.16666 2025b.
Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng. Yongyuan Liang, Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, et al. Magma: foundation model for multimodal ai agents. arXiv preprint arXiv:2502. 13130, 2025a.
Shuai Yang, Hao Li, Yilun Chen, Bin Wang, Yang Tian, Tai Wang, Hanqing Wang. Feng Zhao, Yiyi Liao, and Jiangmiao Pang. Instructvla: Vision-language-actionn instruction tuning from understanding to manipulation. arXiv preprint 2rXiv 2025b.
Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyza- guirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, et al. Re-thinking temporal search for long-form video understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8579 8591, 2025a.
Seonghyeon Ye, Joel Jang, Bycongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettle- moyer, Dieter Fox, and Minjoon Seo. Latent action pretraining from videos. In The Thirteenth International Conference on Learning Representations (ICLR), 2025b.
Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. In European conference on computer vision, pp. 69-85. Springer, 2016.
Wentao Yuan, Jiafci Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. Robopoint: langual model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406. 10721, 2024.
Michat Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied thhon-hf- reasoning. arXiv preprint KXiv:2407.0869) 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11975-11986, 2023.
Wenbo Zhang, Tianrun Hu, Yanyuan Qiao, Hanbo Zhang, Yuchu Qin, Yang Li, Jiajun Liu, Tao Kong, Lingqiao Liu, and Xiao Ma. Chain-of-action: Trajectory autoregressive modeling for robotic manipulation. arXiv preprint arXiv:2506.09990 2025.
16

---

Published as a conference paper at ICLR 2026
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma. Song Han, Chelsea Finn, et al. Cot-vla: Visual chain-of oftohhugttthttttt reasoning for vision Dision-languuuo-actunn models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1702-1713, 2025.
Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé III, Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-tempone awareness for generalist robotic policies. arXiv preprint arXiv:2412.10344 2024.
Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tięjun Huang, Lu Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. arXiv preprint 2506.
2025a.
Zhongyi Zhou, Yichen Zhu, Junjie Wen, Chaomin Shen, and Yi Xu. Vision-language-action model with open-world embodied reasoning from pretrained knowledge. arXiv preprint arXiv:2505.21906
2025b.
Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng, Ran Cheng, Yaxin Peng, Chaomin Shen, et al. Chatvla: Unified multimodal understanding and robot control with visionlanguageeactiin model. arXiv preprint arXiv:2502.1 14220, 2025c.
Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang. Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang. Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang, Jiapeng Luo, Yi Wang, Conghui He, Botian Shi, Xingcheng Zhang. Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu. Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Internv13: Exploring advanced training and test-time recipes for open-souree multimodal models, 2025. URL wwt.iiv.......///..//.........vv org. abss 4504 10479
17

---

Published as a conference paper at ICLR 2026
Supplementary Materials ST4VLA: Spatially Guided Training for Vision-Language-Aciin Model
Table of Contents
Projection-space Similarity (PSS)
19
B Additional Experiments
19
B.1 Further Study for Spatial Prompting
19
LIBERO Benchmark
Ablation Studies
20
Impact of Spatial Grounding Pre-training
20
The Impact of Cotrain Loss Weight
21
C,3
Backbone Backbone-Agnosti Agnostic Generalization and Training Method Contribution
22
C.4
Scaling Laws of Spatial Priors]
22
Ablation Study on Spatial Prompt Formulations
22
D
Experiments setup
23
D.I
Evaluation SimplerEny
23
D.2
Evaluation Simulated Large scale Pick and place
24
D.3
Real-world Pick and and--llce Manipulation Setup
24
D.4
Real-world Long horizon Manipulation Setup
26
D.5
Real-world Robot Setup)
27
Case Study
28
E.1
Case Study for Public Benchmarks
28
E.2
Case Study for Simulated Large- scale Fick-andppp place Manipulation
31
E,3
Case Study for Real world Pick and place Manipulation
31
E,4
Case Study for Real-world Long-horizon horizon Manipulation
33
F.5
Failure Case Study
36
Data
36
Spatial Grounding Data for Pre-training
36
P2
Synthetic Data For Action Post-Pre-traiming
37
F3
Scalable Synthetic Data Engine for Instruction Instruction-Follovinn
38
G
Post- Pest-Processing of Teleoperated Data
39
G.
Real Teleoperated Data Processed for Evaluating Long-horizon and Interactive Tasks 39
18

---

Published as a conference paper at ICLR 2026
A PROJECTION-SPACE SIMILARITY (PSS)
Setup. To quantify the alignment between the optimization directions of spatial grounding and robot manipulation tasks, we analyze the similarity of their loss gradients with respect the shared parameters. Let 0 E Rdxn denote the shared parameters of the model (i.e., the VLM backbone). We fix two probing bini-batches: batch of grounding data Bspat and batch of action data Bact. We then compute the gradients of the respective losses with respect to @ for each batch, resulting in the following gradient matrices:
Gapat V VoLspat(Bsppat 0) E Rdx" Gact = Va Ca 0) E Rdxx.
(I)
Projection-space similarity (PSS). To capture structural alignment between the spatial grounding objective and the action manipulation objective, we compare the subspaces spanned by Gspat and Gaet via Singular Value Decomposition (SVD). Let Pspat and Pact be the orthogonal projectors onto range( (Gege spas) and range(Gacd respectively. Using the Moore-Penross pseudoinverse (.)*,
Pa spat
Gspat Gspar:
Pact
Gact G
Denote T Tspat = rank (ppat)) and Fact = raakk (Gact). The projection-space similarity is define as:
tr(Pspat Pact).
PSS (aapt:: Gact) =
E
[0,1],
(3)
min Tact)
which equals the mean of squared cosines of the principal angles between the two subspaces. value of 1 indicates identical subspaces.
Protocol. Given the billion-scale parameters of the VLM backbone, computing gradients for the entire model would be computationally prohibitive. Therefore, restrict analysis to single layer: the projection in the self-attention module of the final layer of the Qwen language model, whose parameter E IRd is 2048 x 2048 weight matrix. We focus this particular layer because lies at the interface between the language model backbone and the action expert, making it the most informative point for capturing the interaction between the two components.
During training, we periodically compute PSS using the fixed probing evaluation sets (batch size 64 for each type data). A higher PSS indicates that the action policy optimization well-aligned with the features learned through multimodal grounding
B ADDITIONAL EXPERIMENTS
B.1 FURTHER STUDY FOR SPATIAL PROMPTING
To address the question of whether spatial priors merely accelerate convergence or are essential for final performance capability, extended the training horizon of all models to 100k steps. As hypothesized, standard baselines might require longer training to fully saturate; however, our extended analysis demonstrates that the performance gap remains significant even after convergence.
Figure 6 illustrates the training dynamics across 100k steps for both WidowX and Google Robot environments. While the Vanilla VLA and Vanilla Co-training baselines continue show marginal improvements early on, they ultimately converge to substantially lower plateaus compared to our method. These results conclusively show that explicitly injecting spatial priors does not simply act as a warm-up for faster learning; it fundamentally alters the optimization landscape, allowing the policy to generalize to a higher performance ceiling that standard multimodal co-training fails reach.
B.2 LIBERO BENCHMARK
LIBERO. LIBERO is a language-conditioned manipulation suite built on Franka arm with diverse scenes and expert demonstrations. We evaluate four task sets: LIBERO- -Spttiall (same objects, different spatial layouts), LIBERO-Object (fixed layout, different objects), LIBERO-God (fixed objects and layout, different goals), and LIBERO-Long (also known as LIBERO- 10; longer tasks that span multiple objects, layouts, and operations).
19

---

Published as a conference paper at ICLR 2026
success Nate on Google Robut (VP))
suc cass Batte on Google (VA)
Succoss Ratte Widoo
Vonilo VIA
Vanila
Vanilo VIA
Vosile Go-kdinng VIA
Vacilo Coitrining
Vonil Co-noiinn VLA SIEVLA
SIPVLA
Training Stops (k)
Training Steps (k)
Training Steps (k)
(o) WidowX VM
(b) Google Robot VM
[c] Google Robot VA
Figure 6: Extended training prolonged training, baselines saturate at a significantly lower performance level compared to our method (ST4VLA), confirming that spatial grounding improves the policy's upper bound rather than just convergence speed.
up 100k steps WidowX and Google Robot tasks. Even with
Table Result comparisons of robotic manipulation on LIBERO (Franka) benchmark.
Models
Spatial
Objects
Goal
Long
Avg
VLe Kim et al. 2024
84.7
88.4
79.2
$3.7
76.5
SpatialVLA Qu et al. 2025
88.2
89.9
78.6
55.5
78.1
CoT-VLA Zhao al. 2025
87.5
91.6
87.6
69.0
83.9
GROUT N Bjocck
2025
94.4
97.6
93.0
90.6
93.9
m
Black et al. 2024)
96.8
98.8
95.8
85.2
94.2
no
Pertsch al.
(2025
96.8
88.6
60.2
85.5
No.5
Kl Driess al. 2025
98.0
97.8
95.6
85.8
94.3
Vanilla VLA
98.8
98.0
81.4
88.0
91.6
ST4VLA
98.0
99.0
93.8
92.6
95.9
Experimental setups. Following Kim et al. (2025), filter out failed demonstrations and pause frames. During training, the policy takes as input both wrist-mounde and third-person camera views. We fine-tune the model each suite independently using 8 A100 GPUs with batch size of 128 and an action chunk size of 8. Training runs for roughly 30K steps, lasting about 20 hours. Each suite is evaluated with 500 trials.
Result analysis. The primary experimental results on the LIBERO benchmark are presented in 35 Compared to previous strong baselines, such GROOT N1 and To. the ST4VLA framework achieves notable improvements, particularly on the spatial and long-horizon tracks, with success rates of 98.0% and 92.6%, respectively. These results demonstrate the efficacy of our proposed method in managing complex, multi-step manipulation tasks. Specifically, for object placement, ST4VLA attains 99.0% SR, which highlights its robust object grounding capability.
C ABLATION STUDIES
As described in Section ST4VLA achieves significant performance improvements on the Sim- plerEnv benchmark. this section, we conduct ablation studies examine the contribution of each critical component. Specifically, we investigate: (1) the impact of different pretrained models used in the first-stage pre-traine pre-training: (2) the ratio of multimodal data during the second-stage post-training (3) whether incorporating spatial prompts into task instructions brings improvement.
C.1 IMPACT OF SPATIAL GROUNDING PRE- TRAINING
We first analyze the impact of different pre- traiirann data configurations in the first-stage pre-training. Our model is evaluated under three settings: 1) Using the official QwenVL-2 5-33-Instted ruct. weights without additional spatial pre-training: 2) Pretraining with general multimodal grounding
20

---

Published as a conference paper at ICLR 2026
data (e.g., VanaVA-O-eeiiiei and RefCOCO); 3) Pretraining with ST4VLA robotic grounding data. The proportions of each data type used are provided the Appendix Section
Table Performance comparison under different pretraining data settings.
Robotie Grounding (pre-training)
Robotic Manipulation
Where2place
Refit-testB IoU@0.5
AO Maniskill
Google Robot
WidowX
Pretraining Data
Acc
1.2 Dist.
VM/VA
VM
No Additional Pretraining
69.0
66.1/63.5
54.9
+ General Grounding Data
30.7
74.9
72.6/70.3
65.2
+ Robotic Grounding Data
60.5
83.4
3.6
84.3/75.9
73.1
Result analysis. We assessed the pre-training Vision-Language Model (VLM) using the Grounding Yuan et al. (2024) for point prediction, RoboRefi et al. (203)) ) for bounding box prediction, and AO ManiSkill Xu et al. (2025b) for trajectory pre-
dataset for grounding performance, Where2Place YaaerlacceYYaneeea
diction. As shown in the upper section Table 6 the base model Qwen -
demonstrates the ability to detect bounding boxes for operation-related objects, but struggles to accurately point objects or predict trajectories. Despite these limitations, a Vanilla VLA built upon this model still achieves competitive performance in SimmerEnv compared To (e.g., 54.9 48.3). By pretraining the VLM with open-source multimodal grounding data (e.g.. RefCOCO), we observe improved object recognition capabilities, specifically increase in Box @.5 from 69.0 to 74.9. This enhancement leads to a significant performance gain on the WidowX benchmark (54.9 65.2). demonstrating that visual grounding pretraining effectively improves downstream manipulation accuracy.
Furthermore, incorporating robotic grounding data such as ST4VLA equips the VLM with the ability to interpret point, box, and trajectory keypoints for object interaction. These advancements contribute the of-te-of-the-nr performance of ST4VLA on SimmerEnv, including 12.4% improvement on the WidowX benchmark.
C.2 THE IMPACT OF COTRAIN LOSs WEIGHT
In this section, we examine the influence of cotraining strategies during the post-training stage. We find that while cotraining significantly affects model performance, the ratio between robotic loss and multimodal loss plays crucial role. We ablate different loss mixing ratios introduced in post-training and summarize the results in T
Table Performance comparison under different pretraining data settings. "Input image resized to 224 to align with prior work Black et al. (2024): Kim et al. (204 2025).
Spatial grounding
Robotic Manipulation
Loss weight ratio (grounding vs. action)
Where2place Point-Ace
Refit-testB I.U@0.5
AO Maniskill MAE Dist.
Google Robot VM/VA
Widow
VM
1:1
50.3
80.2
3.5
52.4/42.4
47.2
1:5
48.3
80.0
4.0
63.8/52.5
58.3
1:10
42.3
80.4
5.5
80.7/76.0
71.7
1:15
38.5
75.8
80.7/70.2
71.8
1:20
31.3
74.1
6.0
78.3/65.2
68.3
Results analysis. The results indicate that cotraining ratios such as 1:1 or 1:5 can further enhance the VLM's robotic grounding capability, but lead to a considerable decline manipulation performance (e.g., 73.2 47.2). However, when the cotraining ratio is increased to 1:15 or I:20. the manipulation performance also declines. This indicates that the relationship between multimodal cotraining and manipulation is not simple trade-off. The optimal ratio is observed be 1:10. We hypothesize that this ratio corresponds approximately to the proportion between the action chunk length and the average next-token prediction length in the multimodal data.
21

---

Published as a conference paper at ICLR 2026
C.3 BACKBONE-AGNOSTIE GENERALIZATION AND TRAINING METHOD CONTRIBUTION
To assess whether the effectiveness of our proposed training framework depends on the capacity of the underlying VLM backbone, we conducted two complementary evaluations. First, we rebuilt ST4VLA using Florence-2 Xiao et al. (2024). considerably weaker VLM compared to Qwen2.5-V2. or the GROOT backbone, and compared it against GROOT N1.5 and a Vanilla Co-training baseline. Second, to isolate the contribution of our spatial grounding training stage from backbone capacity, we performed controlled ablations where all models share the identical Qwen2.5-VL-33 backbone.
Xixao
Table 8: Comparison across different VLM backbones (Florence-2 vs. Qwen2.5-VL-3B) and training methods.
Put Spoon on Towel
Put Carrot on Plate
Stack Green Yellow
Put Eggplant Basket
Backhone
Model
Average
Eagle-2.5
GROOT NI.5
75.3
54.3
57.0
61.3
61.9
Vanilla Co-training VLA
75.2
31.3
3.1
75.0
46.1
Florence-2
ST4VLA
79.6
70.5
28.3
93.0
67.9
Vanilla VLA
56.6
63.3
27.0
71.8
54.7
Qwen2.5-Ve VL-33B
Vanilla Co-training VLA
70.3
68.4
20.5
85,2
61.1
ST4VLA
80.2
79.2
35.4
98.0
73.2
Result analysis. Across both settings, shown in Table 8 the evidence consistently shows that the benefits of ST4VLA do not stem from backbone capacity. With much weaker Florence-2 backbone, ST4VLA still surpasses GROOT N1.5 (67.5 .9% vs. 61.9%), while the Vanilla Co-training baseline collapses on difficult tasks (e.g.. 3.1% 1% for Block Stacking). Under controlled conditions with identical Qwen2.5-VL-3B backbones, clear improvement trajectory is observed: Vanilla VLA achieves 54.7, Vanilla Co-training reaches 61.1, and ST4VLA further improves to 73.2. This progression confirms that the performance gains arise from our spatial grounding training stage rather than from the backbone capacity.
C.4 SCALING LAWS OF SPATIAL PRIORS
Finally, we investigated the scaling behavior of spatial priors by varying the Spatial Grounding Pre-training data volume from OM to 3M pairs, followed by standard Post-training on OXE.
Table 9: Ablation on the scaling of Spatial Grounding Pre-training data volume.
Pre-training Scale
Google Robot VM
Google Robot VA
WidowX VM
Average
66.1
63.5
54.7
61.4
0.5 M
66.1
61.2
55.6
61.0
1.0 M
68.9
65.5
55,8
63.4
2.0 M
72.8
72.9
67.3
71.0
3.0 M
84.6
75.9
73.2
77.9
Result analysis. Our experimental results, shown in Table 9 reveal nonlinear relationship between spatial data scale and model performance. Performance gains remain modest when spatial grounding data is below 1.0M pairs. However, once the data scale surpasses 2.0M pairs, observe dramatically increasing returns. At 3.0M pairs, the model achieves substantial improvement, with average performance rising from 61.4 to 77.9- remarkable 26.9% relative gain. This suggests that critical mass of spatial grounding data is required to unlock the VLM's full manipulation potential.
C.5 ABLATION STUDY ON SPATIAL PROMPT FORMULATIONS
In our default implementation, ST4VLA employs single unified spatial prompt across all tasks: "Figure out how to execute then locate the key object needed." This design encourages the model to
22

---

Published as a conference paper at ICLR 2026
attend to spatial features without strictly enforcing specific output format (e.g.. bounding boxes points) during the action prediction phase.
To investigate whether the specific phrasing the imposition of explicit spatial constraints influences manipulation performance, we conducted an ablation study comparing our unified prompt against the following four variants:
Unified Prompting (Default): "Figure out how to execute it, then locate the key object needed." Random Padding: Uses non-semantic sequence to test if gains are due to sequence length alone:
xxx, XXx, XXX, xxX, xXX, xxx"
Box Prompting: Appends specific constraint demanding bounding box coordinates: " "Figure out how to execute it, then locate the key object needed. Give the box coordinates according to the instruction".
Point Prompting: Appends a specific constraint demanding list of tuples for points: "Figure out how execute it, then locate the key object needed. Your answer should be formatted as list of tuples".
Trace Prompting: Appends a constraint demanding a trajectory prediction: "Figure out how to execute it, then locute the key object needed. Based the lask description predict the trajectory that the end effector should take".
Table 10:
Ablation analysis of different spatial prompt formulations SimplerEnv, comparing the default Unified Prompt against non-semantic and explicit formatting constraints.
Prompt Type
Google Robot VM
Google Robot VA
WidowX VM
Average
Random Padding
64.2
60.8
50.6
58.5
Unified Prompting (Ours)
84.6
75.9
73.2
77.9
Box Prompting
80.9
73.0
75.8
76.6
Point Prompting
80.8
70.7
73.3
74.9
Trace Prompting
79.6
70.9
71.2
73.9
Result analysis. The results in Table 10 provide two key insights:
1. Semantic content matters: The Random Padding baseline significantly underperforms the Unified Prompting (58.5% 77.9%). This confirms that the performance gains in ST4VLA stem from the model explicitly attending to spatial semantics, rather than merely from the computational overhead of processing extra tokens.
2. Unified prompting is sufficient: Our default Unified Prompting achieves the highest average success rate (77.9% ),) outperforming variants that enforce strict output constraints like Box (76.6%) Point (74.9%), or Trace (73.9%). This suggests that while spatial awareness is critical, rigidly forcing the VLM to format its internal reasoning into specific coordinates during action inference is unnecessary and may even slightly constrain the policy's flexibility.
Therefore, our unified prompt serves as an optimal, task-agnostic instruction that robustly activates spatial attention across diverse manipulation tasks.
D EXPERIMENTS SETUP
D.1 EVALUATION IN SIMPLERENV
Experiment Setup. As described in Section 2.2 we post-train ST4VLA on subset of Open- X Embodiment (OXE) (including fractal_rt.. and bridge_vl), with co-training on spatial grounding data (Figure []. The VLM takes the primary observation image, task instruction, and an auxiliary spatial prompt input, while the action expert predicts actions with action chunk size of 16. For multimodal data, the model follows an SFT-style questionnansweriing format. Training is performed on 16 NVIDIA A100 GPUs for 50k steps (~2.5 epochs), with batch sizes of 16 for robot action data and 4 for multimodal data, optimized with a summed loss over both data types. All evaluations conducted within SimplerEnv using its official evaluation protocol.
23

---

Published as a conference paper at ICLR 2026
D.2 EVALUATION IN SIMULATED LARGE-SCALE PICK-AND-
Evaluation settings. As illustrated in Figure 7 our evaluation consists of four distinct tracks: (1) In-distribution, which evaluates the model's pick-and-place capability identical object instances under varied layouts corresponding to post- post-training scenarios; (2) Unseen object, where in each of the 200 scenes the graspable target object replaced with one not encountered during training. thereby testing the model's generalization to novel object instances; (3) New background, in which the table and background textures of the in-distribution scenes are altered assess visual robustness; (4) Unseen instruction, where the original template instruction "Move objl to the top of containerl" reformulated using GPT-4o-mini, using prompt as shown in the box below, introducing variations in object attributes and grammatical structures to evaluate the model's capacity generalize novel linguistic commands. In these tasks, the graspable target object is randomly placed within a 20 x 35 region in front of the robot base, while the container is randomly positioned within 40 x 70 cm area. Nine additional background objects are scattered across the tabletop at random. A data generation pipeline constructs each testing layout, ensuring that every configuration remains solvable for successful grasping and placement. Each track includes 200 distinct scenes, with maximum of 600 steps permitted per trial. trial deemed successful if the object placed atop the designated container within the step limit.
Experiment setup. The observation space consists of two RGB images: one captured from fixed third-person viewpoint and the other from first-person camera mounted on the Franka end-effector. Both images are resized to 224 x 224 before being input to the model. For comparison, the baseline methods additionally incorporate a 7-dimensional representation of the Franka joint states. The VIAA model outputs an 8-dimensional continuous action vector, where 7-dimensions correspond to the incremental deltas of each joint and one dimension encodes the binary signal for gripper control. Each action vector has a temporal chunk size of 16 and, after temporal ensembling, is applied to the Franka robot in the simulation environment.
In-distribution
Unseen Objecl
New Background
Unseen Instruction
Moke the yellow bott o to the top of the boord.
Move the flower the lop of the bowl.
Move the bottle to the lop of the first oid kit.
Trainter the item with the Idd on the borrel.
Figure 7: Evaluation settings for generalizable pick-and-place in large-scale simulation.
D.3 REAL-WORLD PICK-AND-PLAC PLACE MANIPULATION SETUP
Evaluation settings. To evaluate generalization, we divide all available object and container assets into disjoint seen and unseen sets, as illustrated in Figure g The training phase uses only the seen set, while testing incorporates both sets to assess the model's ability to handle novel objects. We examine real-world pick-and-place generalization across several conditions: in-distribution, unseen object, unseen object position, unseen object orientation, and unseen instruction. Among these, the unseen instruction and unseen object settings (depicted in Figure 9 introduce complementary reasoning challenges. The unseen instruction setting involves two key reasoning types, namely spatial reasoning and attribute identification, whereas the unseen object setting encompasses three categories, including new object instances. similar distractors, and new backgrounds. (1) Spatial reasoning. where the robot must act based on relative spatial relationships (c.g.. "Place the object closest to the robot base into the brown box"); (2) Attribute identification, which requires grounding instructions in specific visual attributes such as color or shape (e.g., "Move the green fruit into the white fruit plate"); (3) New object instances, where novel objects not encountered during training must be manipulated (e.g., "Put the small chips into the brown fruit plate"); (4) Similar distractors, which probe the model's ability to disambiguate between nearly identical objects (e.g., distinguishing a blue Orco from other cookies); (5) New backgrounds, where the robot must adapt to altered visual contexts while grounding instructions (e.g.. "Move the pear onto the pink fruit plate"). Together, these variations establish comprehensive and challenging benchmark that evaluates instruction
24

---

Published as a conference paper at ICLR 2026
following across perception, reasoning, and generalization. In contrast, the unseen object position and unseen object orientation settings involve seen objects whose grasping or placing positions, as well orientations, shifted during testing, such that they differ from the regions rotational angles encountered during training.
23 Seen Objects
27 Unseen Objects
Seen Containers
Unseen Containers
Figure Overview objects and containers used in enstruction-oollviing pick-and-place.
For each model, we conduct a total of 300 rollout evaluations. Each trial may correspond to one or more testing settings, and we ensure that each setting evaluated at least 50 times. Each trial allows up to three consecutive attempts. For each trial, three containers are chosen and placed at fixed tabletop locations within a 60 x 90 em workspace, and a larger collection of objects is randomly scattered between them. This configuration ensures that the robot must rely on precise perception and instruction grounding. rather than memorized placements, to correctly execute the instructed pick-and-plase actions. We report the success rate (SR), defined the fraction of trials in which the specified object is successfully placed into the designated container. A higher SR indicates better performance. To ensure fair comparisons across models, fix the positions of the objects and containers for each task during testing.
Experimental setup. We collected six hours of teleoperated demonstration data with seen objects and containers to used as model inputs. For comparison, the baseline methods additionally incorporate a 6-dimensional representation the end-effector's 'ss position and orientation. The VLA model outputs a 7-dimensional continuous action vector: six dimensions correspond to the incremental deltas of the end-effector's position and orientation, and one dimension encodes the binary signal for gripper control. Each action vector organized into temporal chunk of size 16, which, after temporal ensembling, is applied during execution.
post-training real-world data. The two RGB views were resized to 224 x 224 and
Prompt Description: Attibbate based Instruction Rewriting
to opeimize existing instruction for robor model cahance untrlbute-ppciife grounding.
Input:
Pick objj description: Ohjecr description obrained from sirulation (e.g.. apple).
Contuimer description: Container description obusine freen simulation.
Original instmetion (ag. Mose oby? the top of comainer)).
Rewrite guideline: Focus materials, color, shape.
Rules:
1. There are many items on the desktop. Ensure the rewritten instruction specific enough to unumbigueusly idemtify the targel object and container.
2. If the attribunes mentioned in the original instrction teach shape, color, or material) are not sutticient to uniquety ideatify the object, add extra festares additional
(e.g. relative pasition,
properties)
ambignity.
object's common name g.p. say "spple" "cup").
4. The rewrittes instruction must sound natural and fluest while preserving the odiginal menting.
Output:
Provide examples of optimioed instructions in ISON format (a list of strings).
Keep examples simple, clear, eary understamd.
Example:
inpus: Place the apple to the top of place.
puli [
*Put the spaere top the
plate.",
*Soack the ane red object onto the large white plate.",
*Swl the aed fruit on the eiscular place.
*Poiitoo the eniny spheze
white ceranic plate,", plate."
object
25

---

Published as a conference paper at ICLR 2026
Spatial reasoning: Put the object closest to the robot bose into the brown box
Attribute ideniifiction: Move the green Irai heeee while full piate.
Unseon object
New New.objeclInstancee the smal chipsinto the brown fruit plate.
original puple plate.
background: pear the pink plate.
Figure 9: Evaluation settings showcases for real-world generalization pick-and-place
D.4 REAL-WORLD LONG-HORIZON MANIPULATION SETUP
Evaluation settings. We evaluate model performance under three distinct settings: in-distribution. physical interference and task replanning:
Physical interference. External disturbances are introduced during task execution. For example, during the sorting items into drawers task, the drawer is manually closed after the robot opens or the target object is displaced during grasping. This evaluates the model's ability to perceive environmental changes and adapt accordingly.
Task replanning. New instructions are issued mid-execution. For instance, after placing object in the drawer but before closing it, the robot is told: "Also put the cow toy into the top drawer." This tests the model's 's ability to incorporate new subgoals and dynamically adjust its plan.
The tasks illustrated in Figure 10 include:
Desktop sorting. The Franka robot is tasked with sorting objects into containers based on high- level semantic categories, aiming to ensure that all items on the desktop are eventually placed into the correct containers. Both objects and containers are scattered within 60x90 em region in front of the robot base. The setup includes five seen containers and five object categories: fruits, toys, vegetables, bottles, and snacks. Each evaluation instance requires sorting objects from one to three categories into their designated containers, with each trial comprising three sequential lik-and-place actions. For every method, evaluations are conducted more than 30 times across the three settings, ensuring that each individual setting is tested at least 10 times. A success is recorded upon the completion of each individual pick-and and and--lace operation, and we report the final overall success rate accordingly.
26

---

Published as a conference paper at ICLR 2026
Sorting items into drawers. The Franka robot is required to (i) open designated drawer (either lower or upper), (ii) place the target objects into it, and (iii) close the drawer. This task demands precise temporal reasoning and articulated manipulation. The objects placed within 35x35 em area located to the front-right of the robot base. As in the previous setting, the number of trials remains the same; however, here report stepwise execution success, where step is deemed valid only if all preceding steps have been successfully completed.
Making sandwiches. The Franka robot is instructed assemble sandwiches following a pre- defined meal recipe. Ingredients and plates are placed within 50x70 em region in front of the robot base. We define five types of sandwich recipes the seen set: beread-lettuce-- bread ]. meet--eee---ttte--eeet---------e
J. mread-meat-meet--vee bread J, and breead We report success rates on both the seen set and an unseen set involving real-time environment interaction, using the same success definition as in the drawer sorting task. Math calculation. The Franka robot is prompted to solve a math problem and press the color- coded button (red, yellow, or blue) that corresponds to the correct answer based arithmetic reasoning. The buttons are randomly placed within a 40x40 cm area in front of the robot base. Goods purchase. The ARX LIFT2 dual-arm robot is tasked with identifying and placing into basket the object bearing the correct price tag, given a numerical cue ranging from 1 to 9. We report the success rate of correctly placing the item corresponding the queried price into the basket.
(bread-lettuee-mmtt-rveee bread
Experimental setup. We collected 22 hours of teleoperated demonstrations (400-500 per task) for Long-horizon training, segmenting trajectories into subtasks with atomic actions. We introduce zero-action vectors padding after each subtask segment. This allows the model stop upon subtask completion and then be prompted to predict the transition to the next subtask. Unlike prior VI.A models relying on external planners, ST4VLA jointly trains on multimodal inputs, task decomposition, subtask identification, numerical reasoning, and action supervision, for unified planning and action prediction.
D.5 REAL-WORLD ROBOT SETUP
bus pa
Ol shren))
Forka Am
Reatonas De35 (Wsr
lobetie
Figure 11: The real-world Franka setup.
Figure 12: The Real-world ARX LIFT2 setup.
As illustrated in Figure we utilize Franka Research 3 robot equipped with a Robotiq-2F- 85 gripper to evaluate real-world tasks, including short-range pick-and-place, long-horizon object sorting, opening and closing drawer, and making sandwiches. In our experimental setup, two RealSense D435 cameras capture RGB images for visual input: one is positioned at a rear-side, third-person perspective, and the other is mounted on the Franka's 's end-effector. Furthermore, as shown in Figure 12 we conduct pick-and- ald-e experiments in shopping scenarios using the dual-arm ARX LIFT2 platform. Each arm is equipped with RealSense D405, while RealSense D455 is mounted on the head to capture RGB imagery from a frontal viewpoint. All model inferences are executed workstation powered by an NVIDIA RTX 4080 GPU with 16 GB of VRAM.
27

---

Published as a conference paper at ICLR 2026
prediction execulion: sandwich stees)
Put piece of bun on the plate.
Put piece of leffuce on plate.
Put piece of bun on the plate
Stop.
Multicallegry snacks.
Start.
Put all the snacks into the white bosket.
the loys into brown basket
Put the toys the brown bosket.
Put all the fruits inlo the wooden basket
Stop.
Task Beplonning: Sort the brush into closed drower. Ihen sort gnother hiopo tov on sudden human requed steps)
Human: Wall, also clecchee he hippo toy into the drower
Collect the brush fo, the upper
C Claar the hippo toy to the
Open the upper drower.
Clese the upper drawer.
upper
Mathematical Reasoning: Do arithmetic exercise Purchase goods (2 steps)
deject 3001
9x7-1-72
Press the blue botton.
$4.
Fut the object the Dasket
Figure 10: Showcases and results for long-horize and reasoning manipulation.
CASE STUDY
To complement the quantitative results presented previous sections, we provide qualitative case studies across simulation and real-world benchmarks illustrate the versatility, robustness, and reasoning capabilities of ST4VLA in diverse manipulation scenarios. For video visualizations, please refer the videos provided in the appendix and our ourour website
E.1 CASE STUDY FOR PUBLIC BENCHMARKS
E.I.1
CASE STUDY FOR SIMPLERENV
Figure 13 presents representative examples from the SimplerEny benchmark, illustrating ST4VLA's performance across its canonical manipulation tasks. Each sequence shows the progression from initial scene perception and language instruction interpretation to successful task execution. These qualitative examples underscore the model's ability to ground natural language commands into actionable behaviors in visually varied environments.
28

---

Published as a conference paper at ICLR 2026
Coke Can
Move Near
Open/Closs Drawer
Open Top Drawer and Place Apple
Put Spoon Towel
Carrol on Plate
Stack Green Block on Yellow Block
Put Eggplant in Yellow Basket
Figure 13: Eyaluation showcases for SimplerEnv.
29

---

Published as a conference paper at ICLR 2026
E.1.2 CASE STUDY FOR LIBERO
Put the black bowl in the bottom drawer of the cabinet and close it.
Open the middle drawer of the cabinet.
Push the plate to the front of the stove.
Pick up the black bowl between the plate and the ramekin and place it on the plate.
Figure 14: Evaluation showcases for LIBERO benchmark.
Figure 14 presents representative examples from the LIBERO benchmark. These cases collectively demonstrate ST4VLA's 's ability to interpret language instructions, adapt to spatial and semantic variations, and execute long-horizon plans with robust generalization.
30

---

Published as a conference paper at ICLR 2026
E.2 CASE STUDY FOR SIMULATED LARGE-SCALE MICK-AND-PLACE MANIPULATION
Figure 15: Showcases for simulated large-scale pick-and- aad---ddppcc placee manipulation.
Beyond small-scale benchmarks, Figure 15 illustrates task executions in our large-scale simulated pick-and-place benchmark, which involves over 200 tasks and thousands of objects. These examples show that ST4VLA effectively parses instructions and manipulates previously unseen objects, while maintaining strong spatial grounding and generalization capabilities in cluttered, visually complex scenes.
E.3
CASE STUDY FOR REAL-WORLD PICK-AND-PLACE MANIPULATION
Figure 16: Showcases for real-world large-scale pick-and-place manipulation.
31

---

Published as a conference paper at ICLR 2026
Novel Container
Without co-training
With co- co-training
Novel Pose
Without co-training
With co-training
Figure 17: Showcases for real-world large scale pick-and-place manipulation w/wo co-training.
We further evaluate our framework in real-world cluttered tabletop environments. As shown in Figure [6] ST4VLA accurately interprets natural language instructions and completes single-step pick- and-place tasks involving numerous objects and containers under in-distribution, new background, unseen object, and unseen instruction conditions. Figure 17 further compares performance with and without co-training on VLM data, and shows that co-training substantially enhances robustness and generalization to novel objects and spatial configurations encountered during deployment.
32

---

Published as a conference paper at ICLR 2026
E.4 CASE STUDY FOR REAL-WORLD LONG-HORIZON MANIPULATION
We also evaluate ST4VLA a range of long-horizon, reasoning-intensive tasks in real-world environments. Figures 18 to 22 collectively illustrate the model's reasoning and decision-making processes across diverse scenarios, including multi-step manipulation, numerical reasoning, object sorting, goods purchasing, and spatially constrained organization. These results demonstrate that ST4VIAA effectively integrates perception, reasoning, and action planning, enabling robust execution of complex long-horizon tasks in dynamic real-world settings.
OC' AUTOE
200 cumeero
the
Sook ael the
For
Figure 18: Showcases for making a sandwich.
Figure 19: Showcases for math calculation.
33

---

Published as a conference paper at ICLR 2026
Figure 20: Showcases for sorting objects.
6
Figure 21: Showcases for purchasing goods.
34

---

Published as a conference paper at ICLR 2026
Figure 22: Showcases for sorting to drawer.
35

---

Published as a conference paper at ICLR 2026
E.5
FAILURE CASE STUDY
the decoralive
X
Locate the largel object
Grosp torget object
locale target container
Move the water boltfe to the the microwave oven.
X
Locale the largal object
Grasp the torgel object
locale the targel container
T!
Move the headphones to the top the trough.
X
Grosp by the right grosp pose
Transeer the container
locate the target container
Move the bowl the top of the book.
Grosp by the right grasp pose
Transfer the conteiner
locale the target container
Figure 23: Failure case study.
To better understand the limitations of ST4VLA, we analyze representative failure cases during real-world follooiion-oolloving pick-and-place tasks. As shown in Figure 23 In cases, the robot executes an incorrect grasp or misidentifies the target container, leading to task failure. These errors highlight challenges in robust perception and action grounding under cluttered environments. While some failures may stem from sensor limitations, integrating additional modalities, such as depth sensing and proprioceptive feedback, could improve performance. We leave this future work to further enhance the reliability of instruction-condiiinn manipulation in complex settings.
F DATA
This section introduces the datasets used in ST4VLA, covering pre-training, mid-training, and post- training stages. For VLM pre-training, we construct large-scale spatial grounding datasets with point, box, and trajectory annotations to enhance spatial perception and lision-language alignment. Mid-training employs synthetic manipulation data bridge pre-training knowledge and robotic execution. Post-training uses both simulated and real-world instruction-follwiin data, including large-scale tabletop tasks and real-robot demonstrations for long-horizon manipulation.
F.1 SPATIAL GROUNDING DATA FOR PRE-TRAIE TRE-NIGIN
The multimodal training dataset for our model comprises over 3M data, categorized into four distinct types: General Question Answering (General QA), Bounding Box Question Answering (Box QA), Trajectory Question Answering (Trajectory QA), and Point Question Answering (Point QA), as shown in Figure 24 and detailed in Table II Notably, more than 2.3M of these data dedicated to spatial reasoning datasets. These categories ensure robust multimodal understanding while supporting adaptation to embodied tasks in tabletop robotic scenarios. Below, describe each category:
36

---

Published as a conference paper at ICLR 2026
General QA. Sourced from LLaVA-OneVisinn Li et al. (2024)) and Intern VL3 Chen al. (2024);
Zhu et al. (025), this category is sampled to cover diverse multimodal tasks, including image captioning, visual question answering (VQA), optical character recognition (OCR), knowledge grounding, and creative writing.
Bounding Box QA. We curate diverse collection of multimodal grounding datasets, including RefCOCO Yu et al. (201 6);; Mao et al. (2016), ASv2 Wang et al. (2024), and COCO-ReM [Singh et al. 2024). sourced from InternVL3 Chen al. (2024); Zhu et (2025). Additionally, incorporate the ST4VLA Manipulation dataset, generated via scalable synthetic data generation as
Appendix Section
M3 and the RoboReflt dataset et al. (2023). a specialized dataset for roboties
grounding.
This category integrates the ManiSkill subset Xu et al. (2025a), the trajectory point subset from the ST4VLA Manipulation Dataset, and the MolmoAct dataset Lee et al. (2025) to enable precise end-effector trajectory prediction. The AO ManiSkill subset provides high- quality, object-centric trajectory data, where small objects move in coordination with the robotic arm's gripper. These trajectories can be approximated as end-effector movements for tabletop manipulation tasks.
Trajectory QA.
Point QA. For precise point localization, we integrate multiple datasets, including the Pixmo- Points dataset Deitke al. (2024), the RoboPoint dataset Yuan et al. (2024), the RefSpatial dataset Zhou et al. (2025a), and a point subset extracted from the ST4VLA Manipulation Dataset, each subjected to tailored preprocessing. Specifically. the Pixmo-Poiie Points dataset is filtered to exclude images with resolutions exceeding 1024 pixels and restricted to maximum of 10 points per image. Additionally, we prioritize the extraction of object reference and region reference data from the RoboPoint and RefSpatial datasets to enhance grounding accuracy.
All point coordinates are converted to absolute coordinates to align with the Qwen2.5- VL, et al.) (2025a) SmartResize prediction framework Bai et al. (2025b). Predicted coordinates are formatted in JSON and XML to support robust learning and adaptive processing of spatial instructions for diverse robotic tasks.
General VQA Dala
Spotial Grounideg Data
General VQA
Provide graph caplion of the giveni Image.
you
light
A: "rerrpooss" "uno. hove found the light brown lon now!: "sustenk") up light tyown
mage
Bunnnin de
Pre- Pre-training Data 3,032K
[|67.
Roncboxe [[30 350j)</bou>>
HoHww lypees onmobr are represorted thes picturef
Pul the lotlcee the plate.
A: A:/hesporse" "Suro I'm placing the leth on the plate now: "subtos
sp
lath:cseborr]j55s, 381]j</box>>
green
VOA
BOX QA
Polet GA
Injjectoy
Figure 24: Overview of the pre-training data for the dision-languee model. The data comprises two main parts: general VQA data to maintain the model's general multimodal capabilities, and spatial VQA data focusing on robotic-related grounding and spatial perception in a VQA format.
F.2 SYNTHETIC DATA FOR ACTION POST-PRE-TRAININS
To bridge the gap between VLM and VLA, we introduce a Post-Pre- Post-Pre-Trainin phase, where large- scale simulated data used to pre-train the VLA after VLM pre-training. This stage initializes the action head and facilitates the learning of action representations. maintaining diversity both at the instruction and object levels. We leverage GenManip as our data synthesis pipeline to construct large-scale pick-and-place dataset which comprises 244K closed-loop samples. Specifically, we adopt the same object set and positional distributions as in InternData-Ne Data Contributors (2025), and process them through scalable data pipeline. Each synthesized sample is rigorously validated to ensure correctness and consistency. To further enhance visual diversity, we introduce controlled randomization in lighting conditions and texture mappings.
Dost-Pre-Training requires
37

---

Published as a conference paper at ICLR 2026
Table 11: Pre-training multimodal datasets categorized by subtype (VQA, BBox-QA, Trajectory-QA, Point-QA), with scenario type and annotation method.
Subtype
Dataset
Samples
Scenario
Annotation
AOKVQA
33k
Web
manual
ShareGPT4V
182k
Web
automatic
Intern Proprictary Dataset
225k
Web
automatic, manual
VQA
COCOTextV2
16k
Web
manual
VQAv2
82k
Web
manual
TallyQA TallyQA.COCO COCO
99k
Web
manual
InternData-M (Bbox)
431k
synthetic,real
automatic,
RoboRefit
36k
real
manual
BBox-QA
ASv2
128k
Web
manual
COCO-ReM
117k
Web
manual
RefCOCO
167k
Web
manual
Inter (Traj)
78K
real
manual
Trajectory-oe QA
AO (ManiSkill)
35k
synthetic
automatic
MolmoAct (Traj)
571k
synthetic
automatic
InternData-M1 (Point)
114k
real
manual
Re(Spatial
200k
synthetic
automatic
Point-QA
RoboPoint
422k
synthetic
automatic
PixMo-Points
96k
synthetic
manual
Total
3.032 M
automatic, manual
Simulation Assets
Layout generation
Physics Simulation
Meto Data
Generation
sayoul
Grasp Generation
Plnniing Data
Motion Planning
MPth
Object
Sim
Actiss
Gepper Action
Large-scale Simulation generalizable pick-place: <Obj1> to <Relation> <Obj2>
Visual Rendering for VLM and VLÁ data synihesis
Synthetic Spatial Grounding QA
Synthelic Demonstralion
Isaac Sim
plicee
noa
Dephh
Koac and Spen
dropp Rerd
244 Esiedds
Simulation data synthesis pipeline. The pipeline generates diverse robotic manipulation data from large asset library, converts intermediate representations into VQA data, and separates physics from rendering to reduce wasted failures and improve efficiency.
Figure 25:
F.3 SCALABLE SYNTHETIC DATA ENGINE FOR COORUUCTION-FOLLOWIIII
To support large-scalc end end-to-end data generation for VLM pre-training, we build a highly scalable, flexible, and fully automated simulation pipeline top of GenManip Gao al. (025 and Isaac Sim Makoviychyyhhk et al. (2021).
Automatic task synthesis for generalizable pick-and-place. We develop scalable simulation pipeline (shown in Figure 25) that generates diverse manipulation trajectories from randomized object layouts and lighting conditions. By leveraging privileged simulation signals, including object poses,
38

---

Published as a conference paper at ICLR 2026
object meshes, and robot arm state, the system rapidly generates scene layouts via seene graph solver and computes candidate grasps based on object meshes Liang et al. 2019). Each candidate trajectory is then executed once in physics for closed-loop verification, after which scene- bene-graph validator checks whether the task goals are achieved. Only trajectories that both execute successfully and pass validation are accepted, ensuring that all collected data are physically feasible and task-complete.
Synthesis of VLM data and VLA data for Spatial Grounding, For higher efficiency, robot planning and rendering are fully decoupled in our framework. The planner records structured scene and trajectory data, including joint states, object positions, and action information, which are later replayed by the renderer under randomized lighting, materials, and viewpoints. To align the simulation with the real world, we calibrate all cameras using ArUco markers, ensuring that their intrinsic and extrinsic parameters match those of real-world cameras, thus maintaining consistent viewpoint geometry. In addition to high-resolution images, the renderer produces rich intermediate outputs, such as object bounding boxes and 2D end-effector trajectories. These signals provide dense supervision for action learning and facilitate the creation of auxiliary datasets for tasks such as spatial grounding. affordance reasoning, and trajectory prediction. Our asset library includes 14K annotated objects, 211 tables, 1.6K textures, and 87 dome lights, offering data with high visual and physical diversity critical for developing generalizable models.
G POST-PROCESSING OF TELEOPERATED DATA
G.1 REAL TELEOPERATED DATA PROCESSED FOR EVALUATING LONG-HORIZON AND INTERACTIVE TASKS
Data annotation. To gather diverse tabletop task data, placed single-arm Franka robot lightweight mobile platform akin to DROID Khazatsky et al. (202)) enabling the robot to be easily transported and data to be captured across various scenes. We collected both short-horizon and long-horizon task data, with short-horizon tasks primarily involving pick-and-place operations, and long-horizon tasks including sandwich preparation, item sorting, and placing objects into drawers. For the long-horizon tasks, manual annotations were required to mark transitions between subtasks, and we preserved the action sequence annotations throughout the data collection process. After data collection, we segmented the data by marking specific time points. For example, the long-horizon task "make classic sandwich" was decomposed into subtasks such as "place the bun on the plate," " "put the meat in the sink," and "put the bun back the plate." This decomposition allows the policy to first learn individual skill components in isolation before combining them into more complex behaviors.
Construction of long-horizon tasks. To enhance system efficiency and reduce the computational overhead caused by meaningless reasoning in the Visual Language Model (VLM), we adopt an agent-based approach to organize the VLM and VLA modules. VLM is triggered only when a human command is received or when the robotic arm stops moving. While VLA handles short-horizon tasks, VLM performs high-level semantic planning, enabling the system to support more complex long-horizon tasks. The data generation pipeline follows the structure outlined below.
Keyframe extraction and data augmentation. We begin by extracting keyframes from the collected data, including the first frame, last frame, and manually annotated task-switching frames. Given the limited amount of data, we extend the keyframes both forward and backward to maximize data utilization. For the forward extension, we identify moments of gripper changes near the keyframe and extract a portion (M%) of data from the segment between the gripper change moment and the keyframe, enhancing data diversity. Similarly, for the backward extension, we extract the next segment (N%) of data after the keyframe. For data balance, is set to 60% and N to 40%, considering the data bias introduced by manual breakpoints.
Scene layout extraction and action adjustment. Once the data is augmented, extract scene layout information, categorizing actions into three types: actions that have already occurred, actions that are yet to occur, and potential actions. For actions that are yet to happen. we adjust them by adding or removing actions. New actions are extracted from a potential action library, resulting in a new task list.
39

---

Published as a conference paper at ICLR 2026
Synthetic user instruction generation and response handling. By combining this updated task list with the original instructions, we generate synthetic user instructions using GPT-4o-minn or simple to-do list concatenation method. These instructions are further rewritten using GPT-4o-mini to handle vague or incorrect instructions and generate appropriate rohot responses.
Reasoning data generation and scene analysis. Based the actions that have occurred, actions that will occur, and the basic scene information, we generate reasoning data through templates or GPT-4o-mini. This reasoning data includes descriptions of the scene layout and action instruction analysis, which contributes to the large model's predictive content.
Consolidation and formatting for model prediction. Finally, we consolidate human instructions, robot responses, reasoning content, and next actions, and format them for the large model's ' predicted output. This ensures the system has a clear execution plan and can handle complex, long-horizon tasks.
40


Reviews:
[{"review_id": "ZUnkCgfptI", "reviewer": "Reviewer_MEHz", "summary": "This paper presents SP-VLA, a dual-system VLA framework that integrates action learning with spatial priors. The framework consists of two components: Sys2, a VLM (Qwen2.5-vl-3B), and Sys1, a DiT-based diffusion model. A Q-former module links the two systems through cross-attention, compressing Sys2’s tokens into a fixed-length representation that is then injected into Sys1.\n\nTraining occurs in two stages: spatial grounding, where the VLM acquires spatial priors (points, boxes, and trajectories) from a large dataset, and spatially guided action post-training. Experimental results in both simulated and real-world environments demonstrate that SP-VLA outperforms existing baselines.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-organized and clearly written, making it easy to follow. The authors provide extensive implementation details to support reproducibility and include numerous illustrative examples that enhance understanding. The experiments are reasonable and solid.", "weaknesses": "The introduction of spatial priors to enhance VLA performance is conceptually intuitive and has been explored extensively in prior research (e.g., LLaRA [1], TraceVLA [2], Magma [3], among others). Similar strategies, such as dataset curation via object detection, have also been commonly adopted. Moreover, the dual-system design has been previously discussed in works like TinyVLA [4] and $\\pi_0$ [5]. Consequently, the paper’s contribution appears to offer limited novelty relative to existing literature.\n\nNonetheless, the work provides substantial implementation details and practical engineering insights, which could be valuable to the research community. Therefore, I recommend a weak acceptance.\n\nReference\n\n1. Li, X., Mata, C., Park, J., Kahatapitiya, K., Jang, Y. S., Shang, J., ... & Ryoo, M. S. (2024). Llara: Supercharging robot learning data for vision-language policy. arXiv preprint arXiv:2406.20095.\n\n2. Zheng, R., Liang, Y., Huang, S., Gao, J., Daumé III, H., Kolobov, A., ... & Yang, J. (2024). Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint arXiv:2412.10345.\n\n3. Yang, J., Tan, R., Wu, Q., Zheng, R., Peng, B., Liang, Y., ... & Gao, J. (2025). Magma: A foundation model for multimodal ai agents. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 14203-14214).\n\n4. Wen, J., Zhu, Y., Li, J., Zhu, M., Tang, Z., Wu, K., ... & Tang, J. (2025). Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation. IEEE Robotics and Automation Letters.\n\n5. Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., ... & Zhilinsky, U. (2024). $\\pi_0 $: A Vision-Language-Action Flow Model for General Robot Control. arXiv preprint arXiv:2410.24164.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mG9IJPMnSI", "reviewer": "Reviewer_uJaW", "summary": "The paper proposs SP-VLA, using spatial prior to bridge the language space and action space. The SP-VLA has two stages, the first train the VLM with image, language, and spatial priors like bbox, points or trajectories. The second stage use the latent generated in the first stage to train a DiT policy for control.", "questions": "See weakness part, and furthermore, small questions:\n1. for the real world experiment, section 3.4, it seems all task is pick and place. It is not clear clarified what is the experiment setting? Did the author use standard DROID? If so, why the pi-zero is so bad on unseen object, with my own experience, at least pi-zero should have better results than what record in Table 4.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The motivation of the paper is good, which using spatial prior to pretrain the vlm, then further tune the vlm for downstream tasks, which compare with mainstream vlm which focus on daily question or specific QA task, spatial information seems more important for control and robotic task.\n\n2. The paper deliver thourough experiments and ablations to claim the spatial priors both in pretraining and post-training will help the embodied control.", "weaknesses": "1. The idea of use spatial priors to pretrain or post train or cotrain a vlm is not novel and actually used by many recent papers like , so i do not think the proposed spatial prior will help both pretraining and post training is novel enough. \n\n2. In the experiment design, the author try to verify with spatial prior in pretraining or post training, the SP-VLA can perform better than other fundational VLAs, but it seems the compared method like pi and openvla, these paper does not claim they use specific priors, The author might need to compare with more related work as mentioned in 1.\n\n\n[1] Wen, Chuan, et al. \"Any-point trajectory modeling for policy learning.\" \n[2] Niu, D., Sharma, Y., Biamby, G., Quenum, J., Bai, Y., Shi, B., ... & Herzig, R. (2024). Llarva: Vision-action instruction tuning enhances robot learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wimWq5idbQ", "reviewer": "Reviewer_QqBv", "summary": "The authors introduce a novel vision-language-action model, SP-VLA, that uses a two-stage training approach to ground action learning in spatial reasoning: 1) a pre-training stage, in which large-scale internet vision-language corpora and robot-specific datasets are formatted into a unified Question-Answer template and used to train a VLM, and 2) a post-training stage, in which a prompt that queries the spatial relationships between objects is appended to the task instruction and the VLM is trained to predict actions. The experiments evaluate SP-VLA on SimplerEnv, Libero, a new simulated benchmark based on IsaacSim, and short-horizon and long-horizon pick and place tasks in a real-world set-up. SP-VLA performs favorably compared to many VLAs on SimplerEnv, and is mainly compared to $\\pi_{0}$ and GR00T N1.5 on the other benchmarks. The ablations show that SP-VLA relies on both training stages for its best performance.", "questions": "l. 147-148 What are the outputs of the model in symbolic form? What is the loss during each stage?\nl. 188 + l. 196 How do you get the spatial prompt? + Relatedly, can you explain the lightweight latent spatial prompting strategy?\nl. 339 Are all the models in Tables 2 and 3 trained on the same amount/set of data?\nl. 372 \"benchmarks\" <-- do you mean tasks?\nl. 376 \"post-pretraining\" Is this different from the post-training phase, i.e. stage 2, of SP-VLA?\nRegarding experiments in section 3: Did you finetune $$pi_{0}$$ and GR00T N1.5 on the tele-operation demonstrations before inference?\nTable 4: The abbreviations are confusing, can you clarify them?\nl. 1106 What prompt do you use to rephrase the instruction? Can you give an example of a rephrased prompt?\nSection E.3: l. 1129 \"unseen object position\" Does that mean that the seen object is in a position unseen during training, or that an unseen object is placed in an unseen position? \"unseen instruction\" Does that mean it was rephrased or is it a novel task? l. 1161 Are positions of objects and containers fixed across trials or across models?\nl. 1258 - 1260 How do the buttons correspond to the correct answer?\nl. 1267 \"be prompted\" Is there a prompt from the user to tell the robot to continue to the next subtask? Also, are the subtask instructions generated by the model?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- SP-VLA explicitly introduces a spatial grounding training phase and includes a loss over spatial metrics (box, point, trace).\n- Strong results on multiple simulated and real-world benchmarks show that SP-VLA sets a new SOTA on SimplerEnv, Libero and the constructed benchmarks.", "weaknesses": "- Unclear whether the SOTA performance is due to the proposed training method or due to the VLM backbone. The comparisons between SP-VLA, $$pi_{0}$$ and GR00T N1.5 in Section 3 are fair if they have seen the same data during pre-training, but if that's not the case then it's possible the improvement comes from the strength of the VLM backbone, not necessarily the proposed training method. \n- The details on the experiments and the losses are not very clear – see the questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NBCx6YWD5t", "reviewer": "Reviewer_BHsq", "summary": "This paper proposes SP-VLA, a Spatially Guided Vision-Language-Action model that introduces latent spatial priors to bridge perception and action. The model conducts Spatial Grounding Pre-training on large-scale real and synthetic data to acquire spatial understanding, and then applies Spatially Guided Action Post-training to maintain such spatial priors during policy learning. It achieves new state-of-the-art results on SimplerEnv, and further validates the approach with a large-scale custom simulation benchmark.", "questions": "- How exactly is Figure 3(a) evaluated? Is the grounding metric still measured through spatial-ground prompts during Stage 2?\n- Have you tried use co-training and your methods together.\n- Why does the Spatially Guided Action Training curve in Figure 3(a) continue to improve, even though Stage 2 seemingly lacks explicit grounding supervision? This behavior seems counter-intuitive and deserves clarification.\n- Do the authors plan to release the custom simulation benchmark used in large-scale experiments?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Achieves new SOTA performance on the SimplerEnv benchmark.\n- Provides clear analysis of Figure 3, effectively demonstrating how spatial priors benefit action generation.\n- Conducts extensive experiments on a large-scale, custom-built simulation to verify the generality of the approach.", "weaknesses": "- In Figure 3(b), Vanilla VLA’s curve may not have saturated. Given its long zero-phase and upward trend, it’s possible that Vanilla VLA could reach similar performance with longer training. Likewise, the Co-training setup doesn’t show a clear drop and might continue improving. I suggest extending training to 80–100 k steps or until convergence for all settings. It necessary to justify whether spatial priors mainly accelerate convergence or are essential for final performance.\n- Moreover, it would be insightful to extend experiments of Figure 3(b) to Google Robot (VM/VA) setting\n- Consider analyzing different scales of spatial prior pretraining data, not only training steps. More interpretability analysis on how spatial priors influence the latent representation would further strengthen the contribution.\n- Related work currently misses several key papers that employ chain-of-thought style or multi-step reasoning in robot learning. Please cite:\n1. Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation\n2. CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models\n3. Efficient Robotic Policy Learning via Latent Space Backward Planning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a1o1cXypt6", "reviewer": "Reviewer_qHTz", "summary": "This paper proposes SP-VLA, a spatially guided vision–language–action (VLA) model that introduces spatial grounding during both pretraining and post-training. By combining large-scale vision–language grounding data with robot-specific datasets, SP-VLA effectively aligns spatial and action objectives, leading to a robust and generalizable framework for robot learning that achieves significant improvements over vanilla VLA baselines on multiple benchmarks.\n\nOverall, SP-VLA presents a well-motivated and promising approach to integrating spatial priors into general-purpose action representation learning. The reported gains are substantial, and the concept of latent spatial prompting is particularly compelling. However, the paper would benefit from clearer methodological exposition, more detailed attribution of performance gains, and improved consistency in presentation. With these refinements, it could make a strong contribution to the field.", "questions": "1. Why not directly fine-tune a pretrained VLM that already exhibits strong spatial grounding capabilities, instead of performing an additional spatial grounding pretraining stage? What are the concrete advantages of the proposed approach?\n\n2. In Section 3.1, how are the spatial grounding and action losses weighted in the co-training baseline?\n\n3. In Section 3.1, what specific perception task is used on RefCOCO-g, and how is vanilla VLA evaluated there without a perception head?\n\n4. Could the authors clarify what kinds of spatial prompts are used (point, box, trajectory, etc.), and whether they vary across datasets or tasks?\n\n5. How much do the diffusion transformer and hierarchical planner contribute independently to overall performance? Could similar benefits arise from these architectural components rather than spatial grounding alone?\n\n6. Why do many popular baselines (Tables 2 and 3) underperform the “vanilla VLA” baseline used in this paper? What efforts are made to ensure that these baselines are evaluated under comparable settings?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. SP-VLA achieves notable improvements over vanilla VLA across multiple settings.\n2. The discussion around latent spatial prompting is particularly interesting. The authors recognize the difficulty of obtaining high-quality spatial annotations and instead design a lightweight strategy that implicitly guides visual representations. The qualitative motivation and empirical validation for this are compelling.\n3. Stronger transfer to unseen objects and paraphrased language instructions; good long horizon reasoning.", "weaknesses": "1. Models such as GR00T and π0 are not cited when first introduced. Similarly, based on context (line 140-145), DiT appears to refer to a Diffusion Transformer but is never expanded upon. Clarifying these early and in figures would improve accessibility for readers new to the area.\n\n\n2. The paper transitions quickly from the framework description to experiments without clearly specifying details like: (a) The training procedure and optimization details (loss functions, objective balancing). (b) Architectural modifications relative to existing VLAs, etc. These are crucial for reproducibility and clarity.\n\n3. There is some ambiguity in experimental setups. For example: In Experiment 3.1, how are training losses weighted between spatial grounding and action data for the vanilla co-training baseline? In the RefCOCO-g perception task, what specific perception task is used, and how is vanilla VLA (without a dedicated perception head) evaluated on it?\n\n4. SP-VLA includes multiple architectural components (spatial prompting, latent planner, diffusion-based actor, hierarchical reasoning). However, the paper attributes most gains to spatial grounding. It would strengthen the claims to discuss or experimentally isolate the contributions of each module. For example, how would performance change if the diffusion actor were replaced with a simpler transformer or MLP policy?\n\n5. The paper does not clearly describe what forms of spatial prompts are used or whether they are consistent across tasks. Are these limited to the ones shown in Figure 2? Appendix D alludes to an ablation while making some conclusions about spatial prompts, but corresponding experiments seem missing. Given that spatial prompting is the key contribution, these things deserve a more detailed exposition (potentially in the main text).\n\n6. Typographical and presentation issues: \n* “WindowX” appears to be a typo for WidowX.\n* Table 1 could improve readability by highlighting best results (e.g., bold/underline consistently to table 2).\n* While the main text has a good number of visual explanation and diagrams, I think the ones related to spatial prompts, the ‘lightweight’ prompting module, etc. that center around the main contributions of the paper are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)