Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation
Authors: Yifu Yuan
Haiqin Cui
Yaoting Huang
Yibin Chen
Fei Ni
Zibin Dong
Pengyi Li
YAN ZHENG
Hongyao Tang
Jianye HAO
Venue: ICLR 2026 Poster (2026)
Keywords: Embodied Reasoning
Vision-Language Model
Embodied AI
Reinforcement Learning
Zero-shot Generalization
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose EmbodiedR1, a VLM using reinforced fune-tuning, achieving 87.5% zero-shot success in robotic manipulation across diverse scenarios.

Abstract: Generalization in embodied AI is hindered by the "seeing-to-doing gap", stemming from data scarcity and embodiment heterogeneity. To address this, we pioneer "pointing" as a unified, embodiment-agnostic intermediate representation, defining four core embodied pointing abilities that bridge high-level vision-language comprehension with low-level action primitives. We introduce Embodied-R1, a 3B Vision-Language Model (VLM) specifically designed for embodied reasoning and pointing. We use a wide range of embodied and general visual reasoning datasets as sources to construct a large-scale dataset, Embodied-Points-200K, which supports key embodied pointing capabilities. Then we train Embodied-R1 using a two-stage Reinforced Fine-tuning (RFT) curriculum with specialized multi-task reward design. Embodied-R1 achieves state-of-the-art performance on 11 embodied spatial and pointing benchmarks. Critically, it demonstrates robust zero-shot generalization by achieving a 56.2% success rate in the SIMPLEREnv and 87.5% across 8 real-world XArm tasks without any task-specific fine-tuning, representing a 62% improvement over strong baselines. Furthermore, the model exhibits high robustness against diverse visual disturbances. Our work shows that a pointing-centric representation, combined with an RFT training paradigm, offers an effective and generalizable pathway to closing the perception-action gap in robotics.

Published as a conference paper at ICLR 2026
EMBODIED-RI: REINFORCED EMBODIED REASONING FOR GENERAL ROBOTIC MANIPULATION
Yifu Yuan 1.f Haiqin Cui', Yaoting Huang' Fei Ni', Zibin Dong', Pengyi Li'
Yan Zheng', Hongyao Tang', Jianye Hao
"College of Intelligence and Computing, Tianjin University
Project Leader. *Corresponding authors: Jianye Hao (jianye.hao aalaaaa@@tu.eeuuu... edn.edu.cn).
ABSTRACT
Generalization embodied Al is hindered by the "seeing-to-doing gap", stemming from data scarcity and embodiment heterogeneity. To address this, we pioneer "pointing" as a unified, embodiment-agnoste four core embodied pointing abilities that bridge high-level vision-language com- prehension with low-level action primitives. We introduce Embodied-RR 3B Vision-Language Model (VLM) specifically designed for embodied reasoning and pointing. We use wide range of embodied and general visual reasoning datasets as sources construct a lcage-scale dataset, Embodied Poieeedd-oints220000 which sup- ports key embodied pointing capabilities. Then we train Embodied-RI using two-stage Reinforced Fine-tuning (RFT) curriculum with specialized multi-task reward design. Embodied-RI achieves state-of-the-art performance on 11 embod- ied spatial and pointing benchmarks. Critically, it demonstrates robust zero-shot generalization by achieving a 56.2% success rate in the SIMPLERENV and 87.5% across real-world XArm tasks without any task-specific fine-tuning, representing 62% improvement over strong baselines. Furthermore, the model exhibits high robustness against diverse visual disturbances. Our work shows that pointing- centric representation, combined with an RFT training paradigm, offers an effective and generalizable pathway to closing the perception-action gap in robotics. More visualizations and datasets are available on website.
intermediate representation, defining
INTRODUCTION
Recent advancements in Vision-Language Models (VLMs) (Bai et al., 2025b; Team et al., 2025; Ni et al., 2025; 2024) have inspired new wave of Lisioo-Language-Attinnn (VLA) models (Kim et al., 2024) aimed at enhancing generalization in robotic manipulation. While these models exhibit strong visual perception and excel at imitating expert demonstrations, their manipulation performance degrades significantly in novel settings. This disparity is widely recognized the "seeing to-doing gap" (Yuan et al., 2025): a failure to reliably translate rich perceptual understanding into effective robotic actions. This gap is largely attributed to key challenges: embodied data prevents from sufficiently grounding language and vision with physical actions (Walke al., 2023; Lin al., 2024), and (b) heterogeneity, where diverse robot morphologies pose significant challenge to knowledge transfer.
(a) data scareity, where limited
To bridge this gap, we propose pointing as an intuitive and effective paradigm to connect high-level un- derstanding with generalizable action. A point-centric representation (Yuan et al., 2024b; Deitke et al., 2024) unifies semantic and spatial information into a compact, embodiment-agnosiin format. This approach is highly scalable, overcoming data scarcity by leveraging broad visual datasets datasets-including weal-worldkynthheiic robotic data, and internet data. Simultaneously, enables knowledge transfer across diverse robot platforms, resolving the heterogeneity challenge. This paradigm offers a promising path, avoiding the fundamental mismatch between physical actions and pre-trained data in end-to-end VLAs (Black et al., 2024) while mitigating the cascading errors of multi-model pipelines in modular methods (Liu et al., 2024a; Huang et al., 2024c). Despite a surge in related work, the pointing information from existing methods is often incomplete and too simplistic for complex geeision-making leading to limited generalization. They often offer only narrow forms of guidance, such as affordance point (Ji al., 2025), object visual trace (Xu et al., 2025), or target
mmbodiment-agnostte nature

---

Published as a conference paper at ICLR 2026
Embodied-K1 Moddl
Overall Performance
Move the mahe the right
<think-11 should first grasp handle and lift it from the hase. tabletop. positioned fürther back, closer wall. need to avaid obstacles and carefully move moka from current position: right side drower </think>
Image Encoder
taTdeel
U LUUU Bon-pro
CAPE
Embodied-R1
Accorey Rewwndd
Ronasl
Raward
Visal Trace Ronasl
Ensineriee
sthinl tihhin
Fanswer -pain- (855 1650,410000000 (0]]:440]]s]]iisee "irwwr
Beward
canswarr... stanwwar
Robet
Pars Afterd
SHPLIGGE
152
Kobofoon
GITA
Rebirnn
Pointing Evalus
Zero-shot Robotic Manipulation
Relerrig Expresice Groseding
Objjct Frmvtiinal Greanding
8 World Tasks
AMMPLKRE Tasks
Baples Groendine
Visal Tract Geseration
Diverse Visual Disturhanees
Background Gemeralization
Lighting Generalizatio
Height Generalization
New Embedhnnst
New Sianlatien
Hasnnndror Sketch
Figure 1: Overview of the Embodied-RI framework and its zero-shot manipulation performance. Embodiid-R1 performs explicit reasoning generate "pointing" commands, enabling robust execution across comprehensive suite of spatial reasoning benchmarks, pointing evaluations, and diverse real-world robotic tasks.
region (Yuan al., 2024b). Crucially, even advanced models like FSD (Yuan al., 2025), which anchor instructions through reasoning, are constrained by rigid CoT templates learned via Supervised Fine-Tuning (SFT). This rigid approach fundamentally limits their ability to generalize to novel tasks.
Therefore, we first systematize embodied pointing into four key capabilities in Fig. 2: Referring Expression Grounding (REG), Region Referring Grounding (RRG), Object Functional Grounding (OFG), and Visual Trace Generation (VTG). These abilities cover object identity ("What is this object?"), functional affordance ("How use/grasp it?"), target location ("Where should it be placed?"), and can even implicitly convey the execution process ("How to complete the task?") through a visual trace. To develop these multi-task abilities, we constructed Embodied Brabodied-Points--ee Points 200k, large-scale dataset of high-quality instances with verification methods, curated from diverse sources.
introduce Embodied- R1, an advanced embodied reasoning VLM trained entirely with Reinforced Fine-tuning (RFT) to master manipulation through "pointing," shown in Fig. 1. This training paradigm provides two critical advantages over standard SFT. First, it enables flexible, free-form reasoning beyond the rigid CoT templates, significantly enhancing generalization novel tasks. Second, it resolves the inherent multi-solution dilemma in embodied pointing. For instance, an instruction mark point in an "empty space" has many valid solutions. While SFT struggles with this ambiguity and tends to overfit to single data point, RFT can positively reinforce any correct answer, fostering genuine understanding of the task's spatial constraints rather than mere memorization. With only 3B parameters, Embodied-R achieves of performance on multiple spatial reasoning and precise embodied pointing benchmarks. achieves robust zero- shot manipulation by generating pointing signals as an intermediate representation, grounding its reasoning the VLM's universal perception capabilities. This method preserves the model's inherent generalization by avoiding the prediction of low-level, embodiment-speecfie actions. Empirically, Embodied- RI delivers statc-of-the-at performance in the SIMPLERENV (Li al., 2024b) simulation, attains remarkable 87.5% success in real-world XArm tasks, and shows notably strong robustness against common visual disturbances such as changing light and backgrounds.
unified, embodiment-agnosse representation and defining core embodied pointing abilities to bridge perception and decision; constructing the comprehensive Embodied- Pointss-ooi dataset for these capabilities; proposing Embodied-R1,
Our contributions include: pioneering "pointing"
2

---

Published as a conference paper at ICLR 2026
Referring Expresiton Grounding
Object Grounding
Embodied Pointing Abilities
Move the maka pot the right side of the drower
Referring Expression Grounding (RFG)
Object Functional Grounding (OFG)
Wher 6t?
How it?
go?
How to perfioes it?
Region Reference Grounding (RRG)
Let
Visual Generation (VTG)
Regian Refervaze Grounding
Figure 2: Overview of four embodied pointing abilities.
VLM trained with RFT resolve the multi-solution dilemma for embodied pointing, delivering powerful reasoning. With only 3B parameters, Embodied-R attains state-of-the-ar performance on 11 diverse spatial and pointing benchmarks and enables robust zero-shot robotic manipulation, achieving 56.2% success in SIMPLEREnv simulation and 87.5% in 8 real-world XArm tasks, representing a 62% improvement over strong baselines, without task saec-fpec fine-tuning.
2 RELATED WORK
Embodied Reasoning in Robotic Manipulation Enhancing the reasoning capabilities Large Language Models (LLMs) is pivotal research direction (Chu et al., 2025; OpenAI, 2024; Guo et al., 2025; Kimi-Team et al., 2025). In embodied AI, recent works have integrated reasoning into robotic manipulation, primarily through Supervised Tine-Tuning (SFT) with templated Chain-of- Thought (CoT) approaches. These works involve various forms of templated CoT, such as language prompts (Zawalski al., 2024), visual subgoals (Zhao al., 2025), spatial relation graphs (Yuan et al., 2025) to guide execution. While newer efforts explore Reinforcement Fine- Tie-TTning (RFT) (Lu al., 2025; Liu al., 2025a; Chen et al., 2025b) or latent planning (Huang et al., 2025), they are often limited to simulation or online learning, In contrast, our VLM stimulates free-form reasoning by integrating pointing with RL, avoiding fixed templates. Furthermore, Embodied- R1 utilizes pointing to precisely anchor reasoning within the scene and directly guide manipulation.
Spatial Reasoning with VLMs Spatial intelligence is essential for general-purpose embodied AL, enabling robots for precise manipulation (Yuan et al., 2024b; Song et al., 2024) and navigation (Hong al., 2023; et al., 2024a). Current methods enhance the spatial reasoning of VI.Ms primarily through SFT on custom datasets (Du et al., 2024; Ray et al., 2024; Cheng et al., 2024; Chen et al., 2024). Some approaches also employ spatial CoT for step-by-step reasoning (Yuan et al., 2025; Liu et al., 2025b). Differently, Embodied-RI employs RL to elicit emergent reasoning, which we show leads to stronger out- out-of-diisribution (OOD) generalization compared to SFT-centric approaches.
Visual Auxiliary Signals for Robotic Manipulation Using visual auxiliary signals (Bharadhwaj et al., 2024; Wen et al., 2023; Xu et al., 2024; Zheng et al., 2024; Yuan et al., 2024a; Dong et al., 2024b) is promising paradigm for abstracting away embodiment-speed cross-robot generalization. Prior works have explored various signals, including keypoints (Yuan 2024b;; 2025), affordance maps (Huang al., 2024a; 2023; Li et al., 2024c: Wang et al., 2025a), bounding boxes (Liu et al., 2024a; Huang et al., 2024b), optical flow (Xu et al., 2024; Wen et al., 2023), and visual trajectories (Yuan al., 2025; Ji al., 2025; Li al., 2025; Gu et al., 2023). In contrast to these methods, which typically generate a single type of visual signal, we propose unified "pointing" definition to express diverse, multi- -rattlgranularr manipulation intents. We adopt an RL paradigm to explicitly improve zero-shot generalization in novel environments.
details and enhancing
Improving VLM Reasoning via RFT Enhancing the reasoning capabilities of large models via pivotal research direction, demonstrating
RFT (Chu et al., 2025; Liu et al., 2024b) has emerged superior performance in complex mathematics (Yu et al., 2025) and coding (Jimenez et al., 2024) tasks. This paradigm has been effectively extended to the multimodal domain, with numerous studies confirming that RFT significantly boosts multimodal perception and understanding capabilities (Tan et al.: Feng et al., 2025). Specifically, applications such as Seg-Zere (Liu et al., 2025c) introduce cognitive reasoning to guide segmentation, while VisionReasoner (Liu et al., 2025d) proposes a unified visual perception framework based on RFT. Building upon VLM foundation models, post-training
3

---

Published as a conference paper at ICLR 2026
with curated data using algorithms like GRPO (Shao et al., 2024) or PPO (Schulman et al., 2017) can achieve performance and generalization capabilities that surpass standard SFT. This methodology has been successfully applied across diverse domains, including affordance prediction (Wang et al., 2025a), video understanding (Park et al., 2025). However, the application of RFT in embodied AI, particularly in robotic manipulation, remains underexplored. In this work, we propose an RI-style VLM tailored for robotic manipulation. By integrating a fine-grained and comprehensive "pointing" mechanism with RFT, we bridge the critical gap between "seeing" and "doing".
EMBODIED-RI: ADVANCING EMBODIED REASONING VIA RFT
This section first details the architecture and embodied pointing capabilities. We then describe dataset construction and training methodology, concluding with its deployment in real-world scenarios.
3.1 THE ARCHITECTURE AND CAPABILITIES OF EMBODIED-RI
Embodied-RI is built upon the Qwen2.5-Ve VL architecture (Bai al., 2025a) and is specifically optimized for embodied manipulation by mastering four fundamental pointing abilities. These abilities all generate image coordinates p (p:q) E w0, w] x [0, h).. but differ in their semantic purpose and output structure. I Referring Expression Grounding (REG) localizes an object from a linguistic description by generating a point within its segmentation mask. Region Referring Grounding (RRG) identifies a spatial region from relational language (c.g.. "the space between objects") by generating point suitable free-space location. Furthermore, Object Functional Grounding (OFG) identifies functionally critical parts of an object (i.e., affordances), such as handle for grasping, by marking a point on that area. Finally,
Visual Trace Generation (VTG) produces an ordered sequence of points, T= (P: t= 1, 2, ... T). T denotes the sequence length, to form complete, object-centric manipulation trajectory. This embodiment-agnosti path provides comprehensive spatial plan for the robot follow. Visualizations presented in Fig. 2.
3.2 ENHANCCNN THE EMBODIED REASONING ABILITIES OF VLM
To develop general embodied pointing capabilities, Embodied-Ri trained on three data types detailed in Fig. 3: embodied spatial reasoning for foundational awareness, general reasoning to preserve existing skills, and embodied pointing to learn the four key abilities.
General and Spatial Reasoning Data The foundation of reasoning data is Embodied-Spatial-e -84K
an embodied spatial awareness dataset aggregated from SAT (Ray et al., 2024) and WhatsUp (Kamath et al., 2023). For objective evaluation and verifiable rewards, all source data were converted into unified multiple-choic format. Furthermore, to counteract the issue of catastrophic forgetting and preserve general reasoning during specialized training, supplement this with ViRL-subset-18K a diverse snnerrl-knowledge set. This 18K subset was curated from the ViRL (Wang et al., 2025b) dataset by filtering for difficulty and balancing content component of our composite dataset, creating a balanced curriculum that fosters specialized spatial skills while safeguarding the model's foundational knowledge.
subjects and types. This process yields
the
Embodied Pointing Data To advance a suite of embodied pointing capabilities, we introduce the Saaboddedd-Pints-200000 dataset, high-quality, meticulously curated corpus containing about 200k samples. To address the multi-solution dilemma inherent in embodied pointing problems, we avoid constructing petestioon-nnsweer pairs typical for SFT. Instead, we structure the data as "question- verification" pairs, leveraging RFT for training. Subsequently, pre-defined reward functions for each task evaluate the response based on the verification and calculate the corresponding rewards. We briefly outline the pipeline for generating point data below. Data generation details are in App. A.
Precise localization is critical for robotic manipulation, but traditional bounding boxes suffer from inherent ambiguity. We therefore constructed a point-centric REG dataset, integrating web images from RefCOCO (Kazemzadeh et al., 2014) and embodied data from RoboRefIt (Lu et al., 2023; Yuan et al., 2024b) and RoboPoint (Yuan et al., 2024b) for broad coverage. We critically adjusted the success criterion: the model must output a single point instead of a bounding box. A prediction is considered correct if this point falls within the object's segmentation mask.
REG Data:
4

---

Published as a conference paper at ICLR 2026
Embodled Spatial 84K
K
ViRL-Subaet-18K
Embodiedd-Sattalllie
Stage 1: Training Spatial Reasoning Capability
OFG
REG
AAG VTG
VIRL-
Embodied- Embodled-Points-- Points- 200K
S-18K
Phase 2: Training Embodied Pointing Capability
Figure Overview of training data: In stage we focus on improving model's spatial reasoning capability, while incorporating a small amount general reasoning data. stage 2, we train the model's embodied pointing capabilitiis, which comprise four distinct capability items.
RRG Data: To enable robots to comprehend complex spatial placement commands, we developed automated data generation pipeline for creating relation-aware placement regions. This pipeline processes a large corpus of open-source embodied dataset (about I million), and after rigorous filtering, yields 33,000 high-quality samples. The core process includes: O Region Extraction: extracting the final position of the manipulated object from the terminal frame; Region Refer- ring: calculating the precise placement of the region relative to reference object the scene; and Rendering: rendering this spatial placement information back onto the initial image. To ensure data diversity and quality, we designed heuristic filtering strategy, which covers a rich variety of spatial relationships, object configurations, and scenes.
OFG Data: To cultivate Embodied-RR RR'ss fine-grained understanding of object affordances, we constructed 40K-sample OFG dataset. We leveraged the HandAl. dataset (Guo et al., 2023), converting its meticulous annotations of manipulable parts into bounding boxes for ground-truth verification. To promote generalization, function-related questions (e.g., "Which part should be held when using a knife to cut vegetables?") corresponding these functional parts.
then employed GPT-4o generate diverse set of
VTG Data: We constructed an object-centric visual trace dataset that exclusively tracks the object's movement. The extraction pipeline follows the methodology of (Yuan al., 2025) and consists of three main steps: ① Key Object Proposal: Using GPT-4o to identify the primary object of interest for given task. KeyPoint Identification: A suff-super keypoint extractor (Huang et al.,
2024c), in conjunction with Grounded-SAM (Ren et al., 2024), is used to automatically identify the object's grasping point. 2 Point Tracking and Projection: We used Cotracker3 (Karaev et al., 2024) to compute the dense temporal visual trace originating from the keypoints. Next, the trajectory is then downsampled into 8 equidistant discrete points and projected back onto the initial image, creating "image- visual trace" pair. Notably, using multiple pre-trained vision models in the process inevitably introduces noise. We implemented rigorous rule-based filtering and continually validated our approach using manually annotated set. Based on this feedback, we iteratively refined the filtering criteria to improve the quality of the dataset.
Training Strategy Based on the collected data, Embodied-R adopts a two-stage training process: the first stage focuses on enhancing spatial reasoning, as spatial reasoning serves as the foundation for point comprehension; the second stage trains embodied pointing capabilities using point-centric, multi-task mixed data. The optimization process is powered by the GRPO (Guo et al., 2025) algorithm. For any given input, the behavior policy generates multiple candidate responses, and their relative advantages are determined by normalizing rewards within the group. The model is then optimized using a clipped surrogate loss to maximize expected returns while maintaining training stability.
3.3 MULTI-TASK REWARD DESIGN
To stabilize multi-task training, where simpler tasks dominate policy optimization, we design modular reward system. Each task is assigned a total reward R, which a normalized weighted sum of components from reward library. This scaling is crucial for balancing learning across tasks.
5

---

Published as a conference paper at ICLR 2026
The following is the complete reward function library. Format Rewards: A binary reward Ttormat()) a(t valid(y)) enforces a structured output y, including required tags like <think> and a unified coordinate representation <point> ]
Accuracy Rewards: For general QA tasks, an accuracy reward (v) = T(y g) is used, where is the ground-truth answer. Point in Mask Reward: For pointing tasks, point in mask reward function Ymask is determined by whether the predicted output point p lies within the ground-truth answer mask Mgt. The reward function can be formally expressed as Tmask Mgr) = L(p Mgt).
To improve learning efficiency, we also designed a dense auxiliary reward rdis, which is used to guide the predicted point to approach the target region Mg. The Euclidean dis- tance is d = Ip g gl12, where is the center of the Mgt. Given pixel distance thresholds Dmin thresh and
Point Distance Reward:
Dmax thresh. rais is then defined as Tais (P, Mg)= = min [1.0, max (0.0, 1.0
d- Dan surdh
Dusx dresh Duin tresh
For evaluating generated visual trace, rewards are derived from trajectory similarity metries comparing the predicted trajectory T with the ground-truth trajectory Tgt. First, we compare the number of points in the T and Tgt. Using the longer one as the reference, we interpolate both trajectories to have the same number of points and then proceed with the calculation Root Mean Square Error (RMSE): dRMSE (T, Tg)). Similarly, we use the DRMSE min and DRMSEL mux hyperparameters to ensure that the reward remains between 0 and 1. The reward is calculated as: dussa (oasu(t,,7 (g) DiossE_mi
Visual Trace Reward:
Tincee (T, Tg))
min (1.0, max (0.0, 1.0
DRMSE mu DRMSE_ min
The total reward R for each task is formulated as a combination of these individual reward components. We define the reward function library F {Iformat: Taccs T'ask, I'dis N'mace J. Each component function evaluates a specific aspect of the model's performance. Since we conduct mixed training on multiple tasks, in order to ensure that each task is equally and sufficiently trained, we constrain the total reward R. for each task to the range of 0 to 1, which is implemented as follows. R formulated as weighted-sun
Total Reward:
combination: R = Erer Wr -r. The task-specific weights wi, are normalized to sum to one (Erer W, = 1). This structure guarantees that the total reward R is also bounded within the range [0, 1] and allows to tailor the reward signal for each task's specifie needs. For example, the RRG task requires simultaneously satisfying the format requirements, ensuring that the predicted points are within the specified region, and accelerating training by employing dense distance rewards. RRRG is defined as RRRG 0.1 + 0.2rais + 0. 7r.7mask- This consistent scaling of rewards across tasks is crucial for stabilizing. We refer to App. B for detailed hyperparameters.
3.4 ACTION EXECUTOR OF EMBODIED-RI
Through its pointing mechanism, Embodied-RI can be flexibly integrated with various downstream action executors. This architecture allows Embodied-RI to initiate reasoning at any task stage, dynamically seleet the required pointing ability, and couple its output with motion planner for zero-shot robotic control. Besides, similar to Hamster (Li et al., 2025), Embodied-R can also serve as high-level planner that integrates with learning-based based methods to enable closed-loop control. Here, we implement this through two primary pipelines. Affordance Points Branch (-P): In this branch, Embodied-RI leverages its RRG and OFG capabilities to predict key points for grasping and placement. These points then serve as targets for a CuRobo (Sundaralingam et al., 2023) motion planner, which generates collision-fre trajectory for the robot's end-effector. Visual Trace Branch (-V): This branch utilizes the object-centric visual traces generated by VTG. The 2D trace is first mapped to 3D Cartesian coordinates using the pinhole camera model and initial depth information. These discrete 3D points are then interpolated to form continuous motion trajectory in SE(3) space, which the robot follows for execution, similar to the FSD (Yuan et al., 2025) methodology.
4 EXPERIMENTS
To validate Embodied-RI generalization in robotic manipulation, we conducted extensive experi- ments evaluating its Seeing (spatial reasoning and pointing capabilities) and Doing (manipulation tasks) dimensions. Our evaluation encompassed 11 QA benchmarks, 4 simulated tasks (SIM- PLEREnv) (Li et al., 2024b), and 8 real-world robot (xArm platform) tasks. We used the Qwen2.5- VL-3B- Instructt (Bai et al., 2025a) model as the initial model. First, we trained using the Embodied- Spatial-84K and subLst 18K datasets. Then, we continued training with the second-stage

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison spatial reasoning benchmarks. Bold indicates the highest value among open-source source models, and underlined values show the second second-highest. scores.
CNBrnch
CRRPE
FarhSp-
Count
20Rel
301kpp
3000s
Sung
Prod.
Val
Real
MV
Rellepth
SpRee
Ob)
Test
Cloudd models
CPP
62.4
71.1
79.5
08.3
10.4
76.7
05.1
05.5
70.1
48.5
20.7
55.60
59.7
72.7
54.9
0017
16.1
CBrT4n
65.9
85.5
$7.8
78.3
75.4
81.9
71.8
75.6
75.8
49.4
57.5
50.2
74.2
199.2
59.8
05.9
49.1
Open source modelss
58.2
46.6
51.0
47.8
51.4
57.8
54.2
55-2
35.6
51.4
41.6
41.4
55.2
09.9
52.5
56.2
15.1
SyT-DDmumic-133
RubaPoin:
55.5
53.3
51.4
57.5
FSD-4 5B
13.3
46.7
RubuBBrais-78
64.3
718
45.3
69.3
Qwen22 SVL-1B
68.4
72.8
71.6
71.0
76.0
48.7
45.1
48.4
61.7
62.8
Emboddsd-SSig
65.4
92.3
85.8
11.1
76.7
21.3
75.8
75.3
59.3
68.5
90.4
54.9
06.1
65.1
37
Rmbbedid-R11 wio
20.4
90.2
14.5
K1.0
81.5
80.3
699
75.4
75.2
700
13.9
47.
72.6
76.7
56.6
64.1
65.4
3.4
Embodied R1
78.6
90.8
$4.7
54.
K2.7
822
70.7
75.4
76.1
7000
76.
51.1
76.8
90.
57.4
66.4
67.4
21
Table Performance on Pointing benchmarks. The Table 3: Performance on VABench-V. V. Lower score is the accuracy of points falling within the target region. values are better for RMSE/MAE, higher is better for LLM Score.
Model
RoboRelit
Where2Place
VABench-P
Part-Afford
4P
15.28
29.06
9.30
10.15
Model
RMSE +
MAE
LLM Score 1
ASMv2
48.40
22.00
10.07
13.75
RobuBrain
10.10
16.60
7.00
25.25
GPT-4o
136.1
113.5
4.4
RobosPoint
19.09
DINOv2 Predictor
117.5
FSD
56.73
45.81
61.82
9.55
RoboBrain
121.6
103.8
4,5
Qwen2.5VL
74.90
31.11
9.89
23.42
78.3
63.4
6.2
Emnhadied
41.25
Embodicd-SFT
109.4
65.2
6.2
Embodied-Rt R1
85.58
69,50
66.00
56.63
Embodied RI
77.8
45.0
7.3
EmbodiedPPints-2006 dataset. For all experiments, focus comparing SFT models trained with the same batch size and data, which we refer to as Embodied-SFT. For training details, please refer to App. B and App. C. We also provide additional experiments in App. F.
4.1 EVALUATION OF SPATIAL REASONING CAPABILITIES
Setup: To evaluate the foundational spatial reasoning from our first training stage, we bench- marked Embodied-R1 five diverse benchmarks: CVBench (Tong et al., 2024), BLINK (Fu et al., 2024), CRPE (Wang et al., 2025c), SAT (Ray et al., 2024), and EmbSpatial-Bench (Du et al., 2024). Baselines included leading closed-source models (GPT-4o, GPT-4V) and various open-source, spatially-enhancedd VLMs such as SAT-DDnamie (Ray et al., 2024), RoboPoint (Yuan et al., 2024b), and FSD (Yuan et al., 2025). We also included two key ablations: Embodied-R1 w/o CS, which excludes the ViRL common sense dataset, and Embodied-STP SFT, a variant trained only with SFT.
Results: As shown in Tab. 1. Embodied-R1 demonstrates of-e-of-the-ar performance among open-source models with only 3B parameter. It achieves an average rank of 2.1, significantly outperforming its variants trained without common-sense data (Embodied-R) w/o CS, Rank 3.4) or with only SFT (Embodied-SFF) Rank 3.7). We believe that more diverse data can stimulate exploratory reasoning capabilities. Embodied-R RR surpasses larger, specialized embodied models, including RoboBrain-7B and FSD-13B, underscoring the effectiveness of our RFT training strategy.
4.2 EVALUATION OF POINTING CAPABILITIES
Setup: To comprehensively evaluate Embodied-RR -R1 embodied pointing abilities, we tested it across our four defined capabilities. For REG, we used RoboReflt (Luu et al., 3023), which challenges models with relational references between similar objects. For RRG, we selected Where2Place (Yuan et al., 2024b) and the more complex, reasoning-intensive VABench-P (Yuan et al., 2025). We assessed (Myers et al.,
OFG on our custom Part-Affort Benchmark, derived from RGBD- SaaDD--art-Affrddaaae 2015) by filtering 2000 grasp-related affordances. This benchmark encompasses 105 types of kitchen, workshop, and gardening tools, designed to evaluate the generalization capability of affordance prediction in OOD scenarios. Finally, followed the VABench-V V (Yuan al., 2025) evaluation methodology for VTG capacity, measuring MAE, RMSE, and LLM Scores.
Results are presented in Tabs. 2 and 3, with visualizations in Fig. 4. Key observations follows:

---

Published as a conference paper at ICLR 2026
lhe stamitr
Coullis
horin on winumm
Nitto tn
Nortle oo
dediaa
ViBench-
Visal
Figure 4: Visualizing Embodied-RI's Performance on Various Pointing Tasks, model can follow diverse text instructions and generalize its capabilities novel, environments.
(OI) Powerful general VLMs perform poorly pointing tasks, such GPT-4o and Qwen2.5 VL, indicating the necessity of specialized training with data for these capabilities. This indicates that is necessary to train embodied VLMs with strong spatial reasoning and pointing abilities.
(02) Embodied-RL demonstrates superior performance across key benchmarks. Tabs. and 3, all benchmarks for REG, RRG, OFG, and VTG, Embodied-RI consistently outperforms both general and specialized baselines, including other pointing-focused models like FSD and RoboPoint. As shown in Fig. single Embodied-RR R1 model masters this diverse skill set, demonstrating high accuracy even with small objects and complex spatial relationships in cluttered scenes.
(03) Embodied-RI generates highly accurate visual traces for robotic manipulation. On the VABench-V V benchmark, Embodied-RI achieves the lowest RMSE and MAF, indicating its ability to produce precise point sequences traces, a crucial aspect for reliable action execution. Embodied- RI also demonstrates significant improvement, indicating that RI focuses on reasoning about the ideal visual trajectory rather than rote memorization. refer to more visualization App. E.
(04) Embodied-RR significantly outperforms models trained solely with SFT. Compared to the Embodied-SFT, Embodied-R demonstrates substantial improvements across these tasks, validating the benefits of RFT in developing strong generalization capabilities for embodied pointing.
Table 4: SimplerEny Evaluation on WidowX X Robot. Each task is tested 24 episodes. Most of the results for end-to-end VLAs are sourced from Chen et al. (2025a), while the results for the remaining models are reproduced in accordance with the official code.
Put Spoon on Towel
Put Carrot on
Stack Green Block on Yellow Block
Eggplant Yellow Basket
Type
Model
Avg
Octo (Team et al., 2024)
41.7
8.2
0.0
56.7
26.7
No (Black et al., 2024)
29.1
0.0
16.6
27.1
Ro-fast (Pertsch et al., 2025)
29.1
21.9
48.3
End-to-end
OpenVLA (Kim al., 2024)
4.2
0.0
5.2
VLAs
OpenVL.A-OFT (Kim al., 2025)
34.2
30,0
30.0
72.5
41.8
ThinkAct (Huang et al., 2025)
58.3
37.5
70.8
43.8
Magma (Yang al., 2025)
37.5
31.0
12.7
60.5
35.4
Modular
MOKA
al., 2024a)
45,8
41.6
33.3
Methods
Sofar (Qi al., 2025)
55.5
56.9
62.5
40.2
53.8
RoboPoint (Yuan et al., 2024b)
16.7
20.8
8.3
25.0
17.7
Affordance
Methods
FSD (Yuan et al., 2025)
41.6
50.0
33.3
37.5
40.6
Embodied-RL
62.5
68.0
56.2
4.3 EVALUATION OF EMBODIED-BR R1 FOR ROBOT MANIPULATION
SimplerEnv Simulation. To rigorously assess the zero-shot generalization capability Embodied- R1, we conducted experiments the WidowX arm using SimplerEnv (Li et al., 2024b), employing CuRobo (Sundaralingam et al., 2023) planner for execution. We compared performance against a comprehensive suite of baselines across three categories: (1) End-to-end VLAs, including standard models (Octo, Open VLA, mT) and stronger variants -fst, Open VLenVLA-OR A-OFT, Magma, ThinkAct); (2) Modular Methods (MOKA, Sofar); and (3) Affordance Methods (RoboPoint, FSD). As presented
8

---

Published as a conference paper at ICLR 2026
Left: Partial Real-world Experimental Tasks
Mid: Visual Disturbances Tasks
Right: Simulation, Novel Embodiment and Hand- mondn-ddmnnnnn Sketches
Figure Left: Snapshots from a variety of our real- experimental manipulation tasks. Middle: Demon- stration of the robustness under significant visual disturbances, such as background and lighting changes. Right: Specifically designed tests evaluating the zero-shol generalization of VTG in entirely OOD scenarios, including simulation, novel robotic embodiment, and draww sketches. See App. for complete execution process.
Table 5: Real-world experimental evaluation results. The first two tasks were conducted 5 times each, while the other tasks were conducted 6 times cach. The best results are highlighted in bold. [x]: The instruction for each trial is a randomly selected color. Nearest object": the object closest to the camera's ' viewpoint.
Pkis
Move
Move the vise
Place the Rirk
Pick the [d] toothoush
Mone the sceres
Put Bhe screedive:
Mose the coka
A/2
up the
ogg
and place
object right
betwees
pot
Stace
Star:
Groep
Statt
Grasp
Stace
Comeel City
Sac.
Carmel City
Sacc
Canap
Sace
Crrup
Ster:
Sier.
MOKA
0,0%
15.7%
83.35
0.00%
40.0%
16.7%
12.5%
20.055
50.075
66.75
11.75
16.7%
16.7%
16.7%
0.0%
0.05
0.05
A6.75
13.155
16.7%
25
100.055
66.75
90.75
198.8%
198.0%
100J15
81.3%
1ALIS
100.5%
100.5%
180.8%
10.05
15.3%
$3.95
Easbodied-At RL--T
100.0%
100,0%
198.9%
108.9%
108.8%
106.9%
10000%
66.7%
10LDS
100.0%
100.0%
100.0%
33.3%
33.3%
87.5%
Table 6: Performance Comparison between SFT and
Table 7: Performance of Embodied-R1 under visual disturbances. BC: Background Change, LC: Light Change, HC: Height Change.
RL on RRG benchmarks.
RL
Think
Where2Place
VABench-P
Disturbance Grasp (%)
Succ. (%)
65,50
65.39
Original
100
100
x
63.00
60.50
BC
100
100
41.25
47.67
BC+LC
83
83
X
36.85
50.46
BC+LC+HC
83
83
in Tab. 4, Embodied-Ri achieves state-of-the -fte------hh--rttt average success rate of 56.2%. Notably, while recent end-to-end methods such as Ta-sas (48.3%) and Open VLA-OFF (41.8%) show improved robustness compared to the base OpenVL. (5.2%), Embodied-RI still outperforms them significantly. Furthermore, Embodied-R1 surpasses the strongest modular baseline (Sofar, 53.8%) and affordance baseline (FSD, 40.6%). These results suggest that explicit visual reasoning provides superior zero-shot generalization compared to end-to-end policy learning, particularly when facing unseen instructions and background variations without domain-specific fine-tuning.
Real-World Robot Evaluation. We conducted zero-shot real-world evaluations on an xArm robot across eight tabletop manipulation tasks. The setup used third-person person Intel RealSense L515 camera 4840x480) with all objects, scenes, and tasks being OOD to test generalization, As shown in Tab. 5, Embodied-R1 achieves an 87.5% zero-shot success rate, improvement of over 60% compared to the RoboPoint and FSD baselines. We attribute this significant improvement to the baselines' poor performance on tasks requiring spatial reasoning (e.g.. moving the nearest object) and their low success rates in grasping challenging rigid objects like screwdriver. In contrast, the Embodied-RI correctly identified these cases and achieved high success rates, demonstrating the effectiveness of reasoning process. Overall, Embodied R1-V generated more accurate annotations than Embodied-RI -I--. resulting slightly higher average success rate. In App. D, we conducted an in-depth analysis of failure cases and execution time.
To further test robustness, we selected a task requiring spatial reasoning and introduced zero-shot visual disturbances, including changes in background, lighting, and height. As shown in Tab. 7, Embodied-R demonstrated outstanding generalization against all disturbances. Surprisingly, located the target and completed the task even under the poorest lighting conditions, while background changes had no impact on performance. This experiment validates that pointing serves as a universal representation that maintains both performance and robustness under visual distractions.
Qualitative Analysis in Complex Scenarios. To further validate the versatility of Embodied-R conducted qualitative experiments across long-horizon, reasoning-intensive, and contact-rich

---

Published as a conference paper at ICLR 2026
Long Horizon Tasks: Place the three ducks onto plates of matching colors
pink duching the pist duckking yellow
Long Horizon objects
the plate
Reasoning
sponge problom on the whiteboard
Move the boujetto the plune
Nove the spoeige into the plate
Contact-rich Wipping Blackboard
Figure Qualitative Analysis on Long-horizon, Reasoning and Contact-Rich Tasks.
For complex scenarios such as table organization and color-matched duck sorting, we employed Gemini-2.5-Prr as high-level planner to decompose global instructions into sub-tasks. Embodied-R1 then generated precise visual traces for each stage, successfully managing multi-step sequences. (2) Reasoning- Driven Tasks: In tasks requiring logical deduction (e.g., "Place sponge on the answer the math problem") "), the model demonstrated robust spatio-logical reasoning. It successfully localized semantic targets derived from internal calculations, effectively bridging the gap between abstract reasoning and spatial grounding. (3) Contact-Rich Manipulation: Embodied-RL serves as flexible interface that decouples high-level perception from low-level dynamics. In blackbourd wiping, Embodied-RI's visual traces were executed an impedance controller to maintain consistent contact force. For bottle pressing, we integrated visual localization with a force-aware Diffusion Policy (Chi et al., 2023) trained on 50 demonstrations. This synergy allowed the system adapt varying bottle positions while maintaining smooth, force-sensitive execution. These results indicate that while pointing serves as a geometric abstraction, it functions a pivotal bridge linking high-level semantic understanding with diverse low-level control strategies, enabling robust generalization in complex embodied tasks.
tasks (Fig. 6). (1) Long-Horizon Tasks:
4.4 FURTHER ANALYSIS AND ABLATIONS
Embodied-RI Exhibits Strong Generalization. Despite being trained exclusively on real-world data, Embodied-RI demonstrates remarkable zero-shot generalization VTG tasks across entirely unseen scenarios (Fig. 5 (right)). makes accurate predictions in simulations, suggesting a promising direction for sim-to-real transfer. Furthermore, Embodied-RI adapts different embodiments, highlighting the advantage of our embodiment-agnostic visual traces. Notably, its accuracy holds in abstract scenarios, correctly marking real screwdriver a hand-drawn box.
Ablation on SFT vs. RL. To dissect our training paradigm, we compared RL against SFT and analyzed the impact of an explicit reasoning process. We trained four variants RRG benchmarks. The results in Tab. show that RL-based models consistently outperform their SFT counterparts, highlighting the crucial role of RL in OOD generalization. Our full model (RL w/ Think) performed the best, confirming that the reward-driven paradigm is most effective when combined with reasoning. In App. we provide further experiments, including an analysis of integration with learning- basedd methods, an ablation study on the benefits of mixed training, and results using RGB-D inputs.
5 CONCLUSION
We introduce Embodied-RI. an embodied reasoning VLM that bridges the critical "sccing-to-doing" gap robotic manipulation. By training Embodied-RI with two-stage RFT paradigm on our large-scale curated dataset, we significantly enhance its spatial reasoning and embodied pointing abilities. Through its core pointing mechanism, Embodied-R masters suite of capabilities, including grounding, spatial referencing, affordance marking, and visual trace generation, which are further applied to downstream robotic manipulation. Empirically, Embodied-B R1 achieves state-of-the of-the-art results across multiple benchmark tests and demonstrates robust shot generalization in robotic manipulation tasks, offering a promising pathway toward more capable and general-purpose purpose embodied AI. detailed discussion of limitations is provided in App. G.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work is supported by the National Natural Science Foundation of China Youth Student Basic Research Project (Grant No. 625B2128). This work also supported by the National Natural Science Foundation of China (Grant Nos. 62422605, 62533021, 92370132) and the National Key Research and Development Program of China (Grant No. (E9YFE021090)) Yifu Yuan is also supported by the Young Science and Technology Scientists Sponsorship Program by CAST Doctoral Student Special Plan. Pengyi Li is supported by the Basic Research Project (Grant No. 624B2101). We would like thank Zhongwen Xu, Liang Wang, Shuyang Gu, and Chen Li for their participation in the discussions of this paper and for providing valuable insights. In addition, we would especially like to thank Yiyang Huang for the constructive suggestions on improving the figures in the manuscript.
REPRODUCIBILITY STATEMENT
To promote transparency and reproducibility within the scientific community, we provide detailed training parameters and resources in the appendix. The complete code for both training and in- ference has been uploaded the GitHub repository: https: /tt::://giti /github. com/ httppxiiixiiappppppocopppppppppp Embod i ed R1. In addition, we provide video demos of model execution and more visualization examples on the project homepage; we refer readers to ht /mmbodi ed- r1 g thub. i for more information. We have made datasets and model checkpoints publicly available.
ETHICS STATEMENT
This paper is dedicated advancing the field of robotic manipulation towards the creation of more effective and versatile robotic assistants. Our research strictly adheres to responsible practices and aligns with the ICLR Code of Ethics. All training data was sourced from large-scale, open- open-access robotics datasets, with all assets utilized in full compliance with their original licensing and terms of service. We recognize that while the intended applications of this research are positive, the long-term teng-eerm societal impacts of increasing robotic autonomy warrant careful consideration.
access
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li. Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025a. URL https: www.ivivivvv////i///aaa/aaaaaava org/ /bs//550000 13923.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 139233 2025b.
Homanga Bharadhwaj. Roozbeh Mottaghi, Abhinav Gupta, and Shubham Tulsiani. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation. arXiv preprint wwXiv:2405.0152.. 2024.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. no: iision-language-action flow model for general robot control. arXiv preprini 2.Xix:2410024164. 2024.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing languae models with spatial reasoning capabilities. In Proceedings of the IEEF/CVF Conference on Computer Vision and Patlern Recognition, pp. 14455-14465. 2024.
Xinyi Chen, Yilun Chen, Yanwei Fu, Ning Gao, Jiaya Jia, Weiyang Jin, Hao Li. Yao Mu, Jiangmiao Pang. Yu Qiao, Yang Tian, Bin Wang, Bolun Wang. Fangjing Wang. Hanqing Wang, Tai Wang. Ziqin Wang, Xueyuan Wei, Chao Wu, Shuai Yang, Jinhui Ye, Junqiu Yu, Jia Zeng, Jingjing
11

---

Published as a conference paper at ICLR 2026
Zhang, Jinyu Zhang, Shi Zhang, Feng Zheng, Bowen Zhou, and Yangkun Zhu. Internvla-mi: A spatially guided vision-languuge-aae action framework for generalist robot policy, 2025a. URL https: /arxivxivvv//./////v//aa/////aia org/ /btp///.....11111
Yuhui Chen, Shuai Tian, Shugao Liu, Yingting Zhou, Haoran Li, and Dongbin Zhao. Conrft: A rein- forced fine-tuning method for vla models via consistency policy, arXiv preprint arXiv:2502.05450, 2025b.
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatialrgpt: Grounded spatial reasoning in vision language model. arXiv preprins arXiv:2406.00 01544, 2024.
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273668. 2023.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong. Saining Xie, Dale Schuurmans, Quoe V Le, Sergey Levine, and Yi Ma. Sft memorizes, r1 generalizes: A comparative study of foundation model post-training. Xrviv preprint arXiv:2501.17661, 2025.
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems,
11757999-581,, 2023.
Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Moham- madreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of- fftte-of-thearte fe-the--rt art multimodal models. arXiv preprint 2XXv:2449.11146 2024.
Zibin Dong, Yifu Yuan, Jianye Hao, Fei Ni, Yi Ma, Pengyi Li, and Yan Zheng, Cleandiffuser: An easy-to-use modularized library for diffusion models in decision making. Advances in Neural (166899-86228 2024a.
Information Processing Systems,
Zibin Dong, Yifu Yuan, Jianye Hao, Fei Ni, Yao Mu, Yan Zheng, Yujing Hu, Tangjie Lv, Changjie Fan, and Zhipeng Hu. Aligndiff: Aligning diverse human preferences via rehavior-customisahe diffusion model, 2024b. URL https::///// http///xxvvv.oo org/ abs /2300 02054.
Mengfei Du, Binhao Wu, Zejun Li, Xuanjing Huang, and Zhongyu Wei. Embssatial--ench: Bench- marking spatial understanding for embodied tasks with large vision-language models. arXiv preprint 0rXiv:2406.05756. 2024.
Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-rl: Reinforcing video reasoning in milmss arXiv preprint arXiv:2503.21776. 2025.
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang. Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference Computer Vision, pp. 148-166. Springer, 2024.
Jiayuan Gu, Sean Kirmani, Paul Wohlhart, Yao Lu, Montserrat Gonzalez Arenas, Kanishka Rao, Wenhao Yu, Chuyuan Fu, Keerthana Gopalakrishnan, Zhuo Xu, et al. Ri-trajectory; Robotic task generalization via hindsight trajectory sketches. arXiv preprint arXiv:2311.01977, 2023.
Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeffrey Smith, and Stan Birchfield. HANDAL: A dataset of woal-world manipulable object categories with pose annotations, affordances, and reconstructions. In IROS, 2023.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Pciyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint arXiv:2501.12949 2025.
Yining Hong. Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36:20482-2044 2023.
12

---

Published as a conference paper at ICLR 2026
Chi-Pin Huang, Yuch-Hua Wu, Min-Hung Chen, Yu-Chiang Frank Wang, and Fu-En Yang. Thinkact: laooonllanagg-aattoon reasoning via reinforced visual latent planning. arXiv preprint arXiv:2507.16815, 2025.
Siyuan Huang, Haonan Chang, Yuhan Liu, Yimeng Zhu, Hao Dong, Peng Gao, Abdeslam Boularias, and Hongsheng Li. A3vIm: Actionable articulation-awe aware vision language model. arXiv preprint 17Xi::2406.07549 2024a.
Siyuan Huang, laroslav Ponomarenko, Zhengkai Jiang, Xiaoqi Li, Xiaobin Hu, Peng Gao, Hongsheng Li, and Hao Dong. Manipvqa: Injecting robotic affordance and physically grounded information into modti large language models. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7587. IEEE, 2024b.
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. Voxposer: Composable 3d value maps for robotic manipulation with language models. arXiv preprint arXiv:2307.05973, 2023.
Wenlong Huang, Chen Wang. Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio-temporal rea- soning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652, 2024c.
Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, Xinda Xue, Qinghang Su, Huaihai Lyu, Xiaolong Zheng, Jiaming Liu, Zhongyuan Wang, and Shanghang Zhang. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. arXiv preprint arXiv:200..22257, 2025.
Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024. URL https: /tttps://axiv... wrg.orrg//ao.o.o. 22b0/ 06770.
Amita Kamath, Jack Hessel, and Kai-Wei Chang. What's "up" with vision-language models? /bs/
investigating their struggle with spatial reasoning, 2023. URL 2310. 19785.
Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410. 11831, 2024.
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. ReferltGame: Referring to objects in photographs of natural scenes. In Alessandro Moschitti, Bo Pang. and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 787-798, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.31 10.3115//1/144-1000 1086. URL /eattss//aclll /tt.:://///aaaattoooooo.ooo/////
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source laiion-anguage-action model. arXiv preprint arXiv:2406. 09246, 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-actiin models: Optimizing speed and success. arXiv preprint (XXi:25002.19455 2025.
Kimi-Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501.12550 2025.
Chengzu Li, Caiqi Zhang, Han Zhou, Nigel Collier, Anna Korhonen, and Ivan Vulić. Topviewrs: Vision-language models as top-view spatial reasoners. arXiv preprint TTT:22406.00537,, 2024a.
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sich, Sean Kirmani, et al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint 20X5v:2405.05941 2024b.
13

---

Published as a conference paper at ICLR 2026
Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Raymond Yu, Caelan Reed Garrett, Fabio Ramos, Dieter Fox, Anqi Li, et al. Hamster: Hierarchical action models for ppen-world robot manipulation. arXiv preprint 25X.5:250..05485, 2025.
Yicong Li, Na Zhao, Junbin Xiao, Chun Feng, Xiang Wang, and Tat-seng Chua. Laso: Language- guided affordance segmentation on 3d object. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14251-14260, 2024c.
Fanqi Lin, Yingdong Hu, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation. arXiv preprint vrXiv:2410.18647, 2024.
Fangehen Liu, Kuan Fang, Pieter Abbeel, and Sergey Levine. Moka: Open-worid robotic manipula- tion through mark-based visual prompting, 2024a. URL htt.::pp///xxxv..rrggoo/o/ 22003. 03174.
Jijia Liu, Feng Gao, Bingwen Wei, Xinlei Chen, Qingmin Liao, Yi Wu, Chao Yu, and Yu Wang. What can rl bring to vla generalization? an empirical study. arXiv preprint arXiv:2505. 19.5/9990 19789, 2025a.
Jinyi Liu, Yifu Yuan, Jianye Hao, Fei Ni, Lingzhi Fu, Yibin Chen, and Yan Zheng. Enhancing robotic manipulation with ai feedback from multimodal large language models. arXiv preprint arXiv:2402. 14245, 2024b.
Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, al. Spatialcot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning. arXiv preprint arXiv:2501. 10074 2025b.
Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprini arXiv:2503.06520. 2025c.
Yuqi Liu, Tianyuan Qu, Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, and Jiaya Jia. Vision- reasoner: Unified visual perception and reasoning via reinforcement learning, arXiv preprint arXiv:2505.12081 12081, 2025d.
Guanxing Lu, Wenkai Guo, Chubin Zhang, Yuheng Zhou, Haonan Jiang, Zifeng Gao, Yansong Tang, and Ziwei Wang. Vla-rl: Towards masterful and general robotic manipulation with scalable reinforcement learning. arXiv preprint 2tt.:22555118199 2025.
Yuhao Lu, Yixuan Fan, Beixing Deng, Fangfu Liu, Yali Li, and Shengjin Wang. VI-grasp: a 6-dof interactive grasp policy for language-oriented objects in cluttered indoor scenes. In 2023 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 976-983. IEEE, 2023.
Austin Myers, Ching Teo, Cornelia Fermüller, and Yiannis Aloimonos. Affordance detection of tool parts from geometric features. In 2015 IEEE international conference on robotics and automation (ICRA), pp. 1374-1381. IEEE, 2015.
Fei Ni, Jianye Hao, Shiguang Wu, Longxin Kou, Yifu Yuan, Zibin Dong, Jinyi Liu, MingZhi Li, Yuzheng Zhuang, and Yan Zheng. Peria: Perceive, reason, imagine, act via holistic language and vision planning for manipulation. Advances in Neural Information Processing Systems, 37: 17541-17571, 2024.
Fei Ni, Min Zhang, Pengyi Li, Yifu Yuan, Lingfeng Zhang, Yuecheng Liu, Peilong Han, Longxin Kou, Shaojin Ma, Jinbin Qiao, et al. Embodied arena: A comprehensive, unified, and evolving evaluation platform for embodied ai. arXiv preprint arXiv:2509. 15273, 2025.
Jake O'Neill, Abraham Arthurs, Fâbio Avila Belbute-Peres. Julian Balaguer, Sarah Bechtle, Gemma Bidoia, Kyle Burden, Erwin Chang, Sheila Chen, Todor Davchev, et al. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864. 2023.
OpenAl. Openai ol system card,
URL /
Jinyoung Park, Jechye Na, Jinyoung Kim, and Hyunwoo Kim. Deepvideo-rl: Video reinforcement fine-tuning via difficulty-aware regressive grpo. arXiv preprint arXiv:2506.07664, 2025.
14

---

Published as a conference paper at ICLR 2026
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision ssioo-language-action models, 2025. URL https:// arxia. httpg///bb//// /b0..00..
Zekun Qi. Wenyao Zhang, Yufei Ding, Rumpei Dong, Xinqiang Yu, Jingwen Li, Lingyun Xu, Baoyu Li, Xialin He, Guofan Fan, et al. Sofar: Languago- grounded orientation bridges spatial reasoning and object manipulation. arXiv preprint 2rXhv:2502.13145 2025.
Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, et al. Sat Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao. Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.141555 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: arXiv: 1707.06347, 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.033000 2024.
Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. arXiv preprint arXiv:4411.16337, 2024.
Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis, Alexander Millane, Helen Oleynikova, Ankur Handa, Fabio Ramos, et al. Curobo: Parallelized collision- robot motion generation. 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 8112-8119, IEEE, 2023.
Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Xiansheng Chen, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang, Reason-rft: Reinforcement fine-tuming for visual reasoning of vision language models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Xiansheng Chen, Minglan Lin, Zhiyu Li. Zhou Cao, Pengwei Wang, Enshen Zhou, et al. Robobrain 2.0 technical report. arXiv preprint arXiv:2507.02029, 2025.
Octo Team, RT-X Team, Anthony Brohan, Noah Brown, Lauren Chen, Michael Cheng, Krzysztof Choromanski, Eamonn Cullina, Gabe Dalal, Chelsea Fu, Florian Golemo, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2403 10164, 2024.
Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal Ilms, 2024.
Homer Rich Walke, Kevin Black, Tony Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning, pp. 1723-1736. PMLR, 2023.
Hanqing Wang. Shaoyang Wang, Yiming Zhong, Zemin Yang, Jiamin Wang. Zhiqing Cui, Jiahao Yuan, Yifan Han, Mingyu Liu, and Yuexin Ma. Affordance-rl: Reinforcement learning for general- izable affordance reasoning in multimodal large language model. arXiv preprint Wtt:25508.06200 2025a.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of languaee models with reinforcement learning, 2025b. URL http://awwxv.......oooa/e
fitps://arre org/ 2504 08837.
15

---

Published as a conference paper at ICLR 2026
Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-seeing project v2: Towards general relation comprehension of the open world. In European Conference on Computer Vision, pp. 471-490. Springer, 2025c.
Chuan Wen, Xingyu Lin, John So, Kai Chen, Qi Dou, Yang Gao, and Pieter Abbeel. Any-point trajectory modeling for policy learning. arXiv preprint arXiv:2401.00025, 2023.
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: convolutional neural network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199, 2017.
Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gordon Wetzstein, Manuela Veloso, and Shuran Song. Flow as the cross- coss-domain manipulation interface. arXiv preprint arX 2024.
Rongtao Xu, Jian Zhang, Minghao Guo. Youpeng Wen, Haoting Yang, Min Lin, Jianzheng Huang, Zhe Li, Kaidong Zhang, Liqiong Wang, et al. A0: An affordance-awarc hierarchical model for general robotic manipulation. arXiv preprint arXiv:2504. /2636, 2025.
Jianwei Yang, Reuben Tan, Qianhui Wu, Ruijie Zheng, Baolin Peng. Yongyuan Liang. Yu Gu, Mu Cai, Seonghyeon Ye, Joel Jang, et al. Magma: A foundation model for multimodal ai agents. arXiv preprint arXiv:2502. 13130, 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source IIm reinforcement learning system at scale. arXiv preprint arXiv:2503.
2025.
Chengbo Yuan, Chuan Wen, Tong Zhang, and Yang Gao. General flow as foundation affordance for scalable robot learning. arXiv preprint arXiv:2401.. 1439, 2024a.
Wentao Yuan, Jiafei Duan, Valts Blukis, Wilbert Pumacay, Ranjay Krishna, Adithyavairavan Murali, Arsalan Mousavian, and Dieter Fox. Robopoint: A dision-languaee model for spatial affordance prediction for robotics. arXiv preprint arXiv:2406. 10721, 2024b.
Yifu Yuan, Jianye Hao, Yi Ma. Zibin Dong, Hebin Liang, Jinyi Liu, Zhixin Feng, Kai Zhao, and Yan Zheng. Uni-rlhf: Universal platform and benchmark suite for reinforcement learning with diverse human feedback. arXiv preprint arXiv:2402.0244 2024c.
Yifu Yuan, Haiqin Cui, Yibin Chen, Zibin Dong, Fei Ni, Longxin Kou, Jinyi Liu, Pengyi Li, Yan Zheng, and Jianye Hao. From seeing to doing: Bridging reasoning and decision for robotic manipulation, 2025. https:/ htt.tttpss//aaii...... org/ /2a05///555... 08548.
Michat Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied tffin-of- reasoning. arXiv preprint arXiv:2407.00 240.06907,, 2024.
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma. Song Han, Chelsea Finn, al. Cot-vla: Visual chain-of-thougle reasoning for Assion-language-actio models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1702-1713 2025.
Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé III, Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint 2wxiv:2412.10344 2024.
Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. arXiv preprint arXiv:2506.044: 04308, 2025a.
Enshen Zhou, Cheng Chi, Yibo Li, Jingkun An, Jiayuan Zhang, Shanyu Rong, Yi Han, Yuheng Ji, Mengzhen Liu, Pengwei Wang, et al. Robotracer: Mastering spatial trace with reasoning in vision-language models for robotics. arXiv preprint 2xXi:25112113660 2025b.
16

---

Published as a conference paper at ICLR 2026
A AUTOMATIC DATA GENERATION PIPELINE
In this section, we provide additional explanations regarding the generation of certain datasets. The generation processes of both the RRG and VTG datasets are improved based on Yuan et al. (2025).
3D RRG Data Generation using Isaac Gym Simulation Furthermore, we leveraged the Isaac Gym simulation engine to generate a synthetic dataset of 3D object rearrangements, equipping the model with 3D spatial awareness. In this task, which takes RGB-D images as input, the model is required to place objects in the correct relative positions according to instructions. Task success is automatically determined and fed back by the simulation based on the true physical state. The dataset comprises 10,028 tasks, each situated in a tabletop scene containing multiple objects. The input for each task consists of processed RGB and depth images, accompanied by a language instruction describing the desired target position of an object within the scene (c.g., "place the cup between the book and the spoon"). Based on these instructions, the model is required to output the 3D position of the target object in the camera coordinate system, specified by its pixel coordinates (X, Y) and depth value D in millimeters. The D value is obtained either through monocular depth estimation or by reasoning from the scene geometry. The generation process of our dataset informed by the methodology of Open6Dor Qi et al. (2025). The object set utilized contains over 200 items spanning more than 70 distinct categories, originally sourced from the YCB Xiang al. (2017) and Objaverse-XI XI. Deitke et al. (2023) datasets. These objects underwent rigorous selection process to ensure their physical integrity and semantic suitability for tabletop arrangements. All selected objects were subsequently ccale-normalize and uniformly represented using consistent mesh format.
In terms of scene configuration, between two and five objects were randomly selected from the object set and placed on a tabletop with random initial poses. For each configured scene, we rendered both RGB and depth images. The depth values represent ground truth measurements, with the scene's s depth range spanning from 600 mm to 1700 mm. For subsequent processing convenience, the depth images were normalized to an 8-bit grayscale format (0-255). We filtered out low-quality scenes, such as those exhibiting implausible object placements severe occlusions. To augment the dataset's s quality and volume, we expanded a subset of high-quality, filtered data by algorithmically generating variations in task descriptions, such as substituting directional prepositions altering object relations. Then, the task instructions are formulated in two primary types: basic directional commands (e.g.. left, right, top, behind, front) and relational commands (e.g.. "between," "center of"). All instructions adhere to a standardized template, for instance, "Place object A in front of object B," where A and B are objects present in the scene. During training, the model receives RGB and depth image inputs and is required to output the target's 's coordinates (X, Y) and depth value D. The simulated environment then executes and evaluates the predicted position, giving positive rewards for correct predictions.
VTG Dataset Generation Pipeline For each video sequence, we first process the initial frame to perform instance segmentation on the manipulated object, thereby obtaining its pixel-wise mask. Instead of relying on a single tracking point, which is susceptible to tracking failure from occlusion or rapid motion, we sample a set of three distinct query points from within this mask. This multi-point initialization serves
redundancy measure, significantly enhancing the robustness of the tracking process. These points are strategically chosen to represent the object's initial state before they are passed, along with the full vidco sequence, to the tracking model for trajectory prediction. The core of the trajectory generation is handled by the CoTracker model Karaev et al. (2024), which takes the initialized query points and video input. The model concurrently tracks each point throughout the sequence, yielding a set of three candidate trajectories. As these trajectories may vary in quality and completeness due transient tracking errors, a selection criterion is required to identify the single most representative path. We employ a path length heuristic for this purpose, calculating the total Euclidean distance of each trajectory. The trajectory with the longest path selected as the definitive motion path for the object. The rationale behind this criterion is that the longest trajectory is most likely to have successfully tracked the object through its entire course of motion without premature termination. Following the selection of the representative trajectory, two-stage refinement process is applied to produce the final visual trace. The raw trajectory, composed of discrete frame-by-frane frramm coordinates, is first smoothed using cubic spline interpolation. This step transforms the discrete points into a continuous, smooth curve, effectively filtering out high-frequence noise and jitter inherent in the tracking process. From this smoothed curve, then uniformly sample eight equidistant points. This final set of eight points constitutes the visual trace- a structured, discretized representation of the object's motion, suitable for downstream analysis and model consumption.
17

---

Published as a conference paper at ICLR 2026
We found that this process inevitably faces prediction errors from pre-trained visual models, such as incorrect object grounding or incomplete motion trajectory tracking. To mitigate these issues, we employ stringent rule-based filtering methods using hyperparameters such as size thresholds and trajectory length thresholds. Before annotating each dataset, we iteratively adjust these rules and conduct manual sampling inspections. Only when the filtering rules robust enough do apply them to the full data generation pipeline, ensuring the high quality of the data.
B IMPLEMENTATION DETAILS OF EMBODIED-RI
Training Hyperparameters: We conducted model training on eight NVIDIA A100 40G GPUs. The first phase was trained for epochs, and the second phase for epoch, with each phase taking approximately 48 hours. The backbone model used Swwn2.5-VLL33B-mmm
with a maximum context length of 4096 and maximum response length of 2048. The optimizer selected is AdamW, with learning rate le-6 and a weight decay coefficient le-2. Embodied-R1, we performed reinforcement learning training based GRPO Shao et al. (2024), set the number of samples 8, and introduced a KI. penalty (coefficient le-2), with a global batch size of 128 for each step. For all experiments, we focus comparing SFT models trained with the same batch size and data, which we refer to as Embodied-SFT. As for Embodied-SFT,, we used exactly the same data but trained with supervised learning loss, kept the batch size at 128, and trained for epochs. The Embodied-RI model is trained entirely based on the EasyR1 open-source codebase".
Reward Hyperparameters: To enable stable multi-task training, we constrain the total reward for each task to the range of O to and define the total reward R. as weighted combination R Erer W/ r. Each task utilizes different combination of reward terms. Here, we provide the hyperparameters for each task in the Tab. 8. We would like to add two clarifying points: First, the task output fails to meet the required parsing format, subsequent analysis cannot proceed successfully, the reward is set directly to O. Second, for the VTG task, we introduced additional constraint on the format: the generated visual trace must consist of exactly 8 points. In practice, we found that without this constraint, the model in the VTG task was prone reward hacking behavior. would tend to output only two points to form a straight line, which easily yields a high reward and prematurely terminates exploration. By enforcing the 8-point constraint, compel the model to perform more complex curve interpolation, thereby improving the performance of visual trace generation. We provide a detailed comparison in the App. F.
Table 8: Detailed Reward Functions for Each Task
Task
Reward Function R
General QA
R 0.1 Fformat + 0.9 Face.
Spatial QA
R 0.1 Fformat + 0.9 Facc
REG
R = 0.1 Fformat + 0.9 T'aak
RRG
0. Fformat
+ 0.6 Tmaak
+ 0.3 rdis
3D RRG
R = 0.1 Flormat
+ 0.9 Teav
OAG
0.1 Fformaa
+ 0.8 Tmask
+ 0.1 rdis
VTG
R = 0.1 Florat
+ 0.9 Ptrace
'https://hugging: coee.co//e coo/wwen//vert 5- rust Phttps://gi thub. com/h youga
18

---

Published as a conference paper at ICLR 2026
C
EMBODIED-RE RI PROMPTS FOR EACH TASK
Referring Expression Grounding (REG) Prompt
Provide one more points coordinate of object region this sentence describes: Your In- struction. The results are presented in format <point>[[x1,yI, [x2.y2], ...]//oint>>. You FIRST think about the reasoning process final answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags. The answer consists only of several coordinate points, with the overall format being: <think> reasoning process here </thnn>>casser><poonn>> yl], [x2, y2], ...K/poont>///ssse>>>>>>>
internal monologue and then provide the
Region Referring Grounding (RRG) Prompt
You are currently a robot performing robotic manipulation tasks. The task instruction is: Your Instruction. Use 2D points to mark the target location where the object you need to manipulate in the task should ultimately be moved. You FIRST think about the reasoning process an internal monologue and then provide the final answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags. The answer consists only several coordinate points, with the overall format being: <think> reasoning process here wtthink>-answer<<pponttt/ yl]. [x2, y2], ... //ppooitt///sswee.>..
Object Functional Grounding (OFG) Prompt
Please provide the 2D points coordinates of region this sentence describes: Your In- struction. The results are presented in format <point>[[x1.II], [x2.y2], //.o//tint>. You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process and answer enclosed within <think> </think> and <answer> </answer> tags. The answer consists only of several coordinate points, with the overall format being: <think> reasoning process here <think>>answrr<cpinb///l yl], [x2, y2), ...///poit><///ssve>>>>>
Visual Trace Generation (VTG) Prompt
You currently robot performing robotic manipulation tasks. The task instruction is: Your Instruction. Use 2D points to mark the manipulated object-centric waypoints to guide the robot to successfully complete the task. You must provide the points in the order of the trajectory, and the number points must 8. You FIRST think about the reasoning process an internal monologue and then provide the final answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags. The answer consists only of several coordinate points, with the overall format being: <think> reasoning process here
y1], [x2, y2]. [x8, y8]]</point><<answer
D IN-DEPTH ANALYSIS BASED ON
EXPERIMENTS
Table 9: Real-world Exccution Time (s)
Task
Embodied-RI
Open VLA (FT)
MOKA
FSD
Move the sponge
10s
23s
28s
14s
Pick the cucumber
8s
12s
18s
10s
Real- Real-World Execution Latency. We benchmarked the real-world execution latency of Embodied-Ri against various model archetypes. including the end-to-end Open VLA (O'Neill et al., 2023), the modular MOKA (Liu et al., 2024a), and the affordance- based FSD (Yuan et al., 2025), with results reported in Tab. 9. Although Fmbodied-R1 employs a teesee-theneexecute process, it achieves the
19

---

Published as a conference paper at ICLR 2026
fastest performance. This efficiency stems from its single-pass inference approach, where reasoning is performed only once before execution begins. In contrast, Open VLA requires step-by-step inference and mandatory fine-tuning for new tasks, while modular methods like MOKA incur substantial system overhead due to their multi-componet pipelines. Furthermore, among reasoning bassoning-based models, the 3B-param paraameteer Embodied-R outperforms the 133--prrmmter FSD, demonstrating superior inference efficiency due to its smaller size. Therefore, Embodied-RI strikes an effective balance between powerful zero-shot manipulation capabilities and low real-world latency.
Comparison with the AO Model. We compare Embodied-RI to the AO (Xu et al., 2025) model, which also generates object-centric visual traces but is trained exclusively through Supervised Fine- Tuning (SFT) and lacks reasoning process. In our gero-shot generalization experiments, we found that despite extensive tuning efforts, AO consistently failed generate reliable trajectories, resulting in success rate approaching zero. As shown in Fig. 7, we present a side-by-side visual comparison of the trajectories generated by Embodied-RR RI and AO our real-world robot experiments.
Embodied-
Embodied-4
Embodied-H
iponge andijilace
outside plote.
thbraa bod lice it the bosket det to side of the drower
Pick sponge and place it outside
Pick the green toothbrush and hasket
Move the mearest object to the right side of the drawer
Figure 7: Comparison of Visual Trace Generation between Embodied-Ri and AO Models in Real-World Experiments
Failure Case Analysis. Here, provide analysis of the failure modes for the results in Tab. 5. The MOKA pipeline exhibited low success rates across the majority of tasks. Its failures were often due cascade of errors originating from multiple stages: incorrect object recognition, failure to select the correct keypoints, or erroneous reasoning from the GPT module, which collectively led to extremely poor performance. Furthermore, even when MOKA did succeed in grasping an object, struggled with accurate placement. For RoboPoint and FSD, the core issue was their extremely low success rate on tasks that require reasoning, such as "find the nearest object" or "pick the toothbrush of the correct color." Additionally, these models suffered from limited pointing accuracy. leading to low success rates when attempting to grasp challenging objects. In contrast, Embodied-BR R1 possesses robust spatial reasoning and precise pointing capabilities, allowing it to perform exceptionally well on most tasks and overcome the limitations of the other methods.
E MORE VISUALIZATIONS
Below, we provide visualizations of the real-robot execution process and the VTG tasks. For additional video materials, please
project website: https: / wwwodiedddii github. io/
More Visualization of Real World Manipulation Process We showcase the process Embodied- R1 performing cal-world tasks in Fig. 8. To further evaluate its robustness, visualize the process of Embodied-RI performing Task 6 under different visual disturbances in Fig. 9.
More Visualization of VTG Task We provide additional visualization examples of Embodied-F R1's predicted visual traces in Fig. 10. It can be seen that Embodied-RR R1 achieves accurate visual trajectory prediction across various scenarios.
20

---

Published as a conference paper at ICLR 2026
Task 1: Pick up the strawberry
Task 2: Move egg to the bowl
Task 3: Move the vise the red basket
Task Place the fork in the green bin
Task 5: Pick the [x] toothbrush and place it in the basket
Task Move the nearest object" to the right side of the drawer
Task 7: Put the screwdriver between drawer and the vase
Task 8: Move the moka pot to the right of drawer
Figure 8: The process of Embodied-RI performing real-world tasks.
F ADDITIONAL EXPERIMENTS
The Phenomenon of Reward Hacking in VTG Tasks We carefully designed the reward function so that the reward for each task is only related to the final goal, thereby avoiding reward hacking caused by intermediate rewards. However, we found that in the VTG task, designing the reward solely based on the distance between the predicted trajectory and the target trajectory can still result in reward hacking. The model quickly learned that in visual trajectory generation tasks, accurately predicting the starting and ending points is both crucial and relatively easy, leading it to converge rapidly to outputs with only these two points while ignoring the generation of intermediate trajectory points. We observed that by forcing the model to output multiple points and applying reward constraints for format reward, it becomes possible to generate complete visual traces. Therefore, we explicitly require the model to output visual trace with eight points; otherwise, all rewards
21

---

Published as a conference paper at ICLR 2026
Original Setting
Background Change
Background Light Change
Lightt Change
Figure 9; The process of Embodied-Rl performing Task 6 under different visual disturbances.
move brown chip
jelepeno
Pare the the
the burger
Ul
Put the blue block on orange plate
Put the orange oboect on the pan
the Blus mar the yellow mug
lomador funm sisth la tablie lecoon the thevaaaar
put the red thing inside the sheee conbxiner
Figure 10: Visualizing Embodicd-RRt Prediction on VTG Tasks across Various Scenarios
(since the format reward is satisfied, subsequent analysis will be performed). As shown in Tab. 10, we present the results on VABench-V, demonstrating the performance differences with and without the trajectory point number constraint. After modifying the reward function, the model is better able to fit the visual trace, achieving lower RMSE and higher GPT score.
Table 10: Comparasion of w/ and w/o Point Num Reward, Bolds are better.
VABench- VisualTrrac
RMSEL
MAE]
GPT Scoret
w/ Point Num Constraint
77.83
44.97
7.27
w/o Point Num Constraint
105.2
59.7
5.57
Qualitative Comparison: Embodied-R1 vs. SFT To provide deeper insight into the performance gains of our model, we conducted a qualitative analysis comparing Embodied-RI with the SFT baseline. As illustrated in Fig. I1, the difference in capabilities is stark. In the first task, "move the orange toy inside the right sink in the bin," Embodied-R R1 first articulates clear plan: identifies the toy's initial position, determines the need to move it towards the sink's 's center, and then guides it into the bin. This logical reasoning translates into a precise and successful visual trajectory. The SFT baseline, however, produces erroneous trajectory that fails to place the toy correctly. Similarly,
22

---

Published as a conference paper at ICLR 2026
for the second task, "place blue chip bag into white bowl," Embodied-R1 correctly reasons that the task requires moving to the bag, lifting it, positioning it over the bowl, and then lowering it. This step-by-step plan underpins the generated trajectory, which successfully completes the task. The SFT model again fails, generating a flawed path that does not accomplish the goal. These examples strongly suggest that the explicit ohain-of-thought reasoning in Embodied-RI is critical for understanding complex instructions and planning accurate, multi-stage visual trajectories, capability that the standard SFT approach evidently lacks.
aerer eesaanaee ae re pe woweeee coreeea co o a g g Task: move
Thinking: To move orange inside night the
loy, inside the right sink
first identify
the
position ralative to the
The orange toy
the edge the
moying towards the sink, Thoy aped guide into the bin. this, use reference points to ensure accurate movement.
of the
S edges and comers
Emilodied ind-r
Task: place blue chip
T Tiinking To white bowl, I need to first move towanis the blue chip bag. Then, should lift position Il
the blue chip bag into the
bag inlo while bowl
and
the bowl. Finally, I'll lower the
blue chip bag into the bow). This the bowl.
will
that the hlue chip bag placed inside
Figure 11: Qualitative comparison of Embodied-R1 and the SFT baseline. Our model Embodied-R11 1, leverages chain-of-thouge reasoning (middle column) to generate a successful visual trace (left column). In contrast, the SFT baseline, which lacks an explicit reasoning process, produces incorrect trajectories (right column) for the same tasks.
Table 11: Comparison of Mixed Training of Multiple Datasets and Only Training Corresponding Dataset.
Mixed Training
Unmixed Training
Part-Afford 56.63
51.25
Where2Place 69.50
65.50
VABench-P 66.00
65.39
Advantage of Mixed Training We performed multi-task joint training of the various abilities required for embodied pointing and reasoning in the second stage. The advantage of this approach is that all abilities can share general knowledge of point coordinates and semantic space alignment during training, thereby achieving better performance. To validate this idea, we conducted multiple sets of experiments comparing the performance of mixed training and unmixed training. In the unmixed training setting, only the data corresponding to the benchmark ability is used; for example, training for one epoch only on the HandAL dataset and then testing on the Part-Afford dataset. As shown in Tab. 1 1. joint training consistently improves the success rate across multiple tasks compared to unmixed training. We believe that mixed training enables knowledge sharing among multiple abilities, enhances semantic understanding of spatial information, and thus leads to better generalization.
Case Analysis of Embodied-RI As shown in Fig. 12, we demonstrated the reasoning pathways of Embodied-RI when facing different tasks. Even without any SFT stage, Embodied-RL exhibited a human-like and rational reasoning process: it first infers the target object to focus on based on the task goal, then analyzes the relative spatial relationship between the object and the surrounding environment, and subsequently performs step-by-step reasoning to determine the target location (in RRG and VTG tasks), strictly adhering to a structured process of reasoning before providing the final answer. In some cases, Embodied-R R1 still demonstrates clear and accurate reasoning process even when confronted with cluttered scenarios.
Integration with Baariing-Based Pipeline. To demonstrate the potential of Embodied-Ri in dynamic settings, we significantly expanded our evaluation to the full LIBERO benchmark suites (Spatial, Object, Goal, and Long, a total of 40 tasks). Each task has 50 demonstrations. We benchmarked the integration of Embodied-RP R1 with Diffusion Policy (Chi et al., 2023) based on the CleanDiffuser (Dong al., 2024a) codebase, where Embodied-RI serves planner. Specifically, it synthesizes a complete visual trace based on the initial observation and language instruction, which serves dense spatiotemporal condition to guide the downstream
efficient conditional
23

---

Published as a conference paper at ICLR 2026
Q: identify some points within the free space that beside the glass and the left of the weights.
<thinksTo identify points the space dumbbells leyout
glass cup the of weights, cup placed
on the table.
question asks points The space could entrr surface of conswer>-pointt([3. 298], (346, 297l]//polnt><//nsst/
between objects. However, identifying areas where objects
space, sshuld or any paps between the objects. </think
present.
Q: Provide points coordinate objects region white electric toothbrush.
desoribes:
to pick up the
<thinke he white electric toothbrush is located on the left side of the sink, closer to the edge of the counter. appears to be lying horizontally on the countortop. </thini>
(1eserrspooon)/115,, 304), [158, 307]s//poin></onsserr
marker the pot. Use points mark the manipulated object centric waypoints to guide the robot to successfully complete the task.
<think-To To put the marker in the pot, need first locate the marker and the pot. The marker is likely the table nearby, while the pot is on the table as well. will move towards the marker move from marker's location the
then position Inside pot. This requires pot's location < </think>
conswer--goint--ll6e 300), (654, 2871, (639, 2601, [612, 2401, (598, [38), 250), (586, 280), [S89, 2980)s/point><onsser
Figure 12: Case Analysis: Embodied-RI possesses emhodied reasoning capabilities. It can progressively locate relevant objects and infer spatial relationships according to task instructions, and ultimately provide coordinates through pointing based embadied scene analysis.
policy. Since LIBERO-Long involves the composition of tasks in two stages, the agent queries the Embodied-Ri model every 15 steps to update the visual trace. As detailed in Tab. 12, this integration yields substantial performance uplift, boosting the average success rate from 64.4% (DP Baseline) to 79.4% (DP + Embodied-R11 11,, achieving an improvement of 15%
We attribute this gain to the nature of the guidance: compared to the sparse, goal-oriented signals from language instructions, the visual trace provides explicit, dense path constraints. This helps the policy disambiguate instructions and maintain long-horizon consistency, validating that Embodied-RI can effectively bridge high-level reasoning with low-level learning-based control.
Table 12: Performance on the full LIBERO benchmark suite. Integration with Embodied RI yields significant improvements across all task categories compared to the DP baseline.
LIBERO Goal
LIBERO Spatial
LIBERO Object
LIBERO Long
Avg
Baseline
64.3
76.
71.4
45.9
64.4
DP+ Embodied-RI
80.7
89.6
78.7
68.7
79.4 (115%
3D RRG Capability of Embodied-R1 To explore our framework's capabilities to 3D spatial rea- soning, we developed an RGB-D variant, named Babbodicd-RIRRGBI utilizing synthetic paired data from YCB (Xiang et al., 2017) and ObjaverseXL (Deitke et al., 2023). Addressing the chal- lenge of modal interference often found in direct concatenation, we implemented a Dual-Tower architecture (Zhou al., 2025a). In this design, independent visual encoders process the RGB and depth inputs separately, ensuring that geometric depth cues are distinctively preserved before fusion. We further devised two-stage training strategy to ensure precise 3D understanding: (1) Modality Alignment: We first conducted SFT on the large-scale RefSpatial (Zhou et al., 2025a) dataset (IM samples) align the depth encoder's feature representation with the embedding space of LLM; (2) Reasoning Fine-Tuue Tuning (RFT): We then performed full call-parameter fine-tuning to optimize the model specifically for 3DD RRG tasks. Evaluation on the Dpen6DOR-Positieee
benchmark (Qi et al., 2025) demonstrates the effectiveness of this approach. As shown in Tab. 13, Embodied-RR -e-oddied-1--GBBDD achieves an overall accuracy of 94.3%. Notably, it excels in handling complex spatial relations (Level 1), scoring 74.5%, and achieves 99.29 on basic tasks (Level 0). These results confirm that our designs enable the feasibility of 3D RRG tasks, thereby validating the effectiveness of our paradigm's 's scalability to 3D. We believe that further design and adaptation are still required to extend the pointing paradigm to more 3D tasks and broader scenarios, but its future feasibility is highly promising. This will constitute key direction for our subsequent research.
24

---

Published as a conference paper at ICLR 2026
Table 13: Performance Open6DOR-PPosition Benchmark
Benchmark
Level0
Level!
Overall
GPT-4V
46.8
39.1
45.2
Qwen2.5-VL Bai et al. (2025b)
59.5
36.2
54.9
VoxPoser Huang al. (2023)
35.6
21.7
32.6
SoFar Qi (2025)
96.0
81.5
93.0
Embodied SFT
62,4
44.7
58.9
GimbodiedRR1-RGG
68.5
59.4
66.8
Rimbodied-R1RGBBl
99.2
74.5
94.3
G LIMITATION AND FUTURE WORK
Despite the state-of-the-ar performance achieved by Embodied-R1 across numerous benchmarks and real-world tasks, this work has several limitations that present avenues for future research.
Potential for Integration with Learning-based Policies. Our current approach primarily pairs the perception and reasoning capabilities of Embodied-R( with a classical motion planner. A promising future direction conditional policy, which promises to enhance execution efficiency and reactivity in dynamic environments. While several studies Bharadhwaj al. (2024); Gu al. (2023); Xu et al. (2024) have explored conditioning policies on visual traces to improve performance, these works focus on downstream policy design and do not provide a general-purpont visual trace predictor. preliminary experiments in App. F where simple integration with Diffusion Policy conditions the strategy on visual traces. Notably, Embodied-FR -1 improves performance on LIBERO tasks without any additional fine-tuning. Therefore, this integration path holds significant potential and warrants further exploration.
integrate the model high-level front-end for a learning-bass
present
Untapped Potential in Long-horizon Tasks. The current framework is designed to process single-step instructions and does not natively include a mechanism for decomposing long-horizon commands (e.g., "prepare meal"). However, this could be addressed through modular, hierar- chical approach. Embodied-R R1 well-suited to act as a robust execution module for individual sub-tasks. A high-level embodied planner could first decompose complex instruction into sequence of simpler steps, which would then be passed to Embodied-ER R1 for execution, enabling the system tackle complex, multi-stage problems.
Inherent Limitations of the "Pointing" Representation. While the pointing representation is effective for localization, placement, and trajectory generation, it may be insufficient for the full spectrum complex robotic manipulation. Tasks requiring precise force control, twisting, wiping, or intricate interactions with deformable objects demand richer representation than 2D coordinate points. We believe this issue can be mitigated by coupling the high-level "pointing" commands with Iearnable downstream policy that can translate these targets into complex, dynamic actions. The design for Embodied-R 1 reflects our primary focus on providing a promising solution for zero-shot generalization, for which simplified, embodiment-agnootic intermediate representation is key advantage.
Preliminary Integration of 3D Information. The exploration of RGB-D version of the model is still its early stages. The paper notes that in tasks with complex spatial relations, the performance of the RGB-D variant can be slightly lower than its 2D counterpart. It is hypothesized that "depth map understanding may be more prone to hallucinations," indicating that robustly fusing 3D information into the model requires further development (Zhou al., 2025b).
Physical Common-Sense Alignment. Another highly promising direction for future work is to mitigate physical common hallucinations in embodied reasoning, such as proposing physically impossible interactions or ignoring fundamental physical laws like gravity. We plan to adopt Reinforeement Learning from Human Feedback (RLHF) (Dong et al., 2024b; Yuan et al., 2024c; Liu et al., 2024b) applied to the intermediate thinking process of the model, forcing the model's cognitive process to align with human intentions.
25

---

Published as a conference paper at ICLR 2026
H IN-DEPTH COMPARISON WITH PREVIOUS WORKS
To explicitly delincate the novelty of our "Pointing" paradigm, we provide a comprchensive compari- son with representative state-of-the-art baselines: RoboPoint (Yuan et al., 2024b), RoboBrain (Ji et al., 2025), AO (Xu et al., 2025), and FSD (Yuan et al., 2025). Detailed comparisons across representation types, capability scopes, and training methodologies are presented in Tab. 14, The fundamental distinctions are analyzed across three key dimensions below.
Table 14: Detailed Comparison of Embodied-R with Prior Point-based VLA Methods. Robot-Trace refers to robot-centric visual trace, while Obj- Trace refers to object-centric visual trace.
Feature
RoboPoint
RoboBrain
A0
FSD
Embodied-RR (Ours)
Visuol Representation
Embodinent
Visual Aids Type
Point
BBox / Robot Trace No (Robot-centrri)
Obj Trace Yes
BBox Point/ / Trccc Trace
Point Obj-Trace Yes
Embodinentt-Agan
Yes
Yes
2. Capability Scope
REG
Yes (Point)
Yos (BBox) No No Yes (Robot-Trace
No No No
Yes (BBox/Point) Yes
(Point)
RRG
Yes
Yes Yes
OFG
No
VTG
No
Yes (Obj-Trace)
Yes (caj-Trrcc)
Yes (Obj-Trace)
Merhodology Reasoning
Truining Methud
SFT
SFT
SFT Yes (Fixed CoT) Weak
SFT
RFT Yes (Free-form) Strong
Thinking Process
No
No
No
Spatial/Looical Reasoning
N/A
N/A
N/A
Unified Interface vs. Fragmented Capabilities. While geometric points are common primitive, prior methods utilize them in fragmented or restrictive ways. Specifically, AO focuses exclusively on object-centric visual trace (VTG) but lacks the semantic understanding required for grounding (REG) or functional analysis (OFG). RoboPoint limits points strictly to spatial constraints (REG, RRG). RoboBrain relies heavily on bounding boxes, which are often too coarse for precise manipulation (potentially causing operation offsets), and utilizes robot-centric traces that limit cross- cooss-embodiment transfcrability. In contrast, Embodied-R eschews coarse bounding box signals. Instead, it innova- tively unifies four core capabilities (REG, RRG, OFG, VTG) into single, embodiment-agnostie point-based interface (where visual traces are formulated as ordered sets of points). This unification allows Embodied-R1 to function a "Generalist Pointing Agent," capable of handling diverse manipulation requirements within single model architecture.
Embodied-RI acquires superior comprehension and reasoning abilities through RFT reasoning. A critical distinction lies in the acquisition of comprehension and reasoning abilities. Baselines such as FSD, RoboPoint, and AO rely primarily on SFT. Models lacking an explicit reasoning process (e.g., RoboPoint, A0) are limited to simple, explicit instructions (e.g.. "point to the right of the plate") and fail at tasks requiring multi-step deduction (e.g.. "pick up the object nearest to the robot"). While FSD incorporates reasoning, it is constrained by fixed CoT templates derived from SFT data. As evidenced in our real-world experiments in Tab. 5, FSD suffers from low success rates in tasks requiring flexible spatial understanding or logical reasoning due to this rigidity in OOD scenarios. Conversely, Embodied-RI employs RFT to develop free-form internal thinking and active spatial/logical reasoning. As shown in Fig. 6, this paradigm enables the model to robustly handle long- horizoorizon and reasoning tasks.
Multi- Task Synergy and Efficiency. By unifying these different capabilities into single "Pointing" task during the RFT stage, Embodied-R1 achieves outstanding coordinate understanding and semantic alignment with only 3 billion parameters. Significant performance improvements are observed across more than ten benchmarks, including tasks of spatial understanding and pointing. The ablation studies presented in Tab. II also demonstrate that unified multi-task modeling and learning bring positive improvements.
I USE OF LLM
We utilized LLMs as a writing assistance tool during the preparation of this manuscript. The use of LLMs strictly limited to polishing the text, which included improving grammar, refining sentence structure, and enhancing overall clarity and readability. The core research concepts, methodologies, and conclusions were developed entirely by the authors.
26


Reviews:
[{"review_id": "JCwbjhMQY5", "reviewer": "Reviewer_gZir", "summary": "This paper introduces Embodied-R1, a 3B Vision-Language Model (VLM) for general robotic manipulation. The core contribution is pioneering \"pointing\" as a unified, embodiment-agnostic intermediate representation to bridge the perception-action gap. The authors define four core embodied pointing abilities (REG, RRG, OFG, VTG) and train the model using a two-stage Reinforced Fine-tuning (RFT) curriculum on a custom large-scale dataset, Embodied-Points-200K. The RFT approach is crucial for handling the multi-solution nature of pointing tasks, which is superior to standard Supervised Fine-tuning (SFT). The model achieves state-of-the-art (SOTA) performance on spatial reasoning and strong zero-shot generalization in both simulation and real-world robot tasks.", "questions": "- 2D to 3D Trace Accuracy: The Visual Trace Generation (VTG) outputs a 2D trajectory which is then mapped to 3D using the pinhole camera model and initial depth information. How robust is this 2D-to-3D transformation process, particularly in the context of fine-grained manipulation tasks?\n- I previously tested OpenVLA on the WidowX Robot in SIMPLEREnv using the same four tasks as in the paper, achieving an average success rate of around 40%. However, the paper reports 1.0%, which is a significant difference from 40%. Therefore, I am unsure whether this reported result is accurate.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The work is original for two main reasons. First, the formal definition and systematization of four distinct embodied pointing abilities (REG, RRG, OFG, VTG) as a unified intermediate representation is a novel and intuitive approach to abstracting robot control. Second, the use of RFT specifically to address the multi-solution dilemma inherent in embodied pointing (e.g., many points are valid for an \"empty space\").\n- The paper is written clearly and is easy to follow. The problem statement (\"seeing-to-doing gap\") is well-defined, and the proposed solution using \"pointing\" is logically introduced.\n- The RFT-based, pointing-centric approach provides a new, highly effective, and scalable pathway for training VLMs for robotics, proving that a relatively smaller VLM (3B parameters) can outperform larger, specialized models by focusing on robust reasoning and generalization rather than just mimicking specific actions.", "weaknesses": "- Limited Exploration of Learning-Based Execution: The current main zero-shot robot evaluation (SIMPLEREnv and XArm) couples the VLM with a traditional CuRobo motion planner. While a promising preliminary experiment in Appendix F shows that the visual trace can boost a Diffusion Policy baseline, the core evaluation does not extensively explore this integration. Given that the paper emphasizes closing the \"seeing-to-doing gap,\" relying predominantly on a classical planner might limit the realization of the full potential of the visual trace output, especially for complex or dynamic actions where a learned policy excels.\n- Generalization Scope: The paper acknowledges that the \"pointing\" representation may be insufficient for complex manipulation requiring precise force control, wiping, or interaction with deformable objects. This limits the generality claim for the full spectrum of robotic manipulation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i1ADui3YL0", "reviewer": "Reviewer_uNYS", "summary": "This paper introduces Embodied-R1, a 3B Vision-Language Model (VLM) designed to bridge the \"seeing-to-doing gap\" in robotics, a problem it attributes to data scarcity and embodiment heterogeneity. The authors pioneer \"pointing\" as a unified, embodiment-agnostic intermediate representation to connect high-level comprehension with low-level actions. They define four core embodied pointing abilities—Referring Expression Grounding (REG), Region Referring Grounding (RRG), Object Functional Grounding (OFG), and Visual Trace Generation (VTG)—and introduce a large-scale dataset, Embodied-Points-200K, to train them. A key contribution is the training methodology: a two-stage Reinforced Fine-tuning (RFT) curriculum with a specialized multi-task reward system. This RFT approach is argued to be superior to standard Supervised Fine-Tuning (SFT) as it enables flexible, free-form reasoning and resolves the \"multi-solution dilemma\" (e.g., many valid points for an \"empty space\") inherent in pointing tasks. Embodied-R1 achieves state-of-the-art performance on 11 spatial and pointing benchmarks and demonstrates robust zero-shot generalization, achieving a 56.2% success rate in SIMPLEREnv and 87.5% across 8 real-world XArm tasks without task-specific fine-tuning, a 62% improvement over strong baselines. The model also exhibits high robustness to diverse visual disturbances.", "questions": "See the weakness section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- It pioneers \"pointing\" as a unified and embodiment-agnostic intermediate representation to bridge high-level vision-language understanding with low-level robot actions. This approach is designed to solve the \"seeing-to-doing gap\" by overcoming data scarcity and the challenge of hardware (embodiment) heterogeneity.\n\n- The paper introduces Embodied-Points-200K, a large-scale, high-quality dataset curated from diverse sources to support four newly defined core pointing abilities: Referring Expression Grounding (REG), Region Referring Grounding (RRG), Object Functional Grounding (OFG), and Visual Trace Generation (VTG).\n\n- It proposes a novel \"Reinforced Fine-tuning (RFT)\" training curriculum, which is a key advantage over standard Supervised Fine-Tuning (SFT). The RFT method enables flexible, free-form reasoning beyond rigid templates and effectively resolves the \"multi-solution dilemma\" by reinforcing any correct solution rather than overfitting to one.\n\n- The resulting 3B parameter model, Embodied-R1, achieves state-of-the-art performance on 11 different spatial reasoning and embodied pointing benchmarks. It also demonstrates exceptional zero-shot generalization to robotic manipulation.", "weaknesses": "- The paper's core motivation for using Reinforced Fine-tuning (RFT) over Supervised Fine-tuning (SFT) is to address the \"multi-solution dilemma\" (e.g., many valid points for an \"empty space\") . However, an SFT loss can also be adapted to handle this by, for example, minimizing the distance to the closest valid point in a target mask rather than a single (x, y) coordinate. Could the authors clarify how the \"Embodied-SFT\" baseline was implemented for these ambiguous tasks? Does this baseline already account for the multi-solution problem, or does it naively overfit to a single sampled ground-truth point? This is critical for validating that RFT is a necessary solution to this specific problem, rather than just one possible solution.\n\n- The paper pioneers \"pointing\" as an embodiment-agnostic representation. While this is effective for localization and trajectories, the authors acknowledge it may be insufficient for tasks requiring force, twisting, or interaction with deformable objects . How does the model currently handle instructions for which \"pointing\" is a fundamentally poor representation? For example, if given the command \"unscrew the jar lid\" or \"wipe the table,\" would the model attempt to generate a point or trace, or would its reasoning process allow it to recognize the task's incompatibility with its action space?\n\n- The preliminary results for the Embodied-R1-RGBD variant are concerning. The paper notes that on complex relational tasks, the 3D model performs worse than the 2D RGB-only model, a degradation attributed to potential \"hallucinations\" from the depth map . This seems to contradict the premise that richer, more relevant sensory information should improve performance. Could the authors elaborate on this finding? Does this suggest a fundamental flaw in the model's fusion architecture, where adding modalities can confuse the reasoning process, and how does this impact the future viability of extending this \"pointing\" paradigm into 3D?\n\n- The paper's title claims \"General Robotic Manipulation,\" yet the evaluations are limited to single-step instructions. The authors propose that a high-level planner could decompose long-horizon tasks as future work . While promising, this is a significant limitation. Could the authors provide any preliminary evidence, even a qualitative example, of Embodied-R1's ability to handle a simple, two-step instruction (e.g., \"pick up the strawberry and then move it to the bowl\") to demonstrate its feasibility as a robust execution module for such a hierarchical system?\n\n- The authors astutely note that the automated pipeline for generating the Embodied-Points-200K dataset \"inevitably introduces noise\". How does the RFT training process handle this label noise compared to SFT? Is it more or less robust to an incorrect ground-truth trajectory or point? A brief analysis of the dataset's noise level and its effect on the reward signal would strengthen the paper's claims about the robustness of the RFT methodology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LRdwUls7HN", "reviewer": "Reviewer_4ftQ", "summary": "This paper summarizes prior works with point-based affordances, redefines the concept of \"pointing\" as an intermediate representation, and extends it to four specific interpretations: Referring Expression Grounding (REG), Region Referring Grounding (RRG), Object Functional Grounding (OFG), and Visual Trace Generation (VTG). It collects and relabels datasets from multiple previous works, curates and re-annotates portions of them, specifically enhancing capabilities in these four areas to improve the accuracy of operations using pointing. The pointing format remains the classic xyz-coordinate point; however, by training on data with different semantic meanings, this representation acquires diverse types of knowledge. Downstream motion planners can more effectively utilize points enriched with these four types of information. To address the limitation of generalization when using supervised fine-tuning (SFT) with pointing, the paper employs GRPO for training.", "questions": "1.\tThe difference between the “pointing” representation and previous works should be further illustrated. For example, how do previous studies address the points and the issue of their dataset composition? \n2.\tIt would be better for authors to add the evaluation results for all four LIBERO suites for further comparison. In the ablations, there is only the LIBERO-spatial suite.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThis paper systematically examines previous point-based works and explicitly categorizes the information required in embodied tasks into four distinct types. Although the representation itself is not particularly novel, the proposed classification scheme and the approach of constructing training data according to these four categories are relatively innovative.\n2.\tThe work is solid and thorough, involving extensive data collection and cleaning, as well as comprehensive experimental validation.\n3.\tThe application of RLVR methods in embodied AI is still rare. This paper takes a clever approach by not training end-to-end directly; instead, it uses reinforcement learning to train a perception module, and then passes the perception outputs to a motion planner for action execution.", "weaknesses": "1.\tBaseline coverage could be broader. Although the authors compare to strong baselines like FSD and RoboPoint, some recent VLA models (e.g., VLA-RL, ThinkAct) are only cited but not experimentally compared, which slightly limits the comprehensiveness.\n2.\tUnclear differentiation from prior “point-based” or affordance representations. Although the paper repeatedly emphasises the novelty of the “pointing” representation, it does not clearly delineate how this differs conceptually and functionally from existing notions, such as affordance points, grasp points, or visual trajectory signals, used in prior works (e.g., RoboPoint, FSD). The reader is left uncertain whether “pointing” represents a new learning formulation, a different abstraction level, or simply a unified terminology. A more explicit comparison — e.g., in a figure or a dedicated subsection — between pointing, affordance, and trajectory signals would clarify the actual contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zdnhY8RVFz", "reviewer": "Reviewer_LhSQ", "summary": "In this paper, the author proposed a pipeline of RL post-training of VLM models to perform embodied related perception tasks. The core innovation is \"pointing\" as a unified, embodiment-agnostic intermediate representation that bridges high-level vision-language comprehension with low-level action primitives. The authors define four embodied pointing abilities: Referring Expression Grounding (REG), Region Referring Grounding (RRG), Object Functional Grounding (OFG) and Visual Trace Generation (VTG). To train the model, the authors curate Embodied-Points-200K, a 200,000-sample dataset structured as \"question-verification\" pairs to handle the multi-solution nature of pointing tasks. It draws from diverse sources like RefCOCO, RoboRefIt, HandAL, and synthetic Isaac Gym simulations, augmented by GPT-4o for question rewriting.", "questions": "See the weakness.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper is well-written and easy to follow.\n2. The evaluation benchmarks are comprehensive.\n3. The proposed RL method improves the performance of zero-shot reasoning tasks.", "weaknesses": "1. Lack of novelty. For the whole pipeline, I think it is highly similar to the Seg-Zero [1] and VisionReasoner [2]. The authors finetuned Qwen2.5-VL-3B using Reinforced Fine-tuning (RFT) approach, closely mirroring the training paradigms introduced the two papers. From my perspective, the authors essentially repurposed the reasoning detection, segmentation, and point prediction tasks from prior papers, adapting them into object-manipulation-level pointing prediction tasks. Apart from introducing a few additional reward functions, the overall framework remains fundamentally unchanged. Most importantly, I noticed that the authors did not cite these two papers at all, despite the fact that their publicly released anonymous code is highly similar to the open-source implementations of those two papers. I would like to remind the authors of the principle of academic integrity. Also, treating point or affordance as a unified representation is discussed broadly in previous works.\n\n2. Insufficient literature review. Several highly similar works are neither compared nor discussed, such as Affordance-R1 [3]. \n\n3. Lack of comparisons of scalability. What would happen if you scale up the model size from 3B to 7B or larger ones?\n\n4. Meaningless comparisons. In the VLAs experiments, the authors compared the Embodied-R1 directly with VLAs, where the Embodied-R1 outputs the point prediction and utilizes another planner to solve the path planning. However, the VLAs directly output all-step actions in relatively high control frequency, which is intrinsically more difficult to complete manipulation tasks.  Hence, such kind of comparisons are meaningless from my perspective.\n\n5. Insufficient experiments. How about the dual-arm experiments, does the Embodied-R1 perform good?\n\n6. Overall, I think the paper's motivation is not strong enough, that we know the RL post-triaing strengthens the perception capabilities as the authors claimed. However, how to prove such boost of performance will lead to improvements of real manipulation tasks? In other words, the object-level pointing paradigm is limited compared to the end-to-end VLA paradigm, which I think limits the meaning of this paper. To be more specific, the claimed paradigm and model are useless in manipulation tasks that are not object-centric.\n\n[1] Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement\n\n[2] VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning\n\n[3] Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)