Sim2Real VLA: Zero-Shot Generalization of Synthesized Skills to Realistic Manipulation
Authors: Runyi Zhao
Sheng Xu
Ruixing Jin
Yueci Deng
Yunxin Tai
Kui Jia
Guiliang Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Zero-Shot Sim2Real
Vision-Language-Action (VLA) Model
Long-horizon Manipulation
Primary area: applications to robotics, autonomy, planning
TL;DR: This paper introduces Sim2Real-VLA, a generalist robotic control model that enables zero-shot transfer from synthetic simulation to real-world manipulation tasks.

Abstract: Vision-Language-Action (VLA) models represent a critical milestone toward embodied intelligence in robotic manipulation. To support their training, recent research has developed high-performance simulation engines for data synthesis. However, their effectiveness is still significantly limited by the simulation-to-reality (Sim2Real) gap, as policies trained on synthetic data often fail to generalize reliably to the real world. To address this challenge, we present Sim2Real-VLA, a generalist robot control model trained exclusively on synthetic data, yet capable of transferring seamlessly to real-world manipulation tasks. Sim2Real-VLA features a dual-system architecture: a high-level planner that infers object-centered chains-of-affordances, and a low-level actor that executes and validates these plans in real time via a tokenized action space. This design filters out manipulation-irrelevant features and prioritizes motion-critical dynamics, thereby enhancing Sim2Real domain transfer. Besides, a notable advantage of Sim2Real-VLA lies in its tight integration with automated data generation for manipulation skills, eliminating the need for manual fine-tuning and enabling scalable, hands-free training. Empirical evaluations across bimanual, dexterous, and long-horizon tasks show that Sim2Real-VLA consistently outperforms previous VLA baselines under diverse real-world environments and domain shifts.

Published as a conference paper at ICLR 2026
SIM2REAL VLA: ZERO-SHOT GENERALIZATION OF SYNTHESIZED SKILLS TO REALISTIC MANIPULATION
Runyi Zhao', Chenn Xu' Ruixing Jin', Yueci Deng's Yunxin Tai Kui Jia Guiliang Liu Scchool of Data Science, The Chinese University of Hong Kong, Shenzhen
DexForee Technology, *Shenzhen Loop Area Institute. Trunyizhao, shengxul, muxinngjin, htw.gggiiinnnniiiii.n.. yuecideng) @link. cuhk. edu. cn
yunxintai@@exxorce.. com (kuijia, lwwgui cuhk. edu.cn
ABSTRACT
Vision- LaiionLLnggaag-Acttinn (VLA) models represent a critical milestone toward em- bodied intelligence in robotic manipulation. To support their training, recent research has developed high-performance simulation engines for data synthesis. However, their effectiveness is still significantly limited by the ummulaton-to--eallii (Sim2Real) gap, as policies trained on synthetic data often fail to generalize re- liably to the real world. To address this challenge, we present Sim2Real-VLA. a generalist robot control model trained exclusively on synthetic data, yet capa- ble of transferring seamlessly to features a dual-system architecture: a high-level planner that infers chains-of. affordances, and low-level actor that executes and validates these plans in real time via a tokenized action space. This design filters out features and prioritizes notion-critical dynamics, thereby enhancing Sim2Real domain transfer. Besides, a notable advantage of Sim2Real-VEL VLA lies in its tight integration with automated data generation for manipulation skills, elimi- nating the need for manual fine-tuning and enabling scalable, hands-free training. Empirical evaluations across bimanual, dexterous, and long-horizon tasks show that Sim2Real-VLA. consistently outperforms previous VLA baselines under di- verse real-world environments and domain shifts, The source code is available at https::/qithub: com DexFF orce/ Emwodiihaa....
manipulation tasks. Sim2Real- VLA
INTRODUCTION
Designing precise and scalable robotic manipulation policy represents a key milestone toward realizing artificial general intelligence (AGI) (Smith et al.) 2012). Despite significant advances in hardware design, control algorithms, and simulation platforms, traditional robotic systems remain highly specialized, often requiring task-specific engineering and extensive manual tuning. Recent advances in large foundation models provide promising pathway to address these limitations with the development of generalizable manipulation policies (Zhao al. 2023b]] Zhhang et al., 2024). The Vision- Law---agnggg-ttiinnn (VLA) models, which integrate visual observations, natural language commands, and robotic control actions, have emerged as a prevailing architecture for implementing generalist agents in real-world robotic applications (Ma et al. 2024b: Zheng et al. 2025
As foundation model for robotic control, VLA training typically follows a data-driven pipeline. This process demands a large amount of robot-operating data, the collection of which involves intensive manual effort and access to specialized hardware. While previous studies have demonstrated
the effectiveness of VLA pre-training on internet videos (Luo et al. 2025) and embodimen datasets (O'Neil et al.] 2024), deploying these models typically requires additional rounds of fine-tuning the target robot and task-specific skills. To enable more scalable training, recent studies have explored the use of synthesized or simulated data (Mandlekar et al.) 2023 Deng et al.) 2025 Liu et al. 2025a). A key advantage of such data is that can be generated at large scale using high-performance computing clusters via automatic skill acquisition (Nasiriany et al. [024 Wang al. 2024b Mu al. 2024 Hua al., 2024 Zhao et al. 2024). Nevertheless, models trained exclusively on these datasets are subject to a Sim2Real domain gap when deployed in practice.
"Corresponding author; Guiliang Liu, liuguiliang @ww.wc.cc..cccm...

---

Published as a conference paper at ICLR 2026
To close this domain gap, mainstream research has focused on developing photo-realistic and physics- accurate simulation environments (Hua et al. al. (2024 Puig et al. (024) or world models (Agar) wal et al. 2025). However, accurately modeling real-world dynamics remains significant challenge that has yet to be solved (Bharadhwaj 2024). More importantly, recent studies (Xie et al., 2024 Liu et al.; 2025a) have shown that factors such as lighting conditions and background textures, despite consuming substantial modeling resources, are essentially irrelevant to manipulation performance. These findings call for an alternative approach: instead of focusing on generating high-fidelity data, we propose addressing the Sim2Real by redesigning the VLA architecture.
In this study, we introduce Sim2Real- -VL,,, which, despite being trained solely on synthetic data, demonstrates generalizable and sustained manipulation performance across diverse real-world envi- ronments. To address the enduring domain gap between synthesized and realistic data, Sim2Real-Vi VL.AA integrates generalization mechanism in model design. Specifically, Sim2Real-VL. system design, encompassing a high-level planner and lower-level actor, interconnected by affordance signals in finishing the given task. Such affordances play fundamental role in robotic manipulation within our SimAReal-VLA design because: 1) The key steps in executing a long-horizon task can be abstracted as a chain of affordances, thereby providing a structured basis for embodied reasoning within the planning system; 2) The predicted affordance signals function as critical supervision for producing robot control outputs within the acting system; and 3) Contemporary methods for manipulation skill acquisition (Ma et al.) | [uu et al. [024]] Wang et al.) 2024b) are grounded in affordances, which can be consistently derived in simulation environments and provide supervision signals for training Sim2Real-V LLA. By coupling reasoning and acting with manipulation affordances derived from object-oriented observations. Sim2Real- VLA filters out camipullatinniirelevaan features and concentrates on task-relevant dynamics, thereby effectively closing the Sim2Real gap.
utilize dual
We conduct extensive studies across multiple tasks involving bimanual, dexterous, and oag-horizon manipulation. Our findings reveal that even state-of-the-art VLA models, when trained solely on synthesized data, struggle to perform effectively real-world manipulation scenarios. contrast, Sim2Real- VI.A outperforms baselines with minimal Sim2Real gap, achieving over 35% higher success in realistic environments. More importantly, our quantitative experiments demonstrate that its zero-shot Sim2Real capability generalizes reliably across wide range of domain shifts.
RELATED WORKS
VLA Model for Robot Manipulation. In recent years, VLA models have emerged as a prevailing paradigm in modal foundation models and have been successfully applied to robot control tasks (Ma et al., 2024b Firoozi et al.) 2024). Among these tasks, a critical application is dexterous manipulation, which requires the model to comprehend the given commands, interact with various objects, and dynamically respond to changing environments (Zheng et al.) 2025). Building upon the VIM backbone (Zhang et al.) (024). earlier VIAA models, such as Open (Kim et al.)
2024), Otco (Ghosh et al. 2024). RTs (Brohan et al., 2023 Zitkovich et al.) 2023). RDT (Liu ct al. 2025b), and To (Black et al. 2024), typically utilized an end-to-end model architecture. To enhance the efficiency of policy inference, recent approaches, such as Figure Al (2025).
GrooN(Bjorck al. (2025), Gemini (Team et al.] 2025), ) AgiBot (Bu et al. (2025). ) and other dual-
system frameworks (Shentu et al. 2024 Han et al. 2024 Bu et al. 2024 Chen et al. 2025 Wen
2025 /Zhang al. (025), have adopted two-level architecture. This design comprises low- frequency VLM system responsible for semantic understanding and embodied reasoning. alongside a high-frequency policy model that efficiently predicts control signals faster rate. Despite these advancements, training such models often demands collecting real-world datasets and fine-tuning the
al.
VLA models for specific robotic tasks (Nasiriany et al., 2024 Mu et al., 2024).
Sim2Real Generalization. In the application of robot Manipulation, Sim2Real generalization techniques often involve: 1) Domain randomization tackles Out-of- cittof-Distribution (OoD) scenarios in practical applications augmenting the training dataset with randomized visual and physical fea- tures (Chen et al., 2022). To determine the scale of randomization, recent studies have determined relevant parameters by automatic learning (OpenAl et al.) 2019), active exploration (Mchta et al.) 2019), Bayesian update (Muratore et al. 2021b a), offline inference (Tiooni et al. 2023) and continual
'The simulation system provides complete access to the spatial, physical, semantic, and morphological properties of objects, which substantially facilitates accurate affordance estimation.
2

---

Published as a conference paper at ICLR 2026
learning JJsiiovski et al. 2024). 2) Domain adaptation reduces the gap between simulated and
real-world domains by aligning them within a shared feature space, such as 2D images (Bousmalis
et al. 2018 Zhang et al. 2019), 3D point clouds Globos-Tsunekeen
& Harada 2020 Qin et al. 2022 Chen et al. (023), or environmental dynamics (Memmel et al. 2024 Josifovsk et al. 2025). 3)
Real2Sim projection maps real-world scenes into the simulation environment, enabling the system to
better capture realistic semantics (Villasevil et al. 2024; Liu et al. 2025a Tai et al. (025). However,
lacks principled studies on redesigning VLA models close the Sim2Real gap.
PROBLEM FORMULATION
Environment for Robotic Manipulation. To formulate the problem of Sim2Real robot manipulation, we can formulate the real-world robot deploying environment into a Partially Observable Markov Decision Process (POMDP) M = (S, O,O,P, PT; R, (o. where: 1) The state 8 E S captures the semantic information of a scene, encompassing the configuration (e.g.. layouts, appearance, and physical characteristics) of various types of objects and the robots. 2) The action 0g A indicates control signals for the target robot. We follow diffusion policies (Chi et al. 2023 Liu et al. 2025b), and utilize the joint angles in the robot for presenting actions. 3) The observation o E O represents
the perceptual signals captured by sensors. These observations are typically non-Markovian and provide only partial information about the current state. 4) Transition function Py characterizes the impact of robot action a, to the configuration of current state 8t, thereby projecting the 8, to new scene represented by 81411 5) The reward function R(s a) represents how effectively a robot completes a targeted task after taking action a in state (Appendix A.2 introduces more details). 6) Po denotes the initial state distribution and ? E (0. 1] denotes the discounting factor which weights the importance of future rewards relative to immediate rewards.
Zero-Shot Rim2Real Learning for Control Policy. Within the robot's operational environment, our objective is learn a control policy x(at,.... at+as Og-H..... oc. I) that predicts a sequence of M future actions ....... Gx+ M based on history of H past observations Or-H: Or and tasks annotation (Zhao et al., 2023a). At a time step l, oz captures both proprioception of and visual signals of from nieti-view cameras. By leveraging observations and language- task annotations, the control policy can be effectively instantiated as a VLA model (Kim et al. 2024). In this study, we consider learning control policy under Sim2Real transferable setting:
Definition 3.1 (Sim2Real Transferable Policy Learning) Let M denote the real-world environ- ment, and M = MŠ(ŠAA A, Ô, Pr. Po: 3) denote a corresponding simulated environment. Let denote the policy trained on collection of skill trajectories ē = [ōn, âo, 01, â1, ....... ÖT: ärr generated in M, where E Ô, â Å, and l is a task description such that arg max, E+ (f, ,; 1) Nevertheless, we expect goal π to achieve the optimal performance in the realistic environment M.
Notably, this Sim2Real transfer is conducted in a zero-shot manner, where no real-world demon- strations are used during training, but the learned policy must generalize to and solve real-world manipulation tasks. However, as demonstrated by prior studies (Nasiriany et al. 2024 Wang et al. 2024a), the discrepancy between the simulated environment M and the real-world environment M poses significant challenges (as also evidenced by our experiment results in Section 5. Directly using simulated skills / to supervise the training of real-world policies ı often leads to suboptimal performance due to this domain gap. Although domain randomization and adaptation methods have demonstrated successful examples in general robotic control (Mehta et al., 2019 Muratore al. 2021a Tiboni et al. 2023 Josifovski et al. 2024), their applicability to manipulation tasks in the context of VLA models remains an open question. This work aims to design structured framework that enables learning zero-shot Sim2Real transferable object manipulation policy.
4 VLA MODEL FOR ZERO-SHOT SIM2REAL GENERALIZATION
To bridge the Sim2Real gap, rather than constructing a more sophisticated data engine to replicate real-world dynamics, our study considers model-side solution. This approach acts as a filter for realistic dynamics, focusing exclusively on features critical for manipulation, guided by the simulation
We use J 13 () denote the objective parameterized hy language command l.
3

---

Published as a conference paper at ICLR 2026
Sim2Real VLA Three Steps!
Senser Observations
Ensironment
M
Predict Affordance
Heal Robot Control
2: Execute Affordance
Zero- Shot-Shet Deployment!
Deploy Real Robot
Propriccen Pnppiacaption
Acciong Povrsiattoon
[n Pet-.//0 ..1)
Planning System: Affordance Prediction Model
Reel
Acting System:
Afford. Execution
Objject Masking and Encoding Layar
Masked Obsarvations
Language Enooder
Demuiin
Rendomission,
Pandomized Obba
Mutii Views Obs. &
Masking Visual
Mation Validation
"Mease
Statue Engoder
unfinish, Repeat Repean.Affordancs Alfordance
paper
Historical Observations
Discrini nator
Visual Enooder
Command !
Figure 1: The pipeline of our Sim2Real VLA model consists of two main components: planning
system Section 4.1) that enables embodied reasoning through chain of affordances, and an acting
system (Section 4.2 that drives model by executing the planned affordances. This design allows the model to transfer seamlessly to real-world manipulation tasks.
engine. This strategy not only significantly reduces modeling complexity but, more importantly enables the skills learned from simulated data effectively drive the VLA model.
To implement the Sim2Real VLA model, we design affordance-driven, object-orientet framework that integrates planning and acting, structured follows:
4.1 PLANNING SYSTEM: ROBOTIC MANIPULATION AS CHAIN-OF-AFFRRDANCE
The goal of the planning system is to reason about the essential steps for finishing the given tasks described by the language command Specifically, our planning system predicts a chain of affordance Igo..... 9K] to represent these essential robot motions. To support the characterization and prediction of affordance, we adapt the visual observation into the object-oriented representation and design an affordance reasoning process to finish the given tasks.
Object-Orient Adaptation in Observation Space. Since some key waypoints in manipulation trajectories are inherently object-centric, synthesizing feasible manipulation trajectories requires access to each object's ' pose and morphology (e.g., position, orientation, and shape parameters). While such information is typically unavailable from real-world cameras, simulation. Given a fixed camera configuration, we render observations as ô; = -fe(eo 61; c.,...) where e, denotes an object (c.g., including those with articulated, rigid and soft body) the simulated scene and fe is the rendering function parameterized by the semantic configuration & (including layouts, camera view). Performing object-oriented adaptation is essentially learning the reverse of fc by minimizing the following negative log- likelihood loss function:
be recovered in
T
((0) E [mi
log (pé
(I)
t=O İ=1
where i and mf denote the skill trajectory and object mask. As objects are not necessarily fully observable, we adopt a probabilistic predictor 2 for object recovery, thereby accounting for the underlying uncertainty. Moreover, because training data are generated in simulation and rendered objects may deviate from real-world appearance, pd incorporates Domain Randomization (DR) into the observation o; under the scene configuration Ç. Specifically, it performs the following operations
1) Strategic DR Features Selection. Sim2Real- VLA incorporates a large foundation model (e.g., a vision language model such GPT-5) into the domain randomization (DR) process. It leverages the model's 's reasoning ability to rank DR features and define their sampling ranges (see Table [ using the task description, current observations o,, and the simulated environment configuration For
4

---

Published as a conference paper at ICLR 2026
simplicity, characterize p" as joint uniform distribution the selected features within their respective ranges (Mehta et al. 2019).
Table 1: The set of DR features for characterizing the Sim2Real generalization gap in robotic manipulation tasks (Xie al. 2024).
Level
Domain Randomization (DR) Features
Scene Level
Lighting, Table Texture, Background, Distractors, Object Locations, Object Orientations, Object Texture and Object Shape
Robotic Level
Cameras Position, Cameras Orientation, Cameras Field of View, Initial end-effector pose
2) Flows of DR in Sequence Observation. each time step, unlike previous methods that rely on fixed set of randomized features to generate trajectory Mu et al. (2024), nd develops flow of DR by resampling iationninvariint features, such as lighting, textures, and backgrounds, whose values have limited influence the robot's actions. By performing DR higher frequency, approach enhances the generalization capability of the learned policy. Figure (below) shows an example of DR for the observations of water pouring motions.
Grosp the oup and the bottle
R2 Movee the bottle
U- Rebata the botsa
Figure 2: An example of the DR flow (left three images) and the chain affordances (right three images) generated in the simulated environment for the water-pouring task.
Reasoning via Chain-of-A Aforrdacce. Conditioned on the visual observation and the instruction 1, the planning system infers the appropriate
q .qo...., .K]. Each dr corresponds to series of geometrically structured keypoints that represent key end-effector pose that governs the robot-object interaction necessary to accomplish an atomic task. By predicting q, our planning system of Sim2Real VLA essentially reasons about the sequence of atomic tasks based on the provided instruction, alongside the robot's key motion finishing these tasks. Unlike the embodied Chain-of- Thoughts (CoT) reasoning based on language description (Zawalski et al. (2024), Sim2Real VLA performs reasoning with affordances, thereby better aligning the planned atomic tasks with objects, robot configurations, and commands 1. During implementation, the objective of Sim2Real VLA
learn the predictive distribution p (gk.t..... 9K.t my, of of...o.H. l), where my represents the predicted mask of the target object in equation [ and 9k denotes the next affordance based on the current observation. To facilitate implementation, we decompose the joint distribution into conditional components and optimize the model by minimizing the following loss function:
Σ log (qu.e 1 myy of.................. of 1: - (o(l,,)) ]
C(Φ) = (. [B K.T
Σ
(2)
where pd is the aforementioned domain randomization function. Conditioning on the previous affordance, the target object, and visual observations, and the command, Pa can perform in-time prediction of the future affordance to finish the given task, but at frequency of muster smaller than that the robot acting system 4.2 Figure 2 (right) illustrates the chain affordances generated in the simulation environment.
4.2 ACTING SYSTEM: PREDICTIVE CONTROL AFFORDANCE EXECUTION
Upon receiving the predicted affordance sequence (go...., , qK] from the high-level planning module, iteratively executes each
the low-level acting system TTe (a, a++M 4k. m,, O H
Here, ac-1 denotes the final action taken achieve the previous affordance 9k-1-

---

Published as a conference paper at ICLR 2026
affordance by guiding the end-effector toward the designated target pose, and verifies at each step that the resulting motion successfully achieves the intended affordance.
Affordance Execution. To execute affordances both efficiently and accurately, the acting system leverages a tokenized action space and employs a decoupled estimation strategy for controlling the manipulation actions of bimanual robot. This design enables the system remain both flexible and modular while reducing unnecessary dependencies between the two arms.
Arm-Decuee Amm-Decuupled Estimation. In the decoupled estimation framework, the policy Th is split into two independent components, Tuy and Tw.., which control the left and right arms of the robot, respectively. Although these models are jointly trained to complete bimanual manipulation tasks, they are independently parameterized. Each model only has access to its own relevant visual observations (from arm-mounted and top-down cameras) and the specific affordance target associated with its arm. This independence is crucial for reliable affordance tracking, as it prevents the acting model from misattributing attention to the other arm's goals or state when computing its own actions.
Tokenized Action Space. Rather than operating directly in high- dineeseienssill,, continuous control space. we apply a frequency- -omain representation named Discrete Cosine Transform (DCT) (Ahmed et al.. 2006, Pertsch et al., (025) to normalized segment of continuous actions. This transformation converts the signal into the frequency domain, where we can efficiently compress it. The resulting DCT coefficients are then quantized, and we apply Byte-Pair Encoding (BPE) (Gage) 1994) to compress the sequence of per-dimension coefficients into a compact action token sequence, denoted as a DCT_ This tokenization significantly reduces the complexity of the action space, enabling faster and more robust learning while retaining the essential temporal and spatial characteristics of the original motion.
Affordance Validation. Since the chain of affordances is sequentially dependent, the acting model must successfully execute the earlier ones before implementing the later ones, especially in long-term tasks. To achieve this, we develop a validation model that dynamically determines whether the target pose has been tracked and whether the system should proceed to the next affordance. In this way, the acting system can be reactive to the failure case, which greatly improves the system's 's robustness.
4.3 AUTOMATIC DATA GENERATION PIPELINE FOR SIM2REAL VLA TRAINING
Since Sim2Real- is trained exclusively on simulated data, its performance heavily depends on the efficiency and scalability of simulated data generation. Recent advances in agentic skill acquisition (Ma et al.] [2024a] Wang et al., 2024b) have made it possible to generate such data automatically, without manual intervention additional hardware, thereby enabling a more scalable, efficient, and cost-effective process for generating training data. This section introduces our automated data generation pipeline, highlighting the core components that can, in principle, generate the integration of Sim2Real-Ve VLA with training environment construction, skill dataset generation, and the provision of relevant guiding signals.
1) Real2Sim Data Projection maps descriptive observations of the target task from real-world applications to the simulated environment. Following (Dai et al. 2024 Liu et al. (2025a), the project encompasses static scene information, including the orientation, position, and morphological features of objects as well as their spatial layout, and dynamic action trajectories derived from teleoperation and human demonstration videos. The resulting simulation environment faithfully preserves the semantic structure of both the tasks and their contexts. Check Appendix A.3 for more details.
2) Generative Scene Scaling samples diverse configurations within the simulated environment based on the Real2Sim prior information obtained from the aforementioned projection. The sampling process spans both scene-level and robot-level features in Table 1 Each sampled configuration defines distinct scene, resulting in a set of candidate environments derived from the target scene. example of the Real2Sim projection along with the corresponding environment scaling. The spatial and morphological information of objects is available in the simulation engines, enabling the generation of object masks m; during robot operation. These masks can serve effective supervision signals in the objective (T). We present more details in Appendix [A.4]
Figure 3 illustratee
3) Automatic Skill Acquisition generates operation trajectories for accomplishing the target task. The embodied agent, equipped with a vision- languaee model, decomposes the task into atomic units (Nasiriany al. 2024), identifies the target object from the input instruction (Fang et al. 2023)

---

Published as a conference paper at ICLR 2026
Reviillin
Human Demrerstntion feal2tim
Scarre Sealing Sumpling scrosa Different Postures
Figure 3: An example of our data generation pipeline, which projects scenes and action trajectories from heterogeneous sources (videos or teleoperation) into the simulated environment, and then scales the environment with diverse randomized features.
and produces candidate grasping and manipulation poses for the end-effector (Mu et al., 2024). From these key poses, generalized inverse kinematics (IK) algorithm is applied to compute the corresponding joint angles required to drive the robot arms. The key poses the end of each atomic task are utilized as an affordance supervision (q [yo.... .......... - in the objective (2), which aligns the tracking of the affordance with finishing atomic tasks. The instructions of atomic tasks, the computed joint angles, the identified affordances, and the rendered images from the simulator are utilized the training data for our Sim2Real- VLA model. Appendix A.5 shows more details.
EMPIRICAL EVALUATION
We conduct an extensive experimental study of Sim2Real- VLA, evaluating its performance from the following perspectives: 1) applicability to real-world manipulation tasks and 2) generalizability across diverse domain gaps. Additionally, we visualize attention maps during the operation of robotic manipulation, demonstrating the rationality of Sim2Real- during robotic manipulation.
Our experiments are conducted based on the Agilex CobotMagic robot by following (Fu et al. 2024).
APPLICABILITY: TOWARD ROBUST SIM2REAL PERFORMANCE IN LONG-HORIZON TASKS
While prior studies have demonstrated zero-shot Sim2Real performance in short- tooot-termm tasks such as object grasping and placing (Collins et al. 2019 Xie et al. 2024 Deng et al. 2025), the extent to which this performance generalizes long-horizon manipulation remains largely unexplored. In this study, we evaluate our method using the manipulation tasks summarized in Table 2 For each task, we generate training data in the simulation environment (Section 4.3). learn manipulation policy from the synthesized data, and evaluate its generalization performance across different environments.
Table 2: Task descriptions with decomposed action steps and arm type.
Task
Steps
Arm Type Single-Arm
Single-Arm Water Pour
(1) Grasp bottle - (2) Move bottle to cup + (3) Pour water
(4) Return bottle
Dual-Arm Water Pour
(1) Grasp bottle + (2) Grasp cup -> (3) Move bottle to pouring
Dual-Arm
position + (4) Move cup to receiving position + (5) Pour water
+ (6) Return cup - (7) Return bottle
Table Rearrangement
(1) Grasp fork + (2) Place fork beside plate - (3) Grasp spoon
Dual-Arm
+ (4) Place spoon beside plate
Items Hand-Over and Place
(1) Grasp an item - (2) Transfer the to other hand
Dual-Arm
the air (3) Place the item into the holder
Basket Pick-and-Placc
(1) Grasp an item + (2) Place the item into basket (3) Grasp
Dual-Arm
basket -> (4) Place basket down
Pan Open-Pick Soan-Pick-and-Piie
(1) Grasp lid - (2) Open lid
(3) Grasp item (4) Place Dual-Arm
the item into pot + (5) Close lid
Evaluation Metrics. We evaluate the robotic manipulation policy in both simulation (Sim.) and real- world (Real.) environments. To prevent overfitting and to examine generalization performance, sample different feature configurations from Table [ in both environments at each run. The simulation environment (Sim.) incorporates certain level of randomness to better reflect the variability and
7

---

Published as a conference paper at ICLR 2026
uncertainties the robot may encounter in real-world scenarios. In this way, simply replaying training data is insufficient to achieve strong performance in both simulation and real-world environments. We report the number of runs, the times of success manipulation, and the average number of steps required to complete the tasks in the real-world environment. Additionally, to reflect the efficiency of robotic manipulation, we also report the average number of steps required to complete each task, as more effective policies tend to complete tasks in fewer steps. For unsuccessful trials where the robot fails to complete the task, we report the predefined maximum step limit as an upper bound.
Baselines. To benchmark our Sim2Real-VLA, we compare it against several representative baselines, including 1) Action Chunking with Transformers (ACT) (Zhao et al., 2023a) leverages sequence modeling to learn temporally extended action policies, 2) Diffusion Policy (DP) applies diffusion- based generative models to represent and sample robot actions in continuous spaces. 3) Robotics Decision Transformer (RDT) (Liu et al. 2025b) adapts the Decision Transformer framework to robotic tasks, enabling goal-conditioned policy learning from offline data. 4) Ho (Black et al., 2024) serves strong pretrained policy prior that provides generalizable low-level skills across different domains. 5) GROOT (Bjorck et al. 2025) is a large-scale, foundation model for robot control trained on diverse multimodal datasets. These models fine-tuned the same offline simulator data (FwS indicates Finetuned with Simulated data) generated from our automatic data generation pipeline (Section 4.3 and then deployed zero-shot to real-world applications.
Table 3 presents the evaluation results. Across all six tasks, our method consistently achieves the highest success rates in both simulation and real-world environments, while also completing tasks with fewer average steps. In particular, our method attains an average real-world success rate of 60.8%, significantly outperforming the best baseline with an absolute improvement of over 35% This substantial margin highlights the robustness and generalization capabilities of our approach, especially under domain shift from simulation to the real world. The performance gap is even more pronounced in complex, Hand-Over Over and Place, where baseline models frequently fail or struggle to generalize beyond their training distributions. These tasks require temporally extended reasoning and precise coordination, which our method handles reliably, while others often exhibit brittle or erratic behavior during real- world execution. Moreover, another important finding is that while small-scale models like ACT and DP demonstrate competence in short-horizon or low-variance settings, they consistently underperform in more challenging, long-horizon tasks. Their limited model capacity, lack of hierarchical planning, and inability to generalize across diverse tasks hinder their effectiveness in realistic, multi stage manipulation scenarios. These results collectively demonstrate that our method is not only capable of mastering complex individual tasks but also exhibits the scalability and generalization ability required for real- wolldw multi-task, long-horizon robotic manipulation, which represents a critical step toward building reliable, general-pure purpose robotic agents. Table 3: Robotic manipulation performance (mean = 95% confidence interval) across different long horizon tasks.
tasks such as Dual-Arm Water Pouring, Pan Open and Place, and Items
Tasks
Singe-Arm Water Pouring (200)
Dual-Ar Water Pouring (250)
Table Reaeringement (250)
Methods
Sim.
Real.
Steps
Sim.
Real.
Steps
Sim.
Real.
Stops
W20H100
200.0000TR0
65500019
1/0051 10
1220R0.00
25000H0010
DP
11/0=00.1
2/2000.1
199 63
5150+0.08
2/20-111 13
247 3.73.
7/50+0 10
2/20=0.13
246. 30+5.33
RDTT
33/50=0.13
3/2000.16
197.35e3.44
21/50=0.14
3/20=0166
243 953.9556.99
18/5060
2/20=0.15
248.05+3.16
xilawss
38/0=00.12
6/20+0.20
194.30s4
25150=0.14
5/2020.19
241.70a7.72
11/0000.1
4/20=0.18
237 557.24411.41
xill FASTigaa
31/5000.13
11/2040:22
185.9518.06
30/5010.14
8/202021
223.95115.50
23/0040.14
7/2020.21
1...60
GROUT NI. N1.3
29/50.0.14
9/20110.2
189.0517.05
22:50u0..14
7/20100.21
231 10.181122.81
1665010 13
4/200.118
257 2018.87
46/00e0.08
17/20H0.16
174.6018843
47/50H0.07
16/2000.18
195.15a16.05
44/50+0.09
16/20e11.18
197.05e14.44
Tasks
Isems Hand-Over and Place (100)
Basket Pick- Place (400)
Pan Open and Place (550)
Metheds
Sum.
Real.
Sim.
Real.
Sim.
Real.
ACT.
0/50=0.00
0020+0.00
000.0020.00
0/5050.00
0/20:00.00
400.00s0.00
0/2000.00
550.0033.00
DPipss
0/50g0.00
022010.00
100.0010.00
0/5010.00
0/2020.00
100.00.0.00
015010.00
0/2000.00
550 550.0010.00
RDTmss)
8/50.0.10 10
1/2010.10 10
397.1515.97
12:5010.12
1/20100110 10
396 5017.33
15/501L 13
2/20.0.13
539 55965115.000 15.09
vilawss
12/50e0.12
4/2080.18
388 10:
15:50H0.13
2/2050.13
395 95 83
12/50H0 12
1/20e0.10
546 4.:7.53
FAST
10:0000.11
1/20+0.10
398.70+2.72
13/5000.12
3/20-016
306 35+4.26
11/50H0.1
3/20=1.16
50+5.23
GROOT N1.5
1880=00.15
3/200.116
305,50=5.33
915040.11
2/20-0133
397
17/50H0.12 13
1/20=0.10
3549.73
Sim.Real-VLA
31/50=0.13
8/2010.21
370.20=19.11
29/5020.14
9/20-0022
164.15219.76
30/5050.14
7/2060021
525.35116.34
We conduct a quantitative analysis to evaluate how well Sim2Real-VLA generalizes across different types of configurations in realistic environments. More specifically, we intentionally alter the original configuration of the real-world environment to introduce domain gaps, including variations in background texture, manipulation objects' location, texture, and shape, and the table's surface texture. Besides, we also experiment Sim2Real- VLA robustness to the combination of these gaps. Figure 4 shows the environment configuration the first run of the experiment. For each domain gap, Shows
8

---

Published as a conference paper at ICLR 2026
evaluate Sim2Real-VIE over 20 trials, each with different sampled configuration, and report the number of successful manipulations.
Original Enetorment
Bacigicunll Tecure
Obaec Paatunso
Origing, eneionnnon
Bockground focure
Object Facturt
Tetla Soture
Sigge moge-bm
Phy Down Hteee teee. Dawkim
Watte
Bayion Thelk Bramr andiffses
oegonore
3nd Oke
Tobie Indurand Oirect Ferturn
Testure
Object Haature
Fester
Ocsect Feoture
Céject Featurea
Oiject Featieee
with Dupaain
Figure 4: Visualization of environment configurations under the domain gaps of background texture, object features, and table texture across different manipulation tasks.
Table illustrates the generalization ability of Sim2Real-VL. under different domain gaps. Despite variations in background, object properties, and table texture, as well as their combinations Sim2Real- VLA achieves comparable success rates across most tasks. These results indicate that the model maintains stable performance and demonstrates strong robustness to real-world differences. Another intriguing finding is that, for some tasks, performance actually improves under certain domain variations. This effect can be explained by domain shifts reducing spurious correlations present in original setup, thereby encouraging the policy to rely on task-relevant features. Moreover, some variations may inadvertently simplify specific trials or align more closely with conditions encountered during training, resulting higher success rates.
Table 4: Number of successful/total trials across different manipulation tasks and domain gaps.
Task / Domain Gap
Original
Background
Object
Table
Multiple Gaps
Single-Arm Water Pour
17/20
17/20
16/20
17/20
16/20 (Table Object)
Dual-Arm Water Pour
16/20
16/20
16/20
17/20
17/20 (Table Object)
Table Rearrangement
16/20
15/20
14/20
16/20
15/20 (Table Object)
Item HlandOOvrra Place
9/20
8/20
6/20
8/20 (Table + Object)
Basket Pick-and-Place
9/20
9/20
10/20
9/20
7/20 (Table + Background Object)
Pan Open Pick-and- Panc-
7/20
6/20
7/20
7/20
8/20 (Table Rackground + Object)
useful diagnostic tool for analyzing how and why performance improves through the incorporation of a By inspecting where the model allocates its attention during action prediction, gain insights into whether the reasoning is aligned with task-relevant visual and proprioceptive cues.
Analyzing Attention Maps in VLA Models Attention maps
Rolevant Motions Guide by Attentions (9im.)
Attentions from Sim2Peal-VLL (8im.)
Relavant Motions Guide by Attentionn (5m.)
ROT (Sim.)
SimSReal-VLA (Sim.)
ROT (Sim.)
Figure 5: Visualization attention maps and relevant robot motions during robotic manipulation.
Figure 5 visualizes the attention maps Sim2Real- -V-A''---L''s action transformer blocks and compares them against those from vanilla RDT baseline without affordance integration. The contrast is clear: without the guidance of affordance, the model's attention is broadly distributed, often covering irrelevant background regions, entire objects regardless of their role, robot joints unrelated to the current manipulation step. In contrast, the affordance-drrven Sim2Real-Vir VLA directs its focus to precisely those spatial regions that critical for the current sub-task These observations shows that affordances encourage localized attention, ensuring that each action step conditions on the most informative object parts and motion-critical pixels.
9

---

Published as a conference paper at ICLR 2026
6 CONLUSION
In this work, we introduced Men.Real-VLA, an affordance-driven Vision-Language- Language-AAtton Action model that achieves zero-shot generalization from exclusively synthetic training data to diverse real-world robotic manipulation tasks. By reformulating manipulation as a structured chain-of-affordances and coupling high-level reasoning with low-level execution through dual-system architecture, the framework effectively filters out irrelevant variability and focuses on task-critical dynamics. Our automatic data generation pipeline further enables scalable training without manual demonstrations, while extensive experiments across dexterous, bimanual, and long-horizon scenarios demonstrate significant improvements- over 35% higher real-world success rates compared to competitive baselines. Beyond its empirical performance, Sim2Real-VLL highlights the importance of model-side design choices, rather than solely pursuing high eigh-iideiity simulation, for bridging the long-standing Sim2Real gap. These findings point toward promising paradigm shift: building robotic foundation models that trained entirely in simulation, yet are robust to realistic deployment. Future research will extend our framework multi-agent collaboration, interactive environments beyond tabletop settings, and the integration of reinforcement learning for continual policy refinement.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our study focuses on developing a Vision-Languuge-Actionn (VLA) framework for robotic manip- ulation, trained entirely in simulation and evaluated in controlled real- wolld environments. We have carefully considered potential ethical concerns in accordance with the requirements of ICLR. Specifically:
All training data were either automatically generated in simulation environments or sourced from publicly available datasets released for research purposes. No personally identifiable information or human subject data were used at any stage. Real-world demonstrations involved only non-sensitive household objects, ensuring compromise to privacy human welfare.
To minimize potential harm, the proposed framework strict monitorty protocols. All real-world evaluations took place in controlled laboratory settings, ensuring that the robot operated within defined safety boundaries and posed no risk humans, property, or the environment.
developed and rigorously tested under
The primary goal of this research is to advance embodied AI systems for beneficial real-world applications, including assistive robotics, resource handling, and safe automation. We explicitly discourage any harmful malicious use of this technology. Future deployment should adhere to domain-speciff safety standards and ethical guidelines to ensure responsible use and positive societal impact.
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure the reproducibility of our results. Details of our Vision- Language- Aanguage-Action (VLA) framework, including model architecture and training procedures, provided in Section 4. and Section [42 and further elaborated in Appendix [A] We include compre- hensive descriptions of all simulation environments, task definitions, and evaluation protocols in Section 5) To facilitate reproducibility, we have submitted anonymized robot manipulation videos as part of the supplementary materials. While we have not included the model parameters and environment source code due to their large size, we will open-source them alongside the code on GitHub upon publication. These resources will enable researchers to replicate our experiments and validate our findings.
ACKNOWLEDGMENTS
This work is supported in part by Shenzhen Science and Technology Program un- der grant K/ZD20244090 10400000122 Shenzhen Science and Technology Program under grant 1100200250001130000000 A055000000, GuangDong Key Laboratory of Big Data
CUHK-CUHK(SZ--GDSTT Computing
Joint Collaboration Fund No.
and Guangdong Provincial Key Laboratory Mathematical Foundations for Artificial Intelligence (2023B (13311222010011
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.033
2025.
Nasir Ahmed, T_ Natarajan, and Kamisetty R Rao. Discrete cosine transform. IEEE transactions Computers, 100(1):90-99) 2006.
Homanga Bharadhwaj. Position: scaling simulation is neither necessary sufficient for mn-the-wild robot manipulation. International Conference on Machine Learning, ICML, 2024.
Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GrOOt nl: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503. 14734, 2025.

---

Published as a conference paper at ICLR 2026
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. Thé A vision-language-action flow model for general robot control. arXiv preprint arXiv:241022466. 2024.
Konstantinos Bousmalis, Alex Irpan, Paul Wohlhart, Yunfei Bai, Matthew Kelcey, Mrinal Kalakr- ishnan, Laura Downs, Julian Ibarz, Peter Pastor, Kurt Konolige, Sergey Levine, and Vincent Vanhoucke. Using simulation and domain adaptation to improve efficiency of deep robotic grasp- ing. In IEEE International Conference on Roboties and Automation, ICRA, pp. 4243-42500 2018.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, et al. RT-1: robotics transformer for real-world control at scale. In Robotics: Science and Systems, RSS, 2023.
Qingwen Bu, Hongyang Li, Li Chen, Jisong Cai, Jia Zeng, Heming Cui, Maoqing Yao, and Yu Qiao. Towards synergistic, generalized, and efficient dual dual-system for robotic manipulation. arXiv preprint arXiv:2410.0800,, 2024.
Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xu Huang, Shu Jiang, al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.0660 2025.
Hao Chen, Jiaming Liu, Chenyang Gu, Zhuoyang Liu, Renrui Zhang, Xiaoqi Li, Xiao He, Yandong Guo, Chi- Wing Fu, Shanghang Zhang, et al. Fast-in-slow: A dual-system foundation model unifying fast manipulation within slow reasoning. arXiv preprint (rXXi:2500.009555 2025.
Suyi Chen, Hao Xu, Ru Li, Guanghui Liu, Chi-Wing Fu, and Shuaicheng Liu. SIRA-PCR: sim-to-real adaptation for 3d point cloud registration. In IEEF/CVF International Conference on Computer Vision, ICCV, pp. 14348-14359 2023.
Xiaoyu Chen, Jiachen Hu, Chi Jin, Lihong Li, and Liwei Wang. Understanding domain random- ization for sim-to-real transfer. In International Conference Learning Representations, ICLR. OpenReview.net, 2022.
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems, RSS, 2023.
Jack Collins, David Howard, and Jurgen Leitner. Quantifying the reality gap in robotic manipulation tasks. In 2019 International Conference on Robotics and Automation (ICRA), pp. 6706-6712 IEEE, 2019.
Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Acde: Automated creation of digital cousins for robust policy learning, arXiv preprint 4rX1v:2410.07408, 2024.
Shengliang Deng, Mi Yan. Songlin Wei, Haixin Ma. Yuxin Yang, Jiayi Chen, Zhiqi Zhang, Taoyu Yang, Xuheng Zhang, Wenhao Zhang, et al. Graspvla: a grasping foundation model pre-trained on billion-scale synthetic action data. arXiv preprint arXiv:2505.032233, 2025.
EmbodiChain Developers. Embodichain: An end-to-end, gpu-accelerated, and modular platform for building generalized embodied intelligence., November 2025. URL https: httttp::///ittbbl com/
DexF Embodiiciii.. CiabodiChhin.
Haoshu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains. IEEE Transaction of Robotics, 3955):392933455, 2023.
Yu Fang, Yue Yang, Xinghao Zhu, Kaiyuan Zheng, Gedas Bertasius, Daniel Szafir, and Mingyu Ding. Rebot: Scaling robot learning with real-tu to-sim-to-real robotic video synthesis, 2025. URL https: // www.aawwwv..o....aa.aao wttp:///////..////// 110.5003...11666
12

---

Published as a conference paper at ICLR 2026
Figure AL Helix: A vision-language- model for generalist humanoid control, February 2025.
URL httph: /tttps://www.. fwg.rei.u..... wtt.::.iiiiee.e.aiiaaaaaaa. aww.w/s////aaeeeeaaaaaaa... /hixx Accessed: 2025-04-15.
Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: Appli- cations, challenges, and the future. The International Journal of Robotics Research, 2024. doi: https://doi.org/10... wttp://orr/11117700777/888888000 1177700788369922113383
Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation using low-cost whole-body teleoperation. In Annual Conference on Robot Learning, CoRL, 2024.
Philip Gage. A new algorithm for data compression. C Users J., 12(2):23-3,, 1994. ISSN 0898-9788.
Dibya Ghosh, Homer Rich Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Quan Vuong, Ted Xiao, Pannag R. Sanketi, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy. In Robotics: Science and Systems, RSS, 2024.
ByungOk Han, Jachong Kim, and Jinhyeok Jang. A dual process vla: Efficient robotic manipulation leveraging vlm. arXiv preprint WrXiv:2410.1554, 2024.
Pu Hua, Minghuan Liu, Annahella Macaluso, Yunfeng Lin, Weinan Zhang, Huazhe Xu, and Lirui Wang. Gensim2: Scaling robot data generation with multi-modal and reasoning llms. arXiv preprint arXiv:2410.03645, 2024.
Josip Josifovski, Sayantan Auddy, Mohammadhossein Malmir, Justus H. Piater, Alois Knoll, and Nicolás Vavarro-Guerreet Continual domain randomization. In IEFE/RSI International Confer- ence on Intelligent Robots and Systems, IROS, pp. 4965-4972, 2024.
Josip Josifovski, Shangding Gu, Mohammadhossein Malmir, Haoliang Huang, Sayantan Auddy, Nicolás Navarro-Guerrerd Costas Spanos, and Alois Knoll. Safe continual domain adaptation after sim2real transfer of reinforcement learning policies in robotics, arXiv preprint 4rXiv:2503.109 2025.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Paul Foster, Pannag R. Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-actoo model. In Pulkit Agrawal, Oliver Kroemer, and Wolfram Burgard (eds.), Conference on Robot Learning, CoRI., volume 270, pp. 2679-2713, 2024.
Guiliang Liu, Yueci Deng, Runyi Zhao, Huayi Zhou, Jian Chen, Jietao Chen, Ruiyan Xu, Yunxin Tai, and Kui Jia. Dexscale: Automating data scaling for sim2real generalizable robot control. In International Conference on Machine Learning, ICML., 2025a.
Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. RDT-lb: diffusion foundation model for bimanual manipulation. In International Conference on Learning Representations, ICLR, 2025b.
Yumeng Liu, Xiaoxiao Long, Zemin Yang, Yuan Liu, Mare Habermann, Christian Theobalt, Yuexin Ma. and Wenping Wang. Easyhoi: Unleashing the power of large models for reconstructing hand-object interactions in the wild. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR. pp. 7037-7047, 2025c.
Kenzo Lobos-Tsunckane and Tatsuya Harada. Point cloud based reinforcement learning for sim- to-real and partial observability in visual navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pp. 5871-5878 2020.
Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-hO; vision-lange aanoon-lnguag--cttiin pretraining from large-scale human videos. arXiv preprint arXiv:2507. /55977 2025.
Yecheng Jason Ma. William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. In International Conference on Learning Representations, ICLR, 2024a.
13

---

Published as a conference paper at ICLR 2026
Yueen Ma, Zixing Song. Yuzheng Zhuang. Jianye Hao, and Irwin King. A survey vision- vision-language action models for embodied ai. arXiv preprint arXiv:2405. 14093, 2024b.
Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. In Annual Conference on Robot Learning. CoRL, 2023.
Bhairav Mehta, Manfred Diaz, Florian Golemo, Christopher J. Pal, and Liam Paull. Active domain randomization. In Conference on Robot Learning, CoRL, volume 100, pp. 1162-1176, 2019.
Marius Memmel, Andrew Wagenmaker, Chuning Zhu, Dieter Fox, and Abhishek Gupta. ASID: active exploration for system identification in robotic manipulation. In International Conference on Learning Representations, ICLR, 2024.
Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou, Lunkai Lin, Zhiqiang Xie, and Ping Luo. Robotwin: Dual-arm robot benchmark with generative digital twins (early version). arXiv preprint wwXiv:2409.02920, 2024.
Fabio Muratore, Christian Bilers, Michael Gienger, and Jan Peters. Data-efficient domain ran- domization with bayesian optimization. IEEE Robotics and Automation Letters, 6(2):911-918,
2021a.
Fabio Muratore, Theo Gruner, Florian Wiese, Boris Belousov, Michael Gienger, and Jan Peters. Neural posterior domain randomization. In Conference on Robot Learning, CoRL, volume 164, pp. 1532-1542, 2021b.
Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.0255: 025023, 2024.
Abby O'Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, al. Open x-embodiment: Robotic learning datasets and RT-X models : Open x-embodiment collaboration. In IEEE Interna- tional Conference on Robotics and Automation, ICRA, pp. 6892-6903. IEEE, 2024.
OpenAL, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, Jonas Schneider, Nikolas Tezak, Jerry Tworek, Peter Welinder, Lilian Weng, Qiming Yuan, Wojciech Zaremba, and Lei Zhang. Solving rubik's cube with robot hand. CoRR, abs/1910.07113,, 2019.
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. FAST: efficient action tokenization for vision-language-action models. CoRR, 0abs/2001.09744 2025.
Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yer Yang, Ruslan Partsey, Ruta Desai, Alexander Clegg. Michal Hlavac, So Yeon Min, Vladimir Vondrus, Théophile Gervet, Vincent-Pierra Berges, John M. Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, and Roozbeh Mottaghi. Habitat 3.0: A co-habitat for humans, avatars, and robots. In International Conference on Learning Representations (ICLR), 2024.
Yuzhe Qin, Binghao Huang, Zhao-Heng Yin, Hao Su, and Xiaolong Wang. Dexpoint: Generalizable point cloud reinforcement learning for sim-to-real dexterous manipulation. In Conference on Robot Learning, CoRL, volume 205, pp. 594-605, 2022.
Yide Shentu, Philipp WU, Aravind Rajeswaran, and Pieter Abbeel. From Ilms to actions: Latent codes bridges hierarchical robot control. In 2024 JEEE/RSJ International Conference on intelligent Robots and Systems (IROS), pp. 8539-8546. IEEE, 2024.
Christian Smith, Yiannis Karayiannidis, Lazaros Nalpantidis, Xavi Gratal, Peng Qi, Dimos V. Dimarogonas, and Danica Kragic. Dual arm manipulation - survey. Robotics and Autonomous Systems, 130000000110111100 2012.
14

---

Published as a conference paper at ICLR 2026
Cong Tai, Zhaoyu Zheng, Haixu Long, Hansheng Wu, Haodong Xiang, Zhengbin Long, Jun Xiong, Rong Shi, Shizhuang Zhang, Gang Qiu, et al. Realmirror: A comprchensive, spen-source vision- language-action platform for embodied ai. arXiv preprint arXtv:2509.14687 2025.
Xudong Tan, Yaoxin Yang, Peng Ye, Jialin Zheng, Bizhe Bai, Xinyi Wang, Jia Hao, and Tao Chen. Think twice, act once: Token-aware compression and action reuse for efficient inference in 21200
vision-language-actiion models, 2025.
Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montser- rat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20020, 2025.
Gabriele Tiboni, Karol Arndt, and Ville Kyrki. DROPO: sim-to-real transfer with offline domain randomization. Robotics and Autonomous Systems, 166:104432, 2023.
Marcel Torne, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Reconciling reality through simulation: A real-to-sim--oo-ral approach for robust manipulation, 2024. URL httttt///aaxxxi org/ 2200b.22443.00 03949
Marcel Torne Villasevil, Anthony Simeonov, Zechu Li, April Chan, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. Reconciling reality through simulation: A real-to-sim-ttorral approach for robust manipulation. In Robotics:
Science and Systems, RSS, 2024.
Jun Wang. Yuzhe Qin, Kaiming Kuang, Yigit Korkmaz, Akhilan Gurumoorthy, Hao Su, and Xiao- long Wang. Cyberdemo: Augmenting simulated human demonstration for real-world dexterous manipulation. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR, pp- 17952-17963, 2024a.
Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, and Chuang Gan. Robogen: Towards unleashing infinite data for automated robot learning via generative simulation. In International Conference on Machine Learning, ICML, 2024b.
Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, and Feifei Feng. Dexvla: Vision-language model with plug-in diffusion expert for general robot control. arXiv preprint arXiv:2502.05855, 2025.
Annie Xie, Lisa Lee, Ted Xiao, and Chelsea Finn. Decomposing the generalization gap in imitation learning for visual robotic manipulation. In IEEE International Conference on Robotics and Automation, ICRA, pp. 3153-3160. IEEE, 2024.
Michal Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. Robotic control via embodied ofain-of-thought reasoning. In Annual Conference on Robot Learning, CoRL, volume 270, pp. 3157-3181, 2024.
Jianke Zhang, Yanjiang Guo, Xiaoyu Chen, Yen-Jen Wang. Yucheng Hu, Chengming Shi, and Jianyu Chen. Hirt: Enhancing robotic control with hierarchical robot transformers. In Conference Robot Learning, CoRL, pp. 933-946, 2025.
Jingwei Zhang, Lei Tai, Peng Yun, Yufeng Xiong, Ming Liu, Joschka Boedecker, and Wolfram Burgard. Vr-goggles for robots: Real-to-sim domain adaptation for visual control. IEEE Roboties and Automation Letters, 4(2): 1448811155 2019.
Jingyi Zhang, Jiaxing Huang. Sheng Jin, and Shijian Lu. lanionelanguage models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, TPAMI, 46(8): 6625-5644, 2024.
Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Roboties: Science and Systems, RSS, 2023a.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303. 10..222333.122.... 1(2), 2023b.
15

---

Published as a conference paper at ICLR 2026
Xufeng Zhao, Comelius Weber, and Stefan Wermter. Agentic skill discovery. arXiv preprint arXiv:2405.15019 2024.
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang. Yifei Feng, Mingxin Yang. Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501.12000 2025.
Ying Zheng, Lei Yao, Yuejiao Su, Zhang. Yi Wang. Sicheng Zhao, Yiyi Zhang, and Lap-Pui Chau. A survey of embodied learning for object-centric robotic manipulation. Machine Intelligence Research, pp. 1-39, 2025.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. RT-2: vision- vision-language-action models transfer web knowledge to robotic control. In Jie Tan, Mare Toussaint, and Kourosh Darvish (eds.), Annual Conference on Robot Learning, CoRL, volume 229, pp. 2165-2183, 2023.
A DETAILS OF SETTING AND IMPLEMENTATION FOR SIM2REAL-VLA
A.1 MODEL ARCHITECTURE & KEY PARAMETERS
Regarding the model architecture, we employ DiNOv2 as the visual encoder and T5-XXI as the language encoder. The action expert model comprises approximately 200M parameters, structured with an 8-layer transformer backbone featuring 256 hidden dimensions and 8 attention heads for action processing. The specific archiutecture of our action moodel can be seen the Figure below. This architecture is complemented by two additional transformer blocks of identical configuration dedicated to affordance inference and guidance, alongside multiple MLP adapters that facilitate dimensional alignment across action, observation, and affordance inputs.
Masked Imgs
Astongeashia Tresforeer
Decoupling
FAST taterizer
MAST detolennc
Acting System
Actions Chunce
Afford Execution
[0 Rrad!
Woi
Affordance
Proprioception
Figure 6: The detailed architecture of action model.
Regarding visual observation masking, the mask prediction module utilizes standard CNN-based architecture to process raw visual inputs and yield stable object masks. Through the implementation of joint training and domain randomization, the module ensures robust generalization across diverse objects and environmental conditions.
These masked visual observations, combined with language instructions, are subsequently fed into the affordance prediction model. Structured as a regressive transformer, this model outputs sequence of 2D keypoints projected from key poses into the camera image space as affordances, which effectively serve high-level plans.
To bridge the affordance- predicccooneiiiiinn subsystem and the acting policy, the action model is formally constructed as conditional autoregressive transformer. The pipeline initiates by employing diffusion-basee action expert to generate action trajectories through denoising, conditioned on the aforementioned masked visual observations and proprioceptive inputs. These refined action chunks are tokenized by a pretrained FAST tokenizer and embedded. Utilizing a tokenize- -henn- concatenate strategy, the model fuses these action embeddings with the predicted affordance outputs.
16

---

Published as a conference paper at ICLR 2026
ly, conditioned on language instructions and visual inputs, the transformer predicts logits for action tokens (or the <EOS> token), which are decoded into executable action chunks. Empirical evaluations demonstrate that this binding strategy significantly outperforms alternative architectures.
A pretrained validation model is also needed in affordance chain inferrence. Constructed as a regressive transformer classifier, the validtion modeal takes maksed visual observation and state as input, current target affordance as condation, and output a validation signal to label if the target affordance is achieved.
For the training protocol, we implement a cosine-annealing learning rate schedule with a maximum value of le-5 across 40,000 epochs, incorporating exponential moving average (EMA) enhance training stability. The training configuration utilizes a batch size of 8, requiring approximately 36 GPU hours to complete under these specified conditions.
A.2 CONFIGURING REWARDS IN VLA MODELS
Note that in robotic manipulation tasks, reward signals are not always explicitly defined; instead, they are often implicitly specified by the task goals themselves. For instance, in widely used robot control frameworks such as Lanoon-anguag-Acttnn
(VLA) (Kim et al. 2024). reward information is typically embedded in the language commands I that describe the desired outcome (e.g., "Please close the door"). . In such cases, the reward function can be interpreted as R(s.a) a) = 1 if the robot successfully completes the task, and R(s, a) O otherwise. When more detailed or nuanced reward structures are needed, AI agents can design sophisticated reward functions (Ma et al.] (022a)). These functions are crucial for reinforcement learning (RL). particularly following Supervised Fine-Tuning (SFT) of VLA models.
A.3 DETAILS ON REAL2SIM DATA PROJECTION
The goal of heterogeneous data projection is to map descriptive observations of real- tasks into a simulated environment, ensuring that the generated skills remain applicable to the target application. Following the approach in (Liu al., (202aa), this projection encompasses both scene-level and action-level mappings, as detailed below.
Scene Projection. We project the static scene information from the real world into a simulated environment. Inspired by Digital Cousins Dai et al.] (2024), we first extract per-object information from input RGB images. Each detected object then matched to its corresponding "digital cousin" "-s visually and functionally similar asset from our simulation dataset. For articulated objects (e.g., drawers, boxes), we further post-process them to create fully interactive simulated counterparts by aligning them with CAD models or synthesized assets. However, in cases where three- view images capture only partial scene information (e.g., occluded object surfaces), or when the retrieved scene fails to semantically align with the real-world context, we leverage Vision-Language Model (VLM) to identify problematic objects, revise the scene configuration. and regenerate a corrected version using objective generation model (Zhao et al. 2025). Such a pipeline can be automatically implemented the simulated environment. The specific prompts used to instruct the VLM for this corrective process provided in Listing and Listing below.
Action-Trajecton Projection. Given either an egocentric video of a human manipulating objects or teleoperated demonstrations performed in the real environment, we project both the actions and object interactions onto robot control signals within a simulated environment. These trajectories capture dynamic motion information and serve as seed demonstrations for downstream automatic skill acquisition.
1) Robot-Action Projection. We extract hand movement trajectories from two sources: (1) human hand motion egocentric videos and (2) teleoperated demonstrations using robotic interfaces. These trajectories are then retargeted to robotic end effeectrs, such as parallel grippers dexterous hands, by transforming human or telcoperated hand poses into control signals compatible with the target robot. In this work, we primarily execute manipulations using gripper the end effector. Accordingly, two representative fingers are selected as proxies for grasping, and their trajectories are retargeted to drive the open-close motion of the gripper jaws.
2) Robot- Object Interaction. To accurately capture the interaction between robot and object, we reconstruct the manipulated objects and their spatial relationship with the operator Liu al. (2025c).
17

---

Published as a conference paper at ICLR 2026
You are visual-in nispect on agent responsible for ensuring camera coverage of speci fied objecta.
Task: (task description)
Target object list: (object list]
Instructions:
1. Examine the provided Image carefully.
2. For each object in the target list, determine whether the object is **completely visible** (i.e., no obstruction) in the image.
3. If you find any object that is **obstructedra (partially blocked from view), output **only** the name (s) of those object from the 1ist, in comma-separated 1ist.
fully
4. If no object is obstructed, output "A11 objects fully visible".
Output format (exact) Obstructed objects: [object_namee object_name?, ...] (or)
All objects fully visible
Listing I: Prompt for visual inspection agent to avoid occlusion by ensuring complete object visibility.
You are visual and task-fcasibility evaluator for image-based asset inspection.
Task: (task_descripp
Target object list: /object_lists [ Real-world task context image/asset aaset description: (real world damaageddescrpption
Instructions:
1. Examine the image carefully to check whether each object from the target list is captured without obstruction.
If any object is obstructed, output the names of those object (s) from the list.
Format: Obstructed objects: [...] or "All objects fully visible".
Then proceed to step 2.
2. Assess whether the visual assets (i.e., what shown the image) would realistically support executing the real-world task described (i.e.,
- If you determine that the assets are **insufficiente* to complete match realistic
the real task, identify awwhich abject(s)** observation (for example: object missing, object appearance unrealistic, object placement wrong etc.).
Output: "Task feasibility: No issues with lobject_namet object_name2, ...]"
If the assets are sufficient, output: "Task foasibility: Yes".
Combined output format (exact): Obstructed objects: [object_namel, object_namez, Task feasibility: Yes (or)
Obstructed objects:
object_namel. object_name2, ....
Task teasibility: No - issues with lobject_name?s object_name4, name4, ....
Listing 2: Prompt for comprehensive task feasibility evaluation, combining visibility assessment with real-world context alignment.
18

---

Published as a conference paper at ICLR 2026
This involves determining the 3D pose at which the end effector (e.g., hand or gripper) engages with the object (e.g., a cup), ensuring that the projected action trajectory reflects realistic physical interactions. We reconstruct 3D object meshes and poses from sequences of image frames within the demonstration videos or teleoperation logs. These object trajectories are then jointly optimized with the corresponding end-effector trajectories ensure proper alignment in 3D space. This process is fully automatic and does not require manual intervention. For each frame, the optimization refines the physical contact between objects and end effectors, modeling accurate contact dynamics such as grasp stability and force application. We further smooth and regularize the resulting action trajectories to ensure temporal coherence and realism in the simulated environment.
Clarification on the Real2Sim prior and "zero-shot" Sim2Real transfer. In this work, we follow the convention in recent real-to-Sin2Real approaches (Fang et al. 2025] Torne et al. (024) and use the term zero-shot Sim2Real to indicate that the control policy is learned entirely from simulated experience, without any fine-tuning or gradient- baased updates on real-world robot data. Our Real2Sim module consumes small set of real teleoperation and human video trajectories solely to reconstruct task instances and configure simulation environments (e.g., initial states, camera viewpoints, and target poses). These real trajectories policy network: the policy is trained only on rendered simulated observations and rewards generated in the reconstructed environments. Thus, Real2Sim acts prior over task configurations rather than an additional source of real-robot training data, and our reported results should be interpreted as zero-shot deployment of a policy that has never been updated on real sensor frames or real robot trajectories.
not used as direct supervision signals for the
A.4 DETAILS ON GENERATIVE SCENE SCALING
The goal of generative scene scaling is to bridge the Sim2Real gap and enhance the generalization capabilities of robot policies. Policies trained simulation often fail when deployed in real-world environments due to domain gaps and distribution shifts between the two domains. We primarily adopt the approach described in (Liu et al. 2025a) for scene-level feature sampling, which provides a good initial range for simulation parameters. However, the kinematic constraints of robot manipulators are typically not fully captured by large language models, leading to sampled scene-level features (e.g.. object poses) that may result in kinematically infeasible trajectories.
To achieve more efficient and higher-qualit, trajectory generation, we implement a robot workspace- aware scene scaling method. First, we develop robot workspace analyzer that precomputes the reachable workspace region using uniform or Monte Carlo joint position sampling combined with forward kinematics. This analysis provides the complete reachable Cartesian space of the robot end-effector. Subsequently, we perform pose sampling within the intersection of the reachable workspace and the object distribution range obtained from the initial scene-level feature sampling. This approach ensures that generated object poses remain within the robot's 's kinematic reach, thereby guaranteeing feasible trajectory generation. An example can be seen the Figure 7 Beyond object-level constraints, we extend the workspace-awarr sampling to robouc-level features. We leverage the same robot workspace analyzer to validate that sampled robot end-effector poses fall within the kinematically reachable region. For camera configuration sampling, including position, orientation, and field of view parameters, we compute the intersection ratio between the camera's view frustum and the robot's reachable workspace volume. By constraining the camera placement such that a substantial portion of its view frustum overlaps with the robot's operational space, we ensure optimal visual coverage of task-relevant regions. This workspace-awand camera positioning strategy enhances the quality of visual observations and improves the robustness of vision-based robotic policies.
A.5 DETAILS ON AUTOMATIC SKILL ACQUISITION
To construct the dataset used for training Sim2Real- -LA,, we designed a multi-stage pipeline for automatic skill acquisition, which combines atomic action primitives with large multi-modal language models (MLLMs). The process unfolds follows.
Action Bank Construction. We first curated set of atomic actions (c.g.. grasp. lift, rotate, etc) that serve as the basic building blocks for manipulation. Each action is implemented with standardized
19

---

Published as a conference paper at ICLR 2026
Workspace examination result showcase
context (scale here, identified by VLM) examination
Figure 7: Succees and fail cases of real2sim projection eamined by workspace analyzer and VLM respectively on asset and action level.
20

---

Published as a conference paper at ICLR 2026
interface, enabling consistent invocation by higher-level planning modules. This collection forms Action Bank.
Task Decomposition with Task Agent. For each target task, we employ multi-modal LLM (GPT-4o in this paper) as a task agent, Given a natural- language instruction and visual context, the task agent decomposes the overall objective into sequence of sub-goals. It generates step-by-step plan, specifying both the order of execution and the high-level rationale behind each step.
Action Invocation with Code Agent. To translate high-level plans into executable robot behaviors, we introduce a second MILM, the code agens. Conditioned on the sub-goals and reasoning generated by the task agent, the code agent selects and invokes the appropriate atomic actions with appropriate configurations (c.g., rotation degrees) from the Action Bank. This design separates task reasoning from low-level control, reducing hallucinations and ensuring that generated action sequences remain grounded in available primitives.
Key Pose and Trajectory Generation. Each atomic action is associated with a representative key pose of the end-effector (e.g., pre-grasp pose, lifted pose, rotated pose). For each key pose, we apply generalized inverse kinematics (IK) solver to obtain feasible joint configurations. A trajectory planner then interpolates between consecutive poses, producing smooth and executable motion trajectories.
Success Condition Generation. Beyond producing trajectories, a task is completed successfully. We prompt the task agent to propose success conditions for each sub-goal (e.g., the bottle lifted above the cup). These conditions are refined through human feedback and integrated into the pipeline automatic evaluation signals during data collection.
essential determine whether
Dataset Assembly. Finally, we store the atomic instructions, joint trajectories, success conditions, detected affordances, and rendered visual observations structured training samples. This ensures that each data point captures not only robot motion but also the reasoning and affordance information underlying it.
A.6 SPECIFICATION OF SIMULATION ENGINE
The simulation platform robotics simulation and learning platform designed to accelerate research in robot skill acquisition, Sim2Real transfer, and large-scale training. By integrating GPU-accelerated physics simulation, high-fidelity rendering, modular learning environments, and multimodal large language model (MLLM) agents with embodied reasoning capabilities, EmbodiChain provides a unified framework for developing and benchmarking robotic intelligence at scale. Its architecture emphasizes efficiency. realism, and extensibility, enabling researchers to rapidly prototype and evaluate advanced algorithms across diverse tasks and robot morphologies. Below, we detail its core specifications:
is EmbodiChain (Developers) 2025).
which is next-generation
System Architecture. EmbodiChain is built on a modular, GPU-accelerated framework with three interconnected subsystems:
Simulation Engine: A high-performance rendering and physics backend that supports real-time interaction, large-scale parallelism with extensible APIs, and seamless integration with learning frameworks.
Robot Learning Environments: suite of standardized, OpenAl Gym-compatible environments with modular functionality for domain randomization, affordable and trajectory generation, reward design, offline dataset collection, online data streaming and more.
Embodied Intelligence Framework: A unified architecture for vision- saaion--anguag-acttiinn (VLA) and vision-language model (VLM) design, training, and deployment. It supports both imitation learning from demonstrations and reinforcement learning through environmental interaction, enabling scalable development of multi-modee robotic agents.
A.7 EMPIRICAL STUDY ON AFFORDANCE CHAIN LENGTH
Since the proposed method utilizes the affordance chain for reasoning, it is anticipated that the length of the inferred affordance chain significantly influences the overall performance of the model. To quantitatively assess the impact of the inferred affordance chain length, we conduct real-world
21

---

Published as a conference paper at ICLR 2026
experiments across all six tasks, with models trained using affordance chain lengths ranging from K to K 3. The results are presented in Table below.
Table 5: Number of successful/total trials across different manipulation tasks and affordance chain lengths.
Task Affordance Chian Length
K=1
K=2
K=3
Single-Arm Water Pour
17/20
10/20
11/20
Dual-Arm Water Pour
16/20
11/20
8/20
Table Rearrangement
16/20
12/20
13/20
Hand-Over and Place
5/20
4/20
Basket Pick-and-Place
9/20
9/20
4/20
Pan Open Pick-and-Plaa Place
7/20
3/20
5/20
The result above have indicated that, K=1 produced the best performance across six tasks. We interpret this outcome indicating that, for our domain, extending the chain beyond one affordance introduces redundancy rather than helpful contextual action guidance.
A.8
REAL-WORLD EXPERIMENT SETUP
To enhance the reproducibility of our experiment results, we listed several critical initialization details here:
1. Regarding the random seed, we use seed value of 42 for all random sampling processes.
The initial object poses are sampled independently. the distribution for all tasks outlined in Table and it mirrors the simulation setup, with the same original reference point located at the base of the robot.
Table 6: Task details including object, xy position, and Z axis rotation.
Task
Ohject
XY Position
Axis Rotation
Single-Arm Water Pouring
Bottle
[0.67, 0.83), [0.06, 0.22]
[0, 0]
Cup
[0.67, 0.83), [-0.22, -0.06]
[0.
Dual-Arm Water Pouring
Bottle
[0.67, 0.83], [0.06, 0.22]
[0.
Cup
[0.67, 0.83), [-0.22, -0.06]
[0. 0]
Table Rearrangement
Plate
[0.575, 0.675), [-0.05, 0.05]
[0. 0]
Fork
[0.35, 0.50), [0.11, 0.21]
[455, +45]
Spoon
[0.35, 0.50), [-0.21, -0.11]
[-45, +45]
Items Hand-Over and Place
Pen
[0.52, 0.68), [0.035, 0.195]
[45,+45]
Holder
[0.5, 0.65), [-0.4, -0.2]
[0,
Basket Pick-and-Place
Milk box
[0.81, 0.93), [0.06, 0,22]
[-15, +15]
Basket
[0.65, 0.85], [-0.2, 0.0]
[-15,+15
Pan Open and Place
Pan
[0.4, 0.6], [0.0. 0.2]
[0.
Carrot
[0.51, 0.711, (-0.1, -0.3]
[-15, +15]
3. The initialization of the robotic arm joint angles for these tasks is also detailed in Table 7 which corresponds to the random initial xpos setup in the simulation, with a range of +0.02m in xyz direction for all tasks.
Table 7: Initialization of robotic arm joint angles for each task.
Task
Initial joint (following the parsing order in PhysX)
Singe-Arm Water Pouring
[1B00000AE2222000000000000000000
Dual-Arm Water Pouring
+4301100441222000000000006111111
Table Rearrangement
Items Hand-Over and Place
[0150110001112220011100000000000
Basket Pick-and-Place
0L30:100.L4412000066000000010000
Pan Open Place
[4301100012222000122200000000000
22

---

Published as a conference paper at ICLR 2026
4. The extrinsic parameters of the wrist camera, more precisely, its relative pose the attached link, are taken directly from the official URDF of the CobotMagic. For the main binocular camera, calibration is conducted using the CCTag algorithm, yielding error of 3.8mm.
A.9 ABLATION STUDY ON ARM-DECOUPLING vs. JOINT LEARNING
To assess the effectiveness of the proposed arm arm-decoupling design, we conducted an ablation study comparing with a joint-learning baseline. In the joint-learning setting, a single control module predicted the actions of both arms based on the full visual observation. In contrast, the arm-decoupling design employed two separate control modules, each receiving only the visual feedback associated with its corresponding arm.
We evaluated both models on two representative tasks: (i) a single-arm "Water Pouring" task and (ii) a bimanual "Items Hand-Over and Place" task. For each task, we measured the success rate in simulation, the success rate in the real world, and the average number of control steps required to complete the task. The results summarized in Table
Table 8: Comparison between the joint-learning baseline and the proposed arm arm- dem--dcuulling design on two manipulation tasks. We report success rates in simulation and the real world, as well as the average number of control steps.
Single-Ane Arm Water Pouring steps
Items Hand Over and Place sim
Method
sim
real
real
steps
Joint learning
0.86
0.75
178.6
0.32
0.15
390.0
Arm decouple
0.92
0.85
174.6
0.62
0.40
370.2
As shown in Table 8
the arm-decoupling strategy achieved comparable or slightly better performance than joint learning on the single-an pouring task in both simulation and real-worke settings, while also reducing the average number of control steps. More notably, the arm arm- con-decouplnng design substantially improved both simulation and real-world success rates for the bimanual hand-over task, together with reduction in the average number of steps. We interpret these results as evidence that decoupling reduces cross-arm interference: each arm controller can focus on its own relevant visual feedback, thereby avoiding the redundancy and complexity introduced by processing combined wrist-camera observations for simultaneous joint control.
A. FEW-SHOT WORLD ADAPTATION AND EFFICIENCY
The quantitative results are summarized in Table We analyze the impact of data quantity in Figure and detail the training efficiency in Figure
Impact of Real Data Quantity (Scaling). As visualized in Figure g baseline methods (To and xfa)) exhibit a monotonic improvement with increasing real data, relying heavily on demonstrations to correct their poor zero-shot performance. contrast, our Sim2Real VLA starts with strong zero-shot baseline (85% on Rearrangement). Notably, we observe temporary performance dip at 5 demonstrations (dropping to 60%) before recovering to peak performance (90%) at 10 demonstrations. This suggests that very small amount of real data (5 eps) may initially disrupt the strong simulation prior due to distribution shift ("unlearning" the sim policy), whereas 10 demonstrations are sufficient for the model to effectively adapt and bridge the Sim-to- RR-l gap.
Training Dynamics and Efficiency. Figure 9 (Top Row) details the training curves across different data strategies. While baselines require real data to reach acceptable performance, maintains high success rates throughout the training process. We further analyze the cost required to reach these results in the bottom rows of Figure 9 FL (Tan et al. 2025):
method
We estimate FL.PSs using the formula from
FLOPs = (1 - R) [Lp (And2 + 2d + 2ndm) - (L- Lp) (4m 2n + 20,dm)
Our method proves significantly more efficient in both metrics. As shown in Figure (Bottom Row), the Sim2Real VLA converges to high performance in approximately 4 hours, whereas To requires over 10 hours to achieve comparable results.
23

---

Published as a conference paper at ICLR 2026
Table 9: Success Rates with Few-Shot Real Data. Comparison across Sim Only, Real Only (10 demos), and Sim-then-Real (5/10 demos) strategies. Note the non-monotonic behavior ("dip") in our method at 5 eps compared baselines. Best results per task are bolded.
Model
Data Strategy
Rearrangement
Basket
Sim Only
0.30
0.15
Real Only (10 eps)
0.65
0.40
TO
Sim-then-Real (5 eps)
0.65
0.35
Sim-then-Real (10 eps)
0.70
0.45
Sim Only
0.40
0.25
Real Only (10 eps)
0.80
0.50
fast
Sim-then-Real (5 eps)
0.70
0.45
Sim-then-Real (10 eps)
0.85
0.55
Sim Only
0.85
0.50
Real Only (10 eps)
0.75
0.40
Ours
Sim-then-Real (5 eps)
0.60
0.35
Sim-then-Reat (10 eps)
0.90
0.60
Rearrangement Data Efficiency
Basket Placement Data Efficiency
1.0
1.0
n
0.8
0.8
Sim2Real VLA
Success 40K)
40K)
0.6
0.6
0.4
- 1b
Sim2Real
10
10
Number Real Demonstrations
Number Real Demonstrations
(a) Rearrangement
(b) Basket Placement
Figure 8: Data Efficiency Scaling, Success rates (at 40k steps) vs. number real demonstrations. Baselines improve monotonically. Our method shows "dip" eps (due to distribution shift disrupting the sim prior) but recovers to SOTA performance at 10 eps.
A.11 ROBUSTNESS AND ACCURANCY OF OUR METHOD FACING SIM2REAL PERCEPTION GAP
Our segmentation module CNN-based mask preki-predittion network trained purely on domain- randomized simulation data and jointly optimized with the control policy. The main paper (Table shows that the overall system relying on these masks performs well in real-world manipulation. Here we provide additional quantitative evidence of the robustness and Sim2Real transfer of this segmentation model.
For each of the six manipulation tasks, first sample 20 observation states from simulator rollouts, together with the corresponding proprioceptive states and predicted masks. We then replay these proprioceptive states the real robot collect the corresponding real images and their policy- generated masks. The real images are segmented by SAM, and the SAM masks are downsampled to match the resolution of our model. We compute the mean IoU for two comparisons: (i) real vs. sim masks, and (ii) real vs. SAM masks. The results are summarized Table T0 And example result can be seen in Figure 10 below.
24

---

Published as a conference paper at ICLR 2026
sim2real VLA on rearr
sim2real VLA on basket
0.0
06
am-rialln
0.7
Curceee cate
2.6
013
0.0
10000
15000
20000
25000
30000
25000
40000
10000
15000
20000
25000
30000
20000
40000
nairing steos
mraiiing steps
(a) Training Curves: Data Strategies (Rearr)
(b) Training Curves: Data Strategies (Basket)
real,
real,
0.7
C.4
2.6
[smm-10 11mm100
Ismn-10 VLA numell real)
Computee
(c) Efficiency: Success vs. Compute (Rearr)
(d) Efficiency: Success vs. Compute (Basket)
Task: reart 10 real, x-axis hours
Task: basket real, K-axis in hours
38
0.7
033
pil lsin 110 roal)
pio biin bimn-1Droall
pio Moi (ilm vie reall
plo Hoi (sims wao reall
2rer2
simitroal
25
20
25
30
10
35
20
35
10
Raining Limi (huund)
Taiining Lime (baun)
(c) Efficieney: Success Time (Rearr)
(f) Efficiency: Success Time (Basket)
Figure 9; Analysis of Training Dynamics and Efficiency. (a-b) Training curves of Sim2Real VLA under different data strategies. The Sim-then-Real (10 eps) strategy yields the best final performance. (c-d) Success rate vs. compute (TFLOPs). (e-f) Success rate vs. wall-clock time (Hours). All efficiency plots (c-f) use the Sim-then-Reet Real (10 eps) setting. Our method converges significantly faster (~44 hours) and with less compute than baselines.
Figure 10: Visualization of predicted mask by segmentation model from our method and SAM, first taken simulator, the rest two fromthe real-world replay.
To test generalization across sensor configurations and action distributions, additionally collect 20 teleoperated executions of the single-arm water-pouring task using a camera with different placement and calibration from standard setup. We apply the segmentation model from the pour-water
25

---

Published as a conference paper at ICLR 2026
Table 10: Mean IoU between segmentation oulputs across six tasks. "real vs. sim" compares masks
predicted on real vs. simulated images under matched robot states. "real vs. SAM" compares masks predicted on real images with -eM-geeer pseudo ground-trume
Task
real vs. sim
real vs. SAM
Single-Am Water Pouring
0.85
0.78
Dual-Arm Water Pouring
0.83
0.81
Table Rearrangement
0.76
0.70
Items Hand-Over and Place
0.78
0.75
Basket Pick-and-Place
0.77
0.82
Pan Open and Place
0.65
0.69
pretrained checkpoint and compute IoU between its predictions on the new real images and the corresponding SAM masks. The result is reported in Table 11
Table 11: IoU between segmentation predictions real images from different camera setup and SAM masks for the single-arm water-pouring task.
IoU / Task
Single-Arm Water Pouring
real vs. SAM
0.78
And example result be in Figure below.
Figure 11: Visualization of predicted mask by segmentation model from our method and SAM on out-of-domain images from different robot and camera setup.
26


Reviews:
[{"review_id": "IQkfPAhDXw", "reviewer": "Reviewer_MEpH", "summary": "In this work, the authors propose Sim2Real-VLA, a Vision-Language-Action (VLA) model that is trained on synthetic data but can be applied to real-world manipulation tasks. The method consists of two components: a high-level Affordance Prediction model and a low-level Action system. The proposed approach outperforms existing baselines.", "questions": "Please see Weaknesses.\n\nHave the authors considered evaluating their approach on different robot embodiments to assess generalization across hardware platforms?\n\nHave the authors considered testing the method on deformable object manipulation tasks to further demonstrate robustness and versatility?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The authors tackle the relevant problem of sim2real transfer. They propose a novel approach that uses a Vision-Language-Action (VLA) model to generate affordances and structure the chain of affordances. This is an innovative idea that, in combination with the low-level Acting system, makes the proposed method effective while maintaining architectural simplicity.", "weaknesses": "In the main part of the paper, several important aspects are missing or insufficiently explained and should be clarified to improve clarity and reproducibility.\n\nFirst, the simulation engine is only mentioned in Appendix A6 as EmbodiChain. However, there is no reference provided, and no public information about this engine appears to be available. Given the importance of the simulator in the training pipeline, the authors should include a proper citation or at least a brief technical description in the main text.\n\nSecond, while some details of the VLA model are provided in the appendix, the paper would benefit substantially from a more detailed presentation of these components in the main body. This would help readers better understand how the model operates and how it integrates with the affordance prediction and action systems.\n\nOverall, the proposed method is interesting and demonstrates promising results. I am inclined to recommend acceptance at this stage. However, I would like to emphasize that without sufficient information about the simulator and the data generation process, I would be inclined to lower the score due to concerns regarding reproducibility and transparency.\n\n\nMinor Remarks\n\t•\tLine 68: There is an extra blank space (“Sim2Real-VLA . By coupling”).\n\t•\tLine 206: Domain randomization is reintroduced, though it was already mentioned in line 201.\n\t•\tLine 272: The authors mention Byte-Pair Encoding (BPE) but do not provide a citation or further explanation. A reference or an equivalent description would improve clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uHpWWupzRe", "reviewer": "Reviewer_nkJs", "summary": "Sim2Real-VLA is a dual-system VLA model trained entirely based on simulation data. It designs a framework in which the high-level planner parses language instructions into a \"supply chain\", and the low-level executor tracks and verifies segment by segment through tokenized action space. Cooperate with the automated Real2Sim-SceneScaling data generation workflow. This work focuses on innovation on the data side. It first \"back-projects\" real tasks into simulation, and then uses automatic scaling + skill generation to produce large-scale simulation data. That is, although the method details of Real2Sim are adopted, the overall top-level goal is to bridge the gap between Sim2Real. The experimental scale of the paper is large and the tasks are diverse. It has achieved a success rate improvement of over 35% on real robots for six long-cycle control tasks, providing a new paradigm for zero-shot transfer from pure simulation to the real world. It is suggested that the author explicitly state in one sentence that \"Real2Sim is a means and Sim2Real is the end\", so as to better eliminate the potential misunderstanding caused by the title \"Sim2Real\" but the extensive use of Real2Sim in the article.", "questions": "1. Is the length K of the affordable chain determined through offline statistical analysis or dynamically predicted in real time? To clarify the generation mechanism and selection strategy for the affordable chain length, could you provide details regarding the decision-making logic, underlying network architecture, and the distribution of K across different tasks?\n2. Have there been any instances of failure in the Real2Sim projection within the current framework? If so, what is the observed failure rate? In cases involving unsolvable inverse kinematics (IK) or semantic mismatches, how is the rollback mechanism triggered? It would be helpful to include representative failure cases along with comparative analyses before and after corrective actions.\n3. For future work, would it be possible to release the initialization configuration files used in real-world experiments—such as random seeds, object pose sampling ranges, initial joint angles, and camera extrinsic parameters? Additionally, could you report the control frequencies and measured latency values for high-level affordance reasoning, low-level action token generation, and each component of the robot joint controller? Such information would significantly enhance the reproducibility and engineering applicability of the proposed method.\n4. To strengthen the statistical validity of the results presented in Table 2 of the main experimental section, please include a 95% confidence interval or an equivalent measure of statistical significance. This would support the claim of a 35% performance improvement and ensure robust interpretation of the findings.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Sim2Real-VLA introduces an affordable chain-based design that decomposes complex tasks into verifiable sub-tasks, effectively mitigating error accumulation through a conceptually novel framework.\n2. We have developed an automated data generation pipeline comprising real-to-sim projection, scene expansion, and skill generation. This pipeline eliminates the need for manual demonstration and demonstrates significant engineering potential in terms of scalability and large-scale deployment.\n3. Extensive real-world experiments were conducted, encompassing single-arm and dual-arm robotic systems, rigid and articulated structures, as well as periodic tasks of varying durations. Furthermore, multiple domain variations—including background environments, object configurations, and tabletop setups—were incorporated, thereby enhancing the robustness and credibility of the evaluation.\n4. The study compares against five recent and representative baseline methods, all of which were fine-tuned using identical simulation data to ensure a fair comparison. Additionally, attention visualization is provided to improve the interpretability of the model’s decision-making process.", "weaknesses": "1.At the methodological level, the generation mechanism of the affordable chain length K lacks a detailed explanation, with insufficient algorithmic description and absence of statistical distribution analysis. The Real2Sim projection phase does not include a clear failure detection criterion or a defined fallback strategy. Furthermore, the tokenized action space has not been evaluated for quantization errors; the impact of codebook size and sequence length in DCT+BPE on control accuracy remains unexplored, and no reconstruction error analysis or comparisons under high-precision tasks have been provided.\n2.At the engineering reproducibility level, critical initialization details are missing in real-world experiments, including random seeds, object initial pose distributions, robotic arm joint angle initialization, and camera extrinsic calibration errors. Additionally, there is no reporting on control frequencies and latency across high-level planning, low-level action token execution, and robot joint controllers. Moreover, fault classification statistics and descriptions of the safety monitoring layer are absent.\n3.In the main experimental evaluation, statistical significance analysis is lacking. For the real-world experiment with n=20, it is recommended to report a 95% confidence interval or p-value to assess the reliability of observed performance improvements. Without such metrics, it is unclear whether the reported over 35% improvement stems from systematic gains or random initialization effects. Complementary failure case analyses should also be included to enhance interpretability and robustness assessment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PMAXuNCLRh", "reviewer": "Reviewer_6zd7", "summary": "This paper proposes a Vision-Language-Action model trained entirely in simulation. The method adopts a hierarchical architecture that decouples high-level planning from low-level control execution, and further decomposes tasks into a sequence of affordance-governed atomic subtasks. By leveraging the simulation environment for large-scale data augmentation and affordance extraction, the model mitigates the Sim2Real gap without real-world demonstrations. In the execution module, the authors introduce an arm-decoupled control design for bimanual manipulation, preventing unnecessary coupling and interference between the two arms during task execution. Real-world experiments across multiple long-horizon tasks demonstrate that Sim2Real-VLA significantly outperforms several robotic policy baselines, indicating improved robustness and generalization under domain shift.", "questions": "Please refer to the Weaknesses part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Overall, the paper presents meaningful contributions in both Sim2Real-oriented algorithmic design and VLA architectural innovation. And the paper is well organized.", "weaknesses": "In 4.2, the authors state that a validation model is used to determine whether the current sub-goal has been successfully achieved and whether the system should proceed to the next affordance. Yet, no further explanation is provided regarding its formulation or training procedure. A brief description of this component should be included in the appendix. \n\nIn the experimental section, most evaluated tasks are compositions of pick-and-place style atomic tasks. While the results clearly highlight the advantage of hierarchical planning in long-horizon settings, the diversity of atomic-level skills remains limited.\n\nAdditionally, further ablation studies would strengthen the work, particularly comparisons isolating the effects of several architecture designs, for example: (i) arm-decoupling versus joint learning, and (ii) the proposed action executing validation mechanism versus predefined goal-based termination conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o3zROenXHA", "reviewer": "Reviewer_84R9", "summary": "The paper proposes Sim2Real-VLA, a dual-system vision-language-action framework trained entirely in simulation that aims to zero-shot transfer to real-world manipulation. A high-level planning system predicts a chain of object-centric affordances from multi-view observations and language, while a low-level acting system executes each affordance with a tokenized action policy; the two stages are connected through object masks and validation signals. The method also introduces (i) object-oriented observation adaptation with domain randomization and (ii) a tokenized action space via DCT→quantization→BPE to compress trajectories. Experiments on six long-horizon/bimanual tasks report an average 60.8% real-world success, >35% absolute over the best baseline using only synthetic training, plus robustness under several domain gaps.", "questions": "- Since your “tokenized action space (DCT→quantization→BPE)” is closely aligned with π0-FAST, report the comparison to π0-FAST. Please clarify how your tokenization differs from π0-FAST.\n- **Few-shot protocol with π0 / π0-FAST.** Will you include 0/5/10 demo finetuning per task for π0, π0-FAST, and your method with identical training schedules and observation stacks? \n- **VLA vs. non-VLA affordance→policy.**\nCould you add a baseline that uses your affordance chain but replaces the VLA with a compact policy head (e.g., transformer or MLP) conditioned on language and affordance parameters, trained end-to-end? Please report: (i) zero-shot and 10-demo performance, (ii) sensitivity to synthetic→real mask noise, and (iii) latency/failure-mode breakdowns when subgoal detection drifts.\n- **Robustness & recovery from affordance errors.**\nHow does the controller behave when the affordance predictor is perturbed (e.g., ±3–5 cm pose error, partial/eroded masks)?\n- Analysis on perception at deployment. See weakness.\n- **“Zero-shot” and Real2Sim prior—scope needs clarification.** See weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper offers a clean, end-to-end sim-only training pipeline that links an affordance-conditioned planner with a language-conditioned low-level policy via a tokenized action interface inside a single VLA backbone. While hierarchical affordance→policy decompositions are known, the consolidation of these components into a coherent recipe for zero-shot deployment is well engineered and practically useful. The problem setup and system diagram are clear, and the empirical results show sizable real-robot gains across multiple long-horizon tasks and domain-shift settings. Overall, the contribution is incremental in originality but strong in execution quality and clarity.", "weaknesses": "- **Lack comparison to π0-FAST**, Since your “tokenized action space (DCT→quantization→BPE)” is closely aligned with π0-FAST, this comparison is essential to demonstrate benefits beyond tokenization alone.\n- **Zero-shot only; lacks a practically meaningful few-shot comparison.**\nWhile zero-shot sim→real is interesting academically, in realistic deployments collecting 5–10 real demonstrations per task is routine and inexpensive. The paper does not evaluate whether the claimed advantages persist when each method is allowed a small, equal real-data budget. Please add a few-shot finetuning study (e.g., 0/5/10 demos per task) comparing your method against π0 and π0-FAST under identical conditions. Report success vs. #demos curves, wall-clock adaptation time, and compute.\n- **Unclear advantage over non-VLA affordance→policy pipelines (e.g., AnyGrasp-style)**.\nThe paper motivates a hierarchical VLA backbone but does not isolate why VLA is needed once an affordance chain is available. A strong control would be a non-VLA baseline: a GraspNet-like affordance estimator feeding a learned policy head (language-conditioned, no VLM/VLA backbone). Compare on the same tasks and sensors, and quantify concrete advantages: (i) robustness to affordance localization errors (error injection tests), (ii) long-horizon credit assignment (success vs. subgoal depth), (iii) compositional generalization to unseen language/object combinations, and (iv) closed-loop latency/stability. Without this, it is difficult to attribute gains to VLA rather than to the affordance decomposition alone.\n- Perception at deployment is under-specified. As we know, the perception gap is one of the most challenges in sim-to-real. Object masks are trained in sim with DR, but how robust are they on real images (sensor placement, calibration, segmentation model architecture, failure modes)?\n- **“Zero-shot” and Real2Sim prior—scope needs clarification.**\nThe Real2Sim step maps descriptive observations, including teleoperation and human video trajectories, into sim. Clarify whether any real sensor frames or trajectories directly supervise the policy (vs. only configuring simulation), and ensure the “zero-shot” claim explicitly excludes fine-tuning on real data.\n\n[1] Robust grasping across diverse sensor qualities: The GraspNet-1Billion dataset. IJRR 2023  \n[2] AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domains. TRO 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)