Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
Authors: Yue Liao
Pengfei Zhou
Siyuan Huang
Donglin Yang
Shengcong Chen
Yuxin Jiang
Yue Hu
Si Liu
Jianlan Luo
Liliang Chen
Shuicheng YAN
Maoqing Yao
Guanghui Ren
Venue: ICLR 2026 Poster (2026)
Keywords: World Model; World Action Model; Embodied AI; Vision-language-action; Robotic Manipulation
Primary area: applications to robotics, autonomy, planning

Abstract: We introduce Genie Envisioner (GE), a unified world foundation platform for robotic manipulation that jointly learns visual representations and action policies within a single video-generative framework. At its core, GE-Base is a large-scale instruction-conditioned video diffusion model that captures the spatial, temporal, and semantic dynamics of real-world robotic interactions in a structured latent space. Building on this foundation, GE-Act employs a lightweight flow-matching decoder to map latent representations into executable action trajectories, enabling precise and generalizable policy inference across diverse embodiments with minimal supervision. Trained on over 1 million manipulation episodes, GE supports both short- and long-horizon tasks, and generalizes across embodiments.  All code, models, and benchmarks will be released publicly.

Published as a conference paper at ICLR 2026
GENIE ENVISIONER: A UNIFIED WORLD FOUNDATION PLATFORM FOR ROBOTIC MANIPULATION
Yue Liao"
Pengfei Zhou'
Siyuan Huang"
Donglin Yang Jianlan Luo
Shengcong Chen Liliang Chen
Yuxin Jiang Yue Hu
Jingbin Cai
Si Liu
Shuicheng Yan
Maoqing Yao
Guanghui Ren
AgiBot Genie Team
LV-NUS Lab
BUAA
https://geele--eviiitoooo github. 10
Genie Envisioner World Foundation Model
Observation
Powered by AgiBet World real world robotic manipulation episodes "Building video centric roboties world"
GE-Base
Helty Framse --00
Memory
Pick Place
Sorpper
World Foundation
Leog kabbon
Model
Condition
Generated Video
World Action Model
Astion Chunk
Action Execation
GE-Base Model
Action Moidef
T
$5 Nbx
GE-Act
$4 30 Hs.
GOLd 2Lethes
Figure 1: Overview of the Genie Envisioner World Foundation Model. Genie Envisioner (GE) is unified world foundation model that integrates robotic world generation and manipulation policy learning within a single vidco-generative framework. At its core is GE-Base, a large-scale video world model that encodes the spatial, temporal, and semantic structure of robotic interactions. Building on this foundation, GE-Act functions as a world action model that derives instruction-conditioned policies from the embodied video space enable embodied control.
ABSTRACT
We introduce Genie Envisioner (GE), unified world foundation platform for robotic manipulation that jointly learns visual representations and action policies within a single aadeo-generative framework. At its core, BE-Base is a large-scale instruttio-coondttioned video diffusion model that captures the spatial, temporal, and semantic dynamics of real-world robotic interactions in a structured latent space. Building this foundation, GE-Act employs lightweight flow-matching decoder to map latent representations into executable action trajectories, enabling precise and generalizable policy inference across diverse embodiments with minimal supervision. Trained on over I million manipulation episodes, GE supports both short- and long-howe long-horizon tasks, and generalizes across embodiments.
INTRODUCTION
Embodied agents that sense, reason, and act in the physical world represent the next frontier of AI systems. At its core, a fundamental challenge remains: developing scalable and robust robotic manip- ulation capabilities the ability to purposefully interact with and control the physical environment via selective contact (Mason, 2001). Progress in this domain has spanned analytic methods (Stilman,
Equal Contribution. Project Leader. Corresponding Author.

---

Published as a conference paper at ICLR 2026
"Vellaw candy requires stap, candy requires storp. Fold box, place appropriate candy inside, seal. the box, and apply the cornect stamp hased the candy type.
11
box ready
condy placed the box
Box cLosed
Seal starped
Figure 2: Real-world demonstration of GE-Act on novel robot embodiment, Agilex Cobot Magic, unseen during pretraining. With only one hour of embodiment- and task-specific teleoper- ation data for post-training, GE-Act successfully executes a complex manipulation task involving fin-grained control of deformable objects and memory-based based decision making. Given a general packaging rule, the robot is required to complete the packing process for each item accordingly. Here, we showcase the detailed execution of the first packing cycle. The robot first folds deformable box, places a target object inside based on instruction, and closes the lid, rendering the object no longer visible. then correctly selects and applies the appropriate stamp, matching the object type, relying solely on internal memory. This showcases GE's generalization to new embodiments, its precise handling of deformable materials, and its ability to retain task-relevant memory across steps.
2007; Berenson al., 2009), model-based frameworks (Ebert al., 2018; Janner al., 2019; Naga- bandi et al., 2020), and more recently, data-driven approaches that learn manipulation policies from large-scale datasets (Brohan al., 2023; Kim et al., 2024; Black al., 2024; Bu et al., 2025b).
Although vision-language-aaee ization and task performance, it remains fundamentally constrained by reliance on language-centric representations. Mainstream VI.As compress visual observations into low-handwidth semantic embed- dings, which, while effective for high-level understanding, fail to explicitly encode future dynamics and therefore fall short in supporting fine-grained motor control. Attempts to combine VLMs with diffusion- based policies (Black et al., 2024) often destabilize training, as continuous action losses overshadow linguistic objectives and distort pretrained weights.
(VLA) imitation learning has achieved notable progress in general-
Recently, video-centre world models (Agarwal et al., 2025) have emerged as a powerful alternative, shifting the dominant paradigm from "vision - language" " to "language - future video." This shift enables models to represent motion dynamics, contact evolution, and fine-grained perceptual cues that inherently abstracted away in purely semantic embeddings, making video-centric representations a more suitable foundation for manipulation. Motivated by this direction, several recent works (Wu al., 2023a; Cheang et al., 2024; Hu et al., 2024b; Liang et al., 2025) explore policy architectures that derive actions directly from video diffusion latents or even decode actions directly from pixel- space video predictions. Although promising, these approaches still exhibit important limitations. Most rely on single-view video generation, which is misaligned with the multi-view egocentric perception typical of real robots. Moreover, both pixel-level action decoding and latent-based serial video-to-action actionn pipelines incur substantial inference latency and therefore require aggressive latent compression to remain tractable. Such compression leads to the loss critical fine- ane-grained spatial and contact cues, ultimately constraining the precision required for real-world manipulation.
To this end, we introduce Genie Envisioner (GE), a world foundation model for robotic manipulation that unifies ego-centric visual world modeling and policy learning within a single closed-loop generative architecture (Figure 1). At its core is BE-Base, an instruction cenetiionnconitiinnedd multi-view video diffusion model that explicitly predicts future head-vier and wrist-view observations, enforcing
2

---

Published as a conference paper at ICLR 2026
the
Video
Meomury
Num blocks
Noise
Frames
cross
Frbot
GE-Base
Sel--spatial
Attention
Self--patial
Altention
Self-sparial
Alteution
(1-9) Num blocks
cross attention
timestops
positions
view-spec
HilO regrendie
Left View
Genie Envisioner World Foundation Model General Framework
(b) Causal Block
Figure 3: Overview of the GE-Base. (a) An illustration of the autoregressive video generation process, Given multi-view visual conditions, the model generates the next multi-view video chunk conditioned on a language instruction. (b) A dedicated causal block facilitates information exchange across different views, ensuring spatial consistency during multi-view video chunk generation.
vrews-view consistency across perspectives. This multi-view formulation aligns the generative process with real robot embodiment and constructs a high-fidelity latent visual space rich in spatial, temporal, and semantic structure. GE-Base is trained on 3,000 hours of paired language demonstrations (about 1 million episodes) from BelBott-orld-Btaa (Bu al., 2025a), enabling it to autoregressively simulate manipulation trajectories and provide dense, multi-scale latent representations.
Building this representation, introduce GE-Act, lightweight parallel world action module that is block-wise aligned with GE-Base. Unlike prior serial video-to-action pipelines that compress video latents before policy decoding. GE-Act operates in parallel with the video generator and directly accesses the full-resolution, multi-scale latent features at each DiT block via cross-attention. This parallel, block-aligned design preserves the fine- nee-graindd geometry. motion cues, and contact details that are typically lost in compressed VLA pipelines, enabling significantly more accurate manipulation. To further meet real-time control demands, GE employs an asynchronous inference strategy: GE-Base updates its heavy video diffusion branch using single denoising step at a low frequency, while GE-Act updates at a higher frequency with multi-step action denoising. This slow-fast scheduling provides dense action outputs and sparse video refreshes, dramatically reducing computation without sacrificing control fidelity.
We comprehensively evaluate GE in both embodied video generation and policy learning across diverse real-world manipulation tasks. GE-Act achieves low-latency end-to-end control, generating 54-step torque trajectories within 200 on commodity GPU. It executes tasks precisely on the in-domain AgiBot G1 and generalizes effectively to novel systems such as Dual Franka and Agilex Cobot Magic with only one hour of teleoperated demonstrations, surpassing task -ase-specific basclines (Black et al., 2024; Bjorck et al., 2025; Bu et al., 2025b). GE-Act also performs reliably across industrial and household scenarios. Beyond short-horizon tasks, its visual world modeling supports oonghooizoon, memory-intensive sequences (Figure 2). We further assess GE-Base on EWMBench (Yue al., 2025), benchmarking embodied video generation against state-of-the-ard models. GE-Base consistently outperforms alternatives, highlighting its role as the foundation of GE.
Together, these contributions position Genie Envisioner as a practical, scalable foundation for real-world manipulation, facilitating downstream research.
GE-BASE: WORLD FOUNDATION MODEL
In this section, we introduce GE-Base, the core of Genie Envisioner. It extends general video generation into embodied predictive representation that anticipates robot-environment interactions from task instructions and initial observations. Unlike generic video models, BE-Base is fully egocentric multi-view generator, synthesizing head and wrist perspectives with enforced cross- view consistency to build coherent embodied spaces. It adopts a video DiT with robotic-adaptive pretraining, transferring knowledge from large-scale video corpora the embodied domain.
3

---

Published as a conference paper at ICLR 2026
2.1 BASIC ARCHITECTURE
To model long-horizon robotic manipulation, we adopt an autoregressive chunk-wise video generation framework. At autoregressive step the world model W predicts a chunk of N consecutive frames, denoted x The prediction is conditioned on the initial multi-view observation Xo. the instruction embedding (g), and long-term sparse memory m0:--1. The memory m is constructed by sparsely sampling keyframes from previously generated chunks {xf ( enabling the model retain extended temporal context. The autoregressive generation process is expressed as
xi.N = ( mo:--1, T(1)).
As illustrated in Figure 3, at each step W receives synchronized observations from V onboard cameras, such as a head-mounted view and two wrist-mounted views. Each view provides its initial observation X its sparse memory frames m and view-specific noise map z'. These inputs encoded by shared video encoder E: vó E(x)), and v {(m).-1), )) and each token is enriched with 3D rotary positional embedding and a learnable view embedding:
RoPE(t, h, w) + v² e'iew
The final per-view input visual tokens are u = v vi ll z'). and tokens from all views are concatenated and passed into the DiT backbone, along with timestep and instruction embedding.
To ensure consistency across viewpoints, we introduce cross-view attention in selected transformer blocks. Tokens from all views are temporarily merged into single latent sequence, allowing each view to attend to others and promoting coherent geometry and motion across perspectives, For efficiency, only a subset of blocks use cross-view attention, while the rest process views independently, achieving balanced trade-off between multi-view coherence and computational cost.
To balance efficiency and modeling capacity, we employ the compact LTX-Video (HaCohen et al., 2024) architecture as the backbone for W.
The model is trained with a latent diffusion objective. Given VAE latents of the target video chunk and a noisy latent = (1 -11-0 o1 + c,c, generated using Gaussian noise E ~ NN, I) at timestep т, the world model predicts the denoising velocity va. Supervision is applied only future frames via a conditioning mask M, giving the training loss
Lvideo = w(T) [l(vo (c- o M)
This unified modeling paradigm enables W to jointly capture spatial layouts, temporal dynamics, and semantic intent, yielding coherent and controllable predictions of embodied robotic behavior.
2.2 WORLD MODEL PRE-TRAINING
A core challenge in robotic world models is adapting general video generation to the structured dynamics and semantics of embodied interaction. To address this, we design a multi-stage pretraining framework that progressively aligns spatiotemporal representations with the distribution of real-world robot behavior. Our data pipeline incorporates sparse memory frames randomly sampled from prior history, which increase prediction difficulty and enhance robustness temporal variation, ultimately improving generalization across diverse manipulation scenarios.
Data Curation. We pretrain on the AgiBott-orldd-er World-Beta dataset (Bu et al., 2025a), which contains about IM high-quality dual-arm manipulation episodes. Each trajectory is annotated with language instructions, multi-view observations, and action policies, spanning diverse tasks, objects, and environments. For video-based modeling, we extract synchronized streams from three calibrated cameras and ensure semantic alignment with paired instructions, yielding high-quality text-video pairs that reflect coherent manipulation behaviors.
Stage I: Multi-Resolution Temporal Adaptation (GE-Base-MRR To bridge generic video repre- sentation learning and robotic motion dynamics, we pretrain GE-Base on 57-frame clips sampled at 3-30 Hz. Each sample includes four sparse memory frames to enhance temporal diversity. Clips are compressed into an 8-frame latent space via pretrained video VAE and optimized with denoising objective. This exposes GE- Base-MR to diverse motion speeds and partial observations, enabling spatiotemporal representations invariant to sampling rates.
4

---

Published as a conference paper at ICLR 2026
days
Agibot World
Video Frimes
Vidao Ensodee
Video DiT
Sampling Rato-3-33007
GE-Base Traininng Pipellns
Stage Multt-Resolation Adapetion
Stage 1: Action Pre training
cequenvec 30 its
GE-det Training Pipeline
Video cecnder
D11 Fuustin
Vidao Decoder
Encoder
Video DIT
Video
FransFormsr
1M+ Roal World Trajectory Data
Stage Spect-pecciicc video Mdptattin
24 hours
P
Task specifici Data
Vidro Ennoder
Video DiT
Viden Trames
Sampling Rate=S1t2
Twist
Pack
Close
ctepe actien
Action DiT +
artiion line
Stage Frequency Aligument
Stage Speciffic Action Specialiiation
Figure 4: Training Pipelines of GE-Base and GE-Act. Left: GE-Base is pretrained on AgiBot- World-Beta and training includes domain adaptation with high-frame-rate sampling for robustness, followed by low-frame-rate fine-tuning to align temporal resolution with downstream policy learning. Right: GE-Act is derived from GE-Base through three-stage pipeline using text-video policy triplets. It first performs action-space pretraining, then applies two-stage task adaptation on task data.
Stage II: Low-Frequency Policy Alignment (GE-Base-LF), To align with the temporal abstraction of downstream control, we fine-tune GF-Base-MR on 9-frame clips sampled at Hz, with four additional memory frames context. Sequences encoded into two latent frames by frozen video encoder, while only the generation components are updated. GE-Base-LF is optimized to capture semantically meaningful transitions under sparse sampling, enabling reliable video feedback the granularity of action steps. This provides the foundation for subsequent action model pretraining.
2.3 ROBOTIC MANIPULATION VIDEO GENERATION VIA GE-BASE
We generate dual-arm robotic manipulation videos using GE-Base. Generation follows an autore- gressive scheme: each step produces video chunk conditioned on the initial observation, memory frames, and language instruction, and the process iterates until the instructed task is fully executed. During inference, memory frames are uniformly sampled at fixed intervals from prior chunks to ensure stable temporal dynamics and consistent predictions.
Scene
Motion
Semantics
GE-Base
0.9427
1.6676
2,0907
4,7010
Kling
0,8888
0,9440
2.0370
3.8698
Disenity
Hailuo
0.8577
0.5362
2,0186
3.4125
COSMOS
0.7963
0,7085
1,7824
3,2872
Open
0.9210
0.3442
1.8739
3,1392
ALTU
Tagl
LTX
0.9156
0.4002
1.6518
2,9676
CLIP
(a) Fine-Grained Evaluuaion of Video Generation Models
(b) Aggregated Evaluation Hierarchical Levels
Figure 5: Comparison of GE-Base and Video World Models. We compare GE-Base with SOTA video generation models on EWMBBenc. Evaluation spans scene, motion, and semantic levels.
We evaluate this pipeline on real-world dual-arm manipulation generation on EWMBench (Yue al., 2025), comparing GE-BBase against seven SOTA video generation models (Zheng et al., 2024; Kuaishou, 2025; MiniMax, 2024; HaCohen et al., 2024; Agarwal et al., 2025) under standardized ext-and-image-to-vviee setting. As shown in Figure 5, GE-Base consistently outperforms baselines across multiple dimensions, excelling in temporal alignment and dynamic consistency, two core requirements for generating action-plausio semantics are comparable to generic models, GE-Base achieves substantially higher control fidelity, producing more precise and reliable task executions. This advantage is attributed to pretraining on large-scale robotic data, enabling better capture of task-relevant spatiotemporal dynamics.
and temporally stable robotic behaviors. While its motion
5

---

Published as a conference paper at ICLR 2026
Nateed Action
In Domsin
Video DIT Block
la DoF Whoded Hamusoed Balist
Video Incoka
Action "
Video DIT Block
Bivi
Cras Easteelinn
14 Aossena! Kanjsllos
DVI Block
Narm &
Ie nor Madil Mangsiattr
cvteryy
Video Tiken B GE-Bare
Aciee Takea B BB
Model
Manipulation Trajechary
Figure 6: Overview the GE-Act World Action Model. GE-Act extends GE-Base with a parallel action branch that converts visual latents into structured policy trajectories. It follows the same block design as GE-Base with reduced hidden dimensions for efficiency. Visual features are integrated through cross- cooss-atentionn for semantic grounding, and final actions produced via diffusion-basse how-matching pipeline that refines noisy predictions into coherent trajectories. By default, action blocks cross-aftend to multi-scale video features; at inference, denoised visual latents cached with few denoising steps (M) and reused, whereas the action branch denoises for N > M steps.
3 GE-ACT: WORLD ACTION MODEL
Bridging high-level world modeling and low-level control is critical for deploying world foundation models in robotics. We present GE-Act, a plug-and-plaa parallel module that augments GE-Bar Base with lightweight 160M-paramete autoregressive decoder. GE-Act is block-wiss aligned with GE- Base, enabling it to parallelly access the full hierarchy of multi-scale latent visual features produced throughout the video diffusion backbone. This parallel design preserves rich spatial information across multi-view observations and allows GE-Act to consume high-resolution latent representations without waiting for explicit video decoding. Conditioned on both multi-viww perception and language instructions, GE-Act directly maps these multi-scale multimodal latents into structured action policies, enabling -folloiin without explicit video generation. This tight block- wise coupling between perception and control provides a scalable and efficient solution for real-time manipulation.
3.1 BASIC ARCHITECTURE
As shown in Figure 6, GE-Act mirrors the DiT depth of GE-Base while adopting a reduced hidden dimension for computational efficiency. At autoregressive step i, GE-Base processes visual tokens derived from the initial observation Xg, the sparse memory mo:r-1+ and the language embedding T(q), producing multi-scale latent features across its DiT blocks:
V; = B, Vins T())),
where Vin denotes the fused visual tokens from Xg and mo:r-1- notes that GE-Act uses a different form of memory is sampled directly from the robot's s historical visual observations rather than from previously generated frames (as in GE- GE-Base). This design ensures that GE-Act conditions on accurate, real-world perceptual history when producing actions.
In parallel, GE-Act operates entirely in the latent action space. It initializes action tokens Vact with Gaussian noise and updates them through a sequence of action-specific DiT blocks that attend to the corresponding multi-scale visual features:
paact (Zuet. CrossAttn (Zact- v:)).
ag
where Best is the i-th action DiT block. Because GE-Act is block-wise aligned with GE-Base, it receives multi-scale visual representations at matching depths, rather than relying on the final-layer visual latents in traditional VLA pipelines. This block-level alignment allows GE-Act to exploit high-resolution spatial cues and cross-view correspondences throughout the generation process.
Following the latent diffusion objective used in GE-Base, GE-Act trains its policy decoder with a noise-conditioned velocity-matching loss. Given ground-truth actions u, sampled noise e, and timestep r with noise level o,, form noisy actions
(1 o O+E,

---

Published as a conference paper at ICLR 2026
and GE-Act predicts denoising velocity vgg The supervision target is the diffusion velocity giving the loss
= = | (E u)
where w(( is the timestep weighting function used modern diffusion models. This objective mirrors the GE-Base training formulation and enables GE-Act learn smooth, temporally coherent action trajectories directly in the latent space.
3.2 TRAINING PROCEDURE
We adopt a two-stage training paradigm inspired by standard VLA manipulation frameworks, consist- ing of task-agnostic pretraining followed by task-specific adaptation.
Pre-training. We pretrain the action model on AgiBot- World- Beta to adapt GE-Base for policy learning. The world model is initialized from GE-Base-LF with frozen parameters, while only the action decoder is updated. To reduce cost, video generation is disabled; instead, four low-rate (5 Hz) visual memory frames are used, and the model predicts 54-step high-frequency (30 Hz) actions.
Task-specific adaptation. For downstream tasks, we adopt a two-stage fine-tuning pipeline of video adaptation and action specialization. In the first stage, only video generation components are updated on composite dataset of AgiBot- World and task-specifie data, with the latter upweighted for alignment. the second stage, the full model is fine-tuned on task-specific data capture fine-grained dynamics, following the same sampling strategy.
3.3 ASYNCHRONOUS INFERENCE
To improve the efficiency of parallel GE-Act urchitecture, introduce Slow-Fast Asynchronous Inferencee, which combines two independent forms of asynchrony. The first component is diffiusion- step asynchrony: the video DiT performs only single diffusion step each time it refreshes the visual latent features, while the action decoder continues to apply multi-step denoising to preserve the accuracy and stability required for fine ane-grained control. This design substantially reduces the computational burden of the heavy video module, yet still provides visual representations that are sufficiently informative for policy decoding. The second component is frequency asynchrony: the video diffusion branch operates at low update frequency, whereas the action decoder runs at a higher frequency to support responsive action prediction. These schedules correspond our inference modes, where GE-Act Slow updates both branches at the same rate, and GE-Act Fast updates video Hz and actions at 30 Hz. This slow-fast scheme enables sparse video updates and dense action generation, allocating computation where is most effective and allowing GE-Act Fast support a 54-step prediction window and the execution of 30 action steps within 200 ms on an RTX 4090, During training, hidden states are initialized with Gaussian noise avoid repeatedly loading full videos, and during deployment, the combination single-ste step video denoising and low- www-freuuenc video updates provides lightweight but timely perceptual inputs that enable GE-Act integrate video world modeling with action execution in real time.
3.4
ACTION PLANNING VIA GE-ACT ON AGIBOT G1
GROOT
UniVLA
GE-Act Slow
GB-Act Fast
1.9
0.5
0.2
SR. Pucking detergent from conveyor Figure 7: Comparison of Task-Specifie Real-World Robotic Manipulation Performance the AgiBot G1 Platform. compare GE-Act with state-of-the VLA baselines across multiple real-world dual-arm robotic tasks, using two evaluation metries to assess performance.
SR
E2E
SRR
E2B
SR
E2E
SR
EIE
E2E
Heat food in the microwave
Make sandwich
a cup
table
To rigorously evaluate our approach in real-world robotic manipulation, we conduct experiments across five tasks, each targeting different aspects of control precision, task complexity, and generaliza-
7

---

Published as a conference paper at ICLR 2026
(a)
(0) Agilex Mogic
(c) RoboTwin
Fold closher
1.9
Fold clodiess
Fold curshooard box
1.9
0.8
0.4
0.4
022
0.2
0.
SR
E2B
SR
EZE
SR
0.0
rutier Handover
pot can par
GROOT
UniVLA
GO-1
GE-Act
Figure 8: Comparison of Task-Speciff Manipulation Performance Various Embodiments.
tion. Spanning both household and industrial settings (details in Appendix A. A. these tasks provide a comprehensive benchmark for assessing innruuction-conitiionnd control and closed-loop execution.
Evaluation Protocols. We evaluate performance using two metrics: Step-wise Success Rate (SR) and End-to-End Success Rate (E2E). SR measures the ratio of successful sub-steps total sub-steps, offering fine mne-grained insight into partial task completion. E2E considers only the final task outcome, allowing recovery from intermediate failures, thus better reflecting real-world deployment.
Performance Comparison on the AgiBot G1. We benchmark GE-Act against three leading VLA models, To. UniVLA (Bu al., 2025b), and GROOT NI (Bjorck et al., 2025), on AgiBot GI under identical protocols and fine-tuning data. As shown in Figure 7, GE-Act consistently surpasses baselines on SR and E2E across diverse tasks, benefiting from GE-Base pretraining that provides strong spatiotemporal priors. Qualitative results provided in Appendix A.4.
further compare two inference modes: synchronized and asynchronous, The asynchronous mode achieves comparable or superior results, particularly in latency-sensitive tasks, e.g., dynamic tracking, and significantly outperforms the standard mode on short-horizon tasks (e.g., packing detergent).
CROSS-EMBODIMENT GENERALIZATION WITH GENIE ENVISIONER
Beyond in-domain evaluation AgiBot GI. we test GE's cross-embodiment generalization two widely used platforms, the Franka arm and Agilex Cobot Magic, configured for consistency with our dual-arm framework. Direct deployment is infeasible due differences in embodiment and action space, so we adopt a few-shot protocol: collecting a small set of teleoperated demonstrations to fine- tune both GE and GE-Act. In addition to standard tasks, include challenging deformable-object manipulation, such as "cloth folding" and "box folding" to assess transferability and robustness.
Few-shot Adaptation. We two-stage adaptation strategy. First, the video DiT is adapted to the new embodiment via text-video demonstrations, while CL.P and video encoders remain frozen to preserve pretrained priors. Second, a new action DiT is trained from scratch on task-specific trajectories, reusing the GE-Base but learning task-specific action head tailored to the new platform. This pipeline enables effective transfer of perceptual and motor capabilities with minimal supervision.
4.1 GENERALIZATION TO DUAL FRANKA EMBODIMENT
We evaluate GE-Act for cross-embodiment generalization on the Dual Franka platform by adapting it with 250 telcoperated episodes (1 hour) on cloth folding, collected via a space-mouse interface. Following the same protocol Agilex Cobot, we fine-tune VLA baselines (Bjorck al., 2025; Black et al., 2024; Bu et al., 2025b) on the same dataset. As shown in Figure 8 (a), GE-Act consistently outperforms task-specific baselines on real-world execution. Notably, despite mo and GROOT N1 being trained with large-scale Franka data, GE-Act achieves superior performance.
4.2 GENERALIZATION TO AGILEX COBOT MAGIC EMBODIMENT
We evaluate the generalization capability of GE on the Agilex Cobot Magic using two tasks: "box folding" and "cloth folding". For each task, collect 250 high-quality teleoperated demonstrations (approximately hour data) using the Aloha-bas based telcoperation system (Fu et al., 2024). These demonstrations the adaptation dataset to fine-tune both (F-B and GE-Act.
8

---

Published as a conference paper at ICLR 2026
"Polding the hot"
"Folding
Figure 9: Visualization of Real-World Demonstrations with GE-Act on Agilex Cobot Magic Platform. This shows GE-Act adapted to a novel Agilex Cobot Magic embodiment, performing real-world robotic manipulation tasks, including cloth-folding and box-folding.
As shown in Figure 8 (b), we compare GE-Act with three SOTA VLA models (Bjorck et al., 2025; Black al., 2024; Bu et al., 2025b), all fine-tuned on the same dataset. GE-Act consistently outperforms all baselines. While UniVLA and GROOT N1 perform well on simple tasks such as pick- and-place, they fail on complex, fine-grained tasks, achieving 0% success. To shows stronger results on deformable object manipulation, yet GE-Act significantly surpasses it in complex deformable scenarios. This advantage stems from the large-scale pretraining of GE-Base, which provides robust spatiotemporal priors, enabling superior generalization across diverse platforms and tasks. Furthermore, as shown in Figure 9, real-world executions cloth and box folding demonstrate that the adapted GE-Act completes tasks with high precision and reliability novel embodiment.
4.3 GENERALIZATION TO ROBOTWIN
We further evaluate the cross-embodiment generalization on the dual-arm simulator RoboTwin (Chen al., 2025). We adopt all-in-one strategy, jointly fine-tuning GE-Act on four tasks using 200 demonstrations (50 per task), and directly evaluating this unified model across all tasks. In contrast, baseline methods (Black al., 2024; Bu et al., 2025a) perform task-specific adaptation. As shown in Figure 8 (c), GE-Act achieves better performance than and GO-1 (Bu et al., 2025a) on three of the four tasks, despite not using one-task-one--oen
setting, and is only slightly behind VLA methods on lift pot. This minor gap may be attributed to task interference introduced by joint training.
ANALYSIS
To systematically analyze our GE model, we conduct real-world robotic manipulation experiments on the AgiBot-Gi platform. We select stable and controllable task, "grasping red cylin- der from the table and placing it into paper cup with fixed positions", using dataset of 305 demonstrations. All models are trained for 40,000 steps under the same protocol. Our analysis focuses on the role of pretraining in action policy prediction, comparing general video pretrain- ing with in-domain embodied pretraining (AgiBot- World-Beta). As shown in Table 1, training from scratch adapting from general video model such ITX-Video yields near-zero success.
In contrast, in-domain pretraining achieves 64 SR and 81 E2E, which further improve 76% and 89% when combined with general video pre- training. We further validate the effectiveness of incorporating robot state as input, which yields additional performance gains. However, when applied directly to general video-pretrained mod- els, the inclusion of state information reduces performance due to short-cut learning effects. These results demonstrate that the GE-Base pre- trained world model offers strong representa- tions and serves as a solid foundation for action policy prediction.
Table Experimental Analysis of Pre-training. 'S' denotes inclusion of robot state; 'VidAW" indicates initialization from GE-Base, 'VidAda' indicates task-specific video adaptation statge mentioned in 3.2.
E2E
SR
VidAW
VidAda
w/ S
w/o S
w/ S
w/o S
X
0.15
0.30
0.05
0.11
X
/
0.05
0
0
x
0.81
0.49
0.64
0.26
0.37
0.76
0.37

---

Published as a conference paper at ICLR 2026
6 RELATED WORK
Vision-Languaag--Attiin Policies. Instruction-condiiinnn policies transfer from vision-language pretraining and are fine-tuned via imitation (Driess et al., 2023; Brohan et al., 2023; Kim et al., 2024; Black et al., 2024), or use VLMs as encoders/planners (Nair et al., 2022; Ahn et al., 2022; Huang et al., 2023). These approaches excel at semantic grounding but lack explicit generative rollouts for dynamics or counterfactual reasoning. Our framework conditions a generative world model on language. preserving direct path to control while enabling predictive simulation.
Embodied Video World Models. World models provide predictive structure for perception and plan- ning (Sutton & Barto, 1981; Chatila Laumond, 1985), evolving from analytic formulations (Murray et al,,, 2017) to neural variants in pixel and latent spaces (Ha & Schmidhuber, 2018; Finn et al., 2016; Ebert et al., 2018; Hafner al., 2019; Wu et al., 2023b; Hu al., 2024a). Recent video-centric approaches shift toward language-sfutrr video (Bruce et al., 2024; Agarwal et al., 2025; Russell et al., 2025; Jang et al., 2025) but typically emphasize single-view synthesis without closed-loop control. GE-Base instead targets egocentric multi-view generation with cross-view consistency and sparse-memory autoregression for long-horizon coherence, yielding an embodied latent space aligned with manipulation.
Coupling World Models to Action. Recent manipulation systems that use video generation or world models can be grouped by how perception is coupled to control: (A) Video-as-polie backbones: video model is used purely as a essual-tempraa encoder whose compressed latents are fed into a lightweight action head. Methods in this family (Hu et al., 2024b; Li et al., 2025; Liang et al., 2025; Wen et al., 2024) typically adopt a serial video->action pipeline, which risks losing fine-grained motion and contact cues due to heavy latent compression. (B) Unified video-action generators: these models jointly generate video and actions within single network (Cheang et al., 2024; Wu al., 2023a; Zhu al., 2025), and can be co-trained on action-free videos, but they incur pixel-level generation cost at inference time unless special acceleration strategies are applied. (C) WM-as- intermediate reference: the world model is placed outside the policy loop and produces auxiliary guidance rather than direct action outputs. Dreamitate (Liang et al., 2024), for example, extracts tool trajectories from generated videos and converts them into actions through heuristic planners. GE-Act cross-attends to multi-scale, multi-view latents from GE-Base without re-compression and uses an asynchronous slow-fast schedule for low-latency, precise manipulation.
CONCLUSION
In this paper, we proposed Genie Envisioner (GE) that unifies egocentric visual representation learning and policy learning within single closed-loop video generative world model. Through extensive experiments, we demonstrated that GE-Base provides strong spatiotemporal priors via large-scale embodied video pretraining, while GE-Act enables efficient and accurate action policy prediction through asynchronous inference. GE supports both short- and long-horizon tasks, and generalizes across embodiments only with one-hour demonstrations. Together, these components establish scalable framework that advances the integration high-level world modeling and low-level control, and lay the groundwork for future video-based neural simulators in robotics.
8 ACKNOWLEDGMENTS
This work was partly supported by the National Natural Science Foundation of China under Grant No. 62320106007 and NUS Start-up Grant A-0010106-00-00 We appreciate the AgiBot Genie Team for their invaluable contributions to data collection, real-world evaluation, and the provision of both robotic hardware and software support throughout this research.
ETHICS AND REPRODUCIBILITY STATEMENT
9.1 ETHICS STATEMENT
We follow the ICLR Code of Ethics. Subjects, data, and safety. Teleoperation and policy evaluation were done by trained adult researchers. They were paid above-average local rates. We did not collect
10

---

Published as a conference paper at ICLR 2026
PII biometric data. Any released media will be checked to remove identifying information. We use three data sources: AgiBot- World-Ber B-et under its license, other open academic benchmarks under their licenses, and our own teleoperation data. All robot experiments followed lab safety rules (emergency stop, speed/force limits, safe workspaces, and human-robot separation) We avoid hazardous tasks and add safeguards to reduce risks.
Fairness, environment, and disclosure. Our system does not use sensitive personal attributes, but scene or object bias can still appear. We try to reduce this by using diverse tasks and testing across embodiments. We will release checkpoints to support reuse. We will disclose funding and affiliations in the camera-ready. Sponsors did not influence our study. Upon acceptance, we have released code, checkpoints, data schemas, and filtered datasets with licenses and clear docs to help reproduction, together with known limitations, line with the ICLR Code of Ethics.
9.2 REPRODUCIBILIT STATEMENT
We aim to maximize reprodueibility. The main paper specifies model architecturex(Sectinn 2.1 and Section 3.1), training pipelines and hyperparameters (Figure 4, Section 2.2, and Section 3.2), and inference strategies (Section 3.3). The appendix details tasks, setups, and qualitative visualizations. We provide an anonymized codebase in the supplementary materials, including config files, data loaders, and evaluation scripts. For datasets, we use open-access academic benchmarks as cited, and we will release our weal-world datasets upon acceptance with data schemas, preprocessing scripts, and licenses. Additional qualitative demos are included as supplemental videos.
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge. Jinwei Gu, Siddharth Gururani, Ethan He, Jiahui Huang, Jacob Huffman, Pooya Jannaty, Jingyi Jin, Scung Wook Kim, Gergely Klár, Grace Lam, Shiyi Lan, Laura Leal-Taixe, Anqi Li, Zhaoshuo Li, Chen-Hsuan Lin, Tsung- Lin, Huan Ling, Ming- Yu Liu, Xian Liu, Alice Luo, Qianli Ma, Hanzi Mao, Kaichun Mo, Arsalan Mousavian, Seungjun Nah, Sriharsha Niverty, David Page, Despoina Paschalidou, Zeeshan Patel, Lindsey Pavao, Morteza Ramezanali, Fitsum Reda, Xiaowei Ren, Vasanth Rao Naik Sabavat, Ed Schmerling, Stella Shi, Bartosz Stefaniak, Shitao Tang, Lyne Tchapmi, Przemek Tredak, Wei-Cheng Tseng, Jibin Varghese, Hao Wang, Haoxiang Wang, Heng Wang, Ting-Chun Wang, Fangyin Wei, Xinyue Wei. Jay Zhangjie Wu, Jiashu Xu. Wei Yang, Lin Yen-Chen Xiaohui Zeng, Yu Zeng, Jing Zhang, Qinsheng Zhang, Yuxuan Zhang, Qingqing Zhao, and Artur Zolkowski. Cosmos world foundation model platform for physical AI. arXiv preprint arXiv:2501 2000:::500.03355,, 2025.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as can, not as say: Grounding language in robotic affordances. arXiv preprint arXiv:2204..0060
2022.
Dmitry Berenson, Siddhartha S Srinivasa, Dave Ferguson, and James J Kuffner. Manipulation plan- ning constraint manifolds. In 2009 IEEE international conference robotics and automation, pp. 625-632. IEEE, 2009.
Johan Bjorek, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang. Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t nl: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503. 14734, 2025.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. A vision-language-actioo flow model for general robot control. arXiv preprint arXiv:2410.2416: 2024.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexander Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wei Edward Lee, Sergey Levine, Yao Lu, Henryk
11

---

Published as a conference paper at ICLR 2026
Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong. Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-2: Vision-language-action models transfer web knowledge robotic control. arXiv preprint (rXXv:2307.18/// 2023.
Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holdee Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad ZoIna, Jeff Clune, Nando de Freitas, Satinder Singh, and Tim Rocktäschel. Genie: Generative interactive environments, In International Conference on Machine Learning, 2024.
Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xu Huang, Shu Jiang, al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503. 066669, 2025a
Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Univla: Learning act anywhere with task-centric latent actions. arXiv preprint arXiv:2505.06111, 2025b.
Raja Chatila and Jean-Paul Laumond. Position referencing and consistent world modeling for mobile robots. In Proceedings of the IEEE International Conference Robotics and Automation, volume 2, pp. 138-145, 1985.
Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. Gr-2: A generative video-language-an action model with web-scale knowledge for robot manipulation. arXiv preprint arXiv:2410.061 158, 2024.
Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Qiwei Liang. Zixuan Li, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint 2XX6::5506.1888 2025.
Danny Driess, Fei Xia, Mehdi S. M. Saijadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine. Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. PaLM-E: An embodied multimodal language model. arXiv preprint 2XXv:2023.03378. 2023.
Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. Visual foresight: Model- deep reinforcement learning for vision-based robotic control. arXiv 2018
preprint
Chelsea Finn, Ian Goodfellow, and Sergey Levine. Unsupervised learning for physical interaction through video prediction. In NeurIPS, 2016.
Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. In Conference Robot Learning (CoRI.), 2024.
David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv: 1803. J0122, 2018.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.0010 2024.
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International conference on machine learning, pp. 2555-2565. PMLR, 2019.
Edward S Hu, Kwangjun Ahn, Qinghua Liu, Haoran Xu, Manan Tomar, Ada Langford, Dinesh Jayaraman, Alex Lamb, and John Langford. Learning to achieve goals with belief state transformers. arXiv preprint 24. 2024a.
12

---

Published as a conference paper at ICLR 2026
Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations. arXiv preprint arXiv:2412.14803, 2024b.
Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, and Hongsheng Li. Instruct2act: Mapping multi-modality instructions robotic actions with large language model. arXiv preprint arXiv:2305.11176. 2023.
Joel Jang, Seonghyeon Ye. Zongyu Lin, Jiannan Xiang, Johan Bjorck, Yu Fang, Fengyuan Hu, Spencer Huang, Kaushil Kundalia, Yen-Chen Lin, et al. Dreamgen: Unlocking generalization in robot learning through video world models. arXiv preprint arXiv:2505. /2705, 2025.
Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization. Advances in neural information processing systems, 32, 2019.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09244 2024.
Kuaishou. Kling. https:: //aps://app... http://ppppkkk.gaa.... wwiikkinnnn.................a... htwp/aai..o/////////////// 2025.
Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified video action model. arXiv preprint MrXiv:2503.00200, 2025.
Junbang Liang. Ruoshi Liu, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, and Carl Vondrick. Dreamitate: Real-world visuomotor policy learning via video generation. arXiv preprint arXiv:2406. 21Xv:22406.6662, 2024.
Junbang Liang, Pavel Tokmakov, Ruoshi Liu, Sruthi Sudhakar, Paarth Shah, Rares Ambrus, and Carl Vondrick. Video generators are robot policies. arXiv preprint arXiv:2508.000 00795, 2025.
Matthew T. Mason. Mechanics of robotic manipulation. MIT Press, Cambridge, MA, USA, 2001. ISBN 0262133962.
MiniMax. Hailuo AI. nttps://hailuoai.ve
2024.
Richard Murray, Zexiang Li, and S Shankar Sastry. A mathematical introduction to robotic manipulation. CRC press, 2017.
Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. Deep dynamics models for learning dexterous manipulation. In Conference robot learning, pp. 1101-111 1112. PMLR, 2020.
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. In Conference Robot Learning (CoRL), 2022.
Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. Gaia-2: A controllable multi-view generative world model for autonomous driving, arXiv preprint 1.XXi:25033.022555 2025.
Mike Stilman. Task constrained motion planning in robot joint space. 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3074-3081, IEEE, 2007.
Richard Sutton and Andrew G. Barto. An adaptive network that constructs and uses and internal model of its world. Cognition and Brain Theory, 4(3):217-246, 1981.
Youpeng Wen, Junfan Lin, Yi Zhu, Jianhua Han, Hang Xu, Shen Zhao, and Xiaodan Liang. Vid- man: Exploiting implicit dynamics from video diffusion model for effective robot manipulation. Advances in Neural Information Processing Systems, 37:41051-41077 2024.
Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre- te-trrinnng for visual robot manipulation. arXiv preprint arXiv:23/2.13139, 2023a.
13

---

Published as a conference paper at ICLR 2026
Philipp Wu, Alejandro Escontrela, Danijar Hafner, Pieter Abbeel, and Ken Goldberg. Daydreamer: World models for physical robot learning. In Conference on robot learning. pp. 2226 2240. PMLR, 2023b.
Hu Yue, Siyuan Huang, Yue Liao, Shengcong Chen, Pengfei Zhou, Liliang Chen, Maoqing Yao, and Guanghui Ren. Ewmbench: Evaluating scene, motion, and semantic quality in embodied world models. arXiv preprini urXiv:2505. 09694, 2025.
Zangwei Zheng, Xiangyu Peng, Tianji Yang. Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all, March 2024. URL https: / giggthuubb. com/ wtt.ccoo//////eeteeeee aspeaitech/Open-
Chuning Zhu, Raymond Yu, Siyuan Feng, Benjamin Burchfiel, Paarth Shah, and Abhishek Gupta. Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets. arXiv preprint arX1v:2504.02792, 2025.
A APPENDIX
A.I TASKS DESCRIPTION
The real-world manipulation tasks include: (1) Make a sandwich: sequentially assembling bread, bacon, lettuce, and bread, which tests multi-object coordination, spatial reasoning, and procedural task execution; (2) Pour a cup of tea: involving grasping, precise pouring, and repositioning a teapot, highlighting the need for fine-grained motion control and dexterity in fluid manipulation; (3) Clean the table: requiring the robot to grasp a wiper and perform consistent wiping motions to remove surface stains, evaluating trajectory stability and compliant force application: (4) Heat food in the microwave: operating a microwave door, inserting a bowl, and interacting with buttons, challenging the system's s ability handle articulated objects and multi-stage interface operations; (5) Pack laundry detergent: grasping moving detergent bags from a conveyor belt and placing them into boxes, designed to assess dynamic perception, motion tracking, and industrial-scate manipulation.
A.2 EFFECT OF MULTI-SCALE VISUAL FEATURES CONDITION MECHANISM ON ACTION POLICY
latent features
World Model
Action Model
World Model
Action Model
T
O
O
T
(a) Multi-scale fusion
(b) Last-layer only
Figure 10: Comparison of GE-Act Performance under Two Variants Visual Condition Strategies.
Table 2: Effect of visual conditioning on GB-Act. Multi-scale fusion outperforms conditioningg only the last video layer feature.
Variant
SR 1
E2E +
Last-layer only Multi-scale fusion (ours)
0.64
0.81
0.76
0.89
We study how the choice of visual conditioning affects GE-Act. Prior VLA work commonly conditions the policy only the last visual layer. In contrast, our design injects multi-scale features aligned across blocks, enabled by the matched DiT structure between GE- Base and the action head. compare these two variants shown in Figure 10, under the same
To validate the effectiveness, data, training schedule, and inference setup. We validate this setting with the task, grasping red cylinder from the table and placing into a paper cup with fixed position.
14

---

Published as a conference paper at ICLR 2026
"Pick up the milk from the refrigerator"
"Place the held potato into the plastic hag in the shopping cart"
Figure 11: Multi-View Robotic Manipulation Videos Generated AgiBot G1 by GE-Base. We visualize robotic manipulation sequences generated by GE-Base across two tasks involving varied objects and environments. For each example, videos from three views are presented, i.e., the head-mounted, left-, and right-arm cameras, respectively.
Last-layer only: action blocks cross-attend to the final video layer.
Ours: action blocks attend to scale-aware fusion of video expert latent features aligned across DiT depth.
As shown in Table multi-scale fusion supplies richer cues in block-aligned manner, yielding more stable grasps and fewer recoverable failures.
A.3 REAL-WORLD GENERATION VISUALIZATION ON AGIBOT G1
As shown in Figure 11, GE-Base generates multi-view videos that accurately reflect diverse language instructions. The results highlight the model's 's ability to maintain spatial consistency across views, preserve background and scene structure, and produce stable, step-by-step execution aligned with the instruction semantics.
A.4 REAL-WORLLD MANIPULATION QUALITATIVE RESULTS ON AGIBOT G1
Figure 12 demonstrates GE-Act's ability to execute complex manipulation tasks precisely and reliably from natural language instructions on AgiBot GI.
A.5 REAL-WORLD GENERATION VISUALIZATION ON AGILEX COBOT MAGIC
As illustrated in Figure 13, our adapted GE-Act model generates coherent, Inaruction-conditionn multi-view videos for the cloth folding and box folding tasks. These videos accurately capture
15

---

Published as a conference paper at ICLR 2026
"Use the sponge held right urm wipe the clean"
"Pour water into the cup on the table"
"Make sandwich"
"Packing washing detergent from conveyor
Figure 12: Visualization of Real-Woor World Robotic Manipulation AgiBot G1 via GE-Act. Condi- tioned on natural language instructions, GE-Act generates and executes action policies on the AgiBot GI platform. The visual samples demonstrate the model's capability produce consistent, reliable, and contextually appropriate manipulation behaviors, showcasing its robustness and effectiveness in real-world environments.
both rigid and non-rigid object dynamics with high fidelity. The results demonstrate strong sistency across different camera views and showcase GE-Act's robust handling of complex object deformations.
"Folding grey clothes"
"Folding buee elothes"
"Folding the box"
Figure 13: Multi- View Video Generation on the Agilex Cobot Magic Robotic Platform by GE-Base. Visualization of instructioo-conditionn video generated by GE-Base for two complex folding tasks on the cross-embodiment Agilex Cobot Magic robot. Each row displays temporally sampled frames from multi-view sequence.
16

---

Published as a conference paper at ICLR 2026
A.6 REAL-WORLD GENERATION AND MANIPULATION VISUALIZATION ON DUAL FRANKA
Figure 14 illustrates the cloth folding task on the Dual Franka platform, including both the future- space video predictions by GE- GE-Base and the woaldworld manipulation results executed by GE-Act. The results indicate that GE effectively models task-relevant visual dynamics and generalizes to new embodiments for precise manipulation.
"Fold the yellow clothes
CONGS
(a) Multi- View Video Generation on the Dual Franka
"Fold the blue clothes"
(b) Real-World Robotic Manipulation on the Dual Franka
Figure 14: Visualization Robotic Video Generation and Real-World Manipulation on Dual Franka via GE.
A.7 LIMITATIONS
This work investigates unified world models for real-world robotic manipulation, coupling embod- ied video generation (GE-Bace Base) with an action model (GE-Act). While the framework advances spatiotemporal representation learning and low-latency control, several limitations remain:
Data coverage and source diversity. Pretraining relies primarily on SgiBot-World--ae Beta (real, dual-arm, multi-view) without incorporating heterogeneous sources such as large-scale web video, broader robot platforms, or high-variance simulation. This limits exposure diverse embodiments, sensors, and long-tail scene configurations. Although few-shot adaptation shows promise on Dual Franka, Agilex Cobot Magic, and RoboTwin, systematic robustness to OOD embodiments remains underexplored. Future work can integrate multi- pretraining and study domain-mixing strategies with explicit data provenance controls.
Embodiment scope and dexterity. Our evaluations focus on upper-body tabletop manip- ulation with parallel-jaw grippers and do not cover dexterous in-hand manipulation, tool use with tight tolerances, or mobile whole-body behaviors (navigation + manipulation). Extending GE to articulated hands and mobile bases will require richer actuation spaces, contact-aware objectives, and safety-constrained control.
Real-time video modeling vs. control bandwidth. GE's asynchronous inference reduces latency by running video prediction at Hz and actions at 30 Hz, but it still depends on periodic video denoising and cached latents. Highly dynamic tasks that demand faster updates may require further optimization.
Compute efficiency and accessibility. GE-Base and GE-Act training uses many GPUs. Although inference is efficient
single commodity GPU, the pretraining budget may limit adoption. Future work will pursue token pruning, quantization, low-rank adapters, and student models distilled from GE-Base to democratize training and deployment.
These limitations suggest concrete next steps. And we are leaving these for our future research topics.
17

---

Published as a conference paper at ICLR 2026
A.8 USAGE OF GPT IN PAPER WRITING
In the process of writing the paper, we used GPT only for minor refinement during the editing stage. Specifically, provided the initial draft and used GPT to improve the text by instructing it to "ensure the text is free of grammatical errors" and "refine the logical expression," among other instructions. This approach allowed GPT to enhance the clarity and coherence of the writing while maintaining the original ideas and structure, ensuring the paper met academic writing standards.
18


Reviews:
[{"review_id": "jNn3AuW7fr", "reviewer": "Reviewer_YQVX", "summary": "This paper proposes a novel Foundational World model for robotic manipulation based on Language-guided  Video Generation. The key contribution of the work is a Diffusion-based autoregressive multi-view video chunk generation model that learns to predict egocentric as well as eye-in-hand camera feeds for left and right hands in a bimanual robot setup. The proposed model is trained in 2 stages. The first is for training the model to learn spatially and semantically coherent video generation from sparse frames at sampled at different frequencies to make the generation more robust. The second stage performs a fixed low-frequency fine-tuning to capture sequential information useful for downstream control. The learnt latent space tokens from this video model (GE-Base) are then used for training a decoder for predicting continuous bimanual actions using flow matching. The proposed approach shows promising results both on robotic video prediction compared to other stat-of-the-art video prediction models as well as on subsequent downstream robotic manipulation tasks on a variety of robots in comparison to other state-of-the-art Vision-Language-Action Models.", "questions": "- Adding a mathematical explanation of how the losses are calculated would help improve the clarity of the paper.\n- In Sec. 2.1, what is the difference between the next chunk $x_{1:N}^{(t)}$ and the sparse memory $\\hat{x}_{0:t-1}$? \n- In Lines 197-198, what is the difference between $\\hat{x_t}$ and $v_{\\hat{t}}^{(i)}$? Isn't $v_{\\hat{t}}^{(i)}$ a part of $\\hat{x_t}$? Shouldn't the model output be $\\hat{x}_{t+1}$ if the model predicts the next video chunk? \n- In Figure 5, are the other methods trained/fine-tuned on robotic manipulation data as well? If not then, a fairer comparison is warranted, potentially by comparing against other robot world models as mentioned in the related works.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The multi-stage training especially with different varying frequencies is a clever idea to make the learned video prediction more robust to different execution speeds. The use of a FiLM style injection of the latents into the policy architechture rather than just using the last layer output is quite a nice way to ensure multi-layered information to be used more effectively for the downstream policy learning. The quality of the experiments are high and thorough. The paper is written clearly and understandably making it easy to follow.", "weaknesses": "- I did not fully understand what exactly the multi-view consistency is. Is it just the joint prediction of head, left and right video feeds? Or is in the structure of the attention layers used in the model?\n- The approach requires few-shot fine-tuning for any new robotic embodiment. While this is a problem in general with VLA style policy networks, I would imagine that the video generation model, if trained on a variety of robotic embodiments, such as from Open-X-embodiments or similar datasets, in addition to the AgiBot world dataset, could help reduce this need for fine-tuning the GE-Base world model and require only fine-tuning the action decoder.\n- Conceptually, the proposal of using video-based diffusion for learning latent spaces for robot manipulation has been explored previously. However, there is no mention of this in the paper, leave alone a comparison.\nWen, Youpeng, et al. \"Vidman: Exploiting implicit dynamics from video diffusion model for effective robot manipulation.\" Advances in Neural Information Processing Systems 37 (2024): 41051-41075.\nXu, Huilin, et al. \"Diffusion-Based Imaginative Coordination for Bimanual Manipulation.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025.\n- Notations in Sec. 2.1 are inconsistent with the use of bold font. The subscripts and superscripts aren't adequately explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r1hhI739dZ", "reviewer": "Reviewer_E6vL", "summary": "This paper proposes Genie Envisioner (GE), a large-scale instruction-conditioned video diffusion model consisting of two components: GE-Base, a foundation video diffusion model trained at scale; and GE-Act, a lightweight flow-matching action decoder operating on the latent representations produced by GE-Base.", "questions": "1. In Line 148, what do **$N$** and **$t$** represent in the notation $x_{1:N}^{(t)}$? How is the temporal step $t$ different from the frame index $N$?\n2. In Line 158, does GE-Base load pretrained weights from LTX-Video 2B? If so, how are the cross-view attention layers (described in Line 190) initialized?\n3. In Line 292, what is the difference between **$B_i^{vis}$** and **$W$** in Line 151?\n4. Could the authors provide a more detailed description or pseudo-code for Asynchronous Inference? This appears to be a central design element, yet it is not well explained.\n5. In Figure 7, the terms GE-Act Slow and GE-Act Fast are not introduced in the main text. Are they related to asynchronous inference?\n\nIf the authors adequately address these questions, I would be willing to raise my score. I also strongly encourage restructuring the paper to highlight its core technical contributions more effectively.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper demonstrates strong engineering efforts, including large-scale data and model training.\n2. This experimental performance is strong.", "weaknesses": "1. The macro-level architecture design—an action model built on top of world model representations—was, to my knowledge, first introduced in the GR-1 and GR-2 series. However, these prior works are not cited or discussed, which weakens the contextual positioning of Genie Envisioner within the literature.\n2. The paper’s presentation resembles a technical report more than a polished conference paper. Although it comprehensively covers implementation aspects, it fails to emphasize the key innovations, such as: GE-Act’s fully latent-space operation, and asynchronous inference mechanism,\n   which are novel techinques and deserve more prominence. This writing style also makes many details not clear enough due to the limitation of the number of main text pages (see questions below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dZtgleke3n", "reviewer": "Reviewer_XRta", "summary": "This paper proposes the Genie Envisioner (GE), which consists of a world model, GE-Base, and an action policy, GE-Act. GE-Base aims to unify egocentric visual representations through large-scale pretraining, enabling it to forecast multi-view observations in robot manipulation. Building on GE-Base, GE-Act achieves efficient and accurate action policy prediction via asynchronous inference. The authors provide comprehensive implementation details for both GE-Base and GE-Act, and extensive experiments demonstrate the strong performance of GE-Base and GE-Act.", "questions": "- Does GE-Base support co-tuning of both multi-view and single-view videos? If so, would the performance be improved?\n- World models often exhibit poor performance when generating long-term trajectories. I am curious about how long GE-Base perform on extremely long trajectories.\n- Training is limited to AgiBot-World-Beta, a single-platform dataset. Whether GE-Base retains knowledge from the original LTX model when applied to non-robotic scenes.\n\nI would be happy to raise my score if the authors could address my concerns.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Well-presented paper that provides comprehensive details and well-illustrated figures, enabling readers to fully understand the task. \n- GE-Base supports multi-view observation, which is quite helpful for robot planning tasks. \n- I like the idea of asynchronous inference, which provides a practical solution for incorporating the world model and policy.\n-The video results are impressive, and the extensive experiments prove the effectiveness of GE.", "weaknesses": "- GE-Base and GE-Act are designed for a specific robot configuration (one head camera and two wrist cameras). This weakens the potential of this pipeline to be leveraged in the general robotics community.\n- The two-stage VLA training pipeline, especially the specific task fine-tuning in the second stage, does not showcase the generalizability of the proposed policy. Is there a large performance drop when tested on out-of-distribution tasks?\n- I am missing a comparison between single-view and multi-view world models. Does predicting multi-view robot videos damage overall performance, considering that GE-Base is built on a single-view video generator? An experiment comparing single-view robot video prediction with multi-view robot video prediction (perhaps both using the head view for comparison) could help. \n- The idea of cross-view attention is rather simple, but it is completely fine if the results turn out well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2uzsHHLGAv", "reviewer": "Reviewer_tRPW", "summary": "The authors propose a joint video+action diffusion model based on transformers. One major difference to prior work is that they incorporate memory in the video diffusion model, where it is conditioned on features of previously denoised clips, in addition to the current frame observation and language instructions. The model is first fine-tuned for video generation. Then the video generation model is frozen and an action denoising module is learned, conditioned in the video DiT features. During inference, video is generated at lower FPS compared to actions for improved computational efficiency, while preserving precise control signal. The model is pre-trained on the large AGIBot dataset of robot demonstrations and fine-tuned for target embodiments and tasks. In the real world experimental evaluation it is shown to significantly outperforms VLAs, such as Pi0. In a more reliable sim evaluation on the recent RoboTwin benchmark the improvements over Pi0 are rather marginal.", "questions": "Please rewrite introduction and related work to properly reflect the contributions with respect to the prior work. \n\nPlease report results on an established sim benchmark, preferably on RoboCasa. Compare to sota video+action generation methods on this benchmark(s).\n\nPlease re-do ablations on a mid-size sim benchmark as well, and add the ablation of the memory module.\n\nPlease compare runtime to other efficient video+action diffusion models, like UVA.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is relatively well written and easy to follow. \n\nThe proposed model design is sound. \n\nThe model demonstrates strong results in real world and sim evaluation. \n\nIt also features high computational efficiency for a model based on expensive video diffusion.\n\nA few ablations are provided.", "weaknesses": "The authors completely ignore prior work on joint video and action diffusion. Specifically, VideoPolicy [Liang et al.,  arXiv'25], Video Prediction Policy [Hu et al., ICML'25], UVA [Li et al., RSS'25], Unified world models [Zhu et al., RSS'25], Dreamitate [Liang et al., CoRL'24], to name only a few most relevant ones. The only methodological novelty of the proposed approach with respect to the ones listed above is the introduction of memory, which is never shown to bring any benefits in the paper. Introduction and related work sections require a major revision to properly reflect the contributions. \n\nExperimental evaluation is also inadequate. Specifically, most of the evaluations are on real robots and hence are not reproducible. The only reproducible sim evaluation is on a small subset of the very recent, RoboTwin benchmark, which is not widely used in the literature, hence lacking strong baselines. To the very least the evaluation should be reported on the entire benchmark. Preferably, the authors should report results on the standard LIBERO and RoboCasa benchmarks instead, using their respective protocols. Priority should be given to RoboCasa, since LIBERO is essentially saturated. \n\nAblations are also not very informative as they are performed on a single real world task. The authors should instead ablate the value of large-scale video generation pre-training on a medium-scale sim dataset (in the same way as done in Table 4 in Video Prediction Policy). \n\nThe effect of the memory module - the only notable methodological contribution of this paper, has to be ablated on a reliable sim benchmark as well. \n\nRuntime has to be compared to other efficient video+action diffusion models, like UVA. \n\nThe GO-1 baseline is used in the experiments but it's never explained what it is.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)