World-In-World: World Models in a Closed-Loop World
Authors: Jiahan Zhang
Muqing Jiang
Nanru Dai
TaiMing Lu
Arda Uzunoglu
Shunchi Zhang
Yana Wei
Jiahao Wang
Vishal M. Patel
Paul Pu Liang
Daniel Khashabi
Cheng Peng
Rama Chellappa
Tianmin Shu
Alan Yuille
Yilun Du
Jieneng Chen
Venue: ICLR 2026 Oral (2026)
Keywords: world models
video generation
embodied AI
generative models
Primary area: generative models
TL;DR: By grounding assessment in embodied task success instead of video metrics, World-In-World provides a principled yardstick for future research on generative world models in the context of embodiment

Abstract: Generative world models (WMs) can now simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive perception for decision making. Progress on this question has been limited by fragmented evaluation: most existing benchmarks adopt open-loop protocols that emphasize visual quality in isolation, leaving the core issue of embodied utility unresolved, i.e., *do WMs actually help agents succeed at embodied tasks?*
To address this gap, we introduce World-In-World, the first open platform that benchmarks WMs in a closed-loop setting that mirrors real agent-environment interactions. World-In-World provides a unified online planning strategy and a standardized action API, enabling heterogeneous WMs for decision making.
We curate four closed-loop environments that rigorously evaluate diverse WMs, prioritize task success as the primary metric, and move beyond the common focus on visual quality; we also present the first data scaling law for world models in embodied settings.
Our study uncovers three surprises: (1) visual quality alone does not guarantee task success—controllability matters more; (2) scaling post-training with action-observation data is more effective than upgrading the pretrained video generators; and (3) allocating more inference-time compute allows WMs to substantially improve closed-loop performance. By centering evaluation on closed-loop outcomes, World-In-World establishes a new benchmark for the systematic assessment of WMs.

Published as a conference paper at ICLR 2026
WORLD-IN- WORLD MODELS IN A CLOSED- LOOP WORLD
Jiahan Zhang Muqing Jiang Nanru Dai' Taiming Lu Arda Uzunoglu' Shunchi Zhang Yana Wei Jiahao Wang' Vishal M. Patel Paul Pu Liang'
Daniel Khashabi Cheng Peng Rama Chellappa' Tianmin Shu Alan Yuille' Yilun Du Jieneng Chen 1:f
JHU *KKU *Princeton *MIT
Haarvard
Project Page: https://world wtw.:o......g.........
ABSTRACT
Generative world models (WMs)
simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive perception for decision making, Progress on this question has been limited by fragmented evaluation: most existing benchmarks adopt open- loop protocols that emphasize visual quality in isolation, leaving the core issue of embodied utility unresolved, i.e., do WMs actually help agents succeed at embodied tasks? To address this gap, we introduce World-In-World, the first open platform that benchmarks WMs in closed-loop world that mirrors real agent-environment interactions. Gorld-In--Worlee provides a unified online planning strategy and standardized action API, enabling heterogencous WMs for decision making. We curate four closed-loop environments that rigorously evaluate diverse WMs, prioritize task success the primary metric, and move beyond the common focus on visual quality; we also present the first data scaling law for world models embodied settings. Our study uncovers three surprises: (1) visual quality alone does not guarantee task training with action-observation data is more effective than upgrading the pretrained video generators; and (3) allocating more inference-ting compute allows WMs to substantially improve closed-loop performance. By centering evaluation on closed- loop outcomes, assessment of WMs.
conss--oooollllbiiiiy matters more; (2) scaling post-
establishes new benchmark for the systematic
Do Better Video Models Lead to Higher Embodied Success?
PER Active Recognition
Navigation
S.
World-In-World an open www.oppeppptteee platform
Wan
Cosmos
SVD
Embodied Palicy
SoTA video models
Question Answering
Manipulation
R
Runway
LTX
Hunyuan
Unified Action
Closed-Loop
Interaction
World models are ranked by embodied success, not flawless visuals
succoss
success
Teak auccess
camef
Soen exampies
count
Visuel quality
Leaderboard
Data Scaling
Inference-time Scaling
Correlation
Figure 1:
We introduce the first open benchmark to evaluate world models by closed- task success, analyze the link between task success and visual quality, and investigate scaling laws.

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
Recent advances in visual generation have sparked interest in world generation, a field focused on the creation of diverse environments populated with varied scenes and entities, with applications in entertainment, gaming, simulation, and embodied AI. The rapid progress in video generation (Brooks et al., 2024; Yang et al., 2024b; Wan et al., 2025), 3D scene generation (Fridman et al., 2023; Chung al., 2023; Yu et al., 2024; Koh al., 2023; Ling et al., 2025), and 4D scene generation (Bahmani et al., 2024b; Xu et al., 2024; Bahmani et al., 2024a) has demonstrated high-quality individual scene generation, highlighting the potential of these models as world generation systems.
Building on these developments, recent world generation systems (Yang et al., 2023b; Parker-Holder & Fruchter, 2025: Li et al., 2025c; Ye et al., 2025; Lu al., 2025; He et al.. 2025c) show promise as world models for embodied agents. Given an agent's 's initial observation and candidate action, such systems predict the resulting video, thereby estimating the future state of the environment. These cetion-condiiioned simulators mirror human mental models by forecasting future states and can provide missing context under partial observability. As a result, they offer a pathway to improved decision-making for embodied tasks that rely on perception, planning, and control.
Despite this promise, the community lacks a unified benchmark that evaluates visual world models through the lens of embodied interaction. Existing suites emphasize video generation quality (e.g.. VBench (Huang et al., 2024)) or visual plausibility (e.g., WorldModelBench (Li et al., 2025b)). The recent WorldScore (Duan al., 2025) offers a unified assessment for models that take an image and camera trajectory as input. However, no current benchmark tests whether generated worlds actually enhance embodied reasoning and task performance-for for example, helping an agent perceive the environment, plan and execute actions, and replan based on new observations within such closed loop. Establishing this evaluation framework essential for tracking genuine progress across the rapidly expanding landscape of visual world models and embodied Al.
In this work, we address this gap by proposing
Zero-shot Post-trained
World-In-World, which wraps generative World models In a closed-loop World interface to measure their practical utility for embodied agents. Specifi- cally, we present a unified strategy for closed-loop on- line planning and standardized action API to seam- lessly integrate diverse world models into closed- loop tasks. The online planning strategy allows the agent to look ahead by anticipating environmental changes and task rewards before committing to an
Runn my Gena
Task Task Success Rate (%)
64
Others
63
Wan2.14
61-
SvDt
Cosmes-P21
WanZ.2 A148
58
NWM
LTWVddeot
SVD SEDDS
Hunyuan
Wan2.1
57
Pathdreamer
action. The standardized action API harmonizes in- put modalities expected by different world models, so that each model can be controlled consistently within the same evaluation protocol. addition, we introduce a post- post-training protocol that fine-tunes pre- trained video generators using a modest amount of action-observation data drawn from the same action space as the downstream tasks, which allows us to examine their adaptation potential and to characterize a data scaling law.
56-
Wan2.2 5Bt LTXVidec
55-
Wan2.2
Cosmos 92
0.325 0.350 0.375 0.400 0.425 0.450 0.475 Gen. Quality (Aesthetic+Image Quality)
Figure 2: Task success rate vs. generation quality from VBench. t: post-trained with extra data. We defend that world models live and die by their closed-loop success, not flaw- less generated visuals.
World offers fair, closed-loop world interface to evaluate diverse WMs. We benchmark leading video generators (Wan et al., 2025; HaCohen et al., 2024; Kong et al., 2024) alongside task-focused world models (Bar et al., 2025; Koh al., 2023; 2021) in perception, navigation, and manipulation settings. Our findings reveal three consistent trends: (1) high visual quality does not necessarily translate into strong task success; (2) scaling post-training with cossevoobservttion data is more effective than upgrading the pretrained video generators; and (3) increasing inference- -iime compute via online planning substantially improves closed-loop performance. As shown in Figure world models with strong visual scores do not necessarily bring high success rates, which underscores the need for closed-loop evaluation when judging WM practical value for embodied agents.
Our work makes three main contributions:
2

---

Published as a conference paper at ICLR 2026
We introduce World-In-World, the first comprehensive closed-loop benchmark that evaluates world models through the lens of embodied interaction, moving beyond the common focus on generation quality.
We propose unified closed-loop planning strategy with unified action API, enabling diverse world models to be integrated and assessed within one framework across four embodied tasks.
We discover that high visual quality does not necessarily guarantee task success, and demonstrate how the performance of pretrained video generators can be substantially improved through training- time data scaling and inference-time scaling.
WORLD-N-WORLD A CLOSED-LOOP INTERFACE FOR VISUAL WORLD MODELS
Design overview. Our goal is establish benchmark that evaluates world -ordd-generation methods by their utility for embodied agents. Unlike prior work focused on generative quality, we develop a predictive-control framework to test how well world model supports online seaisinn-making.... The evaluation setting mirrors practical scenarios in embodied AI, emphasizing the interaction between prediction, control, and reward under closed-loop operation.
We detail the unified strategy for closed-loop online planning (Section 2.1) and the unified action API (Section 2.2), which together provide common interface across tasks and models. We then describe our task selection and evaluation protocol (Section 2.3). Finally, we present a post-training recipe that adapts a pretrained video generator into more effective embodied world model (Section 2.4).
Closed-loop online planning
Imagined Interactions + Real Interactions;
proposal
Candidate Action Plan Á, (1)
Unified Action API
g
Candidate Action Plan Á
Og
O3
Candidate Action Plan A (M
Toxt/ Rat/Camera Actions
Embodied Env.
Possible Future State 1
(1)
Possible Future State 2
)
<
D
D
D
Possible Future State M 6 (a))
Trevision
World Model de
At time step f, the agent receives the world state, represented by observation O,, and invokes a proposal policy proposal total of M candidate action plans. The unified action API (@) transforms each plan into the control inputs required by the world model. The world model
Figure 3: Closed-looy online planning in World-In-Wond
produce
2 then predicts the corresponding future states as observations Ó.. The revision policy Trevision evaluates all rollouts and commits to the best, yielding decision D This decision is applied in the environment, closing the interaction loop.
2.1 UNIFIED STRATEGY FOR GLOSED-LOOD ONLINE PLANNING
In Figure 3. we present a unified closed-loop strategy that uses visual world models for decision- making. It cycles through proposal, simulation, and revision. In proposal, the agent generates candidate plans; in simulation, each plan is rolled out by the world model to predict counterfactual futures; in revision, the agent top-scoring plan the environment, coupling model-based planning with real execution.
rollouts and refines its plan. Finally, the agent executes the
Let o, denote the agent's egocentric observation time step t.' Define the agent's future potential action sequence of horizon L starting time step A, [āc+1. de42, ... d.+L], where each
'The observation may be RGB, RGB-D, or another sensory modality. For clarity, use the generic notation throughout.
3

---

Each candidate plan A, (m)) is subsequently transformed by the unified action API C into the control inputs expected by the world model: r.m² C(A(Tm)) where 7/'²)) may include textual prompts, camera trajectories, or low-level action sequences, depending on the required format of the chosen Published as a conference paper at ICLR 2026
elementary action à is specified in either continuous action space discrete action space, i..., E V, with V denoting the set of action primitives available to the agent.
Our unified strategy can be formalized to the number of candidate plans M drawn from the proposal policy proppsaa- At time step t, given the current observation o, and the task goal g, the proposal policy Tprppsal samples M candidate action sequences that serve as future candidate plans:
policy-guided beam search. The beam width corresponds
(m))
Proposal
A Or: g),
M.
world model. The visual world model ge then performs counterfactual rollout based on these control
inputs, predicting the future world states Ó (m)) with horizon L:
O (pm))
gol O0| oz,
)
6 (m) [0 0 1
(m)
Then, the candidate plans and their simulated rollouts (A(m)) Ô (m)) evaluated and revised by the revision policy Frevision: which assigns a score to each trajectory and selects the decision that maximizes the expected reward. In the most general form, we write
D = Trevision (((A)T)) (m))
((()) 3 Oz, 8).
(3)
Here, D denotes the best decision according to Trevision at time step t. Depending on the task. D may represent high-level answer, recognition result, or a refined sequence of low-level actions, which renders the framework more general than classical Model Predictive Control (MPC) (Morari H. Lee, 1999), where optimization is typically restricted to sequences of actions.
common instantiation implements Trevision as score-and-select operator S. When the decision is an action sequence, selection is performed over the M candidate plans produced at time step t:
D = A (m²) where m== max max
((A(m) (m) Ó (!)) Ot, 8)
mE(1....MM)
Here, 5(.) denotes task-specific scoring function that estimates the expected reward or utility of candidate plan based on its simulated outcomes. Alternatively, Trevision may synthesize or update a new decision by aggregating information across the candidate set and their predicted consequences, rather than selecting one candidate verbatim.
Once the best decision D; is executed in the environment, the agent acquires at time step 1- The unified strategy then re-enters the proposal-simulatiin-ervii the newly observed state initiate the next round of proposal, simulation, and revision. In our framework, both proppscl and Trevision can be instantiated flexibly: they may be pretrained modules, such as sarge-scale vision-language models or diffusion policies, or simple rule-based heuristics. In our experiments, we explore multiple instantiations to systematically explore the flexibility and generality of our framework for different tasks.
observation loop, using
2.2 UNIFIED ACTION API
In this section, we present a unified action API that transforms an action sequence A into control inputs/ / that guide the world model, i.e I C(A). The action API is designed to be flexible so that the same interface can serve a wide range of world models and tasks. supports three principal types of control information: (1) text prompt, (2) camera trajectoryvieewooin, and (3) low-level actions, depending on the inputs expected by the chosen world model.
Text prompt. For mmage-and-texxt text-to-vides world models, the controller maps the intended action sequence into a descriptive text prompt. A predefined template converts each primitive action into a phrase, and concatenating these phrases yields the final prompt J
Camera trajectory / viewpoint. For models that consume explicit viewpoints, the controller translates A into camera trajectory, e.g., each translation action moves the camera by 0.2 m, and
4

---

Published as a conference paper at ICLR 2026
each rotation action changes the azimuth by 22.5.. The resulting trajectory is represented sequence [2x,, 3k: (x)) k=1 with (Øx, y/) E I and azimuth OR R.
Low-level actions. For world models that take discrete continuous low-level actions as input, the controller maps the action sequence A to the world model's s action vocabulary, yielding Aoerds This mapping A + Awordd applies the necessary transformations to maintain unique and consistent correspondence between the agent's actions and the inputs expected by the world model.
2.3 COMPREHENSIVE EMBODIED TASKS
To evaluate the practical utility visual world models in embodied tasks, we select a diverse set of tasks that span multiple domains and stress distinct capabilities. We focus on four representative tasks: Active Recognition (AR), Active Embodied Question Answering (A-EQA), Image-Goal Navigation (ImageNav), and Robotic Munipulation, as illustrated in Figure 4. Taken together, these tasks emphasize complementary aspects embodied intelligence, including perception, navigation, and object-level manipulation, and thus provide comprehensive testbed for assessing how effectively a visual world model supports online planning and decision decision-making. Below, we describe the tasks included in our benchmark, and more detailed settings provided in Appendix B.
Active Recognition
Image-Goal Navigation
Navigate as needed and Identiff the object marked by the red bbox.
Step Ti-dioe
Navigate -Geal Image> was captured.
What The
Step
Lourdded by
Active Embodied QA
Robotic Manipulation
Navigate
Stop sironD view
the rabotic slide block onto the blue target.
user's <Query>.
HOW the noffel BIII
Snng?
Figure 4: Top-left: Active Recognition (AR), the agent needs to identify designated target under occlusions or extreme viewpoints while minimizing navigation cost. Top-right: Image-Goal Naviga- tion (ImageNav), the agent reaches the viewpoint matching goal image, emphasizing success rate and path efficiency. Bottom-left: Active Embodied Question Answering (A-EQA), the agent answers open-ended question after active exploration. Bottom-right: Robotic Manipulation, the agent needs to control a robotic arm to complete tasks such as grasping and placement to specified targets.
Active Recognition (AR) is closely related to amodal recognition (Aydemir al., 2013; Liu et al., 2018; Yang et al., 2019; Fan et al., 2024; Bhattacharjee et al., 2025). in which the agent must identify a designated target that may be observed from extreme viewpoints or be heavily occluded. In addition, AR allows the agent to acquire additional observations through active exploration. All AR experiments are conducted in the Habitat-Si (Savva et al., 2019), encompassing 551 episodes across 29 scenes from the validation split of Matterport3D (Chang et al., 2017). Within AR, the visual world model assists two decision-malee makiin processes. For answering, synthetic views provide auxiliary evidence that helps the agent reason about occlusions and extreme viewpoints that impede recognition. For navigation, rollouts simulate the consequences potential actions so that the agent choose path that is more likely yield informative observations.
5

---

Published as a conference paper at ICLR 2026
Image-Goal Navigation (ImageNav), also referred to as goal-conditioned visual navigation, requires an embodied agent to reach a target position in a scene given a single reference image that specifies the goal viewpoint. We construct 144 ImageNav episodes from 87 validation scenes of HM3D (Ramakr- ishnan et al., 2021). In this task, the visual world model exclusively supports navigation decisions. The agent simulates the outcomes of candidate action plans, selects the best option, executes the first segment of that plan, and then replans with the newly observed state in a closed-loop manner.
Active Embodied Question Answering (A-EQA) requires an agent to answer open-ended natural- language questions after actively exploring a 3D environment. Our evaluation set includes 184 questions across 54 indoor scenes from the official OpenEQA split (Majumdar et al.. 2024) and the HM3D validation set (Ramakrishnan et al.. 2021). As in AR, the visual world model supports both question answering and navigation. For answering, synthetic views generated by the world model provide complementary perspectives that help resolve references to occluded or distant objects. For navigation, the agent simulates high-level action plans using the world model's s predictions choose exploration strategies likely to reveal question-relevant information.
Robotic Manipulations are fundamental capabilities for embodied agents that must operate in real- world interaction settings. We study how visual world models contribute to closed-loop manipulation planning, evaluating performance on four RLBench (James et al., 2020) tasks with 50 episodes per task. In our setting, the visual world model supports the agent in assessing candidate 7-DoF gripper actions by providing visual evidence about anticipated object motions and interactions, which enables comparison of alternative plans before execution. The predicted outcomes then guide the selection of actions that are more likely to achieve the specified objective, thereby linking visual prediction accuracy to improvements in manipulation performance.
2.4 EXPLOITING WORLD MODELS VIA POST- TRAINING
To evaluate the feasibility of adapting pretrained video generators for embodied tasks, we introduce post-training procedure that aligns pretrained model with the domain distribution and action space of target environments. We perform fine-tuning separately on data from two simulators, Habitat-Sim and CoppeliaSim, to match the corresponding task domains. For Habitat-Sim tasks (AR, A-EQA, ImageNav), we post-train on a panoramic oostrvvobservtiion dataset collected from the HM3D (Ramakrishnan al., 2021) training split. For CoppeliaSim tasks (Robotic Manipulation), we post-train on task demonstrations generated with RLBench (James et al., 2020). To assess generalization rather than memorization, all Habitat-Sim data used for post-training are sourced from scenes that are disjoint from our evaluation scenes, so the scenes in our evaluation tasks remain unseen by the world models after post-training. Additional details regarding the training objective, dataset construction, and training configuration are provided in Appendices C and D.
3 EVALUATION RESULTS AND ANALYSIS
In this section, we report quantitative results and key observations on the four embodied tasks in Section 3. followed by ablation studies in Section 3.2. We evaluate visual world models spanning based (PathDreamer (Koh et al., 2021), SE3DS (Koh et al., 2023)) and video-based (SVD (Blattmann et al., 2023a), LTX-Video (HaCohen et al., 2024), Hunyuan (Kong et al., 2024), 1 (Wan et al., 2025), Wan2.2 (Wan et al., 2025), Cosmos-Predichs (Agarwal et al., 2025), NWM (Bar et al., 2025)) approaches. covering major control interfaces. For video-based models, compare off-the-shelf versions with their post-trainee post-trained variants, where the additional postfix "4" denotes a post-trained video generator.
3.1 BENCHMARK RESULTS
World models can enhance the performance of the base proposal policy. Across AR, A-EQA, ImageNav, and Manipulation, adding a visual world model consistently improves the performance of the base proposal policy (c.g., VLM policy, heuristic policy, or a 3D diffusion policy), as shown in Tables I to 3. For example, in AR, the best proprietary model (Runway Gen4) attains accuracy of 64.79% while reducing the mean steps per episode to 4.06, compared to the VLM base policy with an accuracy of 50.27% and mean steps 6.24. Similarly, in ImageNav, the best open- model Wan2.1 i achieves a success rate of 45.14% with an average path length of 45.8, outperforming the VLM base policy at 35.42% SR and 47.5 average length. In A-EQA, the top post-trained model

---

Published as a conference paper at ICLR 2026
Table 1: Active Recognition (AR) and Image-Goal Navigation (ImageNav) performance across various models and base policies. Higher success rate (SR%), success weighted by path length (SPL%) and lower mean trajectory length (Mean Traj.) are better. "I" denotes our post- trained video generators. "A14B" denotes a mixture-of-experts configuration of Wan2.2 with an effective model size of 14B during inference.
Model Details
ImageNav
Model Type
Method
Control Type
Input Type
AParam.
SR
Mean Traj.4
SR1
Mean Traj..
SPL
Base Policy
Heuristic (wlo WM)
RGB
39.02
8.81
2.08
59.6
0.63
Video Gen.
SVD:
Action
RGB, Pano
1.58
60.62
5.17
20.83
58.5
11,86
Post-Train
WAN2.1f
Action
RGR, Pano
14H
62.98
4.71
22.92
58.7
11.63
Base Palicy
VLM (wlo WMD
RGB
72B
50.27
6.24
35.42
47.5
25.88
Image Gen.
PuthDreamer
Viewpoint
RGB-D: Puno
0.69B
56,99
5.28
36.80
47.3
26.85
Image Gen.
SE3DS
Viewpoint
RGB-D, Pano
1.18
57.55
5.29
36.11
47.0
26.91
Video Gen.
Trjectory
RGB
SVD
Imnge
RGB
5.29
46.4
Video
Text
RGB
2B
56.08
5.37
36.81
47.5
25.85
Huryuan
Text
RGB
13B
57.71
5,21
36.11
46.8
26.89
Video Gen.
Wan2.1
Text
RGB
5,24
Zeno-Shot
Wan22.
Text
RGB
5B
55.35
5.73
38.88
46.5
28.87
Cosmos: P2
Text
RGB
2B
55.35
5.71
36.81
47.6
25.89
Cosmos:
5.12
Wan2.2
Text
RGB
A14B
59.53
4,91
43.05
45.8
31,46
Runway Gend (proprictary)
Text
RGB
64.79
4.06
SVD:
Action
RGB, Pano
1.58
60.98
5.02
43.05
46.0
30.96
LTX- Videof
Action
RGB: Pano
2B
$7.53
5.49
38.89
47.4
27.47
Video Gen.
Post Train
WAN2.1
Action
RGB: Puno
14B
62.61
4.73
45.14
45.8
32.10
Cosmos-P2i P2)
Action
RGB: Pano
2B
60.25
5.08
41.67
45.5
30.29
Won2.2f
Action
RGR; Pano
56.26
5.15
38.89
46.7
28.24
Wan2.2f
Action
RGB: Pano
A14B
62.43
4.67
46.53
44.6
34.61
Table 2: Active Embodied Question Answering Table 3: (A-EQA) performance.
Robotic manipulation performance across various models and base policies.
Model Details
A A-EQA Performance
Model Details
Manipulation Performance
Model Type
Mathed
Ans, Score
Moan Tni..
SPL.
Model Type
Method
Traj.
Rase Policy
VLM (wio WM)
45.7
20.4
29.6
Base Policy
VLM (wlo WM)
44.5
2.52
Image Gen.
PathDreumer
46.0
20.4
29.3
Image Gen.
SE3DS
45.8
20.3
29.4
SVD
44.0
2.47
Video
LTX. Video
44.5
2.46
Video Gen.
W1n2.1
45.7
20.1
28.8
Hunyuan
44.5
2.44
Wim2.3 (SB)
Wan2.1
44.0
2.51
LTX-Vidoo
46.6
20.8
29.5
Cusmos: P2
44.0
2.50
Video Gen.
Coumos-P2
46.6
21.0
31.3
Hunguan
46.8
20.4
20.9
Video Gen.
SVD1
46.5
2.38
SVD
46.9
20.4
29.7
Post-Train
Cosmos- P2]
45.0
2.40
(A34B)
31.9
SVDF
46.4
21.1
30.1
Base Policy
3D-DP (wlo WM)
24.0
5.21
Video Gen.
Post-Tran
Win? 21 (5B)
Cosmos
Wa
47.5
48.2
20.7
20.6
20.8
31.6
30.7
Video Gen.
Post-Train
Cosmos. P2
SVD
44.7
38.0
4.41
4.79
LTX-Video]
48.6
20.7
31.8
Wan2.2f (A14R)
45.4
20.2
31.9
Wan2.2f A14B reaches an answer score of 48.4 and SPL of 31.9, surpassing the VLM base policy at 45.7 answer score and 29.6 SPL. These results support the effectiveness of our World-In-World online planning framework with world models, in which the world model provides simulated future states that inform better decisions.
World models struggle to simulate precise motion and dynamics in manipulation. The gains are less pronounced for Robotic Manipulations (Table 3), likely because accurately modeling contact-rich interactions and robot kinematics is significantly more challenging than predicting purely view changes. For instance, the best post-trained model on manipulation (SVDf) reaches SR of 46.5% with a mean trajectory length of 2.38, only modestly above the VLM baseline at 44.5% SR and 2.52 mean length. This gap suggests that while current visual world models can effectively guide perception and navigation, capturing fine-grained physical dynamics and action- centionconditiooeed object motion remains an open challenge.
7

---

Published as a conference paper at ICLR 2026
65
65
Zero-shot
Past-trained Others
Runwey
Post-trained
Runway Gend
Task Success Rate (%)
64
Others
63
63
Wan2 11
Task Success Rate
62
Won2.1
61-
SVDI
61
SVDT
Gosmes-P21
60
Ward.2 2 A148
59-
59
KWW
LTXViseot
RESDS
Hunyyan
58
Hangaesdees SESDS
57-
Pothdrsamer
57
Pathdreame"
Wan2 2 TTXVideo
56-
Trxvideo Wanz.2
55
2.2
55-
Cosmos-92
War2.2 58
0.325
0.350 0.375 0.400 0.425 0.450
0.475
0.15
0.20
0.25
0.30
0.35 0.40
0.45
0.50
Gen. Quality IAastheti++Image Quality)
Controllability (1 LPIPS)1
Figure 5: (a) SR vs. generation quality in AR: generation quality is scored as the average of an aesthetic predictor (Akio Kodaira, 2024) and an image-quality predictor (Ke et al.. 2021), both trained match human preferences. (b) SR vs. controllability in AR; controllability is quantified as 1 1 LPIPS between ground-trui and predicted observations.
62.01%
62-
52%
63.34%
62-
60. 98%
Success Rate (%)
60.25%
Success Rate (%)
96%
59.71%
58
58
57.17%
56.62%
56.445
56.26%
55
55
Wan2. 1
54-53.36%
Wan2.11
Wan2.1f
SVD]
SVD+
52
4.0
9.0
11.0
Seen Examples During Training
Avg Inference Count per Episode
Figure 6: SR vs. seen examples during post- training. SR increases consistently with more downstream data, revealing a clear data-scaling trend for adaptation.
Figure 7: SR vs. average number of world- model inferences per episode. Increasing the inference-time computation allocated to each decision step leads to higher SR.
Post-training substantially boosts world-model utility. Our post-training adaptation yields consis- tent improvements. Relative to off-the-shelf Wan2.1, Wan2.1 1 raises AR accuracy from 58, 26% to 62.61% and ImageNav SR from 38.19% % to 45.14% (Table 1). Likewise, SVD$ improves AR accu- racy from 57.71% % to 60.98% and ImageNav SR from 40.28% to 43.05%. In A-EQA, LTX- Vidcot increases the answer score from 46.6 48.6, and Wan2. 1f from 45.7 to 48.2. These gains show that aligning the generative model to the target domain and action space of the specific embodied tasks improves downstream mitisiin-making....
3.2 ABLATION AND FINDINGS
Fine-grained controllability matters more than visuals for task success. Although recent off-the shelf video generators like Wan2.1 produce visually appealing clips, they are driven by text prompts with limited fine-grained low-level controls. Without adaptation, these models yield only small gains on downstream embodied tasks. We further study the relation between controllability and the success rate on AR. Here, controllability defined as alignment between intended actions and the motions in the model's predictions. After action-conditionss post-training, alignment improves substantially and SR rises accordingly. Figure 5(b) shows a clearer positive correlation than Figure 5(a), which depicts SR versus generation quality (aesthetic and image-quality scores), and suggests that models that respond reliably to low-level controls achieve higher SR. These results indicate that precise control, not just visual quality, is critical for embodied world models to support effective recision-making
8

---

Published as a conference paper at ICLR 2026
Data-size scaling for post-trained models. We study how post-training data size affects WM performance (Wan2.2i, Wa SVDf). Each WM is post-trained for one epoch on datasets from 400 to 80K instances. As shown in Figure 6, more post-training data consistently improves AR performance: Wan2.1f i rises from 60.25% to 63.34%, and SVD$ from 56.80% to 60.98%. Wan2.2 (A14B), despite substantially larger web-video pretraining, reaches nearly the same performance as Wan2.1f after 40K post-training instances, suggesting that scaling action-conditione post-training is more effective for embodied utility than upgrading the pretrained generator. Moreover, larger models (Wan2.1f, f, 14B) benefit more and saturate less than smaller ones (SVDf, 1.5B), indicating greater capacity to absorb cediionconditioned supervision.
Inference-time scaling for online planning with world models. Within our online planning framework, the number of world-model inferences (simulated potential futures per episode) directly affects task performance. As shown in Figure 7, increasing the average inferences per episode for AR yields a clear positive correlation with SR. For example, increasing the average inference count from 3 to 11 improves SR from 53.36% to 60.98% for SVDj. This suggests that allocating more inference-time computation to simulate potential futures lets the planner make more informed decisions, thereby improving overall performance.
Global vs. local context for generation. We study the effect of input context format. Specif- ically, we compare post-trained models condi- tioned on panoramic versus front-view input (Ta- ble 4). Panoramic input provides a 360 field of view, whereas front view offers a focused but lim- ited perspective. For fairness, generated panora- mas are converted to perspective views with the same horizontal field of view during evaluation. Although panoramic input offers richer global context, it does not consistently yield large gains across all settings. Likely, penorama-to--pespectiiee conversion introduces resolution loss, degrading downstream perception and planning.
Table 4: Post-training with different input con- texts: front view vs. panorama.
Task
Model
Front View
Panorama
SR
Monn Toij. 1
SR1
Monn Trij I
SND]
60:98
5.02
Wan2.1f
62.25
62.61
Win2..2 (SB)
57.16
5.08
56.26
5.15
Cosmos P21
38.98
4.94
00.25
5.08
SVD:
38.19
47.0
43.05
46.0
ImageNNa
Wan2.11
48.61
43.8
45.14
45.8
Wa
40.97
Cosmos-P2
40.97
Effect of different revision policies. We study how the revision policy affects task per- formance by comparing VLM-based revi- sion policy with a simple L.PIS--based pol- icy that selects the candidate whose predicted observation is closest to the goal image in perceptual feature space. From Table 5, we see that even a simple LPIPS-based revision policy could improve the performance signif- icantly: SVDf obtains 47.92% SR and 39.82
Table 5: Effect of world-model augmentation and revision policy on ImageNav. SR and SPL are higher- be-bett mean trajectory length is power-is-betted
WM Type
SR
Mean Traj
1PL
VLM
None
None
35.42
47.5
25.88
VLM
SVD;
VLM
43.05
46.0
30.96
VLM
Wan2.11
VLM
45.14
45.8
32.10
VLM
SVD:
LPIPS
47.92
41.3
39.82
VIM
Wan2.1
LPIPS
48.61
39.8
42,48
SPL compared with 43.05% SR and 30.96 SPL using a VLM-bassed revision policy and 35.42% SR and 25.88 SPL without any WM augmentation. Augmenting the planner with action-conditioned WMs and applying a simple L.PPS--basss based revision can yield a higher SR and more efficient navigation.
Domain transfer across scene distributions. We evaluate cross-domain generalization by post- training WMs on the synthetic Habitat Synthetic Scenes Dataset (HSSD) and testing them on our AR and ImageNav suites built the real-world scenes in HM3D/MP3D (Table 6). Despite the synthetic-to- real gap, HSSDDttriind WMs still yield clear gains over the VI.M-only baseline (e.g.. SVDi improves AR SR from 50.27% 58.98% 98% and ImageNav SR from 35.42% to 38. 89%). Performance remains be- low in-domain under a stronger distribution shift. These results indicate that post- learns action-conditioos visual representations that transfer across scene distributions, consistent with prior work on adaptable world models (Gao et al., 2025).
Table 6:
post-training: WMs post-trained on HSSD HM3D and evaluated on HM3D/MP3D (val) for AR and ImageNav.
WM Aug.
Post Post-Train Eas.
AR
ImngeNav
1 Mean Traj. 1
SR
SPL. f
w/o WM
None
50.27
6.24
35.42
25.88
+SVD
HSSD
5.24
WWan2.1f
HSSD
62.98
4.78
42.56
31.18
+SVD!
HM3D (train)
60.98
5.02
43.05
30.96
#Wan2.1f
HM3D (train)
62.61
4.73
45.14
30.10
on HM3D (SVD): 60.98% AR SR. 43.05% ImageNav SR), as expected

---

Published as a conference paper at ICLR 2026
4 DISCUSSION AND FUTURE DIRECTIONS
Generalization capacity of world models is critical for practical use. Most video generators are pretrained web videos. In unseen embodied environments, they may revert to training priors or ignore action controls, yielding plausible but physically or semantically inconsistent rollouts (see Figures 13 and 14). These deviations mislead planning and reduce success. Larger models or more pretraining data can partly help, but robust generalization remains central. Future work should prioritize strategies and action representations to improve transfer to novel environments, such as unified action representations (Gao et al., 2025; Wang et al., 2025f; Zhi et al., 2025; Wang et al., 2025e) and curriculum or Nomain-speciife data collection (Zhao et al., 2025).
Long-horizon planning with world models remains challenging. our experiments, visual world models simulate short-term changes but struggle on long horizons due to limited mechanisms for accumulating spatiotemporal history. We attempted to alleviate this issue by replacing front-view inputs with panoramas to provide global context, but gains were inconsistent across models and tasks. Future work should better encode and retrieve long-term dependencies, e.g., spatial memory (Zhou al., 2025b; Xiao et al., 2025; Li et al., 2025d; Yu al., 2025a; Ren al., 2025; Wang al., 2025c) and episode-level memory (Cai et al., 2025; Guo et al., 2025), to maintain scene-level context and enable coherent planning over extended horizons.
Precise modeling of interactions and dynamics remains difficult. For manipulation, capturing contact-rich interactions, compliance. friction, and state changes of articulated or deformable objects is essential. Current visual world models often miss these details, producing rollouts that violate physics and degrade planning and control-consistent with observations and prior analyses (Kang et al., 2024; Li et al., 2025a). Promising directions include poysics-guided motion generation (Wang al., 2025a; Zhang et al., 2025b; Akkerman et al., 2025), inferring or generating physical properties to inform action-conditionee predictions (Cao et al., 2025; Gillman et al., 2025; Zhang et al., 2024b), and physics-aware reinforcement post-training (Wu al., 2025; Liu et al., 2025). Integrating such signals into conditioning pathways may improve fidelity when precise dynamics are required.
Stronger proposal and revision policies set the performance floor. The agent's 's overall performance depends on both world-model fidelity and the strength of the proposal and revision policies that select and refine decisions. While simulated rollouts improve decision-makin be effective to provide a reliable starting point, and strengthening them raises the ceiling. Future work could explore stronger policies (Geng et al., 2025; Kim al., 2025), and integration strategies that deepen synergy between world models and decision-making (Neary et al., 2025), such as more human-aligned reward models (Wang al., 2024; Seneviratne al., 2025; Rocamonde al., 2023; Zhang et al., 2024a; Wang et al., 2025d; Wu et al., 2025).
base policies must
Computational cost and efficiency remain practical concerns. Incorporating world models into planning introduces additional computational overhead because multiple future rollouts must be simulated at each decision step. Although our experiments show that allocating more inference-time computation to the world model improves task performance, this extra cost may be impractical in settings with strict real-time constraints or limited hardware resources. Future work should therefore investigate more efficient world-mode architectures (Yang et al., 2025b; Kodaira al., 2025), training and inference strategies that enable near real-time rollouts (Huang et al., 2025; Cui et al., 2025), and distillation techniques (Wang et al., 2025b; Agarwal et al., 2025) that reduce computational demands while preserving the predictive fidelity of world models.
5 CONCLUSION
We introduce World-In-World, a closed-loop erative world models via embodied interaction rather than isolated visual metries. By unifying heterogeneous controls, our action API enables any world model to serve perception and planning utilities for an embodied agent. Coupled with a unified closed-loop planning strategy that proposes, simulates, and revises action plans, the benchmark measures agent performance four demanding tasks. Our experiments reveal large gaps between visual metrics and task success, underscoring the need for closed-loop evaluation, and show that pretrained video generators improve with post-train post-training data scaling and inference- time scaling. We expect World-In-War to guide world models toward not only striking visual realism but also reliable perception and planning in embodied scenarios.
world interface and benchmark that evaluates gen-
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.0355 2025.
Sayan Goswami Akio Kodaira. Aesthetic predictor v2.5, May 2024, URL hwwph:ttp///gg.. http//ggithbb..ooo di scus 040444444 aesthetic-pp i-ppreeieeeei ctor-v2-5. 5/.
Rick Akkerman, Haiwen Feng, Michael J. Black, Dimitrios Tzionas, and Victoria Feráandez Abrevaya. Interdyn: Controllable interactive dynamics with video diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 12467- 12479,, 2025.
Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in atari. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Alper Aydemir, Andrzej Pronobis, Moritz Göbelbecker. and Patric Jensfelt. Active visual object search in unknown environments using uncertain semantics. IEEE Transactions Robotics, 29(4):98-1100 10088 August 2013. ISSN 1941-0468.
Sherwin Bahmani, Xian Liu, Wang Yifan, Ivan Skorokhodov. Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, al. Tc4d: Trajectory-conditionn text-to-4d generation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 53-72. Springer, 2024a.
Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi. and David B Lindell. 4d-fy: Text-to-4d generation using hybrid score distillation sampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7996-8006, 2024b.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song. Kai Dang, Peng Wang. Shijie Wang. Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025.
Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2025.
Subhransu S. Bhattacharjee, Dylan Campbell, and Rahul Shome. Believing is seeing: Unobserved object detection using generative models, March 2025.
Andreas Blatmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion 2023a.
models to large datasets. arXiv preprint arXiv:2311.151
Andreas Blattmann, Robin Rombach, Huan I Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis, Align your latents: High-resolution video synthesis with latent diffusion models. Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023b.
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Sora: Video generation models as world simulators. OpenAl Blog, 1:8, 2024.
Shengqu Cai, Ceyuan Yang, Lvmin Zhang, Yuwei Guo. Junfei Xiao, Ziyan Yang. Yinghao Xu, Zhenheng Yang. Alan L. Yuille, Leonidas J. Guibas, Maneesh Agrawala, Lu Jiang, and Gordon Wetzstein. Mixture of contexts for long video generation. ArXiv, 2508.21058, 2025.
Ziang Cao, Zhaoxi Chen, Liang Pan, and Ziwei Liu. Physx-3d: 2507.12466 2025.
3d asset generation. ArXiv,
Angel Chang. Angela Dai, Thomas Funkhouser, Macicj Halber, Matthias Niebner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. In Proceedings of the International Conference on 3D Vision (3DV), pp. 667-676, October 2017.
Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16901-16911, 2024.
Jaeyoung Chung. Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free generation of 3d gaussian splatting scenes. arXiv preprint arXiv:2311.1333 2023.
11

---

Published as a conference paper at ICLR 2026
Justin Cui, Jie Wu, Ming Li, Tao Yang. Xiaojie Li, Rui Wang, Andrew Bai, Yuanhao Ban, and Cho-Jui Hsich. Self-forcing++: Towards minute-scale high-quality video generation. ArXiv, 2510.02283, 2025.
Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in neural information processing systems, 36:9156-91111 2023.
Yilun Du, Sherry Yang, Pete Florence, Fei Xia, Ayzaan Wahid, brian ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B. Tenenbaum, Leslie Pack Kaelbling, Andy Zeng, and Jonathan Tompson. Video language planning. In Proceedings of the International Conference on Learning Representations (ICLR), 2024,
Haoyi Duan, Hong-Xing Yu, Sirui Chen, Fei-Fei, and Jiajun Wu. Worldscore: A unified evaluation benchmark for world generation. arXiv preprint arXiv:2504.00900 2025.
Lei Fan, Mingfu Liang, Yunxuan Li, Gang Hua, and Ying Wu. Evidential active recognition: Intelligent and prudent open-world embodied perception. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16351-16361, 2024.
Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. Scenescape: Text-driven consistent scene generation. Advances in Neural Information Processing Systems (NeurlPS), 36:39897-3999 -9917779994....9 14, 2023.
Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. In Advances in Neural Information Processing Systems (NeurIPS), November 2024.
Shenyuan Gao, Siyuan Zhou, Yilun Du Jun Zhang, and Chuang Gan. Adaworld: Learning adaptable world models with latent actions. In International Conference on Machine Learning (ICML), 2025.
Haoran Geng, Feishi Wang, Songlin Wei, Yuyang Li, Bangjun Wang. Boshi An, Charlie Tianyue Cheng, Haozhe Lou, Peihao Li, Yen-Jen Wang, Yutong Liang. Dylan Goetting, Chaoyi Xu, Haozhe Chen, Yuxi Qian, Yiran Geng, Jiageng Mao, Weikang Wan, Mingtong Zhang, Jiangran Lyu, Siheng Zhao, Jiazhao Zhang, Jialiang Zhang. Chengyang Zhao, Haoran Lu, Yufei Ding, Ran Gong, Yuran Wang, Yuxuan Kuang, Ruihai Wu, Baoxiong Jia, Carlo Sferrazza, Hao Dong, Siyuan Huang, Yue Wang, Jitendra Malik, and Pieter Abbeel. Roboverse: Towards unified platform, dataset and benchmark for scalable and generalizable robot learning. ArXiv, 2504.18900 18904, 2025.
Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, and Chen Sun. Force prompting: Video generation models can learn and generalize physics-basee bassed control signals. ArXiv, 2505.19388 2025.
Yuwei Guo, Ceyuan Yang, Ziyan Yang, Zhibei Ma. Zhijie Lin, Zhenheng Yang, Dahua Lin, and Lu Jiang. Long context tuning for video generation. ArXiv, 10589, 2025.
Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Fitan Richardson, Fran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Ltx-video: Realtime video latent diffusion. arXiv preprint Sidi:25000000000 2024.
Hao He. Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation. arXiv preprint 2xXiv:240.002101, 2025a.
Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models. arXiv preprint arXiv:2503.10522 2025b.
Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang, Yifan Zhang, Qi Cui, Fei Kang, Biao Jiang, Mengyin An, Yangyang Ren, Baixin Xu, Hao-Xiang Guo, Kaixiong Gong, Cyrus Wu, Wei Li, Xuchen Song, Yang Liu, Eric Li, and Yahui Zhou. Matrix-game 2.0: An open-source, real-time, and streaming interactive world model. arXiv preprint Xiv:22508. 13009, 2025c.
Jonathan Ho, Ajay Jain, and Pieter Abheel. Denoising diffusion prohabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and
Gianluca Corrado. Gaia-1: A generative world model for autonomous driving, September 2023.
Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test. gap in autoregressive video diffusion. ArXiv, 2506.08009, 2025.
12

---

Published as a conference paper at ICLR 2026
Ziqi Huang, Yinan He. Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21807-21181 2024.
Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J. Davison. Rlbench: The robot learning benchmark & learning environment. IEEF Robotics and Automation Letters, 2020.
Jindong Jiang, Lunan Zheng, Fei Luo, and Zhijun Zhang. Rednet: Residual encoder-decoder network for indoor rgh-d semantic segmentation. arXiv preprint
2018.
Bingyi Kang, Yang Yue, Rui Lu, Zhijie Lin, Yang Zhao, Kaixin Wang, Gao Huang, and Jiashi Feng. How far is video generation from world model: physical law perspective. ArXiv, 2411.02385 2024.
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), pp. 1128-5137,20 2021.
Tsung- Te Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3d diffuser actor: Policy diffusion with 3d scene representations. Arxiv, 2024.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-actiion models: Optimizing speed and success, ArXiv, 25... 19645, 2025.
Po-Chen Ko, Jiayuan Mao, Yilun Du Shao-Hur Sun, and Joshua B Tenenbaum. Learning to act from actionless videos through dense correspondences. arXiv preprint arXiv:2310.08576, 2023.
Akio Kodaira, Tingbo Hou, Ji Hou, Masayoshi Tomizuka, and Yue Zhao. Streamdit: Real-time streaming text-to-video generation. ArXiv. 2507.03745. 2025.
Jing Yu Koh, Honglak Lee, Yinfei Yang, Jason Baldridge, and Peter Anderson. Pathdreamer: A world model for indoor navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
Jing Yu Koh, Harsh Agrawal, Dhruv Batra, Richard Tucker, Austin Waters, Honglak Lee, Yinfei Yang, Jason Baldridge, and Peter Anderson. Simple and effective synthesis of indoor 3d scenes. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 37(1): 169--178,, June 2023. ISSN 2374-3468.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large vidco generative models. arXiv preprint arXiv:2412.03603 2024.
Chenyu Li, Oscar Michel, Xichen Pan, Sainan Liu, Mike Roberts, and Saining Xie. Pisa experiments: Exploring physics post-training for video diffusion models by watching stuff drop. ArXiv, 2503.09595, 2025a.
Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael I Luo, Xiaolong Wang. Hongxu Yin. Joseph E. Gonzalez, Ion Stoica, Song Han, and Yao Lu. Worldmodelbench: Judging video generation models as world models. ArXiv, 2502.20694, 2025b.
Jiaqi Li, Junshu Tang, Zhi- Ting Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo Cao, and Qinglin Lu. condition. ArXiv, 2506. 17201, 2025c.
High-dynamic interactive game video generation with hy hybrid history
Runjia Li, Philip H. S. Tor, Andrea Vedaldi, and Tomas Jakab. Vmem: Consistent interactive video scene generation with surfel-indexed view memory. ArXiv, 1890318903, 2025d.
Lu Ling, Chen-Hsuan Lin, Tsung-Yi Yi Lin, Yifan Ding, Yu Zeng, Yichen Sheng, Yunhao Ge, Ming- Yu Liu, Aniket Bera, and Zhaoshuo Li. Scenethesis: A language and vision agentic framework for 3d scene generation. arXiv preprint arXiv:2505.02886, 2025.
Huaping Liu, Yupei Wu, and Fuchun Sun. Extreme trust region policy optimization for active object recognition. IEEE Transactions on Neural Networks and Learning Systems, 29(6):2253 2258, June 2018. ISSN 2162-2388.
Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu. Qiulin Wang, Wenyu Qin, Menghan Xia, Xintao Wang, Xiaohong I Liu, Fei Yang, Pengfei Wan, Di Zhang, Kun Gai, Yujiu Yang. and Wanli Ouyang. Improving video generation with human feedback. ArXiv, 2501.13918, 2025.
13

---

Published as a conference paper at ICLR 2026
Xiaoxiao Long, Qingrui Zhao, Kaiwen Zhang, Zihao Zhang, Dingrui Wang. Yumeng Liu, Zhengjie Shu, Yi Lu, Shouzheng Wang. Xinzhe Wei, Wei Li, Wei Yin, Yao Yao, Jiangtian Pan, Qiu Shen, Ruigang Yang, Xun Cao, and Qionghai Dai. survey: Learning embodied intelligence from physical simulators and world models. ArXiv, 2507 00917, 2025.
TaiMing Lu, Tianmin Shu, Alan Yuille, Daniel Khashabi, and Jieneng Chen. Generative world explorer. In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
Arjun Majumdar, Anurag Ajay. Xiaohan Zhang, Pranav Putta, Sriram Yenamandra, Mikael Henaff. Sneha Silwal, Paul Mevay, Oleksandr Maksymets, Sergio Arnaud, Karmesh Yadav, Qiyang Li, Ben Newman, Mohit Sharma, Vincent Berges, Shiqi Zhang, Pulkit Agrawal, Yonatan Bisk, Dhruv Batra, Mrinal Kalakrishnan. Franziska Mcier, Chris Paxton, Alexander Sax, and Aravind Rajeswaran. Openeqa: Embodied question answering in the era of foundation models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pottern Recognition (CVPR), pp. 1148-1 2024.
Zhiting Mei, Tenny Yin, Micah Baker, Ola Shorinwa, and Anirudha Majumdar. World models that know when they don't 't know: Controllable video generation with calibrated uncertainty. arXiv, 2512.05922 2025.
Manfred Morari and Jay H. Lec. Model predictive control: Past, present and future. Computers & Chemical Engineering, 23(4)6667-682, May 1999. ISSN 0098-1354.
Cyrus Neary, Omar G. Younis, Artur Kuramshin, Ozgur Aslan, and Glen Berseth. Improving pre-trained vision-langugge-action policies with model-based search. ArXiv, 2508. 12211, 2025.
Jack Parker-Holder and Shlomi Fruchter. Genie 3:
frontier for world URL https: / deepmind grrpmmiiddggggeeee. /ppoole/diccove//bloo gen e 3-a- a-e--ew---------- new- front centier--ie For-wor woor-world-oddee d-modelss s/. Google DeepMind Blog.
models, August 2025.
Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexander Clegg, John M. Turner, Eric Undersander. Wojeiech Galuba, Andrew Westbury, Angel X. Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. In Advances in Neural Information Processing Syxtems (NeurIPS), August 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408 0000424000000144, 2024.
Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-Dave Thomas Muller, Alexander Keller, Sanja Fidler, and Jun Gao. Genăe: 3d-informed world-consistent video generation with precise camera control. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (C (CVPR), pp. 6121 6132, 2025.
Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision-language models are zero-shot reward models for reinforcement learning. ArXiv, 12921, 2023.
Robin Rombach, Andreas Blattmann, Dominik Loreaz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Runway Research. Introducing runway gen-4. https: com/ research/
introduiing sarrodccing-uuwaay www.aaunnooaaooee -on--, March 2025. Research announcement, Runway AI, Inc. Accessed: 2025-09-21.
Kyle Sargent, Zizhang I.,, Tammay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Fric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, and Jiajun Wu. Zeronvs: Zero-shot 360-degree view synthesis from single image. Proceedings of the IEFF/VVF Conference on Computer Vision and Pottern Recognition (CVPR). pp. 1420-9429, 2024.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A platform for embodied rescarch. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9339 9347, 2019.
Gershom Seneviratne, Jianyu An. Sahire Ellahy, Kasun Weerakoon, Mohamed Bashir Elnoor, Jonathan Deepak Kannan, Amogha Thalihalla Sunil, and Dinesh Manocha. Halo: Human preference aligned offline reward learning for robot navigation. ArXiv, 2508.01539, 2025.
14

---

Published as a conference paper at ICLR 2026
Junyoung Seo, Kazumi Fukuda, Takashi Shibuya, Takuya Narihira, Naoki Murata, Shoukang Hu, Chich-IIsin Lai, Scungryong Kim, and Yuki Mitsufuji. Genwarp: Single image to novel views with semantic-preservin generative warping. In Advances in Neural Information Processing Systems (NeurIPS), November 2024.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning (ICML), 2015.
Jiankai Sun, Yiqi Jiang, Jianing Qiu, Parth Nobel, Mykel J. Kochenderfer, and Mac Schwager. Conformal prediction for uncertainty-awwe -ware planning with diffusion dynamics model. In Neural Information Processing Systems, 2023.
Stephen Tian, Chelsea Finn, and Jiajun Wu. A control-centric benchmark for video prediction. ArXiv, 2304.13722 2023.
Vikram Voleti, Chun Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitrii Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani. SV3D: Novel multi-view synthesis and 3D generation from a single image using latent video diffusion. In Proceedings of the European Conference on Computer Vision (ECCV), 2024.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen- Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314. 2025.
Chen Wang, Chuhao Chen, Yiming Huang, Zhiyang Dou, Yuan Liu, Jiatao Gu, and Lingjie Liu. Physctrl: Generative physics for controllable and physics grounded video generation. ArXiv, abs/2509.20358 2025a.
physics-grounded
Hanqing Wang, Wei Liang, Lue Van Gool, and Wenguan Wang. Drcamwalker: Mental planning for continuous vision-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
Hanyang Wang, Fangfu Liu, Jiawei Chi, and Yueqi Duan. Videoscene: Distilling video diffusion model to generate 3d scenes in one step. Arxiv, 2504.01956, 2025b.
Jiahao Wang. Luoxin Ye, TaiMing Lu, Junfci Xiao, Jiahan Zhang, Yuxiang Guo, Xijun Liu, Rama Chellappa. Cheng Peng, Alan Yuille, al. Evoworld: Evolving panoramic world generation with explicit 3d memory. ArXiv, 2510.01 183, 2025c.
Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation. ArXiv, 2503.05236, 2025d.
Yiqi Wang, Mrinal Verghese, and Jeff Schneider. Latent policy steering with embodiment-agnostie pretrained world models. ArXiv, 2507. 13340, 2025e.
Yuang Wang, Chao Wen, Haoyu Guo, Sida Peng, Minghan Qin, Hujun Bao, Xiaowei Zhou, and Ruizhen Hu. Precise action-to-video generation through visual action prompts. ArXiv, 13104, 2025f.
Yufei Wang, Zhanyi Sun. Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. RI-vIm-f: Reinforcement learning from vision language foundation model feedback. ArXiv, 2402.03681, 2024.
Jie Wu, Yu Gao, Zilyu Ye, Ming Li, Liang Li, Hanzhong Guo, Jie Liu, Zeyue Xue, Xiaoxia Hou, Wei Liu, Yan Zeng, and Weilin Huang. Rewarddance: Reward scaling in visual generation. ArXiv, 2509.08826, 2025.
Zeqi Xiao, Yushi Lan, Yifan Zhou, Wenqi Ouyang, Shuai Yang. Yanhong Zeng, and Xingang Pan. Worldmem: Long-term consistent world simulation with memory. April 2025.
Yiming Xic, Chun-Han Yao, Vikram Voleti, Huaizu Jiang, and Varun Jampani. Sv4d: Dynamic 3d content generation with multi-fran frame and multi-view consistency, July 2024.
Dejia Xu, Hanwen Liang, Neel Bhatt, Hezhen Hu, Hanxue Liang, Konstantinos Plataniotis, and Zhangyang Wang. Comp4d: Llm-guided compositional 4d scene generation. arXiv preprint a4X3v:2403. 16993, 2024.
Jianwei Yang, Zhile Ren, Mingze Xu, Xinlei Chen, David J Crandall. Devi Parikh, and Dhruy Batra. Embodied amodal recognition: Learning to move to perceive objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2040-2050 2050, 2019.
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023a.
15

---

Published as a conference paper at ICLR 2026
Mengjiao Yang. Yilun Du, Kamyar Ghasemipour, Jonathan Tompson. Dale Schuurmans, and Pieter Abbeel. Learning interactive wollwwoldd simulators. arXiv preprint arXiv:2310.061 14, 1(2)6, 2023b.
Rui Yang. Hanyang Chen. Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang. Qineng Wang. Teja Venkat Koripella, Marziyeh Movahedi, Manling Li, Heng Ji, Huan Zhang, and Tong Zhang. Embodiedhench: Com- prehensive benchmarking multi-modas large language models for vision-driven embodied agents, February 2025a.
Sherry Yang, Yilun Du, Seyed Kamyar Seyed Ghasemipour, Jonathan Tompson, Leslie Pack Kaelbling, Dale Schuurmans, and Pieter Abbeel. Learning interactive world simulators. In Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2024a.
Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang. Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, and Song Hanand Yukang Chen. Longlive: Real-time interactive long video generation. ArXiv, 2509.2250 22622, 2025b.
Yuncong Yang, Jiageng Liu, Zheyuan Zhang, Siyuan Zhou, Reuben Tan, Jianwei Yang. Yilun Du, and Chuang Gan. Mindjoumey: Test-time scaling with world models for spatial reasoning. ArXiv, 2507. 12508, 2025c.
Zhuayi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng. et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408. 06072, 2024b.
Deheng Ye, Fangyun Zhou, Jiacheng Lv, Jianqi Ma, Jun Zhang, Junyan Lv, Junyou Li, Minwen Deng, Mingyu Yang. Qiang Fu, Wei Yang. Wenkai Lv, Yangbin Yu, Yewen Wang. Yonghang Guan, Zhihao Hu, Zhongbin Fang, and Zhongqian Sun. Yan: Foundational interactive video generation. ArXiv,
2025.
Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine- grained control video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308. 2023.
Hong-Xing Yu, Haoyi Duan, Charles Herraann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from single image. arXiv preprint arXiv:2406. 093344 2024.
Jason J. Yu, Fereshteh Forghani, Konstantinos G. Derpanis, and Marcus A. Brubaker. Long-term photometric consistent novel view synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 7094-7104, 2023.
Jiwen Yu, Jianhong Bai, Yiran Qin, Quande Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. Context as memory: Scene-consistent interactive long video generation with memory retrieval. ArXiv, 2506.03141, 2025a.
Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive vidcos, January 2025h.
Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Behzad Dariush, Kwonjoon Lee, Yilun Du, and Chuang Gan. COMBO: Compositional world models for embodied multi-agent cooperation. In Proceedings of the International Conference on Learning Representations (ICLR), 2025a.
Jiahan Zhang, Qi Wei, Feng Liu, and Lei Feng. Candidate pseudolabel learning: enhancing vision-language models by prompt tuning with unlabeled data. In Proceedings of the 41st International Conference on Machine Learning, pp. 60004 60020, 2024a.
Ke Zhang, Cihan Xiao, Yiqun Mci, Jiacong Xu, and Vishal M. Patel. Think before you diffuse: LIms-guided Ahysics-aware video generation. ArXiv, 2505.21653 2025b.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEFF/CVF International Conference Computer Vision (ICCV), pp. 3836-3847, 2023.
Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y. Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T. Freeman. Physdreamer: Physies-based interaction with 3d objects via video generation. ArXiv, 13026, 2024b.
Zhifang Zhang, Jiahan Zhang, Shengjie Zhou, Qi Wei, Shuo He, Feng Liu, and Lei Feng. Improving gen- cralizability and undetectability for targeted adversarial attacks multimodal pre-trained models. ArXiv, 2509. 19994, 2025c.
Qi Zhao, Xingyu Ni, Ziyu Wang, Feng Cheng, Ziyan Yang. Lu Jiang, and Bohan Wang. Synthetic video cnhances physical fidelity in video synthesis. ArXiv, 2503.20822 2025.
16

---

Published as a conference paper at ICLR 2026
Haoyu Zhen, Qiao Sun, Hongxin Zhang, Junyan Li, Siyuan Zhou, Yilun Du, and Chuang Gan. Tesseract: Learning 4d embodied world models. arXiv preprint arXiv:2504. 20995, 2025.
Hongyan Zhi, Peihao Chen, Siyuan Zhou, Dong Yu, Quanxi Wu, Lei Han, and Mingkui Tan. 3dflowaction: world model. ArXiv, 2506.06199, 2025.
Learning eessdmmmbdi manipulation from
Jensen Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Haa Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. arXiv preprint arXiv:2503. 14489, April 2025a.
Siyuan Zhou, Yilun Du, Yuncong Yang, Lei Han, Peihao Chen, Yan Yeung, and Chuang Gan. Learning 3d persistent embodied world models. ArXiv, 2505.05495, 2025b.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Yue Cao, Yangzhou Liu, Haomin Wang. Weiye Xu, Hao Li, Jiahao Wang. Han Lv. Dengnian Chen, Songze Li. Yinan He. Tan Jiang. Jiapeng Luo, Yi Wang. Cong He. Botian Shi, Xingcheng Zhang. Wenqi Shao, Junjun He, Ying Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Lijun Wu, Kai Zhang, Hui Deng, Jiaye Ge, Kaiming Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Qiao, Jifeng Dai, and Wenhai Wang. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. ArXiv, 1049, 2025.
17

---

Published as a conference paper at ICLR 2026
World-In-World: World Models in a Closed-Loop World
Appendix
CONTENTS
1
Introduction
2
World-In-World: a Closed-Loop Interface for Visual World Models
2.1
Unified Strategy for Closed-Loop Online Planning
2.2 Unified Action API
2.3 Comprehensive Embodied Tasks
2.4
Exploiting World Models via Post- Training
3
Evaluation Results and Analysis
3.1
Benchmark Results
3.2 Ablation and Findings
4 Discussion and Future Directions
10
5
Conclusion
10
A Related Work
Embodied Task Details
B.1 Active Recognition (AR)
B.2
Image-Goal Navigation (ImageNav)
B.3 Active Embodied Question Answering (A- (A-EQA)
B.4 Robotic Manipulation
B.5
Policies in Embodied Tasks
B.5.1
Base policies and proposal policies
B.5.2
Revision policies
B.6 World Models in Embodied Tasks
Post-Training Recipe for Embodied World Models
C.I Problem Formulation
C.2 Post-Training Configuration
Post-Training Dataset Construction
10
D.I Trajectory Sampling.
10

---

Published as a conference paper at ICLR 2026
E Visualizing World Model Predictions
12
F Prompt Templates used in World-In-World
17
F11
Active Recognition (AR) Prompt
17
F2 Image-Goal Navigation (ImageNav) Prompt
18
E3 Active Embedded Question Answering (A-EQA) Prompt
18
F.4 Robotic Manipulation Prompt
21
G Use of Language Models
23
A RELATED WORK
Visual generation. Recent advances in diffusion models (Sohl-Dickstein al., 2015; Ho et al., 2020; Rombach et al., 2022; Brooks et al., 2024) have significantly improved the quality of image generation (Rombach et al., 2022; Zhang et al., 2023) and video generation (Blattmann et al., 2023b;a; Voleti et al., 2024; Xie et al., 2024), enabling temporally coherent and visually rich content synthesis from text prompts or a single image. Image generators (Koh et al., 2021; 2023; Yu et al., 2023; Sargent et al., 2024; Seo et al., 2024) allow viewpoints. Text-to-video generators such as Sora (Brooks et al., 2024) can generate minutes-lone long videos from text. Extensions incorporating camera trajectories as conditioning signals (Yin al., 2023; Bar et al., 2025; He et al., 2025a;b; Zhou et al., 2025a; Bahmani et al., 2024a) push video generation toward dynamic scenes. However, the absence of unified conditioning framework hinders integration into downstream applications (e.g., embodied decision making) and prevents fair cross-mest metsod comparisons. Moreover, these generative methods remain passive: generated worlds are treated as static backdrops and evaluated in an open-loop fashion using visual quality score (Huang al., 2024) or controllability score (Duan et al., 2025). In contrast, our work assesses not only generation quality but also closed-loop task success within physical simulation.
synthesize novel views with conditions targeted
World models. Video-based generative models used as world models have shown effectiveness
across range of domains, including games (Alonso et al., 2024; Yu et al., 2025b; Li et al., 2025c; Ye et al., 2025; He et al., 2025c), manipulation (Du et al., 2023; Ko et al., 2023; Du et al., 2024;
Yang et al., 2024a; Zhen al., 2025), autonomous driving (Gao al., 2024; Hu al., 2023), and navigation (Bar et al., 2025; Wang et al., 2023; Koh et al., 2021), with extensions to broader embodied tasks (Lu al., 2025; Zhang al., 2025a; Long et al., 2025; Yang al., 2025c). However, current evaluation frameworks for these world models are often limited to visual metrics (Duan et al., 2025; a single embodied task in narrow domain (Bar et al., 2025; Zhen et al., 2025). The VP2 benchmark (Tian et al., 2023) moves toward a control-centrie evaluation by measuring the utility video prediction models in model-based planning, However, its simple setting, including limited task diversity and the use of earlier video prediction architectures, limits its relevance to modern video generative models and more complex embodied scenarios. contrast, our work provides a broader evaluation across four closed-loop embodied tasks, systematically benchmarking the practical utility of diverse world models that are pretrained on large-scale Internet video datasets.
Li et al., 2025b)
B EMBODIED TASK DETAILS
This section details the setups for the four embodied tasks evaluated in World-In-World: Active Recognition (AR) in Appendix B. Image-Goat Navigation (ImageNav) in Appendix B.2, Active Em- bodied Question Answering (A-EQA) in Appendix B.3, and Robotic Manipulation in Appendix B.4. We also describe the policies used across these tasks in Appendix B.5 and summarize the world model details in Appendix B.6.
2

---

Published as a conference paper at ICLR 2026
B.1 ACTIVE RECOGNITION (AR)
All AR experiments are performed in Habitat-Sim using scenes from the validation split of Matter- port3I (Chang al., 2017). We foeus on 29 scenes and curate subset of 551 challenging episodes adapted from the dataset released by prior work (Fan et al., 2024). Each episode is manually inspected to ensure that it presents either an extreme viewpoint or heavily occluded target object. These conditions force the agent to actively explore the environment and to rely on its world model for informed decision-making.
Task setup. In the AR setting, the agent is allowed at most K 10 decision steps. At each step 4 the agent receives an RGB observation O, that includes panoramic view and front view with a horizontal field of view of 90 The agent's 's output each step consists of answers two multiple- choice queries: (i) which object category Gs matches the target. (ii) which navigation primitive az E V to execute next. For each query, the VLM selects the token with the highest likelihood, and the associated probability is interpreted as the model's confidence. After choosing ar, the agent executes the action, acquires the next observation, and proceeds step 1+1. The episode terminates when either the step budget K' is reached or the confidence of the predicted category Ot exceeds 95%.
Integrating a world model. Within the AR pipeline, the world model supports gecision-making in two complementary ways that mirror the two queries above. For query (i). the model generates synthetic future views that act auxiliary evidence in addition the real observation o,- These additional cues help the agent reason about occlusions, extreme viewpoints, and other distribution shifts that hinder recognition, as illustrated in Figure For query (ii), agent will first generate M candidate action sequences {A," ) m==1 each of length L. Given each candidate plan and its corresponding predicted observations, the agent estimates the value of alternative low-level control sequences before committing
action in the real environment. Unlike baseline policy that greedily chooses a++1 from o, alone, the agent equipped with a world model compares simulated outcomes for all candidates and executes the sequence that is expected yield the most informative next view. When a world model is used, the planner proposes M 2 candidate action sequences per step, each with horizon I. 4.
Art. 1: Tit.L-Tamtia Left
Ach. 2: Forward
Art. 3: Forward
Act. d: Forward
World Model
Planning Enhancement
What is the target object?
Perception Enhoncement
Figure 8: In AR, the world model supports both queries (perception and planning). In this example, the agent must identify a wooden door that is initially visible only from an extreme viewpoint. For each candidate action sequence, the world model predicts future observations; these forecasts augment the agent's perception and inform the choice of the next action.
Bounding box annotation. The target object marked by a red bounding box overlaid on the image. For the current real observation o,, the box is obtained from Habitat ground-trutt annotations. For the predicted frames {ō, / produced by the world model, apply SAM2 (Ravi et al., 2024) to segment the target, seeding the segmenter with the ground-truth box from the current real observation o, maintain correspondence across time.
Metrics. AR performance is reported using two metrics: (1) Success Rate (SR), defined as the fraction of episodes in which the final predicted label matches the ground-truth label y, and (2) Mean Trajectory Length, defined as the average number of executed actions before the agent either issues its final prediction exhausts the step budget K.
3

---

Published as a conference paper at ICLR 2026
B.2 IMAGE- GOAL NAVIGATION (IMAGENAV)
Image-Gowt Goal Navigation (ImageNav). also known as coal-condi visual navigation, requires an embodied agent reach the target location depicted by a single reference image of the goal. The environment is unknown, so the navigation policy must determine how to explore in order to locate the goal efficiently. To examine how world models can assist, we create 144 ImageNav episodes taken from 87 validation scenes of HM3D (Ramakrishnan et al., 2021).
Task setup. Each episode permits at most the agent receives an RGB observation comprising panoramic view and front view with horizontal field of view of 90" The agent then proposes a sequence of low-level navigation primitives A, [1++1, at+2: .... GC+L] with maximum horizon of L = 5. The first L primitives from the selected plan are executed in the real environment, after which the agent replans based on the newly acquired observation. An episode is successful if, within the budget of K steps, the agent's position enters a sphere of radius Rg = 0.5, m centered at the location specified by the goal image
20 decision steps. As in the AR setting, at step t
Integrating a world model. In ImageNav, the agent answers only the navigation query of which action sequence to execute next; therefore, the world model is used exclusively for planning enhance- ment. The agent first enumerates several candidate action sequences. For each candidate, the world model predicts the future observations that would follow the sequence were executed from the current state. The agent then scores each sequence by assessing how informative its predictions are for locating the goal, and selects the sequence with the highest expected utility. When a world model used, the planner proposes M = 3 candidate action sequences at each decision step, with horizon L= 5. The first I, 2 actions from the chosen sequence are carried out before the next cycle begins.
Metrics. We report three standard metrics for ImageNav: (1) Success Rate (SR), the fraction of episodes in which the agent reaches the goal within the decision budget; (2) Mean Trajectory Length, the average number executed actions across episodes; and (3) Success weighted by Path Length (SPL), which accounts for both success and path efficiency. Formally, for a set of N episodes,
Li max L. LT)
SPL Σ S
X 100%,
where S: E {0, }} indicates whether episode is successful, L; is the shortest path length from the start position to the goal for episode i, and L, is the actual path length executed by the agent in that episode.
B.3 ACTIVE EMBODIED QUESTION ANSWERING (A-EQA)
Active Embodied Question Answering (A-EQA) tasks embodied agent with answering open-ended, natural-language questions after actively exploring an environment. The questions span six broad categories that are common in embodied QA: recognizing objects, recognizing object attributes, recognizing object states, localizing objects, performing spatial reasoning, and performing functional reasoning. Our evaluation set contains 184 questions distributed across 54 indoor scenes drawn from the official OpenEQA split (Majumdar et al., 2024) and the validation set of HM3D (Ramakrishnan al., 2021).
Task setup. In A-EQA, there is no predefined navigation goal, so the agent must design its own exploration strategy gather sufficient visual evidence for answering the question. At every decision step f, the agent receives a panoramic RGB observation that we decompose into four perspective views, each with a horizontal field of view of 105* (see Figure 10). The exploration budget limited to 250 low-level actions; a single decision step can comprise multiple low-level actions, depending on the high-level intent. An episode terminates when the budget is exhausted or when the agent outputs final answer ÿ.
For A-EQA, we implement a two-level policy that separates deliberation and control. The high-level planner periodically issues one of two types of commands: (i) textual instruction (for example, "move to the hallway visible in the front view"), or (ii) the index of a landmark object detected in the current panorama. Once high-level command is produced, execution is delegated to the low-level controller. If the command specifies a landmark, the controller uses depth data together with custom pathfinder to plan and follow route to that landmark. If the command is textual
4

---

Published as a conference paper at ICLR 2026
instruction, the controller generates sequence of low-level actions earry out the instruction. This planner-controlle loop continues until either the 250 atomic actions are consumed or the high-level planner decides to emit the final answer ý.
Candidate plans from WMs Executed plan
How many cushions the red sofa?
Figure 9: Overview of our embodied closed-loop evaluation for A-EQA. For each question, the high-level planner proposes multiple candidate action plans and queries the world model generate the corresponding future observations. The agent then evaluates each plan together with its predicted observations and selects the plan that maximizes the expected reward before executing it in the environment.
Integrating a world model. In A-EQA, the world model is primarily used to strengthen the high- level planner. At each high-level decision point, the planner samples M candidate action plans and queries the world model to produce the corresponding predicted observations, as illustrated in (m) ((m) and chooses the plan
Figure 9. The agent then evaluates each plan-observation pair (A(m) that maximizes the estimated reward under the current question context. This differs from the AR setting, where perception and planning are evaluated through two separate queries. In A-EQA, the high-level planner must both design a long-horizon exploration sequence and decide when to stop exploring output final answer ý. Consequently, the world model supports a single unified query: the predicted observations simultaneously refine the agent's understanding of the scene and provide forecasts for scoring alternative exploration plans. When world model is enabled, the planner proposes M = 3 candidate sequences per step, each with horizon I, = 14. Unlike AR or ImageNav. only the terminal predicted observation at step is returned to the high-level planner for scoring, rather than the full rollout over all L steps.
Landmark detection and labeling. Landmark objects are detected by first rumning YOLO-World to obtain bound- ing boxes and then applying SAM2 to derive instance masks (Ravi et al., 2024; Cheng et al., 2024). This detection pipeline follows the Set-of-Marks (SoM) strategy ( (Yang al., 2023a) shown in Figure 10 and provides discrete set of navigable targets for high-level planning.
Curr. View: Front
View:
Views Right
Curr. View:
Metries. A-EQA performance evaluated with three mer- rics. (1) Answering Score: a large language model (e.g., GPT-4o) compares the agent's 's final answer ý to the ground- truth answer and assigns raw score in [1, 5). where indicates perfect match. We average the raw score across episodes and then linearly map it to [0, 100). (2) Mean Trajectory Length. This the average travel distance the agent covers before either producing its final answer or exhausting the step budget K, lower is better. (3) Success weighted by Path Length (SPL): this metric rewards both answer quality and navigation efficiency. For episodes in which the agent fails return answer, fall back to its
Figure 10: Illustration of the Ser-of- Marks (SoM) representation that en- codes candidate navigable directions. The high-level planner chooses among these discrete landmarks when con- structing candidate action plans.
5

---

Published as a conference paper at ICLR 2026
blind LLM variant and set the SPL contribution to zero. Formally,
N
0
L max Lis Li) x 100%,
SPLA-EQA
N
4
where N is the number of evaluation episodes, oy E [1,5] denotes the raw Answering Score for episode i, Lद denotes the shortest-path length from the start to viewpoint that affords correct answer, and L; denotes the actual path length executed by the agent in episode i. A higher value indicates both more accurate answering and more efficient exploration.
B.4 ROBOTIC MANIPULATION
We study whether world models can improve low-level manipulation, which is a core capability for embodied agents. Our evaluation covers four robotic manipulation tasks in RLBench (James et al., 2020): Push Buttons, Slide Block to Color Target, Insert onto Square Peg, and Stack Cups. RLBench is widely used benchmark for robot learning. Each episode provides natural- -aatuual- instruction that specifies the task objective, and the agent must control a 7-DoF robotic arm to satisfy that objective. We prepare total of 200 evaluation episodes, with 50 episodes for each task.
Task setup. each decision step i, the agent receives an observation og and proposes an action sequence A, [[a-1: ar+2, .... a where each low-level action is parameterized as a [z. V, 2, roll, pitch, yaw, gripper]. We consider two base policy settings with different horizons: L= 5 for a VLM base policy that emits discrete actions, and L = 50 for 3D diffusion base policy that emits continuous actions. An episode is counted success if the specified goal g is achieved within the step budget K.
When VLM is the base policy, directly producing precise low-level controls is challenging for current VLMs. Fol-
Task objective: Slide the red block to
lowing (Yang al., 2025a), we therefore introduce two enhancements. First, we discretize the action space by dividing the position components (I, y. into 100 bins
Ausillary Information of Object Positions:
and the orientation components (roll, pitch, yaw) into 120 bins. Second, we augment the observations with object index markers and provide precise object poses for indexed objects so that the VLM can directly access spatial infor- mation during planning (shown in Figure 1 11). Under this configuration, the manipulation policy is allowed at most K = 15 low-level action steps per episode. In contrast, when using a 3D diffusion policy (Ke et al., 2024) as the base policy, the controller naturally generates continuous low-level actions, so we do not apply the discretization or the additional indexing enhancements. In this configuration,
Pobject 1:: [45, 13, 18),
"soject 2: [72, 20, 18)
'ooject 3: [50,
'soject [36, 181.
lobject 5:: [69, 39, 15])
Figure 11: Illustration of the auxiliary information provided the VLM pol- icy. The objects are marked with in- dices, and their positions are given to the VLM to facilitate decision decision-making.
the manipulation policy is permitted at most K = 8 macro decision steps per episode.
Integrating a world model. As in ImageNav, we use the world model exclusively for planning enhancement. The agent executes propose, simulate, and revise loop so that it can reason about the consequences of alternative plans before applying any action in the real environment. At each decision step, the planner proposes M 5 candidate action sequences. When the length of candidate sequence is shorter than the world model's required action-conditioning length, the unified action API linearly interpolates the sequence to the required length. Conversely, when the candidate sequence is longer than required, the unified action API uniformly samples actions along the sequence to match the world model's is input length. The planner then evaluates the simulated outcomes and selects the sequence with the highest expected reward, and the loop repeats with updated observations.
Metrics. We report two standard metrics for manipulation tasks: (1) Success Rate (SR), the fraction of episodes in which the agent reaches the goal within the decision budget; and (2) Mean Trajectory Length, the average number of decision steps across all episodes.

---

Published as a conference paper at ICLR 2026
B.5 POLICIES IN EMBODIED TASKS
There are three types of policies in paper: the base policy, the proposal policy, and the revision policy, The base policy is an independent policy that interacts with the environment without using world model, and when a world model is enabled, it is always the same as the corresponding proposal policy. When world model integrated, the proposal policy generates multiple candidate action sequences at cach decision step, and the revision policy evaluates these candidates and selects one based on the predicted rollouts produced by the world model.
B.5.1 BASE POLICIES AND PROPOSAL POLICIES
In our experiments, we employ two types of base policies for AR and ImageNav: VLM policy and a heuristic policy. For the VLM policy, we use -eeen225-VL-72--nnsttreee 2025) the default base policy and as the proposal policy when integrated with world model to answer queries. For the heuristic policy, we implement primitive action sampling mechanism that draws actions from the action space according to the previously executed actions and a set of handcrafted rules. Concretely, if there exists previous action, then the next action must not be its inverse (for example, a turn_leff ft cannot be immediately followed by a turn_right) In addition, we prevent excessively long subsequences of turns in the same direction by capping the maximum number of consecutive turns to four. These rules help the heuristic policy to avoid redundant back-and and-forth movements and to explore the environment effectively.
(Bai et al.,
For manipulation tasks, we likewise consider two base policies: VLM policy and a 3D diffusion policy. The VLM policy remains -wen2.5-VL-72B--1
by default. The 3D diffusion policy follows 3D Diffuser Actor (Ke et al., 2024); we train using the authors' official code. To encourage diverse action trajectory proposals, scripts so that task variants occur with equal frequency during training. For each manipulation task, the diffusion policy is trained on 120 demonstrations and used as the proposal policy to generate short-horizon 7-DoF gripper action sequences within the planning loop. When using 3D Diffuser Actor as the proposal policy, we report results on only three manipulation tasks, since we find that Stack Cups is difficult for the diffusion policy to learn reliably.
drop its text input and modify the task-definition
B.5.2 REVISION POLICIES
The revision policy is the component that refines the proposals produced by the proposal policy using the world model rollouts. At each decision step i, the proposal policy outputs M candidate action sequences {A(m)) M 1m and the world model predicts the corresponding future observations (m)
{Ó(
( M
Im=1- The revision policy
Trevision : (I(A, (m) (m)) )
J Or, g)
+ D;
consumes these imagined trajectories together with the current observation o, and goal g, and outputs the final decision DJ. Depending on the task, D may be a pure action decision (ImageNav, Manipulation) or a joint action-answe decision (AR, A-EQA).
Score-and-select for action-only tasks. For Goal Navigation and Robotic Manipulation, the objective is to reach a goal state, and the revision policy only needs to choose which action sequence to execute. In these settings, D; Å; and Trevision is instantiated as score-and-select operator as in Equation (4) of the main paper:
A = A (m²
Sect (A(
(m) O,, g
m arg max me (I....M)
where Saa(.) is an action-centric scoring function that estimates the expected task reward of each imagined trajectory (c.g., progress toward the goal).
In most experiments, we instantiate Sua with VLM-based reward model: we use Qwen2.5-VE 12B-Innstrut-AWWO as the default revision policy to score candidate rollouts and to select the action sequence with the highest predicted utility. For ablations, also replace -wen2.5-VL--22--nnste AWQ with Intern VL3- AWW-78B-AWQ (Zhu et al., 2025); results Table 7 show that world model integration consistently improves performance regardless of the specific VLM used. In addition to
7

---

Published as a conference paper at ICLR 2026
Table 7: Task performance for Intern VL3 variants with and without world model. Higher SR%, %, and Ans. Score are better; lower Mean Traj. is better.
Model Details
ImageNav
A-EQA
Model Type
Method
SR
Mean Traj.
SR1
Tmi...
SPL-1
Score
Mean Traj / SPI-1
Base Policy
Int (wlo WM)
49.91
7.06
13.19
60.30
7.46
47.28
20.45
31,22
Image Gen.
SVDf
55.72
5.37
40.97
$2.50
26.26
47.13
16.78
34.54
VLM-based scoring, consider task-specific reward functions when direct signal is available. For example, in Image-Guu Navigation we also evaluate an APIPS-based reward that measures perceptual distance between predicted observations and the goal image, and use this score in place of the VI.M-based Sact
Joint action-answer refinement for AR and A-EQA. For AR and A-BQA, each episode combines action planning and question answering. Here, the world model rollouts are used not only to guide the next action, also to provide auxiliary visual evidence for the final answer (e.g., multi-view observations that reduce occlusions). This leads to richer instantiation of the revision policy than the pure score-and-seled operator above.
At time step f, the output of Tevisson is decomposed into action component and an answer compo- nent. Let i denote the predicted answer (a category label for AR and a natural- natural-language answer for A-EQA). We write
D; = (Ái, 0t)
(( 3 Or, 8): (()()))
Trevision
In our implementation, the action component A; is still selected by score-and-select rule with an action scoring function Sax:
A
A!! (mm))
m arg max Suct mE(l.....
[A (m)
Ó (mm)
Ot, 8)
while the answer component ÿe is obtained by aggregating predicted futures from all candidates:
Ú: = f Oc: 8, (0(mm)) (m)) 1==1)
Here, Swa(-) again scores trajectories from the perspective of future task performance (for example, preferring trajectories that move the agent toward informative views or closer to the target object), and (-) (+) is an answer head that consumes the current observation, the goal, and the set of predicted futures as multi-view evidence. In practice, lans implemented with the same vision-language model as the proposal policy, which takes the frames as input and outputs the answer.
Thus, for AR and A-EQA, the revision policy operates in two coupled ways: chooses how the agent should move next via Sact and A and simultaneously uses the simulated rollouts as additional context to produce more informed answer ýt- This joint action-and answer refinement is a richer instantiation of Tevision than the score-only operator in Equation (4), and is specific to tasks that require both control and question answering.
B.6 WORLD MODELS IN EMBODIED TASKS
Output format. The world models evaluated in our framework fall into two categories according their native output format: perspective models and panoramic models. Perspective models, such as NWM (Bar et al., 2025), LTX-Videe (HaCohen et al., 2024), and Wan2.1 (Wan et al., 2025). generate frames in perspective view. Panoramic models, including PathDreamer (Koh et al., 2021), SE3DS (Koh et al., 2023), and our post-trained variants, produce equirectangular panoramas. For integration into our closed-loop pipeline, panoramic outputs are decomposed into perspective views, which are then supplied to the agent. In A-EQA, the agent consumes four principal perspective views (front, left, right, back) when they are available. In AR, the agent uses the view that contains the target bounding box; if the box not visible, we discard the generated frames until the predicted box (from SAM2) enters the field of view. Unless otherwise specified, each perspective view image is resized 384 384 pixels before being passed to the agent.
8

---

Published as a conference paper at ICLR 2026
Input format. Panoramic models conditioned equirectangular panorama at resolution of 576 x 1024 pixels. Perspective models, when possible, take the current front-view observation with resolution 480 x 480 as input. Some models require additional modalities. SE3DS expects depth map, while Patmer requires both depth and a per-pixel semantic label map. For all depth-aware models, provide ground-truth depth from Habitat. For PrthDrr the initial semantic map is obtained by running a pretrained RedNet (Jiang et al., 2018) on the initial RGB-D frame to produce per-pixel labels that match the required input specification.
C TRST-INGINING RECIPE FOR EMBODIED WORLD MODELS
In this section, we describe how an off-the-shelf video generation model is adapted, via post-training. into an action-controllable world model suitable for embodied tasks. We first formalize the learning objective and the cetton-observation alignment (Appendix C.I), and then detail the concrete post- training setup used for tasks in Habitat-Sin -imm and for Robotic Manipulations (Appendix C.2).
C.I PROBLEM FORMULATION
Let X1 R denotes the initial RGB frame that conditions the generation process. Our goal to synthesize an N- video X = [11 X2, ..... xv] E xxx/Ixxxx where X represents a plau- sible sequence of future observations after executing sequence of actions
[ai, 022 .... an]. For tasks in Habitat-Sim, we adopt a discrete action space with 0; E V, where V is a finite set of navigation primitives (e.g., Forwa rd, Turn- Left, Turn-Right, Stop). For manipulation, we use continuous action space with E IR? corresponding to 7-DoF end-effector poses. Actions in Habitat-Sim specify relative transformations between consecutive observations. Since a, maps X;-1 to X,, no action precedes the first frame. To maintain one-to-one alignment between frames and actions, we prepend a special token and set d1 ANall- In contrast, for manipulation tasks during post-training, actions are absolute end-effector poses expressed in the world frame, so there is naturally one-to-one correspondence between actions and frames.
We formulate future-observation synthesis with the world model go by learning the conditional distribution pa X x1 C(A)). where C(A) denotes the control signal emitted by the unified action API. This API converts the native action sequence A into the conditioning interface expected by the pretrained video generator (for example, a text prompt, a camera trajectory, or a sequence of low-level controls). This formulation yields conditioond rollouts that evolve from the initial frame X1 according to the specified action sequence, thereby aligning the pretrained model with the domain distribution and action space of the target embodied tasks.
C.2
CONFIGURATION
Table 8: Post-trained (action-conditioned) world models used in our experiments, with repositories and training configurations.
World Model
Domain
Reposibory
Frames (N) Truin Res.
Notes
Hubitat-Sin
Comncs PredezZt OAgarwal et al. 2025)
Hammzn Sim
github. con heri coonos costree-pee -pedee
13
576 1021
Official repo
UIX- (HaCo 2004)
Haiint-Sim
gut now/ et stte/ moe- dep- Tha neer
17
576 1024
Officind
(Was
Hoiini-Sim
github.
570
Oflicis rpo
2 (5B)
3025)
Habitu-Sin
qitbub. coo /model aacope/B pu yy deuddd
670
Officia rapo
Word (A14H)) (Wan 2025)
Hasmm-Sim
gittub. con//aode /wode! del cooge/ /tt (rreytth-Brood
13
570 1094
Officna mpo
SVD) (Riatmann et al., 2023s)
Habin--Si
github..com wwwe///ppppooee seel gun Xerwnd
14
570 1094
Sci-allpil hanadi on ropo
Poet-training on manipulatten
Coenss-rrahha (Agiraal at al. 2015)
Manipahtion
github. mom foor a commoo powern---reddiee github. lavell 30/ ae/evo Xced_
450 460
Oflicis rpp
SVDJ (Blantiann et al., 202333
Manipatatio
14
448 448
Self adapted based repo
For tasks in Habitat-Sim, we use panoramic observations as both the input and the output of the video generators. We fine-tune the pretrained video generation models at a resolution 576 1024 and train them to predict N future frames on our self-collected panoramic action-observation corpus from Habitat-Sim In these tasks, the action space is discrete and comprises four navigation primitives: Forward 0 Turn Left 22. 5", Turn Right 22. 5% and Stop. For manipulation tasks, we use front-view observations as both the input and the output of the video generators. We fine-tune the pretrained video generation models at resolution of 480 x 480 (Cosmos-Predict2)

---

Published as a conference paper at ICLR 2026
448 x 448 (SVD) and train them to predict N future frames with continuous 7-DoF end-effector poses as conditioning.
Unless otherwise stated, post-training uses 40K sampled instances for the Habitat-Sim tasks and for the manipulation tasks. All models are initialized from their official pretrained weights and adapted the corresponding dataset for epoch. We rely the official implementations and the recommended hyperparameters for fine-tuning whenever available; specific post-training details of various world models are summarized below in Tables 8 and 9.
Table 9; All the world models and their details in World-In-Wool "t" denotes post-trained (action- conditioned) variants.
World Model
Model Type
Control Type
Inpot Type
Zero-shot (no post-training)
PuthDreamer et 2021)
Image Gen.
Viewpoint
RGB-D; Pano
0.69H
SE3DS (Kob et al., 2023)
Image Gen.
Viewpoint
RGB-D: Pano
1.1B
NWM (Bar et al., 2025)
Video Gen.
Trajectoey
RGB
18
(Blatmaano al., 2023a)
Gen.
Image
I:TX-Vider (HaCrihen et al., 2024)
Video Gen.
Text
RGB
2B
Hanyuan (Kong al., 2024)
Video Gen.
Text
RGB
130
Wan2.1 (Wan al., 2025)
Video Gen.
Wan2.2 (Wan al., 2025)
Video Gen.
Text
RGB
SH
Wan2.2 (Wan 2025)
Video Gen.
Text
RGB
A148
Cosmoc-Predicd Predier2 (Agirwal
Videa Gen.
Text
2B
Runway Gend (Runway Research, 2025)
Video Gen.
Text
RGB
Post trained (action-condiiooned)
SVDI (Blatimann et al., 2023a)
Video Gen.
Action
RGB: Pano
1.5B
LTX Videof (HaCohen
2024)
Video Gen.
RGB:
W (Wan al., 2012)
Video Gen.
Action
RGB: Pano
14B
Wan2.2| (Wan et ali, 2025)
Video Gion.
Action
RGR; Pano SH
Wan2.2f
2025)
Gen.
RGB:
Cosmas-Predier2: (Agarwal et all, 2025)
Video Gen.
Action
RGR; Pano 2B
In Table 10, we summarize the computational resources required to post-train each world model on ~40k domain-specife clips collected from Habitat-Sim. This post-training stage is intentionally lightweight and is several orders of magnitude less expensive than full pretraining. For [4B-parameter variants, we adopt LoRA fine-tuning to reduce GPU memory usage, while all other models are fine-tuned with full weights.
Table 10: Post-training resources for 440 domain clips per model. The procedure is lightweight and substantially cheaper than full retraining
Model
Model
GPU Memory (peak)
H1000 GPU-bours
SVD
1.5B
84 GB
29
LTX-Video
2B
61 GB
Wan2.1
14B
57 GB
74
Cosmos-Predier2
2B
71 GB
15
D POST-TRAINING DATASET CONSTRUCTION
For the post-training dataset used in manipulation tasks, we rely on the official RLBench code- base (James al., 2020) to generate data. Specifically, produce 200 demonstrations for each manipulation task. Each demonstration includes approximately 150 front-view RGB observations together with the corresponding sequence of 7-DoF end-effector poses. These pose sequences are aligned with the image observations and serve as the action labels during post-training. For the tasks evaluated in Habitat-Sin (Savva et al., 2019), there is no existing pipeline for constructing large-scale dataset of panoramic action trajectories. To address this gap, we build comprehensive post-training dataset by sampling action trajectories from the training splits of indoor scenes in HM3D (Ramakrishnan et al., 2021) and Matterport3D (Chang et al., 2017). Our trajectory sampling procedure is described in Appendix D. summary of the resulting dataset statistics is provided in Table 11.
D.I TRAJECTORY SAMPLING
10

---

Published as a conference paper at ICLR 2026
Algorithm 1 Three-stage construction of the post-training panoramic dataset
Input: scene mesh S, waypoint density p. weight a, filter radius rf, leaf ratio 7)
Output: set of panoramic trajectories T
I/ Stage 1: waypoint selection
1: S f Area(S)
2: Nwp max 1400, IpS])
target number of points
3: P <- UNIFORMS A MPL AMPLENAVIAALLLIEEEE... ANAVIGABLE Nwp)
4: build geodesic distance matrix on
5: for all p; P do
leaf score s(i)
6: ecc(i) + max, Dij - 2, Dij
7:
d(i) f- [P-
8:
s(1) ecc(() a d(i))
9: sort P by s(i) in descending order
higher s(i) = more peripheral
10: W 2-
11: for all p; in sorted P do
D radius-based greedy pruning
12:
if W: Diw ri then
13:
W WU {p:}
// Stage 2: path generation
14: T + -0
15: NEar [nNwwl
16: 4 < W(:Nieal
b unvisited waypoints random start
C RANDOMSAMPLE(4)
18: while U & Ø do
19:
-ar (e GEODESICDIST(C
20:
<- SHORTESTPATH(C, n)
Habitat planner
21:
record panoramic RGB-D frames along 7 and append to T
I/ Stage 3: waypoint dynamic update
22:
for all w E W do
if 3m E T GEODESICDIST(m, w) < ri then
24:
W + W [w}
>> mark as visited
25:
recompute s(.) updated W, then sort in descending order U W[al
26:
b refresh unvisited set
27:
c 71
28:
return T
Our aim is record physically reasonable trajectories that resemble the exploration behavior of real agents in indoor spaces. We follow three guiding principles: (i) Diversity. The trajectories should cover many viewpoints and actions so that the model sees the scene from different perspectives and motion patterns. (ii) Plausibility. The paths must respect physical constraints; the agent must not move through walls or other solid objects. (iii) Manage- ability. The data should be free excessive redundancy so that training remains balanced and efficient.
Statistic
Value
Number of scenes
858
Panorama RGB frames
763,724
Action trajectories Depth recorded
439,213
Camera poses recorded Low-level actions recorded
Table 11: Statistics of the post-training panoramic dataset.
We implement these principles with sampling procedure shown in Algorithm and described below.
1. Waypoint selection. For scene of floor area S we set the waypoint density to -4 -2 and draw
Nwp max 1400, (oS])
navigable points uniformly across the scene. We construct a complete graph whose edge weights Dij are the geodesic distances between points p; and py. Each vertex is assigned a leaf score
s(i) eco(i) d(i),
11

---

Published as a conference paper at ICLR 2026
where ecc(t) max; Diy is the eccentricity, d(i) (PP) -1 Dij is the mean geodesic distance all other vertices, and a 1.7. Sorting vertices by s(i) in descending order, greedily build a waypoint set W that respects a minimum spacing of Ti = 3 m: a candidate is accepted only if Doj 2 7f for every waypoint j already chosen.
2. Path generation. We maintain list U of unvisited waypoints, initialized with the top Neal vertices of W. Starting from a random waypoint E U, we repeatedly move to the nearest unvisited waypoint
7 arg
min GEODESICDIST(C, w), (EM((()
and use the Habitat path-finder to compute the shortest collision-fre path T from c to n. Panoramic RGB-D frames recorded at every step along and appended to the trajectory set T.
with 3. Waypoint dynamic update. After each segment т we label any waypoint GEODESICDiST(m, w) < Tr for some path point m E T as visited and remove it from W. We then recompute s()) the remaining vertices, resort W, and refresh the unvisited list
U < Wi:Near)-
The next segment starts from c <- n, and the loop continues until UI is empty. This dynamic reselection guarantees that peripheral regions covered while avoiding redundant sampling in interior corridors.
woypoinl_idx: 0,
waypoint O, step:
Figure 12: Top-down visualization of sampled waypoints in scene. Red (left) and yellow (right) after radius-based pruning. The proposed strategy places waypoints
dots are the final waypoints throughout peripheral regions while avoiding redundant interior points, yielding diverse and spatially balanced trajectories.
Compared with random sampling of start and end waypoints, the above strategy distributes waypoints across peripheral areas such as bedrooms while avoiding redundant paths through interior corridors. The resulting dataset therefore offers balanced and diverse of viewpoints for post-training (see Figure 12).
E VISUALIZING WORLD MODEL PREDICTIONS
We illustrate the behavior of several world models under identical action sequences generated by the planner. Figure 13 and Figure 14 show example rollouts in which the action sequence consists solely of Forward d actions; well-behaved model should yield pure forward motion. The figures contrast models that follow the commands with those that drift or hallucinate, underscoring the importance of precise action control for downstream embodied tasks. These examples also reveal current limitations of world models in trustworthy prediction (Sun et al., 2023; Zhang et al., 2025c; Mei et al., 2025). For further examples of good and bad predictions, see Figures 15 to 18.
12

---

Published as a conference paper at ICLR 2026
Action Control: Forward
Good Example:
Bad Examples:
Figure 13: Examples of good and bad predictions. The action sequence contains only Forward actions. Models that violate this requirement yield observations that can mislead the planner.
Action Control: Forward
Good Example:
Bad Examples:
Figure 14; Examples of good and bad predictions. The action sequence contains only Forward actions. Models that violate this requirement yield observations that can mislead the planner.
13

---

Published as a conference paper at ICLR 2026
Good Examples:
Bad Examples:
Figure 15: Additional examples of good and bad predictions.
Good Examples:
Bad Examples:
Figure 16: Additional examples of good and bad predictions.
14

---

Published as a conference paper at ICLR 2026
Good Examples:
Bad Examples:
Figure 17: Additional examples of good and bad predictions.
15

---

Published as a conference paper at ICLR 2026
Good Examples:
Bad Examples:
Figure 18: Additional examples of good and bad predictions.
16

---

Published as a conference paper at ICLR 2026
F PROMPT TEMPLATES USED IN WORLD-IN-WORLD
In this section, we provide the exact prompt templates used in our experiments for four tasks in World-In-World: (i) Active Recognition (AR), (ii) Image-Goal Navigation (ImageNav), (iii) Active Embedded Question Answering (A-BQA), and (iv) Robotic Manipulation.
F.1 ACTIVE RECOGNITION (AR) PROMPT
AR Answerer Prompt
Please recognize the object in the image bounded by the red box.
AR Planner Prompt
You are an Al agent tasked with identifying a target object within an image specifically, the object enclosed by a red bounding box. Your objective to navigate toward viewpoint that maximizes the target's visibility and recognition accuracy. Instructions:
1. Based
on the current fobs fobs_key] observation, plan the next <look_aadd _look_ahead_aeee
num> action(s) to take in sequence.
2. Use the following heuristics to guide planning:
red-boxed object appears on the left side of image, turning left often
improves visibility.
If appears on the right side, turning right is usually beneficial.
the object is partially occluded obstructed, consider repositioning to bypass the obstacle and refine your viewpoint.
3. Choose sequence of actions that leads to clear, centered, and unobstructed view the red- rexeboxed object.
AR Answerer Additional Prompt (with WM Rollouts)
You now have a composite visualization formed by stitching imagined views from multiple perspectives around your current position. These perspectives centered the target object (enclosed within the red bounding box).
Use these synthesized views
Improve object identification accuracy.
Make more informed recognition decisions.
AR Planner Additional Prompt (with WM Rollouts)
You are now simulating imagined future trajectories by generating hypothetical actions and their corresponding observations.
Use these imagined observations to:
Evaluate the potential outcomes different action sequences.
Make informed navigation decisions by selecting the next best action based predicted future states and your current state.
Note:
Each imagined frame is annotated with the specific action taken and its index at the top of the image.
Pay attention to the presence of red bounding boxes indicating the target object. If the target is not visible in frame, this indicates poor action.
17

---

Published as a conference paper at ICLR 2026
You should adjust your action selection strategy avoid such failure states.
F.2 IMAGE-GOAL NAVIGATION (IMAGENAV) PROMPT
ImageNav Nav Planner Prompt
You AI navigation agent tasked with locating the position from which the goal image was captured. Your objective is to plan sequence of actions that leads to position where the goal image clearly visible, centered in the front view, and appears have been taken within at your current position.
Inputs: You are provided with sequence of images:
1. First, the current egocentric observation: (obs_ (obs_key).
2. Last, the goal image: a reference image that represents the target viewpoint you are trying to reach.
Task:
1. Based the input images, plan the next < (llokkoaheeddooaeeaeaaaa <looo__atee___ttionn_n ac- tion(s) in order.
2. Optimize for:
Alignment:
The goal image should centered in the front view.
Proximity: Your position should match the goal image's 's capture point.
Visibility: The goal image should appear clear and unobstructed in your current front view.
ImageNav Planner Prompt (with WM Rollouts)
You AI navigation agent tasked with locating the position from which the goal image was captured. Your objective is to plan sequence of actions that leads to position where the goal image clearly visible, centered in the front view, and appears have been taken within at your current position. Inputs:
You provided with:
1. The goal image: reference image that represents the target viewpoint you trying to reach.
Task:
1. Based the input images, plan the next < tion(s) in order.
_um)
2. Optimize for:
Alignment:
The goal image should centered in the front view.
Proximity: Your position should match the goal image's capture point.
Visibility: The goal image should appear clear and unobstructed in your current front view.
F.3 ACTIVE EMBEDDED QUESTION ANSWERING (A-EQA) PROMPT
A-EQA High-Level Planner Prompt
You embodied navigation and question-ansee answering agent specialized indoor scene understanding. Your goal is to either answer the user's question directly from the current observation propose high-level navigation planning to gather more information.
User Query: (question)
18

---

Published as a conference paper at ICLR 2026
Inputs:
You are provided with the following:
A stitched panoramic image with annotations - composed of multiple directional images captured from your current position (the name of each view is labeled the top of the image). Each detected object is annotated with its contour and a unique object index.
2. A stitched panoramic image without annotations- - vissulll identical but without overlays, serving as a clean reference.
3. A dictionary mapping detected objects to their corresponding perspective views and object indices in the annotated image: Format:
{vviewidd
[object_indd)
object_name)
Current mapping: (detected objs}
Note all the provided images are in the formt of (obs key). Task Description:
Your task is to:
1. Analyze the visual information from each perspective direction.
2. Identify all possible exits and doorways in the environment.
3. Give one high-level navigation plan to further explore the scene order to answer the User Query.
4. If the answer to the question is fully evident from the current observation, provide directly. Otherwise, set your current answer "None".
Output Format:
Return your response as a dictionary with the following structure: { 'Reaso' <Your visual reasoning and analysis>,
'Action Plan': <Description of your next high-level
havigation plan>,
'Chosen View': <One of: 'frent' 'left', 'right',
'back" indicating the view you are going to further explore in your Action Plan>,
Chosen Landmark'; <Index of the selected object landmark from the annotated stitched image, or 'None'> Annwer'' <Your answer to the User Query, or 'None'>
Constraints:
Provide exactly one high-level action, including one Chosen View' and one
Chosen Landmark'.
If no suitable annotated object is available in your desired direction, set Chosen Landmark' to None' and describe your intended action in the intion Plan' field.
Each 'Act ion Plan' should include a clear and executable instruction and stop condition.
Good Example: 'Action Plan': "Pass through the doorway (object index "3") the front view, and stop once inside the next room.
Good Example: 'Action Plan': "Approach the sofa (object idx "10") in the left view, and stop once we can see the objects on it."
Bad Example: 'Action Plan': "Move into the kitchen area visible in the view and stop once inside the kitchen."
kitchen area is not specific object and not clear how to get there.
If landmark is selected, it must correspond to visible, annotated object in the stitched image.
19

---

Published as a conference paper at ICLR 2026
Do select unlabeled objects they typically indicate previously visited non- non-informative regions.
Populate 'Anwwer' only when you are confident the question can answered from the current observation. Otherwise, 'Answer' 'None' in the dictio- nary.
Tips:
If you observe a door closed state, means you cannot pass through it.
If the current observation shows that your previous plan has yet been completed, it is acceptable to propose a similar plan again to continue pursuing the same goal.
Leverage human spatial habits to guide your planning. For instance, if the goal involves finding television, selecting nearby sofa may be effective, these often appear together in living spaces.
A-EQA Low-Level Planner Prompt
You are now performing low-level navigation action planning for an indoor scene explo- ration task.
Inputs:
You are provided with:
1. An updated RGB image with annotations, representing the egocentric view of your current environment:
Detected objects are annotated with contours and unique object indices with square text boxes.
2. A high-level navigation plan represented dictionary with two fields:
Action Plan': A description the intended navigation strategy.
Chosen Landmark': The object index the selected landmark from the annotated image approach, None'' if no landmark is selected (in which case follow the Action Plan' description).
The current high-level plan is: {high _evelleeel____ee _lan] Note all the provided images are the format of (obs_key). Task:
Your task is to:
1. Analyze the visual scene and identify your position relative to the goal.
2. Determine the next low-level action(s) to take in sequence, up to a maximum of Blook_aheade action _nnoonn___mmmm_m steps.
Constraints:
You must generate less than [look_ ahead_ actionn num) low-level actions.
The actions sequence should align with the goal descripted in high-level Action
Plan' and Chosen Landmark'.
If the navigation goal or selected landmark in the high-level plan is either:
not visible in the current observation,
already reached (i.e., centered, unobstructed, and close).
then your only action should be "stop"..
Tips:
If the landmark object partially occluded or obstructed, consider repositioning to bypass the obstacle before approaching it directly.
Choose actions that meaningfully move the agent toward the selected landmark or fulfill the intent of the high-level plan.
Maintain spatial awareness: understand the relationship between your egocentric view and the direction of the target.
20

---

Published as a conference paper at ICLR 2026
A-EQA High-Levvel Planner Additional Prompt (with WM Rollouts)
In addition to your current (real) observations, you are now provided with simulated out- comes fow-rsstoowrsootttiinn reconstructions that represent the potential result of executing future navigation plans. These simulated outcomes are designed to help you better understand your surroundings and support more informed navigation planning. Each simulated outcome includes:
Proposed High-Level Plan: A hypothetical navigation strategy used to generate the simulated result.
Simulated Observation: A stitched panoramic image showing what the environ- ment might look like after following the proposed plan.
You should use this information to:
Evaluate the potential effectiveness and correctness of the proposed high-level strategies.
Make informed decisions by selecting your next high-level plan based on both the simulated information and your current real observation.
Notes:
Object indices remain consistent across simulated and real observations.
Simulated outcomes are NOT fully accurate. If you believe you can answer the user query based on simulation alone, you should NOT provide a final answer yet. Instead, select high-level plan that will lead a real observation and validate your answer afterward.
Your current simulated outcomes are:
F.4 ROBOTIC MANIPULATION PROMPT
Manipulation Planner Prompt
You are Franka Panda robot with parallel gripper. You can perform various tasks and output a sequence of gripper actions to accomplish a given task with images of your status. The input space, output action space and color space are defined follows: Input Space You are given the following inputs:
1. Human Instruction: A natural language command specifying the manipulation task goal.
2. Object Dictionary:
Each object represented by unique index (e.g., object 1) and mapped 3D discrete coordinate [X, Y, Z].
3. Annotated Scene Image:
Each object the image is annotated with:
circle point marker with
unique object index, which corresponds to the object dictionary.
There a red XYZ coordinate frame located in the top-left corner of the table.
-The XY plane represents the surface plane of the table (Z (Z=0) 0).
The valid coordinate range for X, Y, Z is: [0, ()]-
Output Action Space
Each output action represented as 7D discrete gripper action in the following format: [X, Y, 2, Roll, Pitch, Yaw, Gripper state].
X, Y, Z are the 3D discrete position of the gripper in the environment. follows the coordinate system as the input object coordinates.
21

---

Published as a conference paper at ICLR 2026
The allowed range of X, Y, Z. is [0, ()).
Roll, Pitch, Yaw the 3D discrete orientation of the gripper in the environment,
represented as discrete Euler Angles.
The allowed range of Roll, Pitch, Yaw [0, (} and each unit represents () degrees.
Gripper state is 0 for close and for open.
Color space
Each object can only be described using one of the colors below:
["red", "maroon", "lime", "green", "blue", "navy", "yell Syellow", "cyan", "magenta", "silver", "gray", "olive", "purple", "teal", "azure", "violet", "rose", "black", "white"],
Manipulation Planner Additional Prompt (with WM Rollouts)
You are now provided with simulated outcomes in addition to your real-time observations. These outcomes are low-resolution predictions of what the scene may look like after executing hypothetical action plans.
They are intended to help you reason about the environment make more informed decisions.
Simulated Outcome Structure
Each simulated-outcome item includes:
Proposed Action Plan: The sequence of gripper actions that led to the simulated result.
Simulated Observation: The simulated result after following the proposed plan.
How to Use This Information
You must consider both:
Your current real observation of the environment, and
2. The provided simulated outcomes.
Use these
Evaluate how well each proposed plan satisfies the task objective. Identify if any proposed plan fully achieves the instruction goal.
If a proposed plan appears valid and effective, you may adopt directly as your final response.
If no plan fully meets the goal, generate revised entirely new action plan, guided by insights from the simulations and the real-world scene.
Additional Notes
Simulated outcomes are approximate. Treat them as helpful forecasts, not absolute truth.
You must analyze these hypothetical action plans and their simulated outcomes in the reasoning_ant and feffection field of the returned JSON (e.g., their differences and why you choose one over another).
Always prioritize correctness and robustness the final executable plan.
You are now given the following simulated outcomes:
22

---

Published as a conference paper at ICLR 2026
G USE OF LANGUAGE MODELS
We used large language models strictly as writing assistants for language refinement: grammar correction, style tightening, phrasing alternatives, and minor reorganization for clarity and brevity. No prompts involved technical ideation, modeling, implementation, data analysis, or result selection. All suggested edits were reviewed by the authors, and the technical content, experiments, results, and conclusions are author-generaie and author-validated LLM assistance did not affect the substance of the work.
23


Reviews:
[{"review_id": "ZBJiOHbuCy", "reviewer": "Reviewer_76LM", "summary": "The paper introduces WoW!, a new open platform to address the flaws of existing world model (WM) benchmarks, arguing that \"closed-loop\" embodied task success, not \"open-loop\" visual quality, is the proper metric to evaluate the WMs. The platform provides a unified planning strategy and action API to evaluate heterogeneous WMs on four categories of embodied tasks. Using this benchmark, the paper finds that visual quality does not correlate with task success, but controllability does, and that performance scales with both in-domain post-training data and inference-time compute.", "questions": "1. See 2 and 3 in the Weakness.\n2. Regarding the limited gains in manipulation tasks, where the WM provides only a marginal boost over the VLM-only baseline as is pointed out, could you provide a deeper analysis? Is it purely a failure to model contact-rich physics, and what are potential solutions? Given that manipulation is a key focus of embodied AI, what other roles might WMs play in these complex tasks where their current planning utility appears minimal?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper's main contribution is shifting the evaluation of world models from open-loop visual fidelity to closed-loop embodied task success. This is a significant and necessary service for the field.\n2. The benchmark, through its Unified Action API, allows for direct, fair comparison of heterogeneous SOTA video generators across comprehensive embodied tasks.\n3. The paper's \"three surprises\" are all well-supported by evidence and can provide insights for the community.", "weaknesses": "1. The \"three surprises\" are to some extent overstated, especially the first and the second ones, as these findings have been pointed \nout by many previous papers, e.g. [1,2,3].\n2. The paper does not directly specify the reward function used to score the world model's imagined trajectories. \n3. The method on how to refine the policy during planning is not that clear (beyond directly using the plan with the highest score).\n4. There already exist some benchmarks that also evaluate the pretrained video generation models' capability on control tasks, e.g. VP2 [3]. Some of the insights in WoW! are also pointed out in VP2, like the first \"surprise\". Although WoW! is indeed a more comprehensive benchmark for evaluating video generation models as WMs on embodied tasks, the paper must reference these prior works and include a discussion or comparison to properly situate WoW!'s contributions within the existing literature.\n\n[1] Gao S, Zhou S, Du Y, et al. Adaworld: Learning adaptable world models with latent actions[J]. arXiv preprint arXiv:2503.18938, 2025. \\\n[2] He H, Zhang Y, Lin L, et al. Pre-trained video generative models as world simulators[J]. arXiv preprint arXiv:2502.07825, 2025. \\\n[3] Tian S, Finn C, Wu J. A control-centric benchmark for video prediction[J]. arXiv preprint arXiv:2304.13723, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HPzzJBvg5n", "reviewer": "Reviewer_8Zz4", "summary": "This article presents the Wow! World Models in a Closed-Loop World, which is the first Visual World Model (WM) evaluation platform for Closed-Loop interactive scenarios. Unlike previous open-loop evaluations that focused solely on the quality of the generated video —such as video fidelity —Wow, taking task success rate as the core index, the practical utility of the world model for decision-making is systematically evaluated across four embodied intelligent tasks (active recognition, image target navigation, active question answering, and manipulator operation). The authors introduce a unified online planning strategy and a standardized action API to enable heterogeneous world models to participate in closed-loop interactions under the same protocol, three key findings were revealed: (1) visual quality ≠ task success, and controllability is more important; (2) fine-tuning with a small amount of action-observation data on the target domain is more effective than upgrading the pre-trained video generator; (3) increasing the number of simulations can significantly improve the performance.", "questions": "- In table 1-3, the baseline strategy (e.g., VLM) already has strong capabilities (e.g., AR task 50% + accuracy). Can the world model still deliver significant improvements with weaker strategies (such as random strategies or simple heuristics)? This helps clarify the marginal contribution of WM.\n- The paper highlights generalization as a key challenge (Section 4), but all experiments were evaluated within the training domain. Have you tried to train the model after testing in an unfamiliar scenario (e.g., new building layout, new object category) or across simulators (e.g., Habitat → Igibson)? Preliminary results, even if negative, are very useful.\n- Figure 7 shows that increasing the number of inferences improves performance, but computing resources are limited in real deployments. What is the “cost performance”(performance/amount of computation) of the different world models on a fixed computing budget (e.g. 5 rollouts per step)? Are there design implications for a Lightweight World Model?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The author has pointed out the “Open-loop bias” in the current world model evaluation,i.e., overemphasis on generation quality and neglect of its practical value in closed-loop decision-making. Wow! For the first time, the focus of assessment has shifted from “Looking like” to “Working with,” a paradigm shift that plays an important guiding role for embodied AI and the world model community.\n- This paper constructs four tasks covering perception, navigation, and operation, and evaluates more than ten mainstream world models (including SVD, Wan, LTX, Cosmos, etc.). Through the standardized action API and online planning framework, heterogeneous models can be fairly compared.\n- The conclusion that “High visual quality ≠ high task success” challenges the current community's prevailing assumptions about generative model capabilities (such as whether Sora-like models are naturally suitable for embodied tasks). The authors further provide empirical support through the controllability quantification (LPIPS alignment), the data size law, and the inference extension, and the results show that the proposed model is more robust than previous models. It points to the optimization direction for follow-up research (e.g., emphasizing alignment of motion conditions rather than simply improving resolution).", "weaknesses": "- In the current framework, both the proposal and revision policies use powerful VLM (such as QWEN2.5-VL-72B). This makes the extent to which task success is attributable to the world model vs. The strategy itself is unclear. Complementary ablation experiments are suggested: for example, fixed-strategy capabilities (such as using weak or regular strategies), observing differences in performance gains across different world models, to more purely assess the decision-aid value of WM.\n- Habitat-sim data comes only from indoor scenes in HM3D and Matterport3D, and RLBench is also limited to desktop operations. This limits generalization to more complex environments, such as outdoor, dynamic obstacles, and multi-agents. The authors mention generalizability as a challenge in the discussion section but do not validate cross-domain transfer in experiments (e.g., in Habitat training and the Gibson test), weakening the persuasiveness of the universal world model claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YRnsn5bq28", "reviewer": "Reviewer_KANk", "summary": "This work addresses a significant gap in evaluating world models for their utility in the development of embodied agents tasks. World models are largely evaluated for their visual quality, motion coherence or controllability, but there is no available benchmark to assess how well a world model can improve embodied reasoning and task success. This work introduces a new general evaluation benchmark for world models, specific for assessing their performance on embodied interactions. The authors provide 1) a unified closed-loop planning strategy 2) a unified action API to allow various world models to be compared and 3) a study on finetuning pretrained video generators for a specific downstream task and the effect of data scaling.", "questions": "1) In Table 2, what does A14B stand for in the Wan2.2 #param column?\n2) nitpick: For clarity, it might be good to mark in the main body text on line 303 that post-trained variants have a special symbol (described in the description of Table 1), and first used on line 313 with Wan2.1.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1) Addresses an existing gap in the research community on assessing world models for their ability to be integrated in decision making embodied agents, presenting a well-rounded general framework for closed-loop evaluation. This allows the research community to make informed decisions when pursuing research goals around improving the utility of world models for embodied scenarios.\n2) Presents a comprehensive set of tasks for embodied agents - 4 tasks (active recognition, image-goal navigation, active embodied question answering, robotic manipulation) - I particularly appreciate the inclusion of robotic manipulation, as it provides an important additional dimension and usability for the proposed evaluation framework.\n3) Conducts an extensive and detailed analysis on 9 world model benchmarks, both using image and video based approaches, highlighting the key results of their evaluation: the use of world models enhance provides performance gains over base policies across tasks, post training aids the utility of world models & the current limitations in manipulation tasks where world models struggle with simulating precise motion and dynamics.\n4) Presents valuable insights on data-size scaling for post-training, highlighting that even a small dataset size (4K) from the task domain can yield significant gains with post training. This is useful for future work in this space.", "weaknesses": "Overall, the work provides a great set of references and supporting experiments. The paper is easy to follow, with detailed result figures and tables.\n\nA couple of minor suggestions:\n- I might have missed this, but I couldn't see the results of the A-EQA task evaluation from Table 2 discussed in more detail in Section 3. There is a stronger focus on the other 3 tasks, specifically AR, ImageNav and robotic manipulation.\n- In Fig 5(b), Wan2.2 5B (post-trained) seems to have high controllability, but performs low in terms of task success rate, which goes against the positive correlation claimed in the paper. Same for Runway Gen4 that has lower LIPIS, but very high SR.\n- At the beginning of section 3.2, you discuss findings presented in Figure 5 (a) - it would be good to reference it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wLQGz91W5x", "reviewer": "Reviewer_Bvsj", "summary": "This paper introduces WoW!, the first open benchmark for evaluating generative world models in a closed-loop, control-focused setting across four domains: Active Recognition, Image-Goal Navigation, Active Embodied Question Answering, and Robotic Manipulation. The authors propose a unified planning strategy and action API to integrate diverse world models into a consistent decision-making pipeline. Key findings include: (1) controllability is more crucial than visual quality for task success; (2) post-training with limited data yields significant gains; and (3) increasing inference-time compute improves performance. This paper shifts the evaluation focus from visual fidelity to embodied decision-making, setting a new standard for world model assessment in interactive environments.", "questions": "- How much impact do the revision policies have on task success?\n- How sensitive is closed-loop performance to action misalignment caused by the unified action API? Could the different action API lead to different experimental conclusions?\n- Compared to the base policy, how much does the inference time increase when introducing world models? How can the computational burden of model-based planning be mitigated?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper is well-written and easy to follow.\n- It addresses a crucial gap in the evaluation of video generation models from a control-centric perspective, an area previously overlooked by benchmarks, which is a valuable step forward in world model development.\n- The benchmark’s task design and baseline selection are thorough and rigorous, incorporating both high-level and low-level decision-making tasks, as well as the latest video generation models.\n- The experimental insights are impactful, especially the finding that controllability is more important than visual quality for decision-making tasks, and the observation that model-based planning is scalable during test-time.", "weaknesses": "- The analysis of proposal and revision policies is limited. Specifically, the revision policy, which selects among simulated rollouts, uses the same VLM as the proposal policy. The paper does not explore how sensitive the results are to the revision policy design, such as whether a simpler reward model would yield different outcomes for world model utility.\n- The unified action API assumes world models can conform to a shared action vocabulary. However, real-world domains often use incompatible action semantics, and mapping between them may introduce misalignment or information loss. The paper does not analyze how such action misalignments affect final task performance.\n- Another concern is the computational cost and inference-time overhead of the proposed framework. Planning-based methods are known for being computationally intensive, and integrating large video generation models into decision-making could significantly reduce the frequency of decision-making due to high computational demands.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)