GenEx: Generating an Explorable World
Authors: TaiMing Lu
Tianmin Shu
Alan Yuille
Daniel Khashabi
Jieneng Chen
Venue: ICLR 2025 Poster (2025)
Keywords: Generative Models
Video Generation
Embodied AI
Primary area: generative models
TL;DR: We propose Genex to allow the agent for imaginatively exploration in a physical world, and acquire imagined observations to update its belief.

Abstract: Understanding, navigating, and exploring the 3D physical real world has long been a central challenge in the development of artificial intelligence. In this work, we take a step toward this goal by introducing *GenEx*, a system capable of planning complex embodied world exploration, guided by its generative imagination that forms expectations about the surrounding environments. *GenEx* generates high-quality, continuous 360-degree virtual environments, achieving robust loop consistency and active 3D mapping over extended trajectories. Leveraging generative imagination, GPT-assisted agents can undertake complex embodied tasks, including goal-agnostic exploration and goal-driven navigation. Agents utilize imagined observations to update their beliefs, simulate potential outcomes, and enhance their decision-making. Training on the synthetic urban dataset *GenEx-DB* and evaluation on *GenEx-EQA* demonstrate that our approach significantly improves agents' planning capabilities, providing a transformative platform toward intelligent, imaginative embodied exploration.

Published as a conference paper at ICLR 2025
GENEX: GENERATING AN EXPLORABLE WORLD
Taiming Lu, Tianmin Shu, Alan Yuille, Daniel Khashabi, Jieneng Chen Johns Hopkins University
ABSTRACT
Understanding, navigating, and exploring the 3D physical real world has long been central challenge in the development of artificial intelligence. In this work, we take step toward this goal by introducing GenEx, system capable of plan- ning complex embodied world exploration, guided by its generative imagination that forms expectations about the surrounding environments. GenEx generates high-quality, continuous 360-degree virtual environments, achieving high loop consistency and active 3D mapping over extended trajectories. Leveraging gen- erative imagination, GPT-assisted agents undertake complex embodied tasks, including goal- -agnstic exploration and goal-driven navigation. Agents utilize imagined observations to update their beliefs, simulate potential outcomes, and enhance their decision-making. Training on the synthetic urban dataset GenEx- DB and evaluation GenEx-E:QA demonstrate that approach significantly improves agents' planning capabilities, providing a transformative platform to- ward intelligent, imaginative embodied exploration.
Website Code ArXiv
https::/www.co. GenEx world/ https://githuu hatpo//..ooo/eecccccee///eeeee
0962
Generative World Explorer
Single Image Input
Interactive Exploration
Real-World Diverse Generation
Augmented Embodied Al
EXPLORATION
Ossenuttes
Clar
Figure 1: GenEx explores an imaginative world, created from a single RGB image and brought to life as
a generated video (top). With interactive and diverse generation, GenEx enables an agent to imaginatively explore large-scale 3D world and acquire imagined observation to augment embodied intelligence (bottom).

---

Published as a conference paper at ICLR 2025
INTRODUCTION
Humans navigate and interact with the three-dimensional world by perceiving their surroundings, taking actions, and engaging with others. Through these interactions, they form mental models to simulate the world (Johnson-Laird, 1983). These models allow for internal representations of reality, aiding reasoning, problem-solving, and prediction through language and imagery.
In parallel, this understanding of natural intelligence has inspired the development of artificial in- telligence systems that create computational analogs of mental models (Ha & Schmidhuber, 2018; LeCun, 2022; Diester et al., 2024). These world models (WMs) (Ha & Schmidhuber, 2018; 8: LeCun, 2022) aim to mimic human understanding and interaction by predicting future world states (e-g.- the existence, properties and location of the objects in scene) to help agents make informed decisions. Recently, generative vision models (Ho et al., 2020; OpenAI, 2024; Bai et al., 2024) have increased interest in developing world models for predictive simulation of the world (Du et al., 2024a; Yang et al., 2024b;c; Wang et al., 2024a). However, these works focus solely on state transition probabil- ities without explicitly modeling agents' observations and beliefs. Explicitly modeling observation and belief is crucial because we often deal with partially observable environments where the true world state is unknown. An embodied agent is inherently a POMDP agent (Kaelbling et al., 1998): instead of full observation, the agent has only partial observations of the environment. To make ra- tional decisions, the agent must form belief, an estimate of the environment is currently in. This belief may be incomplete or biased, but it can be revised through incoming observations obtained by physically exploring the environment.
Typically, in an unfamiliar environment, an embodied agent must acquire new observations through physical exploration to understand its surroundings, which is inevitably costly, unsafe, and time- consuming. However, if the agent can imagine hidden views by mentally simulating exploration, it can update its beliefs without physical effort. This enables the agent to take more informed actions and make more robust decisions. Consider the scenario in Fig. suppose you approaching an intersection. The light ahead is green, but you suddenly notice that the yellow taxi in front has come to abrupt, unexpected stop. A surge of confusion and anxiety hits you, leaving you uncertain about the reason behind its halt. Physically investigating the situation would be unsafe and even impossible that moment. However, by standing in the taxi's position in your own imagination and envisioning the surroundings from its perspective, you sense a possible motivation behind the taxi's puzzling behavior: perhaps an ambulance approaching. Consequently, you clear the path for the emergency vehicle, timely and decisive choice, thanks to your imagination.
To build agents capable of imaginative exploration in a physical world, we propose Generative World Explorer (GenEx), a video generative model that conditions on the agent's ' current egocentric (first-person) view, incorporates intended movement direction as an action input, and generates fu- ture egocentric observation. Although prior works (Tewari et al., 2023) can render novel views of a scene based on 3D models (Yu et al., 2021), the limited render distance and the limited field of view (FOV) constrain the range and coherence of the generated video. Fortunately, video genera- tion offers the potential to extend the exploration range. To address the FOV constraint, we utilize panoramic representations to train our video diffusion models with spherical-consiter learning. As a result, the proposed GenEx model achieves impressive generation quality while maintaining coherence and 3D consistency throughout long-distance exploration.
Furthermore, the proposed GenEx can be applied to the embodied decision making. With GenEx, the agent is able imagine hidden views via imaginative exploration, and revise its belief. The revised belief allows the agent to take more informed actions. Technically, we define the agent's s behavior as an extension of POMDP with imagination-driver belief revision. Notably, the proposed GenEx can naturally be extended to multi-agent scenarios, where one agent can mentally navigate to the positions of other agents and update its own beliefs based on imagined beliefs of the other agents.
In summary, our key contribution is three-fold:
We introduce GenEx, a novel framework that enables agents to imaginatively explore the world with high generation quality and exploration consistency.
We present one of the first approaches to integrate generative video into the partially observable decision process by introducing the iaagination-driven belief revision.
We highlight the compelling applications of GenEx, including multi-agent decision-making.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORKS
Generative video modeling. Diffusion models (DMs) (Sohl-Dickstein et al., 2015; Ho et al,, 2020) have proven effective in image generation. To render high-resolution images, the latent diffu- sion models (LDMs) (Rombach et al., 2022) are proposed to denoise in the latent space. Similarly, video diffusion models (Blattmann et al., 2023b; Wang et al., 2023a; Blattmann et al., 2023a; Song et al. 2025) use VAE models to encode video frames and denoise in the latent space. For controllable synthesis, the conditional demoising autoencoder are implemented with text (Rombach et al., 2022; OpenAI, 2024) and various conditioning controls (Zhang et al., 2023; Sudhakar et al., 2024). We focus on video generation conditioned on the egocentric panoramic view of agent, as panorama (Li & Bansal, 2023; 2024) ensures coherence in the generated world, and the use of egocentric vision is de facto choice in many embodied tasks (Das et al., 2018; Sermanet al., 2024; Song et al., 2025).
Generative vision for embodied decision making, Decision-making in the physical world (Das al., 2018; Sermanet et al., 2024) is fundamental Al challenge. L.MSs provide linguistic reason- ing that aids decision-making (Hao et al., 2023; Min ct al., 2024) and vision-language planning (Cen al., 2024). World models offer predictive representations of future states to inform decisions, though early attempts (Ha & Schmidhuber, 2018; LeCun, 2022) focus on simple game agents and often lack commonsense reasoning about the physical world. Generative vision (OpenAI, 2024; Kondratyuk et al., 2024) and in-context learning (Bai et al., 2024; Zhang et al., 2024) offer new av- enues for using video generation to guide real-world decision-making (Yang et al., 2024c). Several works focus on specific application domains such as autonomous driving (Hu et al., 2023; Wang et al., 2023b; 2024c; Gao et al., 2024a;b) which limit their generality. Others like video in-context learning (Zhang et al., 2024) requires a known demonstration video, which is inefficient for decision- making. Aetion-conditioned video generation models (Du et al., 2024a; Yang et al., 2024b;c; Wang et al., 2024a; Bu al., 2024; Souček al., 2024; Du et al., 2024b) can directly synthesize visual plans for decision-making. These models, however, focus on state transition probabilities without explicitly modeling agent beliefs, which are crucial for reasoning about other objects/agents in par- tially observable environments.
3 GENERATIVE WORLD EXPLORATION
machine explorer, such as a home robot, designed to navigate within its environment and seek out previously unvisited locations. Integrating generative models, present the concept of a generative world explorer (GenEx), enabling spatial exploration within an imaginative realm, akin to human mental exploration. We introduce the macro-design of GenEx in $ 3.1, followed by the micro-design including input representation, diffuser backbone, and loss objective in 3.2.
(a) Maacro-framewer Genex
Genex
RGB
Exploration Configs
orforation
Smun Video of ingiitate Eeplenztion
Goal-agnostic Imaginative Exploration:
(c) Goal-driven Imaginative Exploration: Move to the blue car's position and orientation.
Explore freely to underseand your surounndnggs
rompek
fermenerr oytoeehaust
2ack
Figure 2: GenEx is able to explore an imaginative world by generating video sequence, given RGB observa- tions, exploration direction, and distance (a). GenEx, grounded in physical environment, can perform GPT- assisted goal-agnostic imaginative exploration of the world (b) and goal-driven imaginative exploration (c).
3

---

Published as a conference paper at ICLR 2025
3.1 MACRO-DESIGE OF GenEx
(a) Overview. As shown in Fig. 2, the GenEx framework enables agents to explore within an imaginative world by streaming video generation, based current RGB observations and given exploration configurations. The RGB observation is represented as a panorama image sampled from any location in the world. large multimodal model (LMM) serves the pilot, or the decision maker, to set up exploration configurations. including any 360 navigation direction and distance. GenEx processes the input in two steps. First, it takes the exploration orientation to update the panorama forward view. Secondly, its built-in diffuser generates the forward navigation video. Both view update and diffuser are detailed in 3.2.
GenEx, grounded in physical environment, can perform GPT-assisted goal-agnostic imaginative ex- ploration and goal-driven imaginative exploration.
(b) Goal-agnostic Imaginative Exploration. GenEx can explore freely with an unlimited number of orientations, helping the agent to understand its surrounding environment, as shown in Fig. 2 (b).
(c) Goal-driven Imaginative Exploration. The agent receives target instruction, such as, "Move to the blue car's position and orientation." GPT performs high-level planning based on the instruction and initial image, generating lowelevel exploration configurations in an iterative manner. GenEx then processes these configurations steepby-step, updating images progressively throughout the imagina- tive exploration in Fig. 2 (c). This allows for greater control and targeted exploration.
3.2 MICRO-DESICE OF GenEx
We detail our micro-design as follows. Our diffuser builds upon the standard stable video diffusion (SVD) (Blattmann et al., 2023a) backbone (a), but adapts the input from conventional images to panoramas (b). Furthermore, we propose spherical-consistent learning (c) to ensure coherence in imaginative exploration.
(a) Diffuser backbone. To support exploration illustrated in Fig. 2, we propose a video diffuser that can be seamlessly adapted be world explorer. Given an initial panorama image x" with camera position y", our objective is to generate a sequence of images {x ..... x²) corresponding to a sequence of camera positions {p' "p'......7 ..... The camera positions progress steadily forward, representing navigation in the world. Since the panorama image represents a 360-degree view, the generation should persist the information stored in previous frames maintain world consistency throughout the sequence. Our model uses the pretrained SVD (Blattmann et al., 2023a). The Trans- former UNet (Ronneherger al., 2015; Chen et al., 2021) architecture is as described in Blattmann et al. (2023b), where temporal convolution and attention layers are inserted after every spatial con- volution and attention layer. The pipeline of our model is shown Fig. 3 (a). Given an image condition (encoded from image x² using CLIP image Transformer (Radford al., 2021)), the video diffusion algorithms learn a network 6o to predict the noise added to the noisy image latent 2 ||ea c) - 2
with L
(b) Input image representation. Panorama images are well-suited for generative exploration as they captures all perspectives from an egocentric viewpoint into a 2D image. Essentially, it rep- resents a spherical polar coordinate system S on a 2D grid in the Cartesian coordinate system P, as shown in Fig. 3 (b). Panorama effectively stores every perspective of the world from a single location which preserves the global context during spatial navigation. This allows us to maintain consistency world information from the conditional image, ensuring that the generated content aligns coherently with the surrounding environment. The panorama image also allows for rota- tional transformations, which facilitate world navigution by enabling us rotate the image to face a different angle while preserving its original information. The rotation can be performed using Eq. 1:
T(u, v, AΦ, 20) = fs-P (R (Jr-s(u, v), AQ, A0)),
(1)
where and are positions on the 2D image plane, and o and 0 represent longitude and latitude in polar coordinates. The rotation function R applies a rotation to the spherical representation in any direction, simulating turning around during navigation. Additionally, panorama image can be converted into a cubemap six separate regular images, each representing a face of a cube (front, back left, right, top, and bottom). This panorama-to-cun transformation enhances visual understanding by LMM agents. Full mathematical details of the equirectangular projection are in $ A.1.
4

---

Published as a conference paper at ICLR 2025
(2) Genex Diffuser Backbone
Input Image Representation
SI-VAE
Diffusion Model
Video Panorama
Orbw Dreesoon
Eaeadar
Reead Nonnng Preecn
D
3 Rol 4)
Ultt Ep
Denobing Procss
Diffuser training objective: SCL,
SCL
Image
Panorama
Cavdilia
Figure 3: (a) Diffuser in GenEx, spherical-consistent panoramic video generation model. During training, video is encoded into latent and noised 2.. A conditioned UNet eg predicts and removes noise, resulting zy which decoded z/. The loss Cael in (c) combined with the original noise prediction loss. During inference, random noise is iteratively denoised to generate video z from an image panorama condition. (b) Left: Conversion between Polar and Cartesian coordinates. Right: Rotated spherical panorama can be converted to either 2D panorama six-view images. (c) Spherical learning: randomly sample camera orientation for edge consisteney.
(c) Diffuser training objectíve: spherical-consiseen learning (SCL). We aim to generate images where pixels are continuous spherical space. However, direct training with results in severe edge inconsistency, as generated pixels at the far left and far right of the equirectangular image are not constraint to continuous on the spherical space. To address this, we introduce spherical-consiee consiieent learning an explicit regularization. After generating panoramic video, we apply the spherical rotational transformation function, as shown Eq. 1, to randomly rotates camera to different positions both the generated video and the ground truth. illustrated in Fig. (a). The de- noised diffused video 2; Cø( e) and the ground-truth video 2g are transformed and then passed into pre-trained temporal VAE encoder E, resulting in the latent transformed diffused video (T( c (1t c))) and the latent over transformed ground-truth video (T(())) Each camera view weighted equally in this process ensure consistent representation across all perspectives. We train with the objective Lee to minimize the mean square error over the latent spaces for main- taining uniformity and coherence in the 360-degree output. During training, the overall training objective is to minimize the loss:
C = A|IE(T(D(> - co(z,, c))))
E(T E +11 A))leo(Z,..) eel1?
Lea
Lame
where DD is the temporal VAE (Kingma, 2013) decoder, À is a weighting constant, and T is the spherical rotation transformation shown in Eq. 1.
During inference, one can initialize Ztmo (O, I), iteratively sample 7,-1 po ( |
using the reparameterizatioo trick, producing latent 2/1 which is decoded to panoramic video z/p-
4 GenEX-BASED EMBODIED DECISION MAKING
4.1 IMAGINATION-DRIVER BELIEF REVISION
Embodied agents operate under a POMDP framework (Puterman, 1994; Kaelbling al., 1998). At each time step f, the agent's world state (which represents the complete environment at this specific moment),
E S, and action a' E A determine the next world state via the transition probability | a²). The agent's given goal g E G (e.g., crossing the street) influences the reward = -R(s'' a..'.. g). which drives the agent to achieve its objective. The agent receives an observation of E Ω based on the observation model O s') and maintains belief, represented by a distribution b(s), which the agent's internal estimate of the true state of the world. belief is updated with observations, following the POMDP framework in Eq. 3:
5

---

Published as a conference paper at ICLR 2025
b (s I1 .a)) I ,a) (s5)
Physical Exploraton
The decision a' made at any time becomes more informed as the agent gains clearer under- standing of its surroundings. By navigating through physical space, the agent gathers additional information about its environment (Fan et al., 2024), enabling more accurate assessments and better choices moving forward. However, physically traversing the space is inefficient, expensive, and even impossible in dangerous scenario. To streamline this process, for the agent to simulate outcomes without physically traversing. The key question becomes:
imagination medium
How agent revise its belief through imaginative exploration for more informed decisions?
Imagination-driven belief revision. We propose imagination-drivee belief revision that uses imag- inative exploration enhance POMDP agents with instant belief revision between time steps. In imagination, we freeze the time and create an imagined world, thus dropping the time variable t and defining an imagination space with hat" on the variables. Here the agent can make a sequence of imaginative actions = {å; E A] over imagination time step can make sequential speculation on the unobserved world based on its initial belief and toward ultimate goal, imagine novel observations in a previous unobserved world with pe ( l a'), where 8 d initialization and 0 is parameterized video diffuser generating the imaginative observations. As result, can update its belief with Eq. 4:
{1,..., ..., n}. The agents
= II po Po "0aaaa))a))) aa) (s())
(s')))
(4)
Imaginative Exploration
Different from Eq. 3, we re- place physical with imaginative exploration (Fig. 4). For an proper imagination, we should ex- pect (++((s++)) = (s'), where the imaginative belief approximates the physical belief. As the se- quence of imagination I expands, more observations o is produced, the agent's 's belief will be approach- ing, 6", which is the belief the agent could obtain under a full observation.
Physical
Bodel
Besttion
Imaginative
Figure 4: Imaginative exploration can achieve the same helief up- date as physical exploration.
The agent make actions based on its belief and goal, with policy model π m(a"lb'ss") (s"),g). Through the revised belief, the agent capable making more informed decision toward a" with more refined belief toward b", with more information on the true state of its surrounding environment.
In our work, we apply GenEx for imaginative exploration and LMM as the policy model æ and belief updater b(s), mapping observation to belief, with examples in Fig. 5 and system pipeline in $ A.5.3.
4.2 GENERALIZED TO MULTI-AGENT
Imagination-base POMDP can be generalized the multi-agent scenario. The 1-st agent can imaginatively explore to the location of the k-th agent to predict the agent-k's 's observation òş and infer agent-k's belief bx. following Eq. 4.
Thus, can adjust agent-1's beliefs by aggregating the imagined belief counterpart for other K agents.
= mb bb.,...bk }.
(5)
When exploring another agent's 's thoughts, we can predict what that agent sees, understands, and might do next, which in turn helps us adjust actions with more complete information.

---

Published as a conference paper at ICLR 2025
(a) Single-Agent
Observation
Generative Imagleed Ooservtion
tunning morsection
rotto lights cur skosty moving ahoud. and r'n ursurs Itll Shouls
@
sioeatt stop aood potentsll
The sees a stop sign and will stop. atooud mone to auded hlockenge
collisron car right stmp.
Egocentic View chese
Decision Imagination: Dastmmatiea
Observation
wattin ar
(b) Multi-Agent
wheee kgtt move
alowed The front puth diarr cur is drving fast and tur
LLM Agent
iih1 podessrian crooing What
Generative World
Exploration ine Cinervation
Genex
stanldi car?
Aguct
light
Agas
red, betooddd placa
pedestion, aoliite.
Egocentric Single VeegeeVeewrboee Deciston: Sto0
X
Deciston with Imagination: both partos
Single agent reasoning with imagination and multi-agent reasoning and planning with imagination. (a) The single agent can imagine previously unobserved views better understand the environment. (b) In the multi-agent scenario, the agent infers the perspective of others make decisions based on more complete understanding of the situation. Input and generated images are panoramic; cubes are extracted for visualization.
Figure 5:
We define embodied agents and introduce imagination-drivs belief revision in $ 4.1, followed by multi-agent decision making in $ 4.2, and instantiation of embodied QA in $ 4.3.
4.3 INSTANTIATION IN EMBODIED QA.
While the traditional EmbodiedQA benchmark (Das al., 2018) features well-defined tasks such as navigation, they are not focus on how mental imagination help planning and the lack of multi-agent scenario limits further advancements (it doesn't 't satisfy condition (3)&(4) follows). To the best of our knowledge, no existing benchmark can be used to evaluate our proposed solutions.
To bridge this gap, we aim collect new embodied QA benchmark satisfying four conditions: (1) The agent is planning with partial observation. (2) Questions can't 't be solved by linguistic common- sense alone; agents must physically navigate or mentally explore the environment to answer. (3) Humans can mentally simulate environments to comprehend and answer questions, but it's unclear machines can do the same. (4) The benchmark can be extended to scenarios involving multi-agent decision decision-making making. Accordingly, propose new dataset called GenEx- in $ 5.1.
5 EXPERIMENTS
View
5.1 DATASET CONSTRUCTION
Realistic
Anime
GenEx-DB. We synthesize large-scale dataset generated using Unity, Blender, and Unreal Engine. The full details are in $ A.3. We create four distinct scenes, each representing different visual style (Realistic, Animated, Low-Texture, and Geometry), shown in Fig. 6: We train a model with each dataset, and for the four re- sulting navigational video diffusers, conduct cross- cross- validaaii across all scenes to evaluate their generalization capabilities (de- tailed in A.7).
Low- Texture
Geometry
Figure 6: Examples for 6 different real and virtual scenes.

---

Published as a conference paper at ICLR 2025
We collect an additional test set of panoramic images from Google Maps Street View (header "Street" in Table 2) and Behavior Vision Suite (Ge et al., 2024) (header "Indoor" in Table 2), which serves as benchmark for real-world street and synthetic indoor exploration
Through the proposed GenEx model, the agents perform exploration autonomously, capable of tackling embodied tasks such single agent scene understanding and multi-agent inter- sectional reasoning. Following the four conditions in $ 4.3, we design over 200 scenarios in virtual physical engine test various LMM agents embodied decision-making. We provide comprehensive details in & A.4.1. The dataset generally represent two scenarios:
GenEx-EQA.
Single agent: the agent could infer the egocentric view from any location in its view. The agent could use GenEx to imagine the missing view (e.g. an ambulance blocked by trees or from the back of the stop sign). This extra information enables the agent to make more informed decisions.
Multi-agent: the first agent can imaginatively explore the locations of other agents and use these imagined observations to update its beliefs.
5.2 EVALUATION ON GENERATION QUALITY
We adopt FVD (Unterthiner et al., 2019), SSIM (Wang et al., 2004), LPIPS (Zhang et al., 2018) 8), and PSNR (Horé& & Ziou, 2010) to evaluate video generation quality, with details in $ A.5.
As a strong baseline, we develop a six-view navigator by training six separate diffusion model for each face of the cube, representing still a 360 view, independently (See Fig. 3 (b) sixwv.ew). The implementation detail is shown in & A.6. This baseline may align well with 2D diffusion models but stands in contrast to the panoramic approach, which is particularly ef- fective maintaining consistent environmen- tal context. To enable a fair comparison with GenEx in video quality evaluation, the six-view baseline predictions are reprojected into panoramas. As result, Table shows that our method achieves high generation quality and surpass six-view baseline in all metrics.
Model
Input
FVD 1
MSE
LPIPS
PSNR
SSIM
direct test
CogVideoX VidooX
six-view
4451
0.30
0.94
8,89
0.07
panorama
4307
0.32
0.94
8.69
0.07
SVD
six-view
5453
0.31
7,86
0.14
SVD
759,9
0.15
0.32
17.6
0.68
>auned on GenEx-DB
Baseline
six-view
196.7
0.30
0.09
26.1
GenEx wlo SCL
$1.9
0.05
0.05
29.4
0.9
GenEs
panorama
69.5
0.04
0.03
30.2
0.94
Table I: Video generation quality of different diffusers.
5.3 EVALUATION ON IMAGINATIVE EXPLORATION QUALITY
Inspired by loop closure (Newman & Ho, 2005), we pro- pose a new metric, Imaginative Exploration Loop Con- sistency (IELC), to assess the coherence and fidelity of long horizontal imaginative exploration. Definition: For any randomly sampled path forming a closed loop within the scene, calculate the latent MSE between the initial real image and the final generated image, both encoded by Inception-vé v4 (Szegedy et al., 2017). The final latent MSE is averaged over 1000 randomly sampled closed paths, with each loop differing in number of rotations and lotal distance traveled (refer to Fig. 7). We filter out paths blocked by ob- stacles.
Sample Cycle Navigation
Origin
Position
In our results, we observe strong loop consistency across all exploration paths, shown in Fig. 8. Even in cases of long-range imaginative exploration (distance = 20m) and multiple consecutive videos, the latent MSE remained be- low 0.1, indicating minimal drift from the original frames. We attribute our method's 's strong performance to its preser- vation of spherical consistency in panoramas, ensuring that rotation does not degrade performance.
X Position
Figure 7: Example randomly sampled tra- jectory for loop consistency. It forms a closed loop within the scene, with 9 ro- tations and in 15 meters distance.
Forr training, exclude Google Maps Street View, for to its inconsistent image quality and unpredictable camera movement, and Behavior Vision Suite, for restricted indoor navigation range.
8

---

Published as a conference paper at ICLR 2025
We further conduct more analysis with three findings regarding the zero-shot generalizabil- ity to real-world, the correlation between gen- eration and imaginative exploration, and the emerging 3D consistency below.
COOB
ZPN
6000
AM
Finding I. The better the generation quality, the more consistent the imaginative exploration will be. Fig. 9 shows a strong correlation be- tween imaginative exploration loop consistency and generation FVD, validating our efforts to enhance the diffuser.
c001
Geeerition quwltyyylyyyyy (1S))
Figure 8: Imaginative Exploration Loop Consis- tency (IELC) varying dis- tance and rotations.
Figure Correlation be- tween exploration qual- ity (IELC) and generation quality (FVD).
Finding 2. GenEx, trained on synthetic data, demonstrates robust zero-shot generalizability to real-world scenarios. Impressively, the model trained on UE5 and other synthetic data (Table 2), is generalized well (IELC < 0.1) to indoor behavior vision suite and outdoor google map street view in real world, without requiring additional fine-tuning (See $ A.7 for examples).
GenEx
GenEx wio SCL,
Sixview
IELA.
Realisti
Anime
Low- Texture
Geometry
Realistic
Realittc
0.105
0.131
0.122
0.147
0,131
0.299
Indoor
0.092
0.168
0,303
0.117
0.120
0233
Table Zero-shot generalizability real world. Rows are by zero-shot test scenes Columns are by models and training scenes.
Finding 3. Generative world exploration empowers strong 3D understanding. Our method enables the generation multi-view videos of an object through imaginative exploration with path cir- cling around it. Table 3 not only report the common object-level foreground metric (M Eotj.) but also highlight background evaluation (MS Our model demonstrates superior performance compared with the SoTA open-source models. Importantly, it maintains near-pert rear-perfect background consistency and effectively simulates scene lighting, object orientation, and 3D relationships. Inter- estingly, we demonstrate that our model can reconstruct 3D worlds using additional plug-and-play models (Depth Anything (Yang et al., 2024a) & DUSt3R (Wang et al., 2024b)), as detailed in $ A.8.
Model
LPESS PSNR SSEM MSG,,
(Tochilkin ot al., 2024)
Trposic
0.76
6.69
0.56
0.08
SV3D
(Voleti et al., 2024)
0.75
666
0.53
0.08
Suble Zero 123
(StabiliyAl, 2023)
0.50
14,12
0.7
0,07
0.06
Table 3: GenEx can synthesize novel views of distant objects (and background scene) with minimal difference from the ground truth, surpassing SoTA methods.
Figure 10: Comparison with state-of-the-art 3D reconstruc- tion models for novel view synthesis. Through exploration, our model achieves higher quality in novel view synthesis for objects and improved consistency in background synthesis.
In summary, the robust zero-shot generalizability to real-world, the high correlation between gen- eration and imaginative exploration, and the emerging 3D consistency pave the way for real-world embodied decision-making.
5.4 RESULTS ON EMBODIED QA
Evaluation of embodied QA. For embodied reasoning evaluation, we define three metrics:
Decision Accuracy: this metric evaluates whether agent's decision aligns with the optimal action a fully informed human would take. It measures the degree to which the chosen action successfully addresses the situation or problem.
Gold Action Confidence: this refers to the agent's strength of belief take the most appropriate action based on the available information and context. The confidence is calculated as the averaged normalized logit the agent outputting the correct choice.
Logic Accuracy: this metric tracks the correctness the logical reasoning process that leads to decision. We use I.LM-as-a-judge (GPT-4o) to evaluate the agent's thinking process, with the

---

Published as a conference paper at ICLR 2025
provided correct chain of thoughts. It highlights the sequence of steps, inferences, and reflections an agent makes while navigating toward final action.
In Table 4, we evaluate our single-agent ($ 4.1) and multi agget ($ 4.2) decision making algorithms. We use Unimodal refers to agents receiving only text context, while Multimodal reasoning demon- strate L.M decision when prompted along with a egocentric visual view. GenEx showcases the performance of models equipped as agents with cognitive world model.
Decision Accuracy (%)
Gold Action Confidence (%)
Logic Accuracy (%)
Method
Ainnle-Agent
Multi-Agent
Single-Agent
Multi-Agent
Single-Agent Multi-Agent
Random
25.00
25.00
25.00
25.00
Human Text-only
44.82
21.21
52,19
11,56
46.82
13,50
Human with Image
91.50
55.24
80.22
58.67
70.93
46.49
Human with GenEx
94.00
77.41
90.77
71.54
86.19
72.73
Unimodal Gemini-1.5 1.5
30.56
26.04
29.46
24.37
13.89
5.56
Unimodal GPT-4o
27.71
25.88
26.38
26.99
20.22
5.00
Multimodal Gemini-1.5
46.73
11.54
36.70
15.35
0.0
0.0
Multimodal GPT-4o
46.10
21.88
44.10
21.16
12.51
6.25
GenEx (GPT4-o)
85.22
94.87
77.68
69.21
83.88
72.11
Table 4:
Embodied QA evaluation across different scenarios. For unimodal input, agent is prompted with only text context, and for multimodal input, agent is given its egocentric image view. In all settings, we prompted the agent generate in Chain-of-Thoughts image other agent's belief.
Vision without imagination can be misleading for GPTs. In some cases, the unimodal's response (processing only the environment's text description) surpasses the multimodal counterparts (which includes both text and egocentric visual input). This suggests that vision without imagination can be misleading. When an LLM agent converts its view into a text description and relies solely on language-bass -aaee commonsense reasoning, it tends to make incorrect inferences due to the lack of spatial context. This highlights the importance integrating imagination with visual data to enhance the accuracy and reliability of the agent's 's decision-making processes.
GenEx has potential to enhance cognitive abilities for humans. Human performance results reveal several key insights. First, individuals using both visual and textual information achieve significantly higher decision accuracy compared to those relying solely on text. This indicates that multimodal inputs enhance reasoning. Secondly, when provided with imagined videos generated by GenEx, humans make even more accurate and informed decisions than in the conventional image-only set- ting, especially in multi-agent scenarios that require advanced spatial reasoning. These findings demonstrate GenEx's potential to enhance cognitive abilities for effective social collaboration and situational awareness.
CONCLUSION
We introduced the Generative World Explorer (GenEx), novel video generation model that en- ables embodied agents to imaginatively explore large-scale 3D environments and update their be- liefs without physical movement. By employing spherical-consisten lcarning, GenEx generates high-quality and coherent videos during extended exploration. Additionally, we present one of the first methods to integrate generative video into the partially observable prcision-making process through imagination-driver belief revision. Our experiments show that these imagined observations significantly enhance decision-man making. allowing agents to create more informed and effective plans. Furthermore, GenEx's framework supports multi-agent interactions, paving the way for more ad- vanced and cooperative AI systems. This work marks a significant advancement toward achieving human Juman-like intelligence in embodied AI.
10

---

Published as a conference paper at ICLR 2025
7 ACKNOWLEDGEMENT
This work partially supported by ONR with award N000142412696 to AY and a Siebel Scholar- ship to JC. We are grateful to the anonymous reviewers for their constructive feedback and to the many wonderful researchers at JHU who contributed after our ICLR submission.
REFERENCES
Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan L Yuille, Trevor Darrell, Jitendra Malik, and Alexei Efros. Sequential modeling enables scalable learning for large vision models. In CVPR, 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rom- bach. Stable video diffusion: Scaling latent video diffusion models to large datasets, 2023a. URL https: /ttp///aavvv///////////eee/aaaaa wwwiiii.r.rrreeeeeee wwg/oogggss.......o..o... alss 2211111111 15127.
Andreas Blattmann, Robin Rombach, Huan Ling. Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In CVPR, 2023b.
Qingwen Bu, Jia Zeng, Li Chen, Yanchao Yang. Guyue Zhou, Junchi Yan, Ping Luo, Heming Cui, Yi Ma, and Hongyang Li. Closed-loop visuomotor control with generative expectation for robotic manipulation. arXiv preprint arXiv:2409.09016 2024.
Jun Cen, Chenfei Wu, Xiao Liu, Shengming Yin, Yixuan Pei, Jinglong Yang. Qifeng Chen, Nan Duan, and Jianguo Zhang. Using left and right brains together: Towards vision and language planning. arXiv preprint DeXiv:2402.105330 2024.
Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04300 2021.
Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra. Em- bodied question answering. CVPR, 2018
Ilka Diester, Marlene Bartos, Joschka Bödecker, Adam Kortylewski, Christian Leibold, Johannes Letzkus, Mathew M Nour, Monika Schönauer, Andrew Straw, Abhinav Valada, et al. Internal world models in humans, animals, and ai. Neuron, 112(14):2265 2268, 2024.
Yilun Du, Mengjiao Yang. Pete Florence, Fei Xia, Ayzaan Wahid, Brian Ichter, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B Tenenbaum, et al. Video language planning. ICLR, 2024a.
Yilun Du, Sherry Yang. Bo Dai, Hanjun Dai, Ofrr Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. In NeurIPS, 2024b.
Lei Fan, Mingfu Liang, Yunxuan Li, Gang Hua, and Ying Wu. Evidential active recognition: ligent and prudent open-world embodied perception. In CVPR, 2024.
Intel-
Ruiyuan Gao, Kai Chen, Enze Xie, Lanqing Hong, Zhenguo Li, Dit-Yan Yeung, and Qiang Xu. Magicdrive: Street view generation with diverse 3d geometry control. In ICLR, 2024a.
Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: generalizable driving world model with high fidelity and versatile controllability. In NeurlPS, 2024b.
Yunhao Ge, Yihe Tang, Jiashu Xu, Cem Gokmen, Chengshu Li, Wensi Ai, Benjamin Jose Martinez, Arman Aydin, Mona Anvari, Ayush K Chakravarthy, Hong-Xing Yu, Josiah Wong, Sanjana Sri- vastava, Sharon Lee, Shengxin Zha, Laurent Itti, Yunzhu Li, Roberto Martin-Martin, Miao Liu, Pengchuan Zhang, Ruohan Zhang, Li Fei-Fei, and Jiajun Wu. Behavior vision suite: Customiz- able dataset generation via simulation. In CVPR, 2024.
David Ha and Jürgen Schmidhuber. World models. arXiv preprint 1803 2018.

---

Published as a conference paper at ICLR 2025
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. EMLNP, 2023.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurlPS, 2020.
Alain Horé and Djemel Ziou, Image quality metries: Psnr ssim. In ICPR, 2010.
Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shot- ton, and Gianluca Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309. 17080, 2023.
Philip Nicholas Johnson-Laird. Mental models: Towards a cognitive science of language, inference, and consciousness. Harvard University Press, USA, 1983.
Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, (01(112):9911)) 1998.
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Rachel Hornung, Hartwig Adam, Hassan Akbari, Yair Alon, Vighnesh Birodkar, et al. Videopoet: A large language model for zero-shot video generation. ICML, 2024.
Yann LeCun. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1):1-62, 2022.
Jialu Li and Mohit Bansal. Improving vision-and-language navigation by generating future-view image semantics. In CVPR, 2023.
Jialu Li and Mohit Bansal. Panogen:
Text-conditioned panoramic environment generation for vision-and-language navigation. In NeurIPS, 2024.
So Yeon Min, Xavi Puig, Devendra Singh Chaplot, Tsung-Yar Yen Yang. Akshara Rai, Priyam Parashar, Ruslan Salakhutdinov, Yonatan Bisk, and Roozbeh Mottaghi. Situated instruction following. arXiv preprint arXiv:2407. 12061, 2024.
Paul Newman and Kin Ho. Slam-loop closing with visually salient features. In ICRA, 2005.
OpenAI. Video generation models as world simulators, 2024.
Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming, John Wiley Sons, 1994.
Alec Radford, Jong Wook Kim. Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML., 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In MICCAI, 2015.
Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrish- nan, Christine Chan, Gabriel Dulac-Arnold Sharath Maddineni, Nikhil J Joshi, et al. Robovqa: Multimodal long-horizon reasoning for robotics. ICRA, 2024.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion, 2025. URL https: //ap:ii///iiiv.///a org/ abs 22022 64.
12

---

Published as a conference paper at ICLR 2025
Tomáš Souček, Dima Damen, Michael Wray, Ivan Laptev, and Josef Sivie. Genhowto: Learning to generate actions and state transformations from instructional videos. In CVPR, 2024.
StabilityAI. Stable zero123, 2023.
Sruthi Sudhakar, Ruoshi Liu, Basile Van Hoorick, Carl Vondrick, and Richard Zemel. Controlling the world by sleight of hand. In ECCV, 2024.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander Alemi. Inception-ve v4, inception- resnet and the impact of residual connections on learning. In AAAI, 2017.
Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Josh Tenenbaum, Frédo Du- rand, Bill Freeman, and Vincent Sitzmann. Diffusion with forward models: Solving stochastic inverse problems without direct supervision. In NeurIPS. 2023.
Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from single image. arXiv preprint urXiv:2403.021552 2024.
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric and challenges, 2019. URL https://////. arxiv. org/ abs 1812.111112 01717.
Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Chris- tian Laforte, Robin Rombach, and Varun Jampani. Sv3d: Novel multi-view synthesis and 3d generation from single image using latent video diffusion. arXiv preprint 1cXi:22403.12008 2024.
Boyang Wang, Nikhil Sridhar, Chao Feng, Mark Van der Merwe, Adam Fishman, Nima Fazeli, and Jeong Joon Park. This&that: Language-gesture controlled video generation for robot planning. arXiv preprint Siii:22407.055300 2024a.
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Mod- elscope rext-to-video technical report. arXiv preprint arXiv:2308.0657,, 2023a.
Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Ge- ometric 3d vision made easy. In CVPR, 2024b.
Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. Drive- dreamer: Towards wor-ddordd world models for autonomous driving. arXiv preprint arXiv:2309.09777, 2023b.
Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. In CVPR, 2024c.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP, 2004.
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. In NeurIPS, 2024a.
Mengjiao Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators. ICLR, 2024b.
Sherry Yang, Jacob Walker, Jack Parker-Holder, Yilun Du, Jake Bruce, Andre Barreto, Pieter Abbeel, and Dale Schuurmans. Video the new language for real-world decision making, arXiv preprint arXiv:2402.17133, 2024c.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelnerf: Neural radiance fields from one or few images. CVPR, 2021.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023.
13

---

Published as a conference paper at ICLR 2025
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric, 2018.
Wentao Zhang, Junliang Guo, Tianyu He, Li Zhao, Linli Xu, and Jiang Bian. Video in-context learning. arXiv preprint arX1v:240.00735, 2024.
14

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 PRELIMINARY: EQUIRECTANGULAR PANORAMA IMAGES
W 28
Caberosp
Sphericl keasion
Rotated
Figure 11: Left: Pixel Grid coordinate and Spherical Polar coordinate systems: Middle: rotation in Spherical coordinates corresponds to rotation in 2D image; Right: expansion from panorama to cubemap or composition reverse.
A.1.1 COORDINATE SYSTEMS
Equirectangular Panorama Image captures all perspectives from an egocentric viewpoint into a 2D image. Essentially, it represents spherical coordinate system on 2D grid.
Definition D.1 (Spherical polar coordinate system). S: Taking the origin the central point, point in this system is represented by coordinates (ø, 0,r) E S, where & denotes the longitude, 0 the latitude, and the radial distance from the origin. The ranges for these coordinates are I- r), 0 E [- 2,, म /2, and r >> O.
Definition D.2 (Cartesian coordinate system for panoramic image). P: In this system, a pixel is identified by the coordinates (u, v) E P, where and correspond the column and positions on the 2D panoramic image plane, respectively. Here, ranges from 0 to W and v ranges from to H L.
Definition D.3 Aopheretoo-Cattes Coordinate Transformation). The transformation between the spherical polar coordinates and the panoramic pixel grid coordinates can be defined by the following functions:
W
(6 H
8))
fs-p(e,0)-
+T),
(6)
2T
π
fF-s(u,))
2Tu
π 2
TU H
W
Here, the function fs-P maps the spherical coordinates (c. 0) to the pixel coordinates vu, and the inverse function fr-s maps the pixel coordinates (u, back to the spherical coordinates (Ø) 0). This transformation ensures that the entire spherical surface is represented on the 2D panoramic image.
Panorama effectively stores every perspective of the world from single location. In our work, due to the nature of panoramic images, we are able to preserve the global context during spatial navigation. This allows us maintain consistency in world information from the conditional image, ensuring that the generated content aligns coherently with the surrounding environment.
1.1.2 PANORAMA IMAGE TRANSFORMATIONS
The spherical format allows various image processing tasks. For example, the image be rotated by an arbitrary angle without any loss information due to the spherical representation. Addition- ally, it can be broken down into eubemaps for 2D visualization, shown in Fig. 11.
15

---

Published as a conference paper at ICLR 2025
Definition D.4 (Rotation Transformation in Spherical Polar Coordinate System). Since panorama image is in a spherical format, we can rotate the image to face a different angle while preserving the original image quality. The rotation can be performed using the following formula:
T(u, v, AΦ, A0) fS-P (R (fr=s(( v), Ad, A0))
(8)
Where the rotation function R. is defined as:
R(o, 0, AΦ, A0) (Ø + A (mod 2m), 0 A0 (mod m))
(9)
there is no explicit input, both A and AO can be set to O.
Panorama to cubes A panorama image can be broken down into six separate images, each corre- sponding to face of cube: front, back, left, right, top, and bottom, as shown in Fig. 11. This conversion allows the panorama be viewed as six conventional 2D images.
A.2 HYPERPARAMETERS AND EFFICIENCY OF
We provide the training hyperparameters for GenEx diffuser in Table 5 and computation resource used for training in Table 6.
peperaaraeeerrs
Value
Setting Total GPU Usage GPU Conliguration
Value
learning
Io-3
384 A 1.00 hours
Ir scheduler
Cosine
2. A100 per batch,
output height
576
Model Parallelism
output width
1024
Training Time
0. 12 minutes per step
mixed precision
[p16
Infereace
0.031 per
training frame
25
Ir warmup steps
Table 6: GenEx-Difftser Training and Inference Time.
Table GenEx-Diffuserr Training configuration.
A.3 GENEX-DB
For dataset creation, scenes in four different styles examine how different visual represen- tation affect final model performance.
Realistic:
Using the Sample City from Unreal Engine 5, designed to evaluate the model's ability to handle photorealistic environments.
Animated: Created test the model's performance in stylized, animated settings.
Low-Texture: Used to assess how well the model adapts to environments with minimal texture details, focusing on whether the model can learn relying only on architectures.
Geometry: Composed solely of simple geometric shapes (cubes and cylinders), designed to determine if the model can learn panoramic movement from basic forms.
In chosen 3D environment, sample random position and random rotation. We sample path moving straight forward for 20 meters where there is no collision to any objects and render a video moving in this path with constant velocity for 50 frames. During training, randomly sample from frame, to framez5 as the conditional image with ground truth the navigation in the next 25 frames. Image example is provided in Fig. 12.
We report dataset statistics in Table 7.
A.4 GENEX-EQA
A.4.1 DATASET DETAILS
Generally, the GenEx-EQA could be divided into two categories, Single-Agent and Multi-Agents. In single-agent scenario, the agent should be able make the appropriate decision with only its
16

---

Published as a conference paper at ICLR 2025
Farc
how
Irer
Frec
Rediskl
Gecestry
Figure 12: Datasct examples are four distinct scenes. Each sampled vidco consist of 50 frames. At cach step, 25 frames chosen for training.
Statisties
Value
Engine (Environment)
UES (City Sample), Unity (Low-texture City, Animate), Blender (Geometry)
scenes
40000 +
# frames
2,000,000
# traversal distance (m)
400,000
# total time (s) navigation direction
285,000
+inf
Table 7: The data statistics for GenEx-DB DB.
current observation does mean there is only one agent exist in the scene). In multi-agent scenario, to fully understand the environment state, the agent need to understand what other agent's belief. For each test case, we repeat the scene in a low-texture virtual environment to observe the different in behavior from observation realistic level.
We provide examples on the constructed GenEx-EQA dataset in Fig. 13.
For each scenario, we include control set. For example, if the exploration would end up with an ambulance driving toward the agent, there also exist an setting where the ambulance is driving away from the agent.
report the statistics of GenEx-BQA dataset Table
Statistics
Value
Engine
UES, Blender
Environment
City Sample, Low-texture City
scenes
200 500 +
agents
# average agent per scene # text context # actions
2.7
800
200
navigation direction
+inf
Table The data statistics of GenEx-EQA benchmark.
A.5 QUANTITATIVE ANALYSIS IMPLEMENTATION
For all tested videos, FVD, LPIPS, PSNR, SSIM is calculated by resizing each image to 1024 x 576 pixels and comparing them with the ground truth videos at the same dimensions.
For latent MSE of images, each image is resized 500 x 500 pixels and processed through the Inception v4 model Szegedy et al. (2017) to compute the latent MSE. When comparing IELC, we compare the latent MSE between beginning and ending frames.
17

---

Published as a conference paper at ICLR 2025
Single-Agent
Soeret lerve du edese coan want
Scevet lurtun on int sectim and ward to
Trene ua driving dion stree Aheed then
Seeee: an apeeaadeng up elsnection with sey apotic
the
fee opporste
stepped lane
rooving skowly
aorm prostayy cnweegeryy wirkire
tuafccs
needs mome.
Questorn Wind trould rmsb
Devetor
Ouerton should proceec? Cheieer (4) pas enspard caindyy
Quertonn shoud iesuordd sttasion?? Choecer
Chosser
Chuncerr
(A] stap place wast the
(A) laneece kyp cantlely
(A) vos interuec
since then abetuatios.
(S) carn canto calliioo
204
chaninal rospac carae
(3) Change paz though.
(C) uen srrtict
(C) 2ca doeera
cown
proseding
IC) peliacc sormone.
100 Comiuuty continus
(8) Comtmu proced theough
dew pander cactouutt assum per
lytc grecs
mertionnng
Multi-Agent
Momee pake area ateoo perteet
Semas invoscttio tortooo eoclight.
beean diiing on the partur pedeea crodding
Scree divin sight
pedeationnnn right dio rosd ho vos fect
peoseraa
myknng to coor crive
reat should
Danstooe
What
Choionn
(A] toverd nomet
(A) Spral given
(4) Move
(A) Contrrus foruar paeh the front
(M) pecestian
(8)
(3)
titting
fest
cit
Figure 13: Example GenEx-EQA questions. We generally divide the questions into two categories. (1) single- agent is testing the ability agent make optimal decision independent of social interaction. For example, in the first scene, decision agent need infer what the other car see, but it does not need infer the helief that agent hold. (2) Multi-agent is testing the ability agents measure other agents' belief and their potential interaction. For example. in the first scene in the second row, the agent need to infer the pedestrian's 's belief in its surrounding and also the other car's belief.
A.5.1 LMM PROMPT FOR WORLD EXPLORATION
We prompted LMM to navigate throughout the scene. The format is provided Fig. 14. To handle different number of frames from generation. For example, the diffusion model generate 25 frames at once and one frame means traveling 0.4 meters, travel 4 meter would mean take the first 10 frames.
difference in distance traveling,
18

---

Published as a conference paper at ICLR 2025
LMM Navigation Prompt
Scenario Context:
You are presented with 360-degree equirectangular panorama, type of image that captures the entire spherical view of an environment and flateens it into rectangular format. This image allows you see the entire surroundings from a single viewpoint. The center of the image corresponds to what directly in front of you, while the left and right sides of the image represent the views your left and right, respectively, The far-left and far-right edges of the image connect to show what is directly behind you. The upper part of the image typically displays sky ceiling, and the lower part shows the ground floor.
Requirement:
Navigate through the urban street scene based provided instructions. The enviromment static with no moving objects, which eliminates the need for caution against dynamic changes. Maintain safe distance from objects to avoid collisions and utilize open spaces effectively to aim for the most direct route without unnecessary detours. You do not need follow any road the image, like crosswalk or sidewalk. You can move any direction.
Navigation Process:
This task is part of a multi-step navigation process, At this step, assess your surroundings based on the provided panoramic image. Regin by identifying potential obstacles and evaluating the most direct paths to your destination. If clear, straight pach evident, rocare to align with it and move forward.
Action Choices: (1) Tum left, (2) Turn right, (3) Move forward (4) Stop
Spage-Specific Instruction:
[Instruction]
Analysis Request:
S.ene Analysis: Start with a thorough examination scene. Describe what is visible in the left, front, right, and back views.
2. Target Identification: Identify where the target location is situated, considering the layout and nearby
objects.
3. Path Planning: Plan efficient and clear path. If the pach is unobstructed, align yourself and proceed
straight towards it. Ensure you maintain safe distance from any objects, and detour if necessary.
Decision Reporting: Return your decision at the end the specified format, e.g. [[move forward 1 meter]]- If ronation is necessary to align with the path, specify the angle, for example, [[turn right 30 degrees]]. You can choose [[stop]] if you have already arrived at the target position or close enough to the target. Notice the object could be closer than expected due to the panoramic property, around target position are highly distorted. This only one step of a multi-step navigation process, give only one decision this current step.
ready stop when the objects
Start with your analysis and return at the end by strictly follow the format.
Figure 14: Navigation prompt template.
A.5.2 EMBODIED DECISION MAKING USING LMM
We provide LMMs with context using the prompt format illustrated in Fig. 15. In multimodal scenarios, we also include the egocentric (first-person) view, presented as six separate images, in addition to the unimodal cases.
A.5.3 SYSTEM PIPELINE OF EQA DECISIGN-MAKING
We shows a general imagination-enhaaee LMM POMDP system pipeline shown in Fig. 16.
19

---

Published as a conference paper at ICLR 2025
LLM Prompt
Below are views provided to me from my surroundings.
<Image 1> This the my
<Image 2> This is the view.
I am currently operating a vehicle. Based on this information, and considering the given context, please recommend the most appropriate course of action for
Additionally, evaluate the perspectives and likely reactions of other drivers, pedestrians, or obstacles the environment. How might their actions behaviors influence your recommendation? Your guidance should be updated based on the evolving situation, ensuring that your final recommendation reflects a thorough assessment of the surroundings.
SCENE:
(scene text description)
Question:
[question3
OPTIONS:
[choices]
Scart by providing your reasoning for each possible action, considering both my immediate environment actions of others. Conclude with probability- recommendation. The total probability across all options must equal 100%. Format your response strictly as: [A: XX%), [B: XX%), [C: XX%], [D: XXX), where "XX" represents the percencage probability assigned to each option.
Image provided only Multimodal Setting
Figure 15: Embodied QA prompt template
Obsanvation Module
Belief and Imagination Module
Decision Module
Oiservaation
O
Havigrtie Instructio ccttion
lriggh Imaginstion Coneration
Imaginonion Domaain
Figure 16: EQA answer pipeline. It follows Imagination-enhance POMDP, updating its belief with imagina- tion for more informed decision.
20

---

Published as a conference paper at ICLR 2025
A.5.4 EVALUATION METRIC
Machine Evaluation. We provide the three embodied decision metrics evaluate the benchmarked agents.
1. Decision Accuracy and Confidence. Since we prompted the LLMs to generate in given format, we directly parse the accuracy and confidence. case the LLM failed to follow the format, we filter and remove the cases.
2. Decision Confidence. This represents the agent's 's confidence selecting the most appro- priate action based on the available information and context. Confidence is calculated by averaging the normalized logits corresponding to the agent's correct choice.
3. Ohain-of-Thought Accuracy This metric evaluates the aceuracy of the agent's 's logical reasoning that leads to a decision. We employ GPT-4o as a judge to assess the agent's thought process against the correct chain of thoughts. It highlights the sequence of steps, inferences, and reflections the agent uses to reach final action. The prompts to GPT-4o are provided in Fig. 17.
LLM a judge Prompt
Please assess the logical correctness the following response generated by an LLM.
Your task strictly to evaluate whether the logic in the LLM's response aligns with the provided correct reasoning. Avoid incroducing any personal prior beliefs interpretations. Your sole responsibility is verify the logic the reference accurately reflected the LLM's output.
After completing the assessment, return either [[YES]] or [[NO]], striedl adhering to
LLM Response: (texr)
Correct Logic: (logic]
Figure 17: The prompt template GPT4o-as-a-judge-
Human Evaluation. We present the prompt all human evaluators. In unimodal scenarios (both realistic and stylized) we reuse the same results due to the absence of randomness, similar to fixed temperatures in LLMs. Evaluators are guided through three strict steps to prevent information leakage: (I) text description only, (2) egocentric view, and (3) pre-navigated GenEx generation. This sequential approach ensures consistency and maintains the integrity of the evaluation process.
A.6 COMPARED METHOD: SIX-VIEW EXPL.RRATION
We use the same training configuration and dataset as in the original approach, but instead of working directly with panorama images, Fig. 18, we break the equirectangular image down into six faces of cube. Each face corresponds to specific direction: front, left, right, back, top, and bottom, as Fig. 1, which obtain navigation process by focusing on discrete sections of the scene.
Front view always moves forward.
Left view moves to the right.
Right view moves to the left.
Back view moves backward.
Top view remains stationary for upward and moves forward.
Bottom view remains stationary for downward and moves forward.
Although each face provides clear perspective, the transitions between faces introduce inconsis- tencies as information in cube faces are not shared. However, panorama navigation can preserve a general world context.
21

---

Published as a conference paper at ICLR 2025
Prone
Trune
Frumes
Framed
Frunze
Front
Frame
Cube-wise Navigation
Navigation
Figure six-view exploration baseline, we separate diffuser representing face. Although each individual face remains acceptable quality, the world context is not preserved as in panoramic world exploration.
A.7 DETAILS OF CROSS-SCENE GENERATION
The model shows a strong cross-scene generalization ability. From the loop consistency results in Table the panorama generation works well even for scenes deviates largely from its training
Dataset For each model trained by the dataset described in 5.1, we evaluate its cross-scene gener- ation quality.
Metric We evaluate loop consistency for different scenes when trained on different model and report in Table 9.
Loop Consistency
Street
Indoor
Realistic
Anime
Texture
Geometry
Realistic
0.1051
0.0917
0.0687
0.1248
0.1332
0.2047
Anime
0.1044
0.1679
0.1171
0.0571
0.1347
0.2890
0.1032
0.0508
Geometry
0.1471
0.0722
0.1230
0.1746
0.0855
Table 9: Cross-Seene Loop Consistency ($ 5.2) Latent MSE by training scene and test Columns are by test scenes rows are by training scenes.
Image Example We demonstrate some example of cross-scene generation. For example, When training using the Anime dataset, the model generalize to generate novel view of car in the Low-Texture dataset, although nothing similar exist in its training set. More image examples are provided in Fig. 19.
22

---

Published as a conference paper at ICLR 2025
Testing
Test
Cross-Scene
Google Street
Input View
Output View
Input View
Output View
Input View
Output View
Behavior Vision Suite Evaluation
Input View
View
View
View
Input View
Output View
Example Generations very different trairing)
Input View
Figure 19: Cross-scene generation examples. Neither google street view or indoor seene is used for training (Inputs and outputs are panorama images. We extract cubes for visualization).
23

---

Published as a conference paper at ICLR 2025
A.8 EXTENSION TO 3D REPRESENTATION OF WORLD
3D Egocentric World. We are able to reconstruct egocentric 3D point cloud combining single panorama image with the external tool Depth-Anything-22 (Yang et al., 2024a). Examples are shown in Fig. 20. For each point the image, directly map it to 3D location using depth.
Given a pixel (u, v) with image dimensions W (width) and H (height), each point (X, Y, 2) repre- sents point in the 3D point cloud.:
Compute angles:
Calculate 3D coordinates:
2Tu 0
X = D - cos( cos(#)
m
Y = D sin(
17
2
Z D cos( sin
Input Panorama
Inside Point Cloud
Reconstructed Point Cloud
Depth Map
Input Panorama
Inside Point Cloud
Reconstructed Point Cloud
Depth Map
Input Panorama
Inside the Point Cloud
Reconstructed Point Cloud
Donkl
Depth Map
Figure 20: Egacentric reconstruction with depth map and point cloud using monocular depth estimation tools (Yang et al., 2024a).
3D Exocentric world. To construct a exocentric 3D reconstruction from multi-view image. For any given panorama image, could sample random forward moving direction to generate multiple
24

---

Published as a conference paper at ICLR 2025
panorama image. Breaking down into cubes, panorama images become usable 2d images to be passed in reconstruction model like DUSt3R (Wang et al., 2024b). Examples are shown in Fig. 21.
Generated Images
Input View
3D Reconstruction
Generated Images
Input View
3D Reconstruction
Figure 21: Exocentric 3D reconstruction with DUSt3R (Wang et al., 2024b).
A.9 BIRD'S S-EYE VIEW GENERATION
Our method can generate bird's-eye s-eee view (BEV) maps from single panoramic image by leveraging exploration along the z-axis. By adjusting the exploration pipeline to navigate upwards, we can ex- tract top-down view directly. shown in Fig. 22, examples of the generated BEV maps illustrate the effectiveness of our method in capturing comprehensive top-down perspective from a single panoramic image. This capability enables the agent to imagine third-person perspective through BEV maps, supporting more informed and objective decision- decision-making.
25

---

Published as a conference paper at ICLR 2025
Input View
Generated View
BEV
Input View
Generated View
BEV
Figure 22: By exploration in z-axis, we are able to generate the 2D bird-eye view of the current scene.
26


Reviews:
[{"review_id": "8FhqlefcQf", "reviewer": "Reviewer_GPuR", "summary": "Humans have the capacity to imagine the future and revise their beliefs about the world based on these imagined observations. Building on this concept, the authors have introduced a video generation model, GeNex, which enables an agent to mentally explore future imagined observations. Subsequently, a Large Language Model (LLM) is utilized as the policy model to predict future actions.", "questions": "1. Equation (3) is incorrect\n2. The derivation of Equation (4) is unclear. Could you explain how it was formulated?\n3. Is the LLM policy model fine-tuned or used as is?\n4. The space of 'state' & 'belief' is not clearly defined.\n5. It is unclear whether the diffusion model has been overfitted to the dataset, potentially making it inadequate for handling complex real-world interactions.\n6. The entire framework appears to have little connection with POMDP.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The concept is intriguing and the explanation is clear and straightforward.", "weaknesses": "1. There are some errors in the mathematical formulations presented, particularly in equations (3) and (4), which are confusing.\n2. Although SCL is highlighted as a contribution, its effectiveness is not demonstrated in the experimental results.\n3. The use of latent diffusion with temporal attention is not a novel architecture.\n4. The real-world dynamics of vehicles do not allow for pure rotation, which the paper seems to overlook.\n5, Table 3 presents an unfair comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l8bo4F7dJl", "reviewer": "Reviewer_M4Hx", "summary": "The paper introduces the challenge of planning with partial observation in embodied AI and highlights how humans can mentally explore unseen parts of the world to update their beliefs and make informed decisions. To replicate this human-like ability, the authors propose the Generative World Explorer (Genex), a video generation model that enables agents to mentally explore large-scale 3D worlds and acquire imagined observations to update their beliefs. They train Genex using a synthetic urban scene dataset, Genex-DB, and demonstrate that it can generate high-quality and consistent observations during long-horizon mental exploration and improve decision-making in an existing model.", "questions": "Overall I think this is a good paper that can contribute to the subsequent development of the field of embodied AI.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "+The idea of build a Generative World Explorer is interesting, and I think it will be useful to the development of embodied AI research.\n\n+ It's practical to apply the proposed Genex to the embodied decision making process.", "weaknesses": "-There is a gap between the training data (synthesized with unity) and test data (captured from google street), the degrees of freedom of in the observation perspectives, google street seems to more limited compared to the unity. But the gap between training and test data may not  be always \"bad\", because such gap may show more \"Generalizability\". \n\n-In the following sentence “An embodied agent is inherently a POMDP agent (Kaelbling et al., 1998): instead of full observation, the agent has only partial observations of the environment.” , “a POMDP agent” seems to lack rigor. POMDP (Partially Observable Markov Decision Process) is a modeling framework that can be applied to describe the behavior of an agent in an environment where full state information is not available. Visual observation is only one channel for information acquisition. Saying that incomplete visual observation necessarily leads to a POMDP is also not very rigorous.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0RLxH9w3Gq", "reviewer": "Reviewer_MnVr", "summary": "The paper works on the problem of decision-making in the partial observation setting. To tackle the task, the authors introduce a novel panorama-based video diffusion model which can imagine the observations from different positions. The authors further combine the generative model and the LLM to help the decision making process. To evaluate the decision making performance, they design a benchmark over 200 scenarios in single and multi-agent settings. The results show that their pipeline achieves better performance by augmenting the agent's imagination ability via the generative model.", "questions": "1. How to determine what’s the trajectory to explore if the world is unlimited? And how to make sure the information is enough to make a decision?\n2. Is there better way to evaluate the imagination ability, like the 3D concept error with GT (there is hidden car or not, how much unobserved information is discove", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Leveraging generative models to complete the partial observations to a full “world” understanding is reasonable to utilize the priors learned from the data.\n2. For the panorama representation, they design the spherical-consistent learning during their learning process to improve the consistency of the panorama image. From their results, the panorama truly shows better consistency and leads to better representation of the scene.\n3. The authors conduct extensive experiments and create a benchmark for demonstrating the challenging cases under the partial observation constraints.", "weaknesses": "1. In this work, the authors actually construct an explicit representation for “the imagination prior” to make decision making. However, in the benchmark setting, most questions seem only related to a specific case. For single-agents, just try to avoid some unseen cars. And for multi-agent, try to make the other two agents avoid collision. The task setting seems not challenging and common enough to demonstrate the usefulness of such imagination ability. Also it’s hard to see the real performance through such discrete choice-making decision accuracy. Potentially, the method can serve as a role to generate the bird-eye map from a single panorama image and can reveal the hidden cars not in the observation. \n2. The paper mentions such imagination can be further updated based on new observations, however, in this work, there is no integration of the imagination and the new observations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LFnGJYE927", "reviewer": "Reviewer_GZnb", "summary": "The authors of this study investigate the problem of  planning with partial observation, which is important in embodied AI. To achieve this, the authors propose a video generation model Generative World Explorer (Genex), that allows an agent to simulate the world through panoramic representation. The authors also propose an imagination-driven POMDP framework, where generated images assist the agent in decision-making through question-answering (QA).", "questions": "1. Some typos: L96: imaginatively-imaginative; L186: a-an; Fig.15 Imaginatin-Imagination.\n2. Fig link: L379: Fig.2? Maybe this should be Fig.6.\n\nThe reviewer has identified four major concerns and would like the authors' responses to these points. Please answer each concern in the rebuttal stage. The reviewer will respond according to the authors' rebuttal in the discussion phase.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 4, "strengths": "1. Importance of the work: While world models with front-view and multi-view videos are actively investigated by the research community, the generation of panoramic videos is seldom explored. This paper introduces a novel training strategy specifically for panoramic video generation, contributing valuable insights for the community.\n2. In this work, the authors aim to define an embodied agent with belief revision driven by imagination, which is able to imagine hidden views through imaginative exploration..\n3. Two new datasets called Genex-DB and Genex-EQA have been collected to facilitate the proposed pipeline. The scenarios include a diverse range of styles: Realistic, Animated, Low-Texture, and Geometric.\n4. On the proposed dataset Genex-DB and Genex-EQA, the proposed method Genex achieves favorable results in panoramic video generation and embodied QA, compared to other baselines.", "weaknesses": "1. My main concern is the **actual impact** of the proposed 'imagination' on embodied QA. While the authors show an approach to link the panoramic video generation with embodied QA, the experiments do not explicitly demonstrate the effectiveness of the ''imagination generation''. How about the results of POMDP without imagination？\n2. As far as the reviewer knows, most of the generation models (including the SVD used in this paper) are poor in the reasoning ability, because essentially they are just simulating the probability of objects appearing. In most cases, if there are no explicit constraints like specified object category, the generation model wouldn't expect an ambulance to be here in most cases. This is an open question and the reviewer wants to see the point of the authors. Additionally, could the authors provide more examples of the imagination results, particularly challenging cases like those shown in Fig.12?\n3. Some **concerns about the Genex-EQA questions**. The questions and answers in the dataset are quite subjective. For example in the second row and second column in Fig.12, the gt choice is \"Signal the car to stop for the pedestrian\". This action seems impractical for an autonomous driving vehicle. Further clarification on the methodology and rationale behind the question and answer collection process is needed to understand the dataset's reliability.\n4. For panoramic video generation task, though the method serves as a baseline, it is beneficial to have **some comparisons with previous single-view world models** (because they can also perform panoramic video generation task by just replacing the data) and demonstrate why these models fail to generate panoramic videos.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)