PlayerOne: Egocentric World Simulator
Authors: Yuanpeng Tu
Hao Luo
Xi Chen
Xiang Bai
Fan Wang
Hengshuang Zhao
Venue: NeurIPS 2025 oral ()
Keywords: Video generation
World model
Egocentric understanding
Primary area: deep_learning

Abstract: We introduce PlayerOne, the first egocentric realistic world simulator, facilitating immersive and unrestricted exploration within vividly dynamic environments. Given an egocentric scene image from the user, PlayerOne can accurately construct the corresponding world and generate egocentric videos that are strictly aligned with the real-scene human motion of the user captured by an exocentric camera. PlayerOne is trained in a coarse-to-fine pipeline that first performs pretraining on large-scale egocentric text-video pairs for coarse-level egocentric understanding, followed by finetuning on synchronous motion-video data extracted from egocentric-exocentric video datasets with our automatic construction pipeline. Besides, considering the varying importance of different components, we design a part-disentangled motion injection scheme, enabling precise control of part-level movements. In addition, we devise a joint reconstruction framework that progressively models both the 4D scene and video frames, ensuring scene consistency in the long-form video generation. Experimental results demonstrate its great generalization ability in precise control of varying human movements and world-consistent modeling of diverse scenarios. It marks the first endeavor into egocentric real-world simulation and can pave the way for the community to delve into fresh frontiers of world modeling and its diverse applications.

PlayerOne: Egocentric World Simulator
Yuanpeng Tu 1.2
Hao Luo 2.3
Xi Chen'
Xiang Bai'
Fan Wang²
Hengshuang Zhao *HUST
HKKU DDMO Academy, Alibaba Group
Hupan Lab
http://playerroeegiihuee
First Frames
Human Motion Sequences
Simulated Videos
Figure 1: Simulated videos our PlayerOne. Given an egocentric image as the scene to be explored, we can simulate egocentric immersive videos that are accurately aligned with the user's motion sequence captured by an exocentric camera. All the users have been anonymized and action videos shot with the front camera.
Abstract
We introduce PlayerOne, the first egocentric realistic world simulator, facilitat- ing immersive and unrestricted exploration within vividly dynamic environments. Given an egocentric scene image from the user, PlayerOne can accurately construct the corresponding world and generate egocentric videos that are strictly aligned with the real-scene human motion of the user captured by an exocentric camera. PlayerOne is trained coarse-to- to-fine pipeline that first performs pretraining on large-scale egocentric text-video pairs for coarse-level egocentric understand- ing. followed by finetuning on synchronous notion-video data extracted from egocentriceexocentric video datasets with our automatic construction pipeline. Be- sides, considering varying importance of different components, we design a part-disentangled motion injection scheme, enabling precise control of part-level movements. In addition, we devise joint reconstruction framework that progres- sively models both the 4D scene and video frames, ensuring scene consistency in the long-form video generation. Experimental results demonstrate its great generalization ability in precise control of varying human movements and world- consistent modeling of diverse scenarios. marks the first endeavor into egocentric real-world simulation and can pave the way for the community to delve into fresh frontiers of world modeling and its diverse applications.
'Work during DAMO Academy internship. t Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
World models [36, 28, 35, 13, 1, 34] have undergone extensive research due their ability to model environmental dynamics and predict long-term outcomes. Recent breakthroughs in video diffusion models [30, 12, 17] have revolutionized this domain, enabling the synthesis of high-fidelity action-conditioned simulations that forecast intricate future states. These advancements empower applications ranging from autonomous navigation in dynamic real-world environments the creation of immersive, responsive virtual worlds in AAA game development. By bridging the gap between predictive modeling and interactive realism, world simulators are emerging as critical infrastructure for next-generation autonomous systems and game engines, particularly in scenarios requiring real-time adaptation complex, evolving interactions.
Despite significant progress, this topic remains underexplored in existing research. Prior studies [9, 36, 7] predominantly focused simulations within game-like environments, falling short of replicating realistic scenarios. Additionally, in their simulated environments, users are limited to performing predetermined actions (i.e,, directional movements). Operating within the confines of a constructed world restricts the execution of unrestricted movements as in real-world scenarios. While some initial efforts [19, 26, 1] have been made toward real-world simulation, they mainly contribute to world-consistent generation without human movement control. Consequently, users are reduced passive spectators within the environment, rather than being active participants. This limitation significantly impacts the user experience, as it prevents the establishment of a genuine connection between the user and the simulated environment.
Faced with these challenges, we aim to design an egocentric world foundational framework that enables the user being a freeform adventurer. Given user-provided egocentric image as the world to be explored, it can enable the user to perform unrestricted human movements real-time captured by an exocentric camera and consistent 4D scene modeling in the simulated world. Specifically, we propose the first realistic egocentric world simulator termed PlayerOne. Starting from a diffusion transformer (DiT) model [23]. we first extract the latent of egocentric user-input image. Meanwhile, we select the real-world human motions (i.e., human pose or keypoints) as our motion representation. Considering the varying importance of different body parts our task, the human motion sequence is partitioned into three groups (i.e., head, hands, feet and body) and fed into our part-disentangled motion injection to generate latents that can enable precise part-wise control. Additionally, we developed a joint scene-frame reconstruction framework that can progressively complete scene point maps during the video generation process enable scene-consistent generation. The DiT model takes the concatenation of the first frame latent, motion latent, video latent, and the point map latent as input and conducts noising and denoising on both the video and point map latent. Notably, the point map sequence is not required during inference, ensuring practical efficiency. Moreover, to overcome the absence of publicly available datasets, we curate required motion-video pairs from existing egoxentrrc-exxocetrii datasets using an automated pipeline designed to filter and retain high-quality data. A coarse-to-fi training strategy is also designed to compensate for the data scarcity. The base model is fine-tuned on large-scale egocentric text-video data for coarse-level generation, then refined on our curated dataset to achieve precise motion control and scene modeling. Finally, we distill our trained model [38] to achieve real-time generation. By integrating these innovations, PlayerOne advances the field of dynamic world modeling. Our contributions are summarized as follows:
We introduce PlayerOne, the first egocentric foundational simulator for realistic worlds, capable of generating video streams with precise control of highly free human motions and world consistency in real-time and exhibiting strong generalization in diverse scenarios.
We design a novel part-disentangled motion injection scheme to enhance fine-grained motion alignment, where a joint scene-frame reconstruction framework is introduced to guarantee world-consistent modeling in long-term video generation as well.
We construct effective automatic dataset construction pipeline to extract high-quality motion-video pairs from existing egocentric-exoceen datasets, where a coarse-to-fine training scheme is also introduced to compensate for the data scareity.
2 Related Work
Video generation. The rapid development of diffusion models [27, 14, 41, 3] has driven substantial advancements in video generation. Early researchers [10, 5] adapted existing text-to-image models to

---

enable text-to-video generation to compensate for the limited availability of high-quality video-text
datasets. Subsequently. diffusion transformers based frameworks [37, 23, 17, 12. 30, 29] are proposed. When scaling-up training, they enable more highly realistic and temporally coherent generation results. Among them, Hunyuan Video [17] substitutes T5 with a Multimodal Large Language Model. L-TX- Video [12] modifies the VAE decoder to handle the final denoising step and convert latents into pixels. Wan [30] introduces a full spatial-temporal attention to ensure computational efficiency.
World models. Existing world models [13, 35, 7, 1.3] can be roughly divided into two categories: 1) Agent learning targeted models, 2) World simulation models. For the former, they [13. 35, 34] aim at enhancing policy learning within simulated environments. Among them, Dreamer [13] and DayDreamer [35] solve long-horizon tasks from images purely by latent imagination. MuZero [34] runs the self-play of Monte Carlo tree search to build world models for Atari. Distinct from this direction, world simulation aims to model an environment by predicting the next state given the current state and action. These works focus on human interaction with neural networks through high-quality rendering, robust control, and strong domain generalization to real- world scenarios. With advances in video generation, high-quality world simulation with robust control has become feasible, leading to numerous works focusing on interactive world simulation [9, 26, 19, 36, 7, 1, 28, 6J. Among these works, WORLDMEM [36]. The Matrix [7] proposes the first world simulator capable of generating infinitely long real-scene video streams with real-time, responsive control. Matrix-Gaii redefines video generation as interactive process of exploration and creation. Cosmos [1] presents a seneral-purpor world model and pre-training-the thenng-the-pppe ceen--ost-traune training scheme. Aether [28] designs a unified framework with synergistic knowledge sharing across reconstruction, prediction, and planning objectives. However, these methods primarily focus on virtual game scenarios and are limited to specific directional actions, rather than facilitating high-degree-of-freeee ffffrreooom motion control in real-world environments. To address these limitations, we target at developing a human motion driven realistic world simulator. Given an egocentric image, we can construct real-scene world that immerses users as freeform adventurers with precise and unrestricted human motion control.
[43]
3 Method
In this section, we detail the methodology of PlayerOne. Sec. 3.1 introduces the relevant preliminaries and the overall pipeline. Sec. 3.2 presents the core of our proposed model, followed by Sec. 3.3 introducing our dataset construction and training strategy.
3.1 Overview
Video diffusion models [23, 15] consist of two key processes: a forward (noising) process and a reverse (denoising) process. The forward process gradually adds Gaussian noise, denoted ~ ~N,0 I), to clean latent sample 2o E Re where k, c, h, and w represent the dimensions of the video latents. This transforms 2g into a noisy latent 2, In the reverse process, a learned denoising model eæ progressively removes the noise from 2t to reconstruct the original latent representation.
As shown in Fig. 2, our method comprises two core modules: dirttdisettanged Motion Injection (PMI) and frame Reconstruction (SR). In PMI, we use real-scene human motion as the motion condition enable free action control for the user. The first frame converted into 2 2/rame via a 3D VAE encoder. The human motion sequence is split into three parts based on varying importance, and each part is fed into a 3D motion encoder to obtain latents. These motion latents are concatenated into Zmation E x To improve view alignment, we transform the head parameters of the human motion sequence into a camera sequence, which is then fed into a camera encoder. The output is added to the noised video latents Evideo to inject view-change signals. In SR, we jointly reconstruct video frames and 4D scenes to ensure world-consistent generation in the context of long video generation. We render point map sequence from the ground truth video and feed it into point map encoder with an adapter to obtain žpoint E R x4xx64x b Finally, all latents and conditions concatenated channel-wing The training objective of our method can be expressed as:
EE [[lc-co co (zz, t)][]] where 2, = 20 + 8&c
(1)
Where / = 1, 1,...,T,, o, 8,2 1. Since we only add noise point map latents and video latents, thus 2g = Evideo & Zpoint. 3 denotes the channel-wise concatenation operation, ((()) represents a uniform distribution, and T denotes the denoising steps.
3

---

Denoised Latent
3D VIVE Encodde
Latent
Video Paint Decoder
Fiint
Noised Point
Video Latent
Map Latent
portddisnnaagled Motion Injection
Diffusion Transformer
Prozen
Trannalle
Scene-frame Reconstruction
Noised
Camera Encoder
Video Latenz
Adtipter
Point Map Encoder
Decampose
Sequencc
Sequence
Hands
Part-wrie
Motion Encoder
Nakool Point Map Latent
Denching
Sody&Feet
Part-disentongleed Motion Injection
Reconstruction
Figure 2: Overall framework of PlayerOne. It begins by converting the egocentric first frame into visual tokens. The human motion sequence split into groups and fed into the motion encoders respectively to generate part-wise motion latents, with the head parameters converted into rotation-only camera sequence. This camera sequence is then encoded via a camera encoder, and its output is injected into noised video latents to improve view-change alignment. Next, we render a 4D scene point map sequence with the ground truth video, which is then processed by a point map encoder with an adapter to produce these latents into the DiT Model and perform noising and denoising on ensure conlddconnistent generation. Finally, the denoised latents are decoded by VAE decoders to produce the final results. Note that only the first frame and the human motion sequence are needed for inference.
latents. Then input the concatenation
video and scene latents to
3.2 Model Components
Part-disentangled motion injection. Prior studies [19, 26, 7, 43] typically utilize camera trajectories as motion conditions or are constrained specific directional movements. These restrictions confine users to passive "observer" roles, preventing meaningful user interaction. In contrast, approach empowers users to become active "participants" by adopting real-world human motion sequences (i.e., human pose keypoints) motion conditions, allowing for more natural and unrestricted movement. However, our empirical analysis reveals that extracting latent representations holistically from human motion parameters complicates precise motion alignment. To address this challenge, introduce part-disentangled motion injection strategy that recognizes the distinct roles of various body parts. Specifically, hand movements essential for interacting with objects in the environment, while the head plays a crucial role in maintaining egocentric perspective alignment. Accordingly, we categorize the human motion parameters into three groups: body and feet, hands, and head. Each group is processed through its own dedicated motion encoder, comprising eight layers of 3D convolutional networks, extract the relevant latent features. This specialized processing ensures accurate and synchronized motion alignment. These latents are subsequently concatenated along the channel dimension to form the final aart-aware motion latent representation Zmotion R xx3hxxx
To further enhance the egocentric view alignment, we solely transform the head parameters of the human motion sequence into sequence of camera extrinsics with only rotation values. We out the translation values in the camera extrinsics, assuming the head parameters are at the camera coordinate system's origin. Specifically, suppose the head parameter (0z, Oy. 0z), first normalize the rotation axis as follows:
0 ||]
(2)
Then we construct the rotation matrix as follows:
R sin 0 [u] + cos 0) [u]
(3)

---

Video Pool
Detection Model
Automatic Data Filtering
GG f Camera Sequence
Projection
Small
Keypaint Distance
Low Confidencc X
SMP Sequence
3D Mesh
2D Keypoints
Exocentric Video
SMPL-X
SMPL Model
Large
X
Egocentric
Exocentric Video
OpenPase
2D Keypoints
Figure 3: The overall pipeline of the dataset construction. By scamlessly integrating detection and human pose estimation models, we extract motion-video pairs from existing egocentric-exocentrie video datasets while retaining high-quality data through our automatic filtering scheme.
Where Uy is the cross product matrix of u, which can be denoted as follows:
-1=
[u]
U
Ux
Ug
Then we use Plücker ray [40] to parameterize the camera extrinsics and then feed the output to an extra camera encoder, which shares a similar structure with the motion encoder. Then the latents from this encoder are added the noised video latents to inject the view-change information.
Scene-frame reconstruction. While PMI enables precise control over egocentric perspective and motion, it does not guarantee scene consistency within the generated world. To address this limitation, we introduce a joint reconstruction framework that simultancously models the 4D scene and video frames, ensuring scene coherence and continuity throughout the video. Specifically, it begins by employing CUT3R [31] to generate a point map for each frame based on ground truth video data, reconstructing the n-th frame's s point map using information from frames ttrouugh n. These point maps are then compressed into latent representations using a specialized point map encoder [16]. To integrate these latents with video features, we implement an adapter composed of five 3D convolutional layers. This adapter aligns the point map latents with video latents and projects them into a shared latent space, facilitating seamless integration of motion and environmental data. Finally, concatenate the latent representations from the first frame, the human motion sequence, the noised video latents, and corresponding noised point map latents. This comprehensive input is then fed into diffusion transformer for denoising, resulting in coherent and visually consistent world. Importantly, point maps are only required during the training phase. During inference, the system simplifies the process by utilizing only the first frame and the corresponding humaan motion sequence to generate world-cone cordd-consistent videos. This streamlined approach enhances generation efficiency while ensuring that the resulting environment remains stable and realistic throughout the entire video.
3.3 Training Strategy
Dataset preparation. The ideal training samples for our task are egocentric videos paired with corresponding motion sequences. However, no such dataset currently exists in publicly available repositories. As a substitute, we derive these data pairs from existing egocentric- exxceetricc video datasets through automatic pipeline. Specifically, for each synchronized ggocentric-exoceentre video pair, we first employ SAM2 [25] to detect the largest person in the exocentric view. The background-removed exocentric video is then processed using SMPLest-X [39] to extract the SMPL parameters of the identified individual as the human motion. To enhance optimization stability, an 1.2 regularization prior is incorporated. We then evaluate the 2D reprojection consistency to filter out low-quality SMPI. data. This involves generating a 3D mesh from the SMPI, parameters using SMPLX [22], projecting the 3D joints onto the 2D image plane with the corresponding camera parameters, and extracting 2D key points via OpenPose [4]. The reprojection error is calculated by measuring the distance between the SMPL-projected 2D key points and those detected by OpenPose. Data pairs with reprojection errors in the top 10% are excluded, ensuring final dataset of high-quality motion-video pairs. The refined SMPL parameters are decomposed into body and feet (66 dimensions), head orientation (3 dimensions), and hand articulation (45 dimensions per hand) components for each frame. These components are fed into their respective motion encoders. The dataset construction pipeline is illustrated in Fig. 3. As detailed in Tab. our training dataset combines multiple publicly available datasets to ensure comprehensive coverage of diverse environmental contexts, action types, and intensity levels, thereby enhancing model generalization.
5

---

Table 1: Statistics of datasets used for training our PlayerOne. "quality" particularly refer to the
image resolution. "Ego-Eat Exo" denotes whether the dataset contains egocentric- egocentric-exocentiic video pairs.
Dataset
EgoExo 4D [8]
Nymeria [20]
FE [td
Egohso-Fitnes: Fitness [18]
Egovid-5hi 5M [33]
Size
740
1,200
38,364
1,276
SM
Resolution
1080p
1408p
1080p
1080p
1080p
Ego-Exo
Action
Joint- Train No Pretrain Baseline
Ours
Figure 4: Investigation on coarse-to-fin training. "Joint-Train" and "No Pretrain" denote training with both motion- viidoo pairs and large-scale egocentric videos a one-stage manner and training with only motion-vide video pairs respectively. The Wanx2.1 1.3B is adopted as the baseline.
Coarse-to-fine training. Though we can extract high-quality motion-video training data with our automatic pipeline, the limited scale of this dataset is insufficient for training video generation models to produce high-quality egocentric videos. To address this, we harness the extensive egocentric text-video datasets (i.e., Egovid-5M [33]). Specifically, we first fine-tune the baseline model using LoRA on large-scale egocentric text-video data pairs, enabling egocentric video generation with coarse-level motion aligmment. Then we freeze the trained LoRÁ and fine-tune the last six blocks of the model with our constructed high-quality dataset to enhance fine-grained human motion alignment and view-invariant scene modeling, which can effectively address the scarcity of pair-wise data. Finally, we adopt asymmetric distillation strategy that supervises causal student model with bidirectional teacher [38] to achieve real-tine time generation and long-duration video synthesis.
Experiments
4.1 Experimental Setting
Implementation details. choose Wanx2.1 1 1.3B [30] as the base generator. We set the LoRA rank and the update weight of the matrices as 128 and 4 respectively and initialize its weight following [30]. The inference step and the learning rate are set as and Adam optimizer and mixed-preees precision bf16 adopted. The cfg 7.5 is used. We train model for 100,000 steps on NVIDIA A100 GPUs with a batch size 56 and sample resolution of x 480. The generated video runs at eight frames per second, and we utilize 49 video frames (6 seconds) for training. After distillation, our method can achieve 8 FPS to generate the desired results. All the action videos in this paper shot with the front camera.
10 respectively, where
Benchmark.
Since there is no publicly available benchmark for our task, we construct a benchmark with 100 videos collected from Nymeria [21] dataset, which is not included for training. It consists of coarse-level motion descriptions for each sample and covers diverse realistic scenarios. Considering the information gap between the human motion sequence and the text, we further use Qwen2.5-VL [2] to enrich the caption to generate videos for the competitors for more fair comparisons.
Metrics. On our constructed benchmark, for evaluation of alignment with the given text descriptions, we calculate both CLiP-Score and DINO-Score, where LPIPS [42] is employed evaluate the video fidelity of the generated video. Besides, we calculate the frame consistency to evaluate the temporal coherence and consistency of the generated video frames over time. We further utilize 3D hand pose estimation model [24] to estimate the hand pose of the generated videos and use the results of

---

Action
Entaggedd Conballer
No Cam
Ours
Figure 5: Investigation on part-disentangled motion injection. "ControlNet" denotes injecting motion latents with a ControlNet [41]. "Entangled" and "No Cam" denote inputting the whole motion sequence into a motion encoder without dividing into groups and removing the camera encoder respectively.
Table 2: Quantitative evaluation on the components of PlayerOne. PlayerOne outperforms all these variants. "No Camera" Camera"/"Filtering" denote training without/with the camera encoder/data filtering.
DINO-Score (1) CLor-Score (1) MPJPE (U) MRRPE (L) FVD (L) LPIPS(L)
Baseline
51.3
65.6
376.14
341.01
394.16
0.1421
Pretrain
56.6
74.4
258.05
232.17
301.32
0.1146
+ Pretrain&ConttolNett
57.1
75.2
241.73
218.46
287.52
0.1103
Pretrain&Ennangled
58.0
76.3
235.12
212.53
279.41
0.1060
Pretrain&PMI (No Camera)
60.7
183.25
196.35
257.04
0,0902
+ PMc
62.5
81.3
156.76
175.18
245.72
0.0839
Pretrain&&PM&&FFleeerinng
64.2
83.8
141.56
163.04
230.50
0.0782
+ Pretrain&PMI&
Magrrin@PPPMMMiilleenRRRRoooooo Adapter)
62.7
81.6
176.23
180.10
240.17
0.0919
+ VrcrainneeMMM@@Fileeing&&oooooD
67.5
87.7
129.08
152.22
228.20
0.0685
(laycrOne(ours)
67.8
88.2
127.16
151.62
226.12
0.0663
the ground truth video the labels. Afterward, we follow [24] to calculate two metries: (1) Mean Per-Joint Position Error (MPJPE): the I.2 distance between the predicted and ground truth joints for each hand after subtracting the root joint. (2) Mean Relative-Root Position Error (MRRPE): the metric distance between the root joints of the left hand and right hand.
4.2 Ablation Study
Investigation on coarse-to-fine training. We first evaluate several variants of our yoarse-to-fing training scheme, as depicted in Fig. Specifically, when inputting action descriptions into the baseline model without fine-tuning, the generated results exhibit noticeable flaws, such as hand distortions or the unexpected appearance of individuals. Similar issues can observed when training with only motion-video pairs. We also explore jointly training with both large-scale egocentric videos and motion-vides pairs. Specifically, when inputting egocentric videos, we set the motion latent values zero and extract the latents of the text description to serve as the motion condition, where salanced-sampling strategy used as well. Despite this variant being capable generating egocentric videos, it fails to produce results accurately aligned with the given human motion conditions. In contrast, our coarse-to-fie training scheme delivers much better outcomes compared to these variants.
Investigation on part-disentangled motion injection. Next, we conduct a detailed analysis of our PMI module. Specifically, three variants are included: ControlNet-basee [41] motion injection, inputting motion sequences as a unified entity (the "Entangled" scheme), and removing our camera encoder. As shown Fig. 5, the ControlNet-based scheme suffers from information loss, preventing it from producing results that accurately align with the specified motion conditions. Similarly, the entangled scheme demonstrates comparable shortcomings. Furthermore, removing the camera encoder leads the model's 's inability to generate view-accurate alignments. As depicted in Fig. 5, this variant fails to produce the corresponding perspective change associated with crouching. Ultimately, PMI module successfully generates outcomes that both viewneligned and action-aligned. Investigation on scene-frame reconstruction. Additionally, we conducted a detailed analysis the SR module, exploring three variants: omitting reconstruction, removing the adapter within the SR
7

---

Adapter
DUSIBR
Ours
Figure 6: Investigation on scene-frame reconstruction. Recon"/"No Adapter" denote training without
reconstruction/the adapter. "DUSIR" is replacing CUT3R with DUSIR for point map rendering.
Adion
Rewlt
Action
Resultt
Result
Figure 7:
Qualitative evaluation the motion alignment. We generate simulated vidcos based the same first frame but different motion sequences, Results show that we can achieve accurate motion alignment.
module, and substituting CUT3R [31] with DUSIR [32] for point map rendering. As illustrated in Fig. 6, the absence of reconstruction results in the model's inability to generate consistently simulated results. Moreover, due to the distribution gap between latents of frames and point maps, training without the adapter leads difficulty in loss convergence, causing noticeable distortions. Furthermore, after replacing CUT3R [31] with DUStR [32], our PlayerOne can also produce sceme-consistent outputs, demonstrating its robustness to different point map rendering techniques.
Motion alignment. To verify the alignment capability with the given motion condition, conduct experiments by generating world-simulated videos with the same first frame but different human motion sequences. Fig. shows that our PlayerOne accurately generate corresponding results according to different conditions and produce reasonable interactive changes.
Quantitative comparisons. We provide quantitative results on the core components of our PlayerOne in Tab. 2. All numerical results concur with the visualization outcomes. A significant performance improvement is observed when the model undergoes pre-training on large-scale egocentric text-video datasets. The introduction of PMI yields additional accuracy boost, and it outperforms all of its variants. addition, our designed filtering strategy maximizes performance as well by filtering noisy motion-video pairs. After removing the adapter, the performance suffers from notable degradation due to the distribution gap between latents of video frames and point maps. By introducing our joint scene-frand reconstruction scheme, achieve superior results across all metrics.
4.3 Comparison with State-of-the-arts
Quantitative comparison. Since there no method sharing the same setting as ours, we selected two potential competitors for comparison: Cosmos [1] and Aether [28]. As shown in Tab. 3,

---

nnteripllon; person perszeslivee stedikee my right hand from. Newow Rnd toole dul the will in the middle of the freezer
Desoriztion:: First porspeotive, my sletring digi, Per
Actin
Cosmos- 148 Cosmos-27
Figure 8: Qualitative comparisons between our method and other competitors. Our PlayerOne can achieve the best performance both the motion alignment, video quality.
Table 3: Quantitative comparison between our PlayerOne and other works. Seven metrics are employed for the evaluation. PlayerOne outperforms these methods across all the metrics.
DINO-Score (1) CLIP-Score (t)
MPJPE (1)
MRRPE (J)
FVD ({)
LPIPS(()
Aether [28]
38.0
64.2
415.70
431.05
397.40
0.1856
Cosmos(Diff-7BB [1]
45.3
70.3
301.92
324.12
346.09
0.1630
Cosmos(Dift-14B) [1]
51.6
79.7
256.73
253.06
302.17
0.1351
PlayerOne(ours)
67.8
88.2
127.16
151.62
226.12
0.0663
Table User study on our PlayerOne and existing alternatives. "Quality", "Fidelity", "Smooth", and "Alignment" measure synthesis quality, object identity preservation, motion consistency, and alignment with the text descriptions, respectively. Each metric rated from (worst) 4 (best).
Quality (f)
Fidelity (t)
Smooth (f)
Alignment (1)
Aether [28]
1.32
1.30
1.31
1.34
Cosmos(Dint-70) [1] Cosmos(Diff-14))
2,07
3.02
2.94
2.98
2.71
PlayerOnc(ours)
3,59
3.63
3.65
3.86
PlayerOne outperforms all the baselines by large margins, especially on the metries of motion alignment. Notably, Cosmos [1] exhibits better generalization ability than Aether [28] by explicitly capturing general knowledge real-world physics and natural behaviors. Besides qualitative results, we provide visualization comparisons in Fig. 8, where consistent superiority can be observed in diverse scenarios for both user interaction and world modeling.
User study. Tab. 4, we report the comparison results of human preference rates. We let 20 annotators rate 25 groups of videos, where each group contains the generated video of each method and text description. And provide detailed regulations rate the results for scores of 1-4 from four views: "Quality", "Smooth", "Fidelity", "Alignment". "Quality" counts for whether the result is harmonized without considering fidelity. "Smooth" assesses the motion consistency across the video. "Fidelity" measures ID preservation and distortions within the video, while we use "Alignment" to measure the alignment with the given text descriptions. It can be noted that our model demonstrates significant superiority across all the metrics, especially for "Alignment", and "Smooth".
5 Conclusion
In conclusion, PlayerOne represents significant advancement in interactive and realistic world modeling for video generation. Unlike conventional models that are restricted particular game scenarios or actions, our PlayerOne can capture the complex dynamics of general-world environments and enable free motion control within the simulated world. By formulating world modeling as a joint process of videos and 4D scenes, our PlayerOne ensures coherent world generation and enhances motion and view alignment with the given conditions through part-disentangled motion injection. Experimental results demonstrate superior performance across diverse scenarios.

---

Limitations. Despite the compelling outcomes, our performance in game scenarios is slightly inferior realistic ones, likely due the imbalanced distribution between realistic and game training data, can be addressed by incorporating more game-scenario datasets in future research.
This work was supported by the National Natural Science Foundation of China (No. 62441615, 62422606, 62201484) and DAMO Academy via DAMO Academy Research Intern Program.
Acknowledgements.
10

---

References
[1] Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. Nalv:2501.03575, 2025. 2, 3, 8, 93, 27,
[2] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang. Xiaodong Deng. Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv:2309. 16609, 2023. 6
[3] Andreas Blattmann, Robin Rombach, Kaan Oktay, and Björn Ommer. Retrieval-angmented diffusion models. NeurlPS, 2022.
[4] Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Openpose: Realtime multi-person 2d pose estimation using part affinity fields. TPAMI, 2019. 5
[5] Weifeng Chen, Jie Wu, Pan Xie, Hefeng Wu, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. 1.Xiv:3305.18444 2023. 2
[6] Junhao Cheng, Yuying Ge, Yixiao Ge, Jing Liao, and Ying Shan. Animegamer: Infinite anime life simulation with next game state prediction. arXiv:2504.01014, 2025.
[7] Ruili Feng, Han Zhang, Zhantao Yang. Jie Xiao, Zhilei Shu, Zhiheng Liu, Andy Zheng, Yukun Huang, Yu Liu, and Hongyang Zhang. The matrix: Infinite-horizon world generation with real-time moving control. arXiv:2412.03356. 2024. 2, 3, 4,
[8] Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyl- los Afouras, Kumar Ahuutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and hird-ppesson perspectives. In CVPR, 2024.
6, 27
[9] Junliang Guo, Yang Ye. Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, and Jiang Bian. Mineworld: a real-time and open-source interactive world model on minecraft. 2008:22004008888, 2025. 2,3, 21
[10] Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. In ICLR, 2024. 2
[11] Zihui Guo. Yonghong Hou Hou, Pichao Wang, Zhimin Gao, Mingliang Xu, and Wanqing Li. Ft-hid: A large scale rgb-d dataset for first and third person human interaction analysis. Neural Computing and Applications, 2022. 27
[12] Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richard- son, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Lix-video: Realtime video latent diffusion. arXiv:2501.00100 2024. 2,3
[13] Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. 1912.019/. 2019. 2,
[14] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. arxiv:2006. 11239, 2020. 2
[15] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and J. Fleet. Video diffusion models. arXiv, 2022. 3
[16] Zeren Jiang, Chuanxia Zheng, Iro Laina, Diane Larlus, and Andrea Vedaldi. Gco4d: Leveraging vidco generators for geometric 4d scene reconstruction. wrXiv:2504.07961 2025.
[17] Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. 2rXiv:2412.03660) 2024. 2,

---

[18] Yunn-Ming Li, Wei-Jin Huang, An-Lan Wang, Ling-An Zeng, Jing-Ke Ke Meng, and Wei-Shi Zheng. Egoexo-fitned fitness: towards egocentric and exocentric full-body action understanding, In ECCV, 2024. 6, 27
[19] Hanwen Liang, Junli Cao, Vidit Goel, Guocheng Qian, Sergei Korolev, Demetri Terzopoulos, Konstantinos N. Plataniotis, Sergey Tulyakov, and Jian Ren. Wonderland: Navigating 3d scenes from a single image. arXiv:2412. (tti:2442..2099), /2091, 2024. 2, 3, 4, 23
[20] Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, Kevin Bailey, David Soriano Fosas, C. Karen Liu, Ziwei Liu, Jakob Engel, Renzo De Nardi, and Richard Newcombe. Nymeria: A massive collection of multimodal egocentric daily motion in the wild. ECCV, 2024.
[21] Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqucira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, et al. Nymeria: A massive collection of multimodal egocentric daily motion in the wild. In ECCV, 2024. 27
[22] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman, Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from a single image. In CVPR, 2019.
[23] William Peebles and Saining Xie. arXiv:2212.09748, 2022. 2,
Scalable diffusion models with transformers.
[24] Aditya Prakash, Ruisen Tu, Matthew Chang, and Saurabh Gupta. 3d hand pose estimation in everyday egocentrie images. arXiv:23/2.06583, 2024. 6,
[25] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma. Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam 2: Segment anything in images and vidcos. In ICLR, 2025. 5
[26] Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3e: 3d-informed world- consistent video generation with precise camera control. arXiv:2503.03751, 2025. 2. 3, 4, 23
[27] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. 12. 2021.
[28] Aether Team, Haoyi Zhu, Yifan Wang. Jianjun Zhou, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Chunhua Shen, Jiangmiao Pang, and Tong He. Aether: Geometric-aware unified world modeling. arXiv:2503.18844 2025. 2, 3, 8, 9,
[29]
Genmo Team. Mochi 1. https: : htt///gttttubb com/ htt.////mmommmee......//// 2024. 3
[30] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv:2503.20314, 2025. 2, 6.27
[31] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv:2501. 12387, 2025. 5, 8
[32] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024. 8
[33] Xiaofeng Wang, Kang Zhao, Feng Liu, Jiayu Wang, Guosheng Zhao, Xiaoyi Bao, Zheng Zhu, Yingya Zhang, and Xingang Wang. Egovid-5m: A large-scale video-acti action dataset for egocentric video generation. arXiv:2411.08338 2024. 6,27
[34] Aurèle Hainaut Werner Duvaud. Muzero general: Open reimplementation of muzero. https:
//github.
dww.udddddddddduu wwwddd/muueeooooo muzero-geneeaa, 2019. 2,3
[35] Philipp Wu, Alejandro Escontrela, Danijar Hafner, Ken Goldberg, and Pieter Abbeel. Day- dreamer: World models for physical robot learning. In CoRL, 2022. 2,
12

---

[36] Zeqi Xiao, Yushi Lan, Yifan Zhou, Wenqi Ouyang, Shuai Yang, Yanhong Zeng, and Xingang Pan. Worldmem: Long-term consistent world simulation with memory. UrXiv:250..12369 2025. 2, 3, 21
[37] Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, and Song Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformer. arXhv:2410.1622, 2024. 3
[38] Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025. 2,
[39] Wanqi Yin, Zhongang Cai, Ruisi Wang, Ailing Zeng, Chen Wei, Qingping Sun, Haiyi Mei, Yanjun Wang, Hui En Pang. Mingyuan Zhang, Lei Zhang, Chen Change Loy, Atsushi Yamashita, Lei Yang. and Ziwei Liu. Smplest-x: Ultimate scaling for expressive human pose and shape estimation. arXiv:2501.09782 2025. 5
[40] Jason Y Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani. Cameras as rays: Pose estimation via ray diffusion. In ICLR, 2024. 5
[41] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 7
[42] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The reasonable effectiveness of deep features as a perceptual metric. (eXiv:/801003922 2018.
[43] Yifan Zhang, Chunli Peng, Boyang Wang, Puyi Wang. Qingcheng Zhu, Zedong Gao. Eric Li, Yang Liu, and Yahui Zhou. Matrix-game; Interactive world foundation model. arXiv, 2025. 3,
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer to the contributions in Section 1.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to the limitations in Section 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
Please refer to the assumptions and experimental results in Section 3 and
Section
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer the implementation details in Section 4.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
Please refer to the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
onssionnolcyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. wttpccccpppccccceeececccccccccct /ttpa//ggiiieeeeeeeuuuubiiiiiiii /CodeSubmic ssionnollll onPolicy))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to the implementation details in Section Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please refer to the experimental results in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer the implementation details in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttp//eeerpp.ee//e cc cc.ppullitt/tthhcciilleeelllaea
Answer:
[Yes]
Justification: Yes, conforms the requirements.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer the supplementary material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
17

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
The scope our research does involve data or models with a high risk for misuse. We utilize publicly available, non-sensitive datasets and models that do not require special safeguards for responsible release.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Please refer the supplementary material.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswi tecoieccoo.e.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
18

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This question does not apply to our research no new assets were introduced. Our study solely relies on existing, publicly available resources, and thus, no additional documentation for new assets is necessary.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This question does not apply our research as it does involve crowdsoure- ing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This question is not applicable to our research as no human subjects were involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
We only use the LLM for writing, editing, or formatting purposes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https: htt.::/s...//ooeeeerreee/2/5555 for what should or should not be described.
20

---

Figure 9: The reconstructed scene of our PlayerOne, Our PlayerOnecan achieve relatively precise seene reconstruction by jointly modeling of video frames and seenes.
Appendix
Appendix Overview
This appendix complements the main paper with additional results, analyses, and implementation details:
Visualizations and reconstructed scenes. Qualitative roll-outs and scene reconstructions in diverse scenarios (Figs. 10, 11-13, 14).
Implementation notes. Clarifications of the training/inference pipeline (world/scene consis- tency is preserved at inference), superiority over prior video diffusion models, and detailed distillation settings.
A More Experimental Results
Framework comparisons with prior works. Fig. illustrates framework comparison between our PlayerOne and other competitors. Prior studies [9, 36, 7] have mainly concentrated on simulations within game-like environments, yet they often fail to accurately replicate real-world scenarios. Within these simulated environments, users are typically restricted to performing predefined actions, such
21

---

Text Camera Sequence
Observation (No Interaction)
User
Limited Action
Video
Observer
PlayerOne
User
Human Action Capture
Real-time Interactive Video
Figure 10: Difference between PlayerOneand prior works. Our PlayerOnecan enable freeform move- ments in the simulated world and achieve great world consistency across diverse scenarios.
Forst-persow perspective, stretch out my left hand and high-flved the w.a opposite nAc.
Desoriptiona: Fürst porson perspeative, stroll forward.
Action
Ours
Aether Cosmos- 14B Cosnos-78
Desoriptions: First-person perspeotive,, sie and puct myy hands cheir.
Desoriptions: First: person perspeotive, I sstretsh may right hand to touch the dlgg.
Action
Ours
Aether Connees 48 Coomos-78
Figure 11: More visualization results generated by our PlayerOne. Our method demonstrates great superiority in both motion alignment and environmental interaction across different domains.
directional movements. This limitation confines user interactions to constructed world, thereby
22

---

Desoriptions: First-person perspective, walked forward and stretsheel oust may Left inand to hold the oloor inavalle.
besoriptions: Först-persow perspective, stretohed out may Left hand and high fivest the cut opposite wat
Action
726
Ours
Cosmos-78 7B
Cosmos- 14B
Aether
Figure 12: More visualization results generated by our PlayerOne. Our method demonstrates great superiority in both motion alignment and environmental interaction across different domains.
Descrigtiona: Firah porson perapertioe, atrttshing suc mag right ahalki hinakk moll sppaxite.
Desevigtion:: First peopertion, atrttsh out Left hand high five the oppasite.
Oues
Connoe Hill Cootoe 73
Aetter
Figure 13: More visualization results generated by our PlayerOne. Our method demonstrates great superiority in both motion alignment and environmental interaction across different domains.
restricting the execution of freeform movements akin to those in real-world settings. Existing realistic world simulators [19, 26, 1] often focus solely world-consistent generation, lacking mechanisms for human movement control. result, users are relegated passive observers, rather than active participants, within the environment. This significantly impacts the experience by hindering the
23

---

besoriptiona: First person perpeestvv,, I exteneed myy lonnel shalk nandss with the wonadre. opposice mee.
Desoriptions: First-peripo poripestise, high-fuvell the golden vetriever.
Ous
Aether Counos--1 TB
Figure 14: More visualization results generated by our PlayerOne. Our method demonstrates great
superiority in both motion alignment and environmental interaction across different domains.
Table 5: Quantitative evaluation on the components of PlayerOne on the Nymeria dataset. "No
Camera" Fiiterin"" denote training without/with the camera encoder/data filtering.
Mothod
DINO-Score (1)
CLIP-Score (t)
MPJPE ())
MRRPE (1)
FVD (L)
LPIPS (J)
Baseline
48.72
62.13
402.31
370.27
425.15
0.1570
Pretrain
53.81
71.29
280.51
255.63
328 21
0.1265
Pretmin&ContrlNNet
54.28
72.31
264.10
241.91
314.27
0. 1228
Entaaled&
55.24
73.54
258.88
234.16
308. 308.12 12
0.1174
Pretrain&PM (No Camera)
57.68
76.79
201 34
215.89
9.1017
Pretrain&PMI
59.57
Pretrain& Fitring
61.74
178.64
Pretrain&PPMP Filtering& Recon (No Adapter)
60.13
78.67
191.22
196.54
263.18
1023
Pretrain& PMI& Filtrr&iiittng&&&RRooo Rocon (DUSt3R)
64,02
84.41
140.16
167.23
251.38
0.0788
PlayerOne (ours)
64.87
85.79
137.91
162.74
247.97
0.0761
formation of a genuine connection with the simulated world. contrast to these limitations, our approach enables freeform motion control for users, enhancing their interactive experience.
Why separate processing contributes to precise alignment? Splitting the latent representation into three separate parts-head, hands, and endales the model more effectively capture the unique and often semi-independent motion characteristics of each region. Human body motion is inherently high-dimensional and highly articulated, with different parts frequently moving indepen- dently exhibiting distinct dynamics. Encoding all motion information into a single, entangled latent can cause fine- Ann-grained cues cues-uspeae especially from subtle or fast-move fast-moving regions like hands or head be overshadowed by larger body movements, making precise alignment difficult. By disentangling the motion into part-specific latents, each body region's 's movement can independently modeled and aligned with the corresponding input. This modular approach allows the model to better capture local nuances, improves compositionality. and enhances control over complex motions, resulting in higher motion fidelity and visual quality.
Latency between motion input and video output. After distillation, the measured end-to-end inference latency is approximately 119 milliseconds per generated -neerated-frame, corresponding to about 8.4 FPS. This latency is measured from the moment the motion input received to the generation the corresponding video frame, and includes all major processing steps.
strictly follow the Causvid distillation procedure in our implementation. During the distillation stage, use the entire SMPL-Video paired dataset from the second phase of training- Specifically, we first generate 8,000 ODE pairs and train the student for 30,000 iterations using AdamW with a learning rate of 5 1 We then continue training with our asymmetric
Details of distillation.

---

DMD loss for another 30,000 iterations using Adam with a learning rate of x 10 10-9. A guidance scale of 3.5 is used, and we adopt the two time-scale update rule from DMD2 with a ratio of 5. The entire distillation process takes about 5 days on 32 A800 GPUs.
Details of training pipeline. We clarify that the world/scene consistency mechanism is not removed at inference. During training, a CUT3R-like reconstructor is used only to produce ground-truth point- map sequences corresponding
each video frame; these point maps supervise our scene-frame reconstruction loss and are further encoded to build noised point-map latents. In both training and inference, the model inputs are always the initial egocentric frame and the ouman-motion sequence. while the prediction targets are the video frames together with the point-may sequence.
Concretely, let the raw video be V E R K x3xHxW and its point maps P E Bx After VAE encoding, the video becomes 20 E IR Bxxxxxxx hxw with h= H/8 w = W/8, k= K/4. A 3D encoder plus an adapter maps the point maps to Epoint E Bxxx664xxxx matching the video latent's 's spatio-temporal shape. Both Zvideo and Spoint noised to obtain Vviteo and </rim for denoising training. The motion sequence, passed through our motion encoder, yields zmotion E Bxxx33xhxx The first-frame latent 2hrrst E IR Bx1xexhxu is repeated k times so that ffirst E Bxkxexxx We then concatenate along channels:
Zinput concat (rraat2fr (zhestt Amotion; "vidoo> Z E ERE (xxx((+444
A DiT backbone denoises Zinput into Zourput (same shape). Losses are computed between the predicted (FVideo "point) and the ground-truth (Zvidco; *point) latents via an MSE objective.
Inference. At test time there are no ground-truth point maps and thus no 3D point-map encoder is required. Instead, the network jointly predicts the video and its point-map latents autoregressively: both streams are initialized from noise and co-denoised frame by frame, conditioned on the initial frame and the motion sequence. Because the video and point-map streams are modeled with identical spatio-temporal layouts and are denoised in lockstep, the scene-consistency module remains fully active during inference-no extra imputs are needed and no module is removed. In short, training uses point maps to supervise and align the latent spaces; inference keeps the joint two-stream denoising (video+ + point maps) to maintain world consistency throughout generation.
Superiority over previous video diffusion models. Our approach advances egocentric video generation and simulation in several complementary ways that prior video diffusion systems do not address. (i) Scene consistency and world modeling, Instead of treating frames almost inde- pendently-which often causes background flicker, object drift, and abrupt scene changes- our method explicitly reconstructs and maintains a unified 3D world along the entire sequence. We jointly predict the video stream and point-map stream under an explicit camera representation, so every generated frame is geometrically aligned to an evolving scene state; this yields stable, immersive first-person experiences over long autoregressive roll-outs. (ii) Accurate and fine-grained first-person control. Existing methods commonly rely on single, entangled pose/control vector and therefore cannot model the different roles of head, hands, and feet in egocentric interaction. By employing part- nurt-disentangled motion injection, our system exposes independent, precise controls for head orientation (viewpoint/world alignment), hand motion (object interaction), and foot placement (navigation realism), enabling faithful user-driven control and correct coupling between the actor and the rendered environment. (iii) Efficient and real-time generation. Rather than invoking heavy 3D encoders at inference, we align modalities in the latent space using lightweight adapters inside the scene-frame reconstruction pathway, and we deploy a distilled backbone for decoding. This design removes bulky inference-time modules and substantially reduces compute/memory cost, enabling high-quality egocentric video at interactive rates suitable for VR/AR and gaming scenarios. (iv) Comprehensive validation. We adapt several strong vidco-diffusion baselines to the egocentric setting under a unified protocol and evaluate them on a reconstructed 100-video benchmark and an extended 24 horizon. Across perceptual (DINO/CLIP), geometric (MPJPE/MRRPE), and tempo- ral/fidelity (FVD/LPIPS) metrics, our model consistently outperforms adapted baselines, confirming that the gains come from principled world modeling and fine-grained control rather than from model size alone. Collectively, these ingredients make PlayerOne a practical, accurate, and controllable egocentric world simulator, rather than a generic short-clip generator.
Limitation analysis on our model structure. From the view of the model, one current limitation of our model is the lack of explicit physical interaction modeling between the human body and the
25

---

environment. Structurally, our framework conditions video generation on SMPL motion sequences and static point cloud reconstruction of the scene, but does not incorporate physical constraints or interaction modules such as collision detection, physics engines, or contact reasoning. Consequently, our model may generate unrealistie results, such as hands or body parts penetrating objects, or lack of proper occlusion and collision feedback. We acknowledge this limitation and consider it promising direction for future work.
More visualization results. Here we provide more simulated results in Fig. 11 and Fig. 12. In terms of first-penn first-person action alignment and world consistency, we have achieved outstanding results in both game and real-world scenarios. Additionally, we selected highly dynamic settings, such as driving scenes, where our method successfully models the world with high accuracy while maintaining excellent video fluidity. More visualization results can be referred in the submitted video.
Visualization of the scene reconstruction. This section presents a comprehensive visualization of the reconstructed scenes using our PlayerOne. As illustrated in Fig. 9, our approach adeptly reconstructs both scenes and video frames through a progressive methodology. This ensures not only inter-frame coherence but also overarching scene consistency across diverse array of scenarios. Specifically, the method seamlessly integrates temporal and spatial elements to maintain visual congruity, even in complex environments. The robustness of our technique is further reflected in its capacity to adapt to varying scene dynamics and compositions, thereby offering a reliable framework for generating high-quality, consistent video outputs. Through these visualizations, the effectiveness of our PlayerOnein achieving smooth and coherent reconstructions is clearly demonstrated, highlighting its potential applications in advanced graphical simulations and interactive environments.
Visual and quantitative difference with the "No Adapter" and "No Recon" setting. The large performance gap for 'no adapter' comes from a mismatch in the feature spaces: our point map encoder is fixed, and its latent space is not directly aligned with the VAE-encoded video latent space. When these are simply concatenated without a dedicated adapter, the model struggles to reconcile the different statistics and semantics between the two types of latents, resulting in degraded video quality and scene breakdown. Our adapter module learns to map point map features into the appropriate latent space for effective fusion with video latents, thus improving both fidelity and scene consistency. This design is crucial for effective information transfer between geometry and appearance. For 'no recon', it is primarily due two reasons:
Strong Pretraining: Our model is first pretrained on a large-scale egocentric video dataset. This pretraining stage may equips the model with weak ability to model scene dynamics and maintain certain level of spatial consistency, even before introducing explicit scene- frame reconstruction.
Lack of Direct Metrics: Most commonly used metrics (c.g.. DINO, CLIP, FVD. LPIPS) are not designed to specifically measure scene (world) consistency across frames. These metrics primarily evaluate frame-wise visual similarity, semanties, or overall diversity, but do not directly capture temporal geometric consistency, which is the core advantage of our reconstruction module.
Definition of evaluation criteria. For user study, we evaluate the video quality from four views:
Quality: Defined as the overall visual harmony and realism of the video, regardless of whether the generated content is perfectly true to the input. Annotators were asked: "Does the generated video look natural, coherent, and free from jarring artifacts?"'
Smoothness: Measures how consistently the motion flows across frames, without abrupt transitions or temporal artifacts. Annotators were prompted: "Are the motions and camera transitions fluid and continuous throughout the vidco?"
Fidelity: Refers to the preservation of the subject's appearance (identity, clothing, back- ground) and the absence of distortions glitches. Annotators were asked: "Does the person look like the initial frame and is there minimal distortion?"
Alignment: Defined as how closely the generated motion matches the intended action described in the motion condition. For this criterion, annotators viewed both the generated
26

---

video and the motion condition signal (stick-fige -fiuure pose sequence) side-by side for all 100 videos, and rated: "Does the generated video accurately follow the given pose/motion signal in timing, type of action, and spatial positioning?"
B Broader Impact
The proposed PlayerOne for video generation, designed to facilitate freeform human motion control within environments created from user-provided images while producing world-consistent videos, demonstrates considerable potential across diverse domains. It is particularly adept at generating engaging and dynamic educational content, thereby fostering experiential learning through interactive simulations. Moreover, the model optimizes the production of high-quality, consistent visual content for films, television, and online media, dramatically reducing both production time and costs. It also enables the creation of interactive narratives, allowing users to influence the storyline through their interactions within the generated environments, thus enhancing user engagement and narrative immersion. Beyond these applications, the world model serves as a valuable tool for research on human behavior and interactions within controlled virtual settings, offering insights for fields interaction. By integrating these capabilities,
such as psychology, sociology, and the proposed world model only amplifies existing applications but also paves the way for novel research and development, significantly contributing to technological progress and societal advancement.
C Limitations & Discussion
While significant strides have been made in egocentric interaction and coherent world modeling, certain limitations persist. Despite the compelling outcomes, the performance in game scenarios is somewhat diminished compared to realistic scenarios, likely due to the disproportionate amount of realistic training data available. Moreover, in highly dynamic scenes, predictions may falter, reflecting the constraints inherent in the current base model. Future research endeavors could potentially overcome these challenges by investigating novel action representations, incorporating an expanded dataset for game scenarios, and adopting
robust base model.
D License of assets
Datasets (Apache 2.0 License) Nymeria [21]/FT-HID (iieg [18] (Creative Com- mons Attribution 4.0 International), EgoExo4D [8//Egovid55M [33](MIT License).
Codes The official repository of Aether [28] (MIT License), the official repository of Cosmos [1] (Apache 2.0 License), the official repository of Wan2.1 [30] (Apache 2.0 License).
27


Reviews:
[{"review_id": "OlBpCjpJiu", "reviewer": "Reviewer_jaf9", "summary": "This work proposes an egocentric realistic world simulator that takes a scene image and human motion as input to generate corresponding egocentric videos. The core of the method is a novel video diffusion model incorporating a Part-Disentangled Motion Injection module, which separately encodes motion signals from different body parts to guide video generation. Additionally, a Scene-Frame Reconstruction mechanism is introduced to maintain scene consistency. For training, the paper designs a pipeline to extract motion-video pairs from existing egocentric-exocentric video datasets and further leverage other egocentric datasets to train the video generation model.", "questions": "See weakness.", "limitations": "The generated videos still exhibit noticeable artifacts, particularly during fast or complex motion. Additionally, the work does not explore the potential benefits of incorporating more images or scene information as input to the model.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. This paper presents a novel video diffusion model featuring an innovative module, along with a well-designed dataset collection and training pipeline. The writing is clear and well-structured.\n\n2. The experiments demonstrate significant improvements over the baseline, and the ablation studies effectively highlight the contribution of each component.\n\nWeaknesses and Questions:\n\n1. Could you elaborate on the decision to use 3 body parts? Why not 5 or 6, which might encode more detailed information about body movement?\n\n2. In Figure 7, what causes the failure in the second motion condition?\n\n3. Regarding the visualization results involving interactions with other people or animals—are there examples of actions beyond handshakes or basic interactions?", "was_revised": true, "final_justification": "The rebuttal satisfactorily addressed my concerns about this paper. I would like to maintain my rating of accept.", "rebuttal": "Thank you for the valuable feedback. We appreciate your insights and suggestions. Below, we address the individual points raised and clarify relevant aspects of our work.\n\n---\n\n**Q1.Why use 3 body parts, not more?**\n\n**A1:** Thank you for your question. We choose to divide the human body into three parts—head, hands, and body/feet—based on the following considerations:\n\n- **Empirical Results:** As shown in our ablation studies below, using fewer than three partitions (such as treating the whole body as a single entity) results in insufficient precision in aligning generated motions, especially for actions involving complex hand or head movements. Conversely, dividing into more than three parts provides negligible improvement but increases computational cost and memory requirements.\n\n\n| Part-Number | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|:-----------:|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| \"+Pretrain&PMI\"-1           | 58.0           | 76.3           | 235.12    | 212.53    | 43.9     | 279.41   | 0.1060    |\n| \"+Pretrain&PMI\"-2           | 60.5           | 79.1           | 185.00    | 190.30    | 46.2     | 263.00   | 0.0950    |\n| \"+Pretrain&PMI\"-3           | 62.5           | 81.3           | 156.76    | 175.18    | 48.3     | 245.72   | 0.0839    |\n| \"+Pretrain&PMI\"-4           | 62.7           | 81.4           | 155.10    | 174.80    | 48.4     | 244.60   | 0.0830    |\n| \"+Pretrain&PMI\"-5           | 62.8           | 81.5           | 154.80    | 174.50    | 48.4     | 244.10   | 0.0828    |\n| \"+Pretrain&PMI\"-6           | 63.0           | 81.8           | 152.62    | 172.90    | 48.6     | 243.82   | 0.0819    |\n\n- **Intuitive Reasoning:** This three-part division aligns naturally with the semantic structure of typical human interactions: **head movements primarily control viewpoint direction, hands execute detailed interactions, and the body/feet handle overall positional and movement dynamics**.\n\n- **Why separate processing contributes to precise alignment:** Splitting the latent representation into three separate parts—head, hands, and body/feet—enables the model to more effectively capture the unique and often semi-independent motion characteristics of each region. Human body motion is inherently high-dimensional and highly articulated, with different parts frequently moving independently or exhibiting distinct dynamics. Encoding all motion information into a single, entangled latent can cause fine-grained cues—especially from subtle or fast-moving regions like hands or head—to be overshadowed by larger body movements, making precise alignment difficult. By disentangling the motion into part-specific latents, each body region’s movement can be independently modeled and aligned with the corresponding input. This modular approach allows the model to better capture local nuances, improves compositionality, and enhances control over complex motions, resulting in higher motion fidelity and visual quality. \n\nHence, our chosen partition strikes an optimal balance between computational efficiency, precision, and intuitive semantic clarity.\n\n---\n\n**Q2.Failure of the second motion condition in Fig.7.**\n\n**A2:** Thank you for bringing this to our attention. We carefully reviewed Figure 7 multiple times but could not identify a clear failure case in the second motion condition on either the left or right side. We would greatly appreciate it if the reviewer could clarify which specific motion sequence or frame is being referred to, so that we can better understand the issue and address it appropriately in our revision. We are committed to improving our work and welcome any further details or suggestions.\n\n\n---\n\n**Q3.Examples of actions beyond handshakes or basic interactions.**\n\n**A3:** Thank you for your question. While our current demo primarily showcases simple interactions with people or animals—such as handshakes, high-fives, or lifting a dog(in 1:03 of our demo)—our method is not limited to these basic actions.\n\n**(1) Diverse Interaction Scenarios in Benchmark:**  \nOur benchmark includes a broad range of interaction scenarios involving people and animals. Typical examples include:\n- **Passing objects to another person (i.e., passing basketball/football)**\n- **Dancing with someone**\n- **Petting or playing with animals (i.e., riding a horse and spinning in circles with a dog)**\n- **Feeding animals**\n\nOn these diverse and realistic human-human and human-animal interaction cases, our method achieves a significant performance advantage compared to existing baselines. This demonstrates the strong generalization capability and robustness of our approach. We will highlight these benchmark results and add more qualitative and quantitative examples in the revised version.\n\n**(2) Complex and Challenging Environmental Interactions:**  \nBeyond interactions with people and animals, our method also handles a wide variety of complex actions involving the environment. For example:\n- **Figure 7:** Challenging actions such as diving\n- **Figure 5:** Climbing over a wall and kicking a ball\n- **Figure 8:** Precise actions like picking up milk from a cabinet\n\nOur demo video provides further examples, including:\n- **Driving by operating the steering wheel** (see 0:32 and 0:38)\n- **Riding a bicycle, swimming, climbing over a wall (1:03), and swinging a sword (0:47)**\n\n**(3) Ongoing Enrichment:**  \nWe are committed to further expanding the diversity and complexity of our demonstrations. In the revised version, we will provide additional visualizations featuring even more complex and varied interactions, including richer cases involving people and animals, to better illustrate the generalization and flexibility of our approach.\n\n---\n\n**Q4.Noticeable artifacts, particularly during fast or complex motion.**\n\n**A4:** Thank you for your valuable feedback. The artifacts are mainly due to the limitations of the current adopted base model and replacing it with a stronger backbone can alleviate such issues. Specifically, when using the **Wan 2.1 14B** as our backbone, we can achieve better performance:\n\n| Model | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|:-----------:|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| 1.3B        |    67.8        |    88.2        | 127.16    | 151.62    |  52.6    | 226.12   | 0.0663    |\n| 14B         |    69.1        |    90.7        | 109.34    | 124.20    |  56.0    | 182.55   | 0.0515    |\n\nThus if more powerful backbone architectures become available, we will replace our current backbone in future work. This upgrade could further reduce potential artifacts and improve the fidelity of the generated results.\n\n---\n\n**Q5:Potential benefits of incorporating more images or scene information as input to the model.**\n\n**A5:** We would like to clarify that our current framework is intentionally designed to operate with only a single egocentric first frame and related human motion provided by the user. This design choice was made to simplify the pipeline and ensure a lightweight, user-friendly experience, but it does limit our ability to directly utilize additional images or scene inputs in the present version.\n\nWe agree that introducing richer scene information—such as additional frames, multi-view images, or more detailed environmental cues—could potentially improve the temporal consistency and visual realism of the generated video. Exploring how to effectively integrate such supplementary scene inputs is a promising direction for future work, and we appreciate the reviewer’s suggestion. In future versions, we plan to extend our framework to support and benefit from richer scene context, and we expect this will further enhance the fidelity and robustness of egocentric video generation."}, {"review_id": "z4N50OUDxg", "reviewer": "Reviewer_ktBL", "summary": "This paper introduces PlayerOne, a novel framework for egocentric world simulation that generates immersive videos aligned with real-world human motion. Given a single egocentric image and an exocentrically captured motion sequence, the system constructs a consistent 4D virtual world, enabling freeform and fine-grained human action control in the generated video. The proposed architecture builds upon a video diffusion transformer, with key innovations including a part-disentangled motion injection module for precise body part control and a joint scene-frame reconstruction framework to enforce long-term scene consistency. To address the lack of suitable training data, the authors design an automated pipeline to extract high-quality motion-video pairs from egocentric-exocentric datasets, combined with a coarse-to-fine training strategy leveraging large-scale egocentric text-video data. The system demonstrates strong performance across multiple metrics and outperforms relevant baselines such as Aether and Cosmos.", "questions": "- The length of video generated by the model in one pass, how long videos are generated through concatenation, and whether the point cloud encoding scheme can ensure consistency across the generated video sequences.\n\n- The paper claims real-time performance post-distillation, but only FPS is reported. Could the authors clarify the actual latency between motion input and video output? This would help gauge deployment feasibility in interactive settings.", "limitations": "The paper notes weaker performance in game-like scenarios but does not deeply investigate or visualize these limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "##  Strengths\n- Novelty \n\nThe idea of simulating egocentric immersive worlds controlled by real human motion is novel and underexplored.\n- Technical Contributions\n\nThe paper presents a comprehensive automatic data construction pipeline, a pragmatic contribution to the field.\n\n- Clarity\n\nThe paper is well-organized.\n\n## Weaknesses\n- Consistency in Long-Video Generation\n\nWhether using point clouds as implicit encodings to constrain the consistency of generated results is relatively weak. For the egocentric simulator task described in the paper, the user is expected to freely explore within the generated scene, which requires the model’s capability to generate long-term video sequences. It seems that the current point cloud encoding approach may not adequately ensure consistency across multiple concatenations when generating long videos.\n\n- Limited Discussion of Failure Cases:\n\nThe paper notes weaker performance in game-like scenarios but does not deeply investigate or visualize these limitations.", "was_revised": true, "final_justification": "borderline accept", "rebuttal": "Thank you for the valuable feedback. We appreciate the reviewers’ insights and suggestions. Below, we address the individual points raised and clarify relevant aspects of our work.\n\n---\n\n**Q1.Consistency in long-video generation and whether the point cloud encoding scheme can ensure consistency across the generated video sequences?**\n\n**A1:** Thank you for your comment. We would like to clarify that our method is **not limited to generating only short (e.g., 6-second) videos.** After distillation with Causvid, our model can generate long videos (i.e., more than 30s) in an auto-regressive fashion, producing each frame sequentially conditioned on previous outputs on-the-fly, rather than simply concatenating multiple short clips. This sequential generation, together with our current point cloud encoding approach, allows us to maintain strong scene consistency throughout long video sequences. We acknowledge that the 6-second horizon in the demo was an oversight, and we will include longer video generations in the revised version.\n\n\nTo address your question directly, we have extended our benchmark by increasing the video length to 24 seconds and evaluated our method under this longer horizon. The results below show that **our model maintains comparable performance and visual quality, demonstrating its ability to generate consistent long-term video sequences.**\n\n\n| Method                                         | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|------------------------------------------------|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| Baseline                                       |     48.38      |     60.25      | 418.66    | 385.23    |  32.82   | 443.77   | 0.1687    |\n| + Pretrain                                     |     53.74      |     69.30      | 293.45    | 270.84    |  38.19   | 341.66   | 0.1331    |\n| + Pretrain&ControlNet                          |     54.61      |     70.98      | 275.18    | 256.24    |  39.08   | 325.13   | 0.1279    |\n| + Pretrain&Entangled                           |     55.21      |     71.82      | 268.22    | 249.91    |  40.08   | 317.42   | 0.1229    |\n| + Pretrain&PMI (No Camera)                     |     57.36      |     74.48      | 210.98    | 229.54    |  42.62   | 286.93   | 0.1063    |\n| + Pretrain&PMI                                 |     58.90      |     76.62      | 178.44    | 204.22    |  44.98   | 275.47   | 0.0955    |\n| + Pretrain&PMI&Filtering                       |     61.13      |     78.94      | 162.10    | 190.07    |  46.27   | 258.54   | 0.0898    |\n| + Pretrain&PMI&Filtering&Recon (No Adapter)    |     59.78      |     76.02      | 200.04    | 209.36    |  44.22   | 272.60   | 0.1077    |\n| + Pretrain&PMI&Filtering&Recon (DUSt3R)        |     63.23      |     82.62      | 148.08    | 178.91    |  49.13   | 260.95   | 0.0837    |\n| PlayerOne (ours)                               |     63.92      |     84.03      | 145.26    | 173.35    |  49.70   | 257.62   | 0.0816    |\n\n\n\n---\n\n**Q2.Limited Discussion of Failure Cases.**\n\n**A2:** Thank you for your suggestion. We acknowledge that our current manuscript provides limited analysis and visualization of limitations in game-like scenarios. This is mainly due to the relatively small amount of available training data from such domains. As a result, our model is more likely to exhibit issues such as inaccurate hand alignment with the given motion conditions or background blurring during rapid movements in game environments.\n\nTo better illustrate this, we separately evaluated our model on data from game-like and real-world domains. The results are shown below:\n\n| Domain | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|:------:|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| Game   |    65.4        |    85.7        | 136.80    | 160.41    |  50.9    | 248.95   | 0.0705    |\n| Real   |  67.8 | 88.2 | 127.16 | 151.62 | 52.6 | 226.12 | 0.0663|\n\nWe will include more visualizations and an in-depth discussion of these failure cases in the revised version to provide a more comprehensive and transparent evaluation of our method’s limitations in different domains.\n\n---\n\n**Q3.Clarify the actual latency between motion input and video output.**\n\n**A3:** Thank you for highlighting this important point. After distillation, the measured end-to-end inference latency is approximately 119 milliseconds per generated-frame, corresponding to about 8.4 FPS. This latency is measured from the moment the motion input is received to the generation of the corresponding video frame, and includes all major processing steps. We will include more comprehensive inference performance metrics and latency breakdowns in the revised version to better illustrate the real-time capability and deployment feasibility of our model in interactive scenarios."}, {"review_id": "rDYZzoO04y", "reviewer": "Reviewer_cVUd", "summary": "This paper introduces PlayerOne,  a novel world simulator that allows users to control human motion in real-time within dynamic, user-generated environments. By focusing on egocentric video generation and precise motion control, it creates interactive simulations with high visual and motion consistency. PlayerOne excels in both motion alignment and environmental interaction across diverse scenarios, outperforming previous models.", "questions": "Based on the weaknesses above, the authors are encouraged to revise this article in the following aspects:\n1) Analyze the failure model of the learned model and provide some corresponding visualizations.\n2) Add more discussion about the temporal horizon of generation. Is generating longer videos necessary for this task? How the result  and visualization quality would be if we extend the inference horizon beyond the one in training (6s)? Will extending the horizon be one of a future directions?\n3) About the distillation. Please consider adding more comparisons to demonstrate how would the distillation stage affect the model performance.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- This paper is well-motivated with a clear methodology and good presentation.\n- The key innovation of this paper is its task setting: generate high-fidelity, dynamic, ego-centric worlds where users can flexibly interact with. \n- Adequate qualitative visualizations in both the main paper (Fig 6,7,8) and the supplementary video.\n- High-quality data collection pipeline. As shown in Fig.3, the authors developed a data collection pipeline with the integration of pre-developed detection and pose estimation models and an automatic filtering scheme.\n\nWeaknesses\n- Lack of analysis for failure mode. Analyzing failure modes is a critical aspect of research in generative models to understand the weakness of the learned model in certain usages. However, such a discussion and visualization of the failure mode is missing.\n\n- Limited temporal horizon of generation. According to line 204-205 and the visualizations in the supplementary files, it seems that the proposed model can only synthesize a 6-second video sequence. It seems that current method does not support longer generation beyond the training horizon.\n\n- Insufficient details about distillation. According to line 195-196, the authors applied a distillation stage in the end of model training to improve the inference efficiency. However, details like how the distillation would affect the quality of generation and comparison between distilled and non-distilled model are not included.", "was_revised": true, "final_justification": "Accept. Technically solid with detailed rebuttal that addressed my concerns.", "rebuttal": "Thank you for the valuable feedback. We appreciate your insights and suggestions. Below, we address the individual points raised and clarify relevant aspects of our work.\n\n---\n\n**Q1.Lack of analysis for failure mode.**\n\n**A1:** Thank you for your suggestion. We acknowledge that our current manuscript provides limited analysis in this area. In particular, due to the relatively small amount of training data from game scenarios, our model may more likely exhibit results with slightly inaccurate hand alignment with the given condition or background blurring. To better illustrate this, we separately evaluated our model on data from game-like and real-world domains. The results are shown below:\n\n| Domain | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|:------:|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| Game   |    65.4        |    85.7        | 136.80    | 160.41    |  50.9    | 248.95   | 0.0705    |\n| Real   |  67.8 | 88.2 | 127.16 | 151.62 | 52.6 | 226.12 | 0.0663|\n\n\nBesides, our models may generate results with noticeable artifacts, particularly during extremely fast or complex motion, which can be observed in the provided cases (e.g., jumping from a high place in 1:03 of the demo). These artifacts are mainly due to **the limitations of the current adopted base model** and replacing it with a stronger backbone can alleviate such issues. Specifically, when using the Wan 2.1 14B as our backbone, we can achieve better performance:\n\n| Model | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|:-----------:|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| 1.3B        |    67.8        |    88.2        | 127.16    | 151.62    |  52.6    | 226.12   | 0.0663    |\n| 14B         |    69.1        |    90.7        | 109.34    | 124.20    |  56.0    | 182.55   | 0.0515    |\n\n\nHere we also provide the **limitation analysis on our model structure**. From the view of the model, one current limitation of our model is the lack of explicit physical interaction modeling between the human body and the environment. Structurally, our framework conditions video generation on SMPL motion sequences and a static point cloud reconstruction of the scene, but does not incorporate physical constraints or interaction modules such as collision detection, physics engines, or contact reasoning. Consequently, our model may generate unrealistic results, such as hands or body parts penetrating objects, or a lack of proper occlusion and collision feedback. We acknowledge this limitation and consider it a promising direction for future work. As recommended, we will substantially expand the Discussion/Limitation section and the related visualization of the failure cases in the revision, providing a more thorough analysis of both model and data limitations.\n\n---\n\n**Q2.Limited temporal horizon of generation and related discussion.**\n\n**A2:** Thank you for pointing out this issue. While our original experiments and visualizations focused on 6-second video sequences, this does not reflect a fundamental limitation of our method. After distillation with Causvid, our approach generates video in an autoregressive manner, allowing for the synthesis of arbitrarily long video sequences. The 6-second horizon in the demo was an oversight, and we will include longer video generations in the revised version.\n\nTo address your question directly, we have extended our benchmark by increasing the video length to 24 seconds and evaluated our method under this longer horizon. The results below show that **our model maintains comparable performance and visual quality, demonstrating its ability to generate consistent long-term video sequences.**\n\n\n| Method                                         | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|------------------------------------------------|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| Baseline                                       |     48.38      |     60.25      | 418.66    | 385.23    |  32.82   | 443.77   | 0.1687    |\n| + Pretrain                                     |     53.74      |     69.30      | 293.45    | 270.84    |  38.19   | 341.66   | 0.1331    |\n| + Pretrain&ControlNet                          |     54.61      |     70.98      | 275.18    | 256.24    |  39.08   | 325.13   | 0.1279    |\n| + Pretrain&Entangled                           |     55.21      |     71.82      | 268.22    | 249.91    |  40.08   | 317.42   | 0.1229    |\n| + Pretrain&PMI (No Camera)                     |     57.36      |     74.48      | 210.98    | 229.54    |  42.62   | 286.93   | 0.1063    |\n| + Pretrain&PMI                                 |     58.90      |     76.62      | 178.44    | 204.22    |  44.98   | 275.47   | 0.0955    |\n| + Pretrain&PMI&Filtering                       |     61.13      |     78.94      | 162.10    | 190.07    |  46.27   | 258.54   | 0.0898    |\n| + Pretrain&PMI&Filtering&Recon (No Adapter)    |     59.78      |     76.02      | 200.04    | 209.36    |  44.22   | 272.60   | 0.1077    |\n| + Pretrain&PMI&Filtering&Recon (DUSt3R)        |     63.23      |     82.62      | 148.08    | 178.91    |  49.13   | 260.95   | 0.0837    |\n| PlayerOne (ours)                               |     63.92      |     84.03      | 145.26    | 173.35    |  49.70   | 257.62   | 0.0816    |\n\nWe believe that generating longer videos is crucial for this task for several reasons: \n- **(1)** it enables modeling of more complex and realistic user interactions. \n- **(2)** it allows thorough evaluation of temporal coherence and scene consistency. \n- **(3)** it better supports applications such as virtual reality and embodied AI, where prolonged, immersive experiences are essential. \n\nWhile our implementation uses Causvid for long video generation, there are many emerging approaches[1-3] focused on long-term consistency and video synthesis, and we agree that pushing the boundaries of temporal horizon will be an important direction for future research.\n\n[1] Lin, J., et al. \"Long-Context Autoregressive Video Modeling with Next-Frame Prediction.\" arXiv preprint arXiv:2403.09603, 2024.\n\n[2] Yu, X., et al. \"Long-context State-space Video World Models.\" arXiv preprint arXiv:2405.11590, 2024.\n\n[3] Chen, H., et al. \"FreeLong++: Training-Free Long Video Generation via Multi-band Spectral Fusion.\" arXiv preprint arXiv:2406.03855, 2024.\n\n---\n\n**Q3.Insufficient details about distillation.** \n\n**A3:** Thank you for your question regarding the distillation process. We strictly follow the Causvid distillation procedure in our implementation. During the distillation stage, we use the entire SMPL-Video paired dataset from the second phase of training. Specifically, we first generate 8,000 ODE pairs and train the student model for 30,000 iterations using the AdamW optimizer with a learning rate of 5 × 10⁻⁶. We then continue training with our asymmetric DMD loss for an additional 30,000 iterations, using AdamW with a learning rate of 2 × 10⁻⁶. A guidance scale of 3.5 is used, and we adopt the two time-scale update rule from DMD2 with a ratio of 5. The entire distillation process takes about 5 days on 32 A800 GPUs.\n\nRegarding quality, the distillation stage is mainly aimed at improving inference efficiency, but we observe that **the generation quality is largely maintained compared to the non-distilled model.** In some cases, the distilled model even achieves slightly better temporal consistency. The comparision between the non-distilled model and distilled one for 6s video generation is shown below:\n\n| Model       | DINO-Score (↑) | CLIP-Score (↑) | MPJPE (↓) | MRRPE (↓) | PSNR (↑) | FVD (↓) | LPIPS (↓) |\n|:-----------:|:--------------:|:--------------:|:---------:|:---------:|:--------:|:--------:|:---------:|\n| Distilled   |    67.8        |    88.2        | 127.16    | 151.62    |  52.6    | 226.12   | 0.0663    |\n| Non-distilled      |    68.2        |    88.7        | 125.20    | 149.40    |  53.0    | 221.85   | 0.0647    |\n\nWe will include a detailed comparison and qualitative side-by-side visualizations between distilled and non-distilled models, and discuss in which scenarios quality differences may be most obvious (e.g., rare actions, very long videos) in the revised version to make this clear to readers."}, {"review_id": "S0MEOITM3r", "reviewer": "Reviewer_WmZe", "summary": "The paper presents a method called PlayerOne that takes an egocentric image of a scene together with the exocentric video of a person doing some body motions, and provides as output a synthesized video where the actions from the person in the exo video are mapped into the ego frame.  Both the body movements (typically hands and forearms) plus the surrounding scene are aimed to appear in sync in the output video.  The method is a diffusion model that decomposes the body motion into body regions/parts, represents the camera rotation separately, and also inputs a scene representation for the ego frame using a point map encoder.  The DiT diffusion transformer model learns to condition the output video on these factors.   Results are quantified by the alignment between the output video and its ground truth text descriptions, as well as the hand pose error against the ground truth video.  Baseline models are Cosmos and Aether.  Qualitative examples are shown in the paper and supp video.", "questions": "Please see above.", "limitations": "-\tPlease see questions about clarity above, whose answers could help me better state the limitations.\n-\tThe authors comment on limitations at the end, though I think they could elaborate on aspects of the model design that may cause issues (not just the data shortage).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- Interesting problem definition, which seems to offer a new twist on conditional video generation.  The approach could be used by artists to guide the ego content in video games or movies to a target interaction pose.  (The authors could spend a few sentences somewhere to motivate where they see this being most useful though.)\n\n-\tApproach seems well designed.\n\n-\tResults are nice looking.\n\n-\tWriting is overall good and flows well with informative figures (but see note below on details that were not clear.)\n\n- I consider this between BA and A, pending questions below being clarified.\n\nWeaknesses:\n\n-\tMotivation of technical design was not always apparent:\n\no\tFor example, why is the camera translation removed?  This seems like a limitation on which body motions can be realistically received as input and then shown in the output.  (around L140)  \n\no\tIt is not clear how the scene frame alignment step works.  How does the adapter have the signal to align the point map latents with video latents?  (L158)  What enforces the alignment, what is the loss.\n\n-\tThe related work coverage seems fine, but I would have liked more explicit and precise call outs about what is distinct and what is important+distinct in this work.  For example, it says the existing methods “primarily” focus on virtual game scenarios and are limited to specific directional actions.  If it is only “primarily”, then what are the exceptions and how do they compare to this model?  What do the authors mean exactly by “specific directional motions”?  Do any of the existing methods also condition on human body pose or is that entirely new?  If they do, what’s insightful about how it’s done here instead?\n\n-\tL130  about adopting real-world human motions as motion conditions allowing more natural movement – but the movement is still camera (head) motion, just now we see the body parts overlayed as well.\n\n-\tTechnical novelty – choices seem reasonable but not surprising.  For me, the interesting nature of the specific setting is what is strong about the paper.\n\n-\tI don’t think the steps towards end of Sec 3 are reproducible given what’s written, e.g., about the distillation and the finetuning.\n\n-\tI could not tell from the model design whether it’s expected that the input exo body poses are “projected” to the nearest visually suitable poses in the ego scene thanks to the diffusion model?  Is this intended and is it observed?  For example, if the person is making petting motions in the exo video and they would not align well with the way the dog is posed in the ego frame, is this corrected?  Or should the input poses be very precise to match the target scene (and is this why the self-taken videos are used for qual results?)\n\n-\tIn the results:\n\no\tIt is not explained why the Cosmos and Aether baselines were chosen and why they are complete set of baselines to support the paper’s claims.\n\no\tWhy is Wanx 2.1 the right base model?  How does that interact with the choice of baselines as well in terms of fairness, apples to apples.\n\no\tBenchmark samples from Nymeria – why only from this datatset?  Would seem more complete to sample unseen examples across all the datasets.\n\no\tThe exo videos in the supp seem to be taken by the authors, not from Nymeria.  Unclear how the qual and quant results match up.\n\no\tWhere do the text descriptions come from that are used to evaluate ground truth?\n\no\tWhat about arms or any other visible body parts for the MPJPE metric, vs. just hands?\n\no\tWhat if the hands are not visible in the frame of the proposed method’s output?  Hopefully those instances are not dropped from evaluation.\n\nMinor:\n\n-\tFragment L89 “Among those works…”\n\n-\tIn the supp video it would help to show the ego frame input for each one.  Also it would be nicer to show one video at a time throughout since a person can’t watch multiple videos at once.", "was_revised": true, "final_justification": "The authors' response addresses my questions, and all the clarifications should be made in a final version.", "rebuttal": "Thanks for the valuable feedback. We appreciate your insights and suggestions. Below, we address the points raised and clarify the related issues.\n\n---\n\n**Q1.Why is the camera translation removed.**\n**A1:** Camera translation is removed because SMPL head-parameters only provide rotation,not translation.When converting SMPL head params to camera extrinsics,only rotation is available—so translation can't be included.This matches the SMPL definition and keeps the camera centered at the head in the egocentric-frame.\n\n**Q2.How the scene-frame alignment step works.How does the adapter have the signal to align the point-map latents with video latents?(L158).**\n**A2:** Thank you for your question.In our framework,the adapter maps point-map latents into the same latent space as video latents.We do not use an explicit alignment loss;instead,we follow prior works like[1-2],which show that **simple linear or convolutional modules can effectively project point cloud latents to the video latent space.** This lets the model leverage spatial structure from point-maps while conditioning video generation efficiently.Thus,alignment is achieved through learned mapping and implicit end-to-end training.\n\n[1]Wu,T.,et al.Video World Models with Long-term Spatial Memory.\n[2]Aether.AETHER: Geometric-Aware Unified World Modeling.\n\n**Q3.More explicit and precise call outs about what is distinct and what is important+distinct in this work.** \n**A3:** Thank you for your question.We acknowledge our previous wording may have been unclear on this point.To our knowledge,no prior work at submission used human body pose as the main condition for egocentric video generation in real-world scenarios.A recent related work[1]also uses human pose,but their method does not ensure scene consistency or accurate action-pose alignment;their model also struggles to generalize beyond its training domain.\n\nBy “specific directional motions,” we mean prior models only support basic navigation(forward,backward,turning),without enabling complex,VR-like or full-body actions such as picking up objects or interacting naturally with NPCs. These limits are due to both the restricted motion in game datasets and the lack of modules for fine-grained action alignment and scene consistency.In contrast,our method uses full-body motion as condition,explicitly addresses part-level action alignment and scene consistency,and generalizes well to diverse,realistic settings.\n\n[1]Bai,Y.,et al.Whole-Body Conditioned Egocentric Video Prediction.\n\n**Q4.L130:Movement is still camera(head) motion.**\n**A4:** We'd like to clarify while the output video is the first-person(head/camera)view,our method uses real-world,full-body human motion as driving condition—not just head pose—so the generated video captures both camera motion and detailed body part movements in sync.This results in more natural,immersive egocentric movement(validated by full-body evaluation results in A10).\n\n**Q5.Technical novelty—the interesting nature of the specific setting.**\n**A5:** We appreciate your recognition of our problem setting and application scenario.Although our technical choices build on established methods for robustness,our main contribution is to introduce and solve a highly interactive egocentric world simulation task—supporting flexible,full-body,realistic user exploration,which prior works have not addressed.We will further highlight the significance and potential of this scenario in the revision.\n\n**Q6.Reproducibility for the steps towards end of Sec 3(e.g.,distillation and finetuning).**\n**A6:** For finetuning,we have included all necessary code in the supplement for reproducibility.For distillation,we strictly follow the Causvid protocol,which is standard in long video generation and reproducible with public tools.We will add more details about our distillation process(**Also shown in Q3-A3 of Reviewer cVUd**),including key parameters and scripts,in the revision.After acceptance,we'll release the complete codebase with full documentation for both distillation&finetuning.\n\n**Q7.Whether it’s expected input exo body poses are “projected” to the nearest visually suitable poses in the ego scene thanks to the diffusion model?Is this intended and is it observed?**\n**A7:** In training, motion poses and the first-person frame are strictly aligned. At test time, however, we observe our model shows some robustness when the input pose doesn't exactly match the ego one in target scene. In these cases, the model often adapts the motion, effectively “projecting” it to the most visually reasonable pose in the generated result, as you described. We will add visual examples of this behavior in revision.We use self-taken videos for qualitative results mainly to demonstrate we can offer immersive egocentric experiences(Sunglasses as VR headset in demo),not because strict pose-to-scene matching is required.\n\n**Q8.Why Cosmos&Aether were chosen.Why they are complete set of baselines to support the claims.**\n**A8:** Most world models differ from our setting,as they focus on specific games(e.g.,Minecraft)or tasks(e.g.,autonomous driving).Cosmos and Aether are the closest general-scene simulators to our approach.Therefore,we emphasize thorough ablation studies of our modules rather than direct comparison with domain-specific baselines(That's the reason that we place the ablation-study before the other-baseline-comparisons.).Our experiments evaluate all modules&variants,demonstrating their necessity&effectiveness.\n\n**Q9.Why is Wanx 2.1 the right base model?How does that interact with the choice of baselines as well in terms of fairness,apples-to-apples.**\n**A9:** Thanks for your question.Wanx2.1 is the best open-source video-generation model currently available,so we use it as our backbone.Our approach is not tied to Wanx—when using stronger models,performance further improves:\n|Model|DINO-Score(↑)|CLIP-Score(↑)|MPJPE(↓)|MRRPE(↓)|PSNR(↑)|FVD(↓)|LPIPS(↓)|\n|-|-|-|-|-|-|-|-|\n|CogVideox-5B|67.3|87.7|135.89|157.41|51.2|227.35|0.0724|\n|Wan1.3B|67.8|88.2|127.16|151.62|52.6|226.12|0.0663|\n|Wan14B|69.1|90.7|109.34|124.20|56.0|182.55|0.0515|\n\nWanx as backbone is independent of baseline-selection.Compared to Cosmos(7B/14B)&Aether(5B),our method achieves better results with fewer-parameters(1.3B),demonstrating both effectiveness&efficiency.\n\n**Q10.Why benchmark-samples only from Nymeria?**\n**A10:** Thanks for your suggestion.In response,we have reconstructed our evaluation benchmark to provide a more comprehensive assessment.Specifically,we selected 100 videos featuring full-body actions from the test-sets of four different datasets: **EgoExo-4D,Nymeria,FT-HID,EgoExo-Fitness.** The results,as shown below,demonstrate that our method consistently outperforms both existing approaches and various ablated variants of our method.\n\n|Method|DINO-Score(↑)|CLIP-Score(↑)|MPJPE(↓)|MRRPE(↓)|PSNR(↑)|FVD(↓)|LPIPS(↓)|\n|-|-|-|-|-|-|-|-|\n|Baseline|48.72|62.13|402.31|370.27|33.57|425.15|0.1570|\n|+Pretrain|53.81|71.29|280.51|255.63|39.11|328.41|0.1265|\n|+Pretrain&ControlNet|54.28|72.31|264.10|241.91|40.16|314.27|0.1228|\n|+Pretrain&Entangled|55.24|73.54|258.88|234.16|41.23|308.12|0.1174|\n|+Pretrain&PMI(No Camera)|57.68|76.79|201.34|215.89|43.58|278.24|0.1017|\n|+Pretrain&PMI|59.57|78.56|170.01|191.43|46.02|266.73|0.0912|\n|+Pretrain&PMI&Filtering|61.74|81.11|153.17|178.64|47.37|249.33|0.0850|\n|+Pretrain&PMI&Filtering&Recon(No Adapter)|60.13|78.67|191.22|196.54|45.29|263.18|0.1023|\n|+Pretrain&PMI&Filtering&Recon(DUSt3R)|64.02|84.41|140.16|167.23|50.25|251.38|0.0788|\n|PlayerOne(ours)|64.87|85.79|137.91|162.74|50.72|247.97|0.0761|\n\n|Method|DINO-Score(↑)|CLIP-Score(↑)|MPJPE(↓)|MRRPE(↓)|PSNR(↑)|FVD(↓)|LPIPS(↓)|\n|-|-|-|-|-|-|-|-|\n|Aether-5B|37.5|63.8|418.91|435.24|37.8|400.50|0.1881|\n|Cosmos-7B|45.6|70.1|304.20|327.05|43.4|348.00|0.1617|\n|Cosmos-14B|51.3|80.0|259.17|256.19|47.5|306.85|0.1374|\n|PlayerOne(ours)|64.8|85.7|137.91|162.74|50.72|247.97|0.0761|\n\n**Q11.Unclear how qual and quant results match up.**\n**A11:** Thank you for your question.For the qualitative comparisons in our supplementary materials,we used exo videos recorded by ourselves.The main reason was to better showcase that our model can simulate **a \"virtual reality\"experience**.To highlight this,the subjects in our videos **wore sunglasses to mimic the appearance of VR headsets**.Please note that for all quantitative evaluations and benchmarking,we strictly relied on the official test sets from Nymeria and other datasets.We will make this distinction clearer in the revised version to avoid any confusion between demonstration videos and quantitative results.\n\n**Q12.Where do text descriptions come from that are used to evaluate ground truth?**\n**A12:** For the comparison methods,the text descriptions were generated using the Qwen2.5-VL model based on the corresponding ground truth video frames(Line210-211 in the manuscript).\n\n**Q13.What about arms or any other visible body parts for the MPJPE metric,vs.just hands? What if the hands are not visible in the frame of the proposed method’s output?.**\n**A13:** In most cases from the test sets,only the hands are visible in the first-person view videos,so our previous evaluation of motion-related metrics(such as MPJPE) primarily focused on the hands.Following your suggestion,we have reconstructed our benchmark by selecting cases where the full body is visible in the frame for comprehensive evaluation.The results in A10 show that our method still consistently outperforms existing approaches across all metrics.We will revise the manuscript with these results to provide more comprehensive demonstration.\n\n**Q14.Show ego-frame input for each one and one-video at a time throughout and typos.**\n**A14:** Thanks for your suggestion.We appreciate your feedback and will update related text and demo video in revision.\n\n**Q15.Aspects of model design that may cause issues.**\n**A15:** Please refer to the last paragraph of Q1-A1 for Reviewer cVUd(Due to the space-limit)."}, {"review_id": "vZZShMfYCx", "reviewer": "Reviewer_X28J", "summary": "The paper introduces PlayOne, an egocentric world model that offers fine-grained control to generate egocentric videos conditioned on motion extracted from exocentric videos.\n\nThe paper proposes a novel part-disentangled motion injection method as its main contribution. The part-disentangled motion injection scheme encodes motion into three separate sets of body parts to obtain three separate latent representations. The authors claim this leads to improved video alignment.", "questions": "Will the dataset be released?\n\nAll other short comments and Questions:\n\n1 - not the first egocentric simulator: GEM, EgoSim, EgoGen, ...\n\n16-18: sounds very LLM written, many words no real meaning, please be more direct what does this work offer the community? Why is it the first work?\n\n20-28: paragraph should be adapted for better understanding of the authors problem. How do the authors define a world model? \n\n25: citations needed or better explanation why these advancements help\n\n26: predictive modeling not explained yet, what is meant by that?\n\n27-28: again not clear why almost these rather novel and less established world models are critical infrastructure. citations or explanation needed\n\n29: unclear what this topic refers to\n\n30: there is a lot of works on realistic scenarios, e.g. GEM, VISTA\n\n31: why is that which works limit users in that sense?\n\n32: GEM seems to allow arbitrary poses drawn as skelton\n\n39-60: very hard to follow. the problem statement is still unclear, and also the research gap is not yet clear to me. Then i get swamped with many detailed information that are very hard to follow.\n\n61: still unclear why its the first, there are some before, so why is yours the first, maybe rather mention on how you improve on previous works\n\n67: do you release the dataset? do you release training code?\n\n106: is k number of frames?\n\n99: the method section can benefit a lot by clearly stating the problem statement. what are the inputs, what are the outputs\n\n109-120: very unclear how this paragraph is connected to its context, e.g. the previous paragraph\n\n109-110 the goal of the core modules is unclear and are introduced without motivation\n\n112: VAE is suddenly introduced. What does it encode? How is related to the human motion sequence? How is it connected to the 3D motion encoder\n\n114: why are the motion latents teh same shape as images? How is that ensured? Why is that high resolution/dimensional encoding needed?\n\n115: What is a camera sequence?\n\n117: how if a 4d scene represented?\n\n118: what is a point map sequence?\n\nEq 1: why is t in U(0,1) but not U(1, T)\n\n121: it is unclear where alpha^2 + delta^2 = 1 comes from\n\n130: where is that empirical analysis?\n\n132: why is splitting the latent into 3 separate latents allowing for better motion alignment? Are those body part poses not highly correlated and thus capturing it in a single latent should be superior?\n\n137-138: completely unclear why separate processing ensures this claim\n\n143: maybe write the axis angles different, i have never seen thetas (angles) being used to represent the rotation axis, seem like you are using euler angles and not a axis representation.\n\n152: still unclear how the 4D scene is represtanted\n\n154: CUT3r not really itnroduced before\n\n158: how is the alignment happening\n\n163-165: how is scene consistency insured during inference if the pointsmaps dont get updated?\n\n169: incorrect. There is nymeria, and also egoExo4D has sequences of egocentric video with ego motion gt, Also there is the EgoSim dataset\n\nMethod: how is the exoview used besides for motion extraction?\n\n4.2 why do the ablations studies come before the main results?\n\n232: do you ensure that the computational complexity of a joint motion encoder is the same as the 3 individual encoders? What are teh dimensions of the latents?\n\n4.3) why are other sota models like GEM not included?\n\nDiscussion and Limitations very short and not detailed enough.\n\nReferences:\n\nGEM - https://arxiv.org/html/2412.11198v1\nVISTA - https://arxiv.org/pdf/2405.17398\nEgoSim - https://proceedings.neurips.cc/paper_files/paper/2024/file/c1017d0a006d31dfbfd4cf1e9189d747-Paper-Datasets_and_Benchmarks_Track.pdf\nEgoGen - https://openaccess.thecvf.com/content/CVPR2024/papers/Li_EgoGen_An_Egocentric_Synthetic_Data_Generator_CVPR_2024_paper.pdf\nNymeria - https://arxiv.org/pdf/2406.09905", "limitations": "Limitations and Discussions are discussed very sparsely. The authors spend less than three lines to discuss the limitations", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Controllable video generation via egocentric pose\n- Novel idea of using exocentric videos as a conditional signal\n- A pipeline for novel dataset generation\n\nWeaknesses:\n- The paper is not clearly written; it's hard to understand the architecture, the research gap, and the motivation behind the model design\n- Some claims are unsupported or lack proper citations\n- Comparison to state-of-the-art models like GEM appears to be missing\n- It remains unclear how the third-person exocentric view is used beyond motion extraction\n- The need for the proposed dataset is unclear, given the existence of datasets that already pair egocentric views with local body pose\n- Many concepts and ideas are introduced abruptly, making the paper difficult to follow\n- The novelty claims are questionable. The authors often claim to be the first, despite the existence of similar prior work", "was_revised": true, "final_justification": "I believe this paper should be accepted, provided that the authors incorporate the clarifications and commitments made during the rebuttal into the camera-ready version. It required multiple exchanges to understand fundamental aspects of the method, such as the inputs and outputs of various modules during training and inference. Additionally, the current figures do not sufficiently aid in understanding the overall approach.\n\nThat said, the authors have ultimately convinced me of the soundness and potential impact of their method. The experimental results are adequate to demonstrate the advantages of the proposed architecture, though the writing and discussion could be improved.\nExperiments and discussion for long sequence video generation and world consistency should be added, and are missing so far.\n\nTL;DR: This is a strong method with solid experimental validation, but the clarity of the writing, especially in explaining the methodology, needs improvement. Addressing this would result in a strong and valuable submission.", "rebuttal": "First thanks for detailed comments.\n\n**Q1.The model,research gap,motivation,and challenges are unclear.**\n**A1:** Most prior world models are limited to narrow domains or simple navigation and cannot align complex human motion or ensure scene consistency in egocentric video.Our motivation is to achieve free-form full-body motion control and immersive egocentric video in realistic environments.Our approach takes a single egocentric image and third-person motion as input and generates consistent egocentric video through two modules:PMI splits motion into codes for different body parts and injects them independently for precise part-level control;SR uses 3D point maps to ensure scene geometry remains consistent.PMI and SR together address the main challenges of motion alignment&scene consistency.\n\n**Q2.Release of dataset and code.**\n**A2:** Training code is in the supplement.Full dataset&code for training&distillation will be released after acceptance.\n\n**Q3.Claim of first egocentric simulator and comparison with GEM,VISTA.**\n**A3:** Egocentric here means world-consistent simulation in diverse environments with free interaction,not just driving.\"Egocentric\" here is from person view.GEM and VISTA focus on driving and cannot support daily-life scenes or real-time user interaction.For EgoGen,it's designed for egocentric perception tasks to generate labels and isn't related to our topic of world simulator.GEM only edits non-player character motion by pose;VISTA is for driving control.Neither allows free egocentric exploration.Our model maintains scene and motion consistency and supports real user motion.Our method outperforms both:\n\n|Method|DINO↑|CLIP↑|MPJPE↓|MRRPE↓|PSNR↑|FVD↓|LPIPS↓|\n|-|-|-|-|-|-|-|-|\n|Ours|67.8|88.2|127.16|151.62|52.6|226.12|0.0663|\n|GEM|40.1|62.7|405.32|391.21|38.2|378.40|0.1789|\n|VISTA|39.5|60.4|431.83|406.06|37.8|391.25|0.1857|\n\nWe'll update our definition&comparison in revision.\n\n**Q4.What does this work offer the community?Why is it first?**\n**A4:** We present first general-purpose egocentric world simulator enabling free-form full-body control and immersive exploration in realistic,diverse settings.Previous work lacks high-degree-of-freedom motion alignment and consistent interactive simulation from minimal input.Our code,models,and data set a new benchmark for embodied AI.\n\n**Q5.What is a world model(Line 20)?**\n**A5:** A world model simulates and interacts with environments and human actions,learning a unified dynamic representation for prediction and interaction,covering spatial,temporal,and causal structure,allowing open-ended exploration.Our definition follows recent works like The Matrix[6] and MineWorld[2].\n\n**Q6.Why do these advancements help(need citations)?**\n**A6:** World models allow game engines to simulate interactive environments that adapt to player actions,not just fixed scripts.This is essential for immersive,scalable games.They enable realistic motion,consistent scene generation,and efficient content creation,as shown in Oasis,MineWorld,Google Genie,Microsoft WHAM.\n\n**Q7.What is predictive modeling in Line26?**\n**A7:** Predictive modeling is learning a model that generates or predicts future environment states from current and past input,typically predicting future frames,actions,or dynamics for planning,control,or simulation.\n\n**Q8.Why are these world models critical infrastructure?**\n**A8:** World models are essential for next-generation interactive simulations&virtual worlds,offering dynamic simulation,generalization,and data-driven learning,which are key for building scalable interactive environments.Prototypes like Oasis&MineWorld show this potential.\n\n**Q9.Statement of topic Line29.**\n**A9:** \"This topic\" refers to world models enabling immersive human interaction,allowing users to explore and interact as participants,not just simple actions or planning.We will clarify scope and emphasize our advances over limited-interaction models.\n\n**Q10.Works on realistic scenes Line30.**\n**A10:** See A3.\n\n**Q11.Why do previous works limit users in this way(Line31)?**\n**A11:** These limits come from data and model design.Prior datasets are from games with basic moves only,so models inherit these limits and cannot generate complex actions.Prior models also lack modules for high-degree-of-freedom motion.Our method addresses both aspects to enable immersive interaction.\n\n**Q12.GEM seems to allow skeleton poses.**\n**A12:** GEM uses pose conditions to edit non-player characters in pre-recorded videos;it does not allow free exploration or user control as main character.See A3 for more.\n\n**Q13.Unclear problem statement&research gap(Line39-60).**\n**A13:** Previous world models only support simple navigation or specific games and do not allow high-degree-of-freedom user interaction in realistic scenes.Our goal is a simulator like a VR game,where users can explore environments with full-body motion,providing a single egocentric image and exocentric video;our model generates immersive egocentric video in real time.\n\n**Q14.Why is this method first(Line61)?**\n**A14:** See A3.\n\n**Q15.Release of dataset/code(Line67).**\n**A15:** See A2.\n\n**Q16.What does k mean in Line106?**\n**A16:** k refers to the number of frames divided by the VAE temporal downsampling ratio,that is,the length of the latent sequence after encoding.\n\n**Q17.What are problem statement,inputs,and outputs(Line99)?**\n**A17:** See A13.\n\n**Q18.How does paragraph in Line109 connect to its context?**\n**A18:** Lines109–120 introduce PMI and SR,crucial for part-level motion control and scene consistency,matching the motivation and pipeline description and directly addressing egocentric video generation.In revision,we will clarify their connection to the overall framework.\n\n**Q19.Goal and motivation of core modules(Line109).**\n**A19:** See A1.\n\n**Q20.Role of VAE?**\n**A20:** VAE is used to encode video frames and does not process human motion sequences,which are encoded separately by our motion encoder.\n\n**Q21.Why are motion latents the same shape as images?How is that ensured and why is high-resolution encoding needed?**\n**A21:** We use eight-layer 3D conv motion coder for this.Same shape lets pixel-to-pixel align for motion and look,so model can set each frame pixel by pixel by motion code for fine part control.Past work[4,5] shows this concat way keeps info and gets better match than others.Matching shapes gives better motion control.\n\n**Q22.What is camera sequence?**\n**A22:** Camera sequence is a series of camera extrinsic parameters(i.e., rotation matrices)from head motion over time,representing the egocentric viewpoint path in video.\n\n**Q23&Q24.What's 4D scene/point-map sequence?** \n**A23:** 4D scene/point-map sequence is modeled as a temporal sequence of 3D point clouds,where each point cloud encodes scene geometry at a given frame.This captures spatial geometry over time,ensuring consistent context in generated videos.\n\n**Q25.Why t in U(0,1) instead of U(1,T) in Eq.1**\n**A25:** In Eq.1,t is sampled from a uniform distribution in [0,1],as is standard in flow matching.The notation t=1,...,T in Line121 will be unified in revision.\n\n**Q26.Where does alpha^2+delta^2=1 come from**\n**A26:** We follow common practice in previous diffusion models,alpha^2+delta^2=1 ensures the trade-off between signal and noise is normalized during forward diffusion.\n\n**Q27–Q29.Empirical analysis in Line130 and why three-separate-latents/separate-processing is better**\n**A27–A29:** See A1 for Reviewer-jaf9.\n\n**Q30.Write axis angles differently in Line143**\n**A30:** Thanks for the suggestion.We'll revise axis angle notation in revision.\n\n**Q31.Unclear how 4D scene is represented in Line152**\n**A31:** See A23.\n\n**Q32.Introduction of CUT3R in Line154**\n**A32:** CUT3R generates per-pixel 3D point maps from images online and is used to create point-map sequences as labels.We will expand details in revision.\n\n**Q33.How is alignment in Line158 happening**\n**A33:** See A2 of Reviewer-WmZe.\n\n**Q34.How is scene consistency kept during inference if point maps do not update**\n**A34:** During inference,the model generates point-map and video-frame latents auto-regressively,each depending on previous results.This ensures the evolving point-map sequence always guides scene geometry,keeping spatial context consistent.\n\n**Q35.Claim of no public datasets**\n**A35:** We acknowledge for the oversight of EgoSim and clarify EgoSim and the subset of EgoExo4D with annotated poses are public but too small.To achieve good generalization on diverse domains, we built an automatic pipeline for more large-scale data.Models trained only on these public data perform much worse.\n\n|Method|DINO(↑)|CLIP(↑)|MPJPE(↓)|MRRPE(↓)|PSNR(↑)|FVD(↓)|LPIPS(↓)|\n|-|-|-|-|-|-|-|-|\n|Limited-data|56.4|76.7|182.42|202.35|46.6|272.01|0.0827|\n|Full-data|67.8|88.2|127.16|151.62|52.6|226.12|0.0663|\n\nWe will add detailed comparisons in revision.\n\n**Q36.Is exoview used for other purposes**\n**A36:** No,exoview data is only used to extract SMPL motion sequences.\n\n**Q37.Why do ablation studies come before main results**\n**A37:** No existing methods match our setting,we show ablation studies first to validate contributions before main comparisons.\n\n**Q38.Complexity of joint motion encoder and latent shape**\n**A38:** Three split motion encoders(2.31M,1.29M,1.97M parameters)are close in size to the joint encoder(4.5M).Both encode SMPL parameters as arrays of shape R(k × 3 × h × w),where k is a quarter of frames(4× downsample),and h,w are spatial sizes.\n\n**Q39.Why were SOTA models like GEM not included in comparisons?**\n**A39:** GEM is designed for autonomous driving,not general environments,and does not support interactive user control.Based on feedback,we add GEM as a baseline(see A3)and will expand comparisons in revision.\n\n**Q40.Discussion and limitations.**\n**A40:** See A1 of Reviewer-cVUd.\n\n[1]OASIS.Yang et al[2]MineWorld.Guo et al[3]GeNIE.Wang et al[4]CatVTON.Chong et al[5]UniAnimate.Wang et al[6]The Matrix.Feng et al"}]

Author rebuttal:

Meta-review: This paper presents PlayerOne, a novel egocentric world simulator that constructs dynamic environments from a single image and generates videos precisely aligned with real-world human motion. The work addresses a significant and novel problem with a sound and sensible methodology, and is considered highly valuable to the community. 

All reviewers and the Area Chair unanimously recommended acceptance. We strongly encourage the authors to integrate the rebuttal discussions and reviewer suggestions into the final version to maximize the paper's impact.
Decision: Accept (oral)