Omni-View: Unlocking How Generation Facilitates Understanding in Unified 3D Model based on Multiview images
Authors: JiaKui Hu
Shanshan Zhao
Qing-Guo Chen
Xuerui Qiu
Jialun Liu
Zhao Xu
Weihua Luo
Kaifu Zhang
Yanye Lu
Venue: ICLR 2026 Poster (2026)
Keywords: unified model; generation helps understanding; 3d scene understanding; novel view synthesis
Primary area: foundation or frontier models, including LLMs
TL;DR: Generation helps Understanding in 3d scene.

Abstract: This paper presents Omni-View, which extends the unified multimodal understanding and generation to 3D scenes based on multiview images, exploring the principle that ``generation facilitates understanding". Consisting of understanding model, texture module, and geometry module, Omni-View jointly models scene understanding, novel view synthesis, and geometry estimation, enabling synergistic interaction between 3D scene understanding and generation tasks. By design, it leverages the spatiotemporal modeling capabilities of its texture module responsible for appearance synthesis, alongside the explicit geometric constraints provided by its dedicated geometry module, thereby enriching the model's holistic understanding of 3D scenes. Trained with a two-stage strategy, Omni-View achieves a state-of-the-art score of 55.4 on the VSI-Bench benchmark, outperforming existing specialized 3D understanding models, while simultaneously delivering strong performance in both novel view synthesis and 3D scene generation. The code and pretrained models are open-sourced at https://github.com/AIDC-AI/Omni-View .

Published as a conference paper at ICLR 2026
OMNI-VIEW: UNLOCKING How GENERATION FA- CILITATES UNDERSTANDING IN UNIFIED 3D MODEL BASED ON MULTIVIEW IMAGES
JiaKui Hu Shanshan Zhao"
Qing-Guo Chen
Xuerui Qiu
Jialun Liu
Zhao Xu² Weihua Luo?
Kaifu Zhang*
Yanye Lu
"Inttitute of Medical Technology,
Peking University
2 Alibaba International Digital Commerce Group
CCSIA
ZZGCA
TeleAI
ABSTRACT
This paper presents Omni-View, which extends the unified multimodal under- standing and generation to 3D scenes based on multiview images, exploring the principle that "generation facilitates understanding". Consisting of understand- ing model, texture module, and geometry module, Omni-View jointly models scene understanding, novel view synthesis, and geometry estimation, enabling synergistic interaction between 3D scene understanding and generation tasks. By design, it leverages the spatiotemporal modeling capabilities of its texture module responsible for appearance synthesis, alongside the explicit geometric constraints provided by its dedicated geometry module, thereby enriching the model's holistic understanding of 3D scenes. Trained with two-stage strategy, Omni-View achieves a state-of-the-art score of 55.4 on the VSI-Bench bench- mark, outperforming existing specialized 3D understanding models, while si- multancously delivering strong performance in both novel view synthesis and 3D scene generation. The code and pretrained models are open-sourced https: / //tp://gtttt//////// com/ Com//IDC-ATI hwt.//mmiiiieei
INTRODUCTION
Recently, unified multimodal models (UMMs) (Team, 2024a; Xie et al., a; Deng et al., 2025) have emerged pivotal area of research. The primary goal is empower multimodal large language models (MLLMs) to both understand and generate visual signals present in our world, laying the foundation for artificial general intelligence.
Numerous methods (Team, 2024a; Wu et al., 2024a; b; Xie et al., a; Chen et al., 2025; Deng et al., 2025; Li al., 2025) have achieved coexistence between 2D image understanding and generation. Among them, some studies (Wu et al., 2024b; Tong et al., 2024) aim to improve model performance by exploring the synergies arising from the interaction between understanding and generation. For example, Metamorph (Tong et al., 2024) has extensively validated the role of understanding in im- proving generation performance. However, the potential and effectiveness of generation improve understanding capabilities within unified models remain underexplored.
Due to the intrinsic geometric and spatiotemporal nature of 3D scenes and their multiview images, generative tasks such as geometry estimation and novel view synthesis are particularly well-suited for facilitating understanding the 3D domain. This attributable to the fact that understanding tasks (Yang al., 2025; Zhang al., inherently necessitate robust geometric and spatiotemporal modeling capabilitics, which can be effectively acquired through these generative tasks. Moreover, biomedical evidence (Maus et al., 2013; Nortmann et al., 2015) suggests that human understanding of 3D environments is governed by the capacity to generate and imagine future sensory and geomet- ric data (Keller al., 2012; Leinweber et al., 2017) in the observed scene. These findings provide us with guidance: the "generation facilitates understanding' paradigm presents a promising approach building a unified model for 3D scene understanding and generation.
This work is done when he interns at Alibaba. Corresponding authors, yanye.lu waku.eddu.cn Shhanrhha000@gmail.coo

---

Published as a conference paper at ICLR 2026
Inspired by the above analysis, this paper aims to fully unlock and maximize the benefits of gener- ation to understanding, thereby constructing a unified model for 3D scene understanding and gen- eration, called Omni-View We emphasize that geometry estimation and novel view synthesis can leverage their inherent geometric measurement and spatiotemporal modeling capabilities to improve 3D scene understanding, localization, and spatial reasoning. Specifically, we achieve this through two key aspects, architectural design and training strategy.
The architectural design aims at unifying 3D scene understanding and generation, leveraging ge- ometry estimation and novel view synthesis to advance 3D scene understanding. Our Omni- View is built on Bagel (Deng al., 2025), a strong unified framework in which the interaction between un- derstanding and generation is facilitated by its shared multimodal self-attention. However, Bagel's generative capacity is limited to RGB images, thus only capturing texture information, whereas 3I) scene generation necessitates inclusion of both texture and geometric structure. In accordance with the dual generative objectives, the generation model in Omni- ules: texture module and geometry module. The texture module receives reference images, list of targeted camera poses, and prompt tokens encoded by the understanding model to generate novel views of the scene. Meanwhile, the geometry module employs the hidden features from the understanding model and the latent output of the texture module to infer geometric information of novel views, such as depth maps and camera poses. This dual-pathway architecture empowers the model to develop both geometric and spatiotemporal modeling capabilities, which are essential for 3D scene understanding tasks.
split into two distinet mod-
The principal goal of training strategy is to comprehensively improve the performance of the model. A two-stage training strategy is employed. The first stage aims augment the benefits of generation for understanding 3D scenes, as introduced by the proposed architecture. The subsequent stage intended to refine the generation performance. In stage the understanding model, geometry module, and texture module are trained simultaneously. Geometry estimation assists the model in comprehending the relative positional relationships among objects, thus directly enhancing the model's 'ss capability to evaluate the relative distances and directions of objects in 3D scenes. The autoregressive generation forces the understanding model to discern the spatiotemporal relations between the generated novel views, thus improving its understanding capabilities. As iterations progress in stage 1, the number of reference images gradually decreases. This progressive shift from dense sparse views supports curriculum-like like, easy-to- to-fffficul training approach, ultimately enhancing the performance of the understanding model. In stage 2, the understanding model is frozen. The generation model is finetuned via RGBB-Depth-Pos joint generation, thereby enhancing its capabilities in 3D scene generation.
scene understanding, spatial reasoning, and novel view synthesis tasks. The model achieves an impressive score of 55.4 on the VSI-Bench, exceeding current MLLMs designed for visual reasoning. It manifests particularly notable improvements subtasks such as Relative Distance and Appearance Order, which require spatiotemporal modeling and the estimation of geometry acquired through generation tasks. It also exhibits superior performance compared to existing 3D understanding MLLMs in the area of 3D question answering (Ma et al., 2023; Azuma et al., 2022). Furthermore, it effectively narrows the performance gap between unified models and specialized models focused on 3D visual localization (Chen et al., 2020). Furthermore, we attain robust results in the domain of novel view synthesis and scene generation, with particular emphasis on enhanced perceptual quality.
We evaluate Omni-View
2 RELATED WORK
Scene understanding. Recent advances in understanding 3D scenes have been greatly provided by incorporating 3D or video input and 3D reconstruction prior. LLaVA-3ID (Zhu et al., 2025) and GPT4Scene (Qi et al., 2025) function within the voxel space and BEV. Video3DLLM (Zheng et al., 2025) improves localization ability by encoding 3D coordinates as position embeddings, while Ross3D (Wang et al., 2025a) improves 3D understanding through visual-centric reconstruc- tion. However, the dependency 3D input of these methods poses practical application challenges. To alleviate this issue, VG-LLM (Zheng et al.) and SpatialMLLM (Wu et al., a) use the features of VGGT (Wang et al., 2025b) input, embedding the 3D piror in the model.
2

---

Published as a conference paper at ICLR 2026
Scene generation. A vital element of scene generation methods is the presence of an efficient proxy to represent the 3D scene, with panoramic images (Team et al., 2025), point clouds (Yu et al,, 2025b), and Gaussian Splatting (Yu et al., 2025a) among the viable options. In consideration of the advances in image and video diffusion models (Rombach et al., 2022; Wan et al., 2025), contempo- rary 31) scene generation strategies reconstruct the scene's s geometry from single view, employing conditioned vidco diffusion models to render the scene's ' texture. ViewCrafter (Yu et al., 2025b) brings explicit 3D information (point cloud) to generation models through iterative reconstruction. Worner (Yu et al., 2024) generates comprehensive set of view sequences. Voyager (Huang al., 2025) reconstructs the scene from single view and uses it as conditions for inpainting.
Unified understanding and generation. In the domain of 3D scenes, unified model for un- derstanding and generation applicable in general scenarios remains absent. Hermes (Zhou et al., 2025) uses BEVs to design unified understanding and generation model for autonomous driving- In contrast, significant progress has been made in 2D vision (Zhang et al., 2025). These methods show primarily variations in architectures and training strategies. Chameleon (Team, 2024a) utilizes VQ-VAE for image tokenization, thereby improving generation competencies. However, its under- standing capabilities
inferior those of Janus (Wu et al., 2024a), which employs SigLIP (Zhai et al., 2023) as visual understanding encoder. VILA-U (Wu et al., b) integrates understanding and generation within the image encoder, bypassing multitask gradient conflicts during MLLM train- ing. BAGEL (Deng et al., 2025) implements task-based hard routing for MILMM, which also avoids gradient conflicts. Harmon (Wu al., 2025) takes advantage of MAE's reconstruction ability and downstream understanding enhancement. BLIP3o (Chen et al., 2025) introduces "understand first, then generation" training paradigm, thus achieving performance gains. Building on the progress in the 2D domain, this paper investigates unified model in 3D scenes and explores how generation aids the understanding scheme within the framework.
METHOD
3.1
ARCUTITECTURES
Omni-View
Geo
Geo
Unified 3d scene understanding and generation model
Tarviiad dogeh róbeas
Poos
Und
Tex
Und
Tex
Geometry Module
Elevviiic mralt-tte iruge miless
Geonatie raise
Understanding Model
Texture Module
The soor
OOOEE
Uamer pracc Neccer encedii
lff stanlng Es tan Backg bossknk Dencigll.
trage Satin
Cuesan panc Misti
Ceptios (Uppueeal
Reforanc
unitbrrty
Tsca inapes
images
Figure 1: Architecture of Omni-Vies Building upon Bagel (Deng et al., 2025), Omni-Vinn View sists of an understanding model and a generation model. The generation model is further composed of two specialized modules: one for texture and one for geometry. Trained via two-stage process, Omni- View shows high effectiveness in scene understanding and novel view synthesis. Crucially, unlocks the benefits of its generative capabilities to enhance the model's understanding performance.
The architecture of Omni-Vise as illustrated in Figure 1, is divided into two distinct components. The left component, termed the understanding model, is tasked with understanding the multiview images of 3D scenes and producing text feedback. The right component is comprised of two integral parts: the texture module and the geometry module, which facilitate the synthesis the RGB image and the associated geometric data.
Understanding Model. The understanding model is required to perform 3D scene or spatial com- prehension tasks, including question answering, localization, and reasoning. This necessitates the
3

---

Published as a conference paper at ICLR 2026
model to extract semantic information from spatial-temporal data while measuring the geometric characteristics of the 3D scene. Specifically, given n multiview images and a question Tques> the understanding model produces the textual answer Tans through the next token prediction. In this process, the images are encoded by SigI.P (Zhai et al., 2023) and the text tokenized by a frozen vocabulary 7()). Subsequently, tokens of multiview images and question q(Tques will be sequen- tially concatenated and fed into the understanding model. Moreover, the understanding model can also produce intermediate features, which will be used in the geometry module.
Texture Module. The texture module in the generation model is tasked with novel view synthesis using flow matching (Lipman al.). This module processes a textual description Taes and some reference images f of the current scene. together with a specified camera pose, to produce con- sistent novel views of this scene. Within this module, the reference images Tref are encoded using the FI.UX-VA VAE encoder (Labs, 2024) e.). and the vocabulary used to tokenize T dess aligns with that used by the understanding model. This process incorporates the camera pose control as delineated in MV-AR (Hu et al., 2025), embedding the Plucker-Ray encoding ri,j (o x d, d) of the camera pose as the absolute position encoding in the model, where o and d denote the origin and direction of the ray, (i.j) represents the pixel coordinate. This camera pose embedding exhibits adaptabil- ity to various image resolutions and demonstrates significant flexibility. The above process can be described as:
Fiex TextureModull ([MM-Head(T(NN) (e: Ntez] + +]);
where Fier is the predicted image noise of the texture module, Net is the random input noise, and LM Head is the text processing module of the understanding model.
Geometry Module. The geometry module in the generation model constructs the geometric aspects of the generated images in the texture module. It synthesizes depth maps through flow matching and employs learnable query strategy accurately estimate camera poses. It receives the latent of novel view images Frez from the texture module as input, which is concatenated with random depth noise Naep and a learnable camera pose query 4eom along the frame dimension. After processing, Ndep will be denoised as depth maps' latent, while Icom will be decoded to reveal the intrinsic and extrinsics of the camera. The features of novel view images Fund, output from the understanding model's 's block, is also integrated into the geometry module through cross-attention. The process in the geometry module can described as follows.
|Feep g] Geometry MommmetryyMoie Moull ([Fiexi Ndep: Jeam]. Fund),
where Foep the predicted depth noise of the geometry module. ğ is the predicted intrinsies and extrinsics of the camera, which is decoded by VGGT's camera decoder (Wang et al., 2025b).
The geometry module has independent parameters and maintains architectural connections with both the texture module and the understanding model. Unlike the connection method in BAGEL, the geometry module uniformly extracts features from the understanding model at the layer dimen- sion as conditions. It utilizes cross-attention for fusion and ensures that the gradients of geometry estimation can be backpropagated to the understanding model. Currently, the geometry module's input only relies the last-layer output latent of the texture module. This approach guarantees that it acquires information closest to the image modality, thereby providing finely aligned features for the estimation of depth map and camera pose.
3.2 TRAINING RECIPE
The Omni- View training recipe has two separate stages, as shown in the upper left of Figure 1.
Stage 1: unify 3D understanding and generation. In stage 1, we train the understanding model, the texture module, and the geometry module simultaneously. It can leverage the fine-grained ge- ometry estimation capability in the geometry module and combines it with the temporll modeling ability within the texture module, thus improving the 3D understanding performance.
Understanding model. For the understanding model. the distribution of the answer text given the distribution of the multiview images and query text. The loss function of the understanding model can be expressed as follows.
the next token prediction to predict
4

---

Published as a conference paper at ICLR 2026
L Lond
Σ log Po(yelue)),
where y is the multimodal sequence that contains tokenized multiview images, query text, and an- swer text.
Texture module. The loss function for the texture module is defined as the Mean Squared Error (MSE) loss between each predicted texture noise Pier generated by the texture module and the provided texture noise Ntex-
Luer
||
Nteal12.
(4)
In contrast to the majority of novel view synthesis methods, the texture model employs autoregres- sive generation framework. Specifically, during the generation of the n-th frame, the model is ex- posed to the visual data of the preceding π - frames, while excluding any subsequent frames. This autoregressive methodology enables the model fully grasp the concept of temporal sequences, thereby enhancing its spatiotemporal modeling proficiency and effectively improving scene under- standing. To optimize the 3D consistency of the sampled images, diffusion forcing (Chen et al., 2024a) is employed when training the texture module.
To acquire this complex spatiotemporal generation capability incrementally, we gradually adjust the reference images. As iterations progress, the reference images are systematically reduced in stepwise manner, transitioning from encompassing all input images, excluding the first image, to including only the first image. This implies that for the model, the reference confidence transitions from dense sparse. We designate this progressive training approach as dense-to-sparse (D2S). This strategy has been shown to be highly effective in facilitating improved understanding.
Geometry module. In stage 1, the geometry module is tasked with estimating the depth and pose of the camera from both the provided images and those synthesized by the texture module. The loss function for the geometry module comprises a sum of the depth estimation loss and the camera pose estimation loss. In terms of depth estimation, we apply the MSE loss the comparison between the depth noise Fdep predicted by the geometry module and the given depth noise Ndep- The estimated intrinsics and extrinsics of the camera ğ optimized directly through the Huber loss.
Lgeo
F| Ndepll2 2 + |1 Igelles
(5)
where Agt is the ground truth of the camera pose and || - -lle denotes the Huber loss.
Ultimately, a weighted summation of the aforementioned losses is conducted derive the loss function in stage L.1.
La Aand + Atna Luta + Agea Legoos
(6)
where Aund, Atez and Ageo represent the weighting coefficients and their default values are 1, 1, and 0.1 respectively.
Stage 2: advance generation. In stage 2, the texture module and the geometry modules are trained. The RGB-Depth- CGB-Depth-Pose (RGBDP) joint learning methodology is used for training, capitalizing on the geometry prior obtained from depth-pose estimation enhance the ability to generate consistent appearances for novel views.
In the texture module, the reference single-view image is used along with its depth map to recon- struct the initial point cloud of the scene. The images rendered from different views, projected through this point cloud, serve as conditions following Voyager (Huang et al., 2025). For the ge- ometry module, it generates the depth map and camera pose from images synthesized by the texture module. Concurrently, no longer relies on the features of the understanding model as conditions for cross-attention.
5

---

Published as a conference paper at ICLR 2026
4 EXPERIENCE
4.1 EXPERIMENTAL SETUP
In our experiments, the understanding model and the texture module in the generation model are initialized using the pre-trained BAGEL-7B (Deng et al., 2025). The geometry module in the gen- eration model is configured to have the same dimensions as the texture module, with depth of four layers. For 3D scene understanding, a filtered dataset comprising 780k valid items, sourced from SQA3D (Ma et al., 2023), ScanQA (Azuma et al., 2022), 3DOD (Zheng et al.,, ScanRe- fer (Chen et al., 2020), VLM-3R (Fan et al., 2025), SPAR (Zhang et al., a) (234k subset) and Ilava-hound4 (Zhang et al., 2024b) (64k subset). is meticulously curated for training. Regarding novel view synthesis, 61k video clips are carefully selected from re10k (Zhou et al., 2018). The corresponding depth maps are synthesized using the Voyager data pipeline (Huang et al., 2025) and captions are synthesized using the VL.Max (Bai et al., 2025). During training, we do not use images from the scene understanding task to train the generation model to fully demonstrate that the understanding performance improvement brought by Omni-Vies does not come from memorizing the data in understanding tasks.
Model training was performed using the Adam optimizer, characterized by 81 0.9, B2 0.95, the peak learning rate of 1 x 10 The warm-up phase that constitutes 5% of the whole training iterations. The training process is completed after one epoch of the understanding dataset. For the understanding model, we do not rely on any 3D scene input to support both 3D scene understanding and spatial reasoning tasks. In main comparisons, we use the same checkpoint for testing all tasks.
4.2 MAIN COMPARISONS
4.2.1
3D SCENE UNDERSTANDING
Benchmarks. We evaluate the 3D understanding performance of the model on question answer- ing (Ma et al., 2023; Azuma et al., 2022) and localization (Chen et al., 2020; Zheng et al.). During inference, we set the frame numbers as 32 following Video3DLLM (Zheng et al., 2025).
Comparison baselines. We compare Omni- View with models specifically designed for 2D or 3D visual understanding tasks (Team, 2024b; Wang et al., 2024a; Zhang et al., 2024b; Huang et al., 2023; Zhang et al., 2024a; Chen et al., 2024b; Zhu et al., 2025; Zheng et al., 2025; Qi et al., 2025; Wang et al., 2025a), as well as with some unified models applicable to video modalities (Deng et al., 2025). Unified models are evaluated after fine-tuning with the same data. Within the results table, we differentiate between models that incorporate explicit 3D input and those that do not. Although the incorporation of explicit 3D input improves model performance, it concurrently restricts appli- cability (Zheng et al., 2025).
Metrics. For SQA3D, we use the EM metric to evaluate the accuracy, which stands for top-1 I exact match EM-R means the refined EM following LEO (Huang et al., 2023). For ScanQA, we use CIDEr (C), BLEU-4 (B-4), METEOR (M), ROUGE (R), and EM for more complete validation. For 3DOD, we use the average F1 score (F1) as a metric to assess the correspondence between the predicted and actual coordinates. For ScanRefer, we calculate the percentage of samples for which the Intersection over Union (IoU) exceeds thresholds of 0.25 and 0.5, respectively, between the predicted and true coordinates. The higher the above metrics, the better.
Results. As shown in Table 1, our analysis leads to four conclusions. (1) Our Omni-View exceeds all current MLLM methods that do not depend on 3D scene input. Within the SQA3D test set, Omni-Viee View achieves an enhancement of 3.3 over SpatialMLLM in EM, while in the ScanQA val- idation set, it surpasses SpatialMLLMM by an increment of 11.2 in CIDEr. For the object detection task, our unified model performs comparably with the 3D understanding model VG-LLM. (2) This performance improvement is mainly attributed to the architectural design and training scheme that we proposed. Compared with the fine-tuned BAGEL, our method still demonstrates substantial im- provements. For example, on the SQA3D test set, Omni- View enhances EM by 2 points compared to the fine-tuned BAGEL (Deng et al., 2025); on the ScanQA validation set. Omni- View improves 7.5 in CIDEr. (3) The efficacy of our approach in the question-answering task is equivalent to ad- vanced MILM methods that require 3D scene input. In particular, in the ScanQA validation set,

---

Published as a conference paper at ICLR 2026
SQA3D
ScanQAva
3DOD
SeanRefer
Methods
3D Imput
EM
EM-R
C
B-4
M
EM
F1
Acc@0.25
Acc@0.
Taas:: specific Models
LEO
50.0
52.4
80.0
11.5
16.2
39.3
21.5
Sheme
57.5
87.7
14.3
18.0
41.6
21.6
55.5
50.2
Grounded 3D-LLM
47.9
44,1
Video-3D-I
20.0
GPT4Scene- HIDM
59.4
62.4
96.3
15.5
18.9
46.5
62.6
57.0
LLaVA-3D
60.1
103.1
16.4
20.8
49.6
30.6
50.1
42.7
VL2-8B
Owen2-VL-78
48.5
53.9
3.0
11.4
29.3
LLVAA-Video-7B
88.7
3.1
17.7
44.6
SPAR-7B
58.1
90.7
15.3
(31.9
43.1 (12.4)
VG-LLM-4B
47.0
53.5 (36.4)
47.5(1
SparialMM..-M44R
58.7
91.8
Unified Models
BAGEL-7B-FT Omni View-7B 7B
59.7
95.5
14.7
18.7
46.3
27.0
41.3
46 (28.0)
41.6(7.7)
59.2
61.9
103.0
16.2
20.1
49.0
29.5
46.4
50.8(32.5)
45.0x9.9)
Table 1: Evaluation of 3D scene understanding. Bold and underline denote the best and second-best models without 3D scene input, respectively. For ScanRefer, the content in "()" indicates results without proposal refinement (Zhang et al., a).
indicates the number is not available for us.
Omni-Vin View achieved a performance comparable to Video3DLLM (Zheng et al., 2025) and LLaVA- 3D (Zhu et al., 2025). (4) However, there remains a notable disparity between methodologies that do not require 3D scene input and those that do, particularly in 3D grounding tasks. The experimental results of VG-LLM (Zheng al.) also substantiate this observation.
4.2.2 3D SPATIAL REASONING
We evaluate the 3D spatial reasoning ability of the model VSI-Bench (Yang et al., 2025). During inference, follow the VSI-Bench to set frame numbers ranging from 32 and frame resolution to 640p.
Benchmarks.
Comparison baselines. We compare our Omni-View with models specifically designed for 2D or 3D visual understanding tasks (Chen et al., b; Zhang et al., b; Bai et al., 2025; Ray et al.: Zhang al., Zheng al.; Wu et al., a), as well as with some unified models applicable video modali- ties (OpenAI, 2024; Team et al., 2024; Deng et al., 2025; Xie et al., b). Unified models are evaluated after fine-tuning with the same data we used.
Numerical Qusetion
Multiple-Choice Question Route Plan
Methods
Avg.
Ohj. Cnt. Ahs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Appr. Order
Task spefic Models
LoLAVILA-BB
29.1
16.7
0.0
29.6
30.7
32.5
25.5
21.6
VA-7V
38.0
16.6
38.9
22.2
33.1
43.3
25.4
15.7
29.2
LLaVA-One OneVonnnn-
43.5
23,9
57.6
37.5
42.5
39.9
32.5
44.6
40.2
LLaVA- VVV-Vide--728
48.9
22.8
57.4
35.3
42.4
36.7
35.0
48.6
40.9
Owen2.5VL-7B
10.7
38.6
38.5
Qwen2.5VL-72B
25.1
29.3
54.5
38.8
38.2
37.0
34.0
28.9
37.0
SAT-LLaVA-Video--P
47.3
41.1
37.1
36.1
40.4
SPAR-8D
41.1
VG-LLM-4B
66.4
36.6
55.2
56.3
40.8
43.4
30.4
39.5
Spatial-MLLM-44
65.3
34.8
63.1
45.1
41.3
46.2
33.5
46.3
48.4
Unified Models
GPT-4o (API)
43.8
38.2
37.0
31.5
Gemini-1.5 Pro (API)
56.2
30.9
64.1
43.6
51.3
46.3
36.0
34.6
45.4
BAGEL-7B-PT
62.8
36.3
56.4
49.7
46.1
49.4
26.8
43.1
46.3
Omni- View-BB
70.3
46.4
68.6
54.7
65.9
54.4
33.5
49.0
55.4
Table 2: Evaluation of spatial reasoning VSI-Bench. <_" indicates the number is not available for us. Bold and underline denote the best and second- best models, respectively.
Results. The results on spatial reasoning tasks more fully demonstrate Omni- View's improvement previous methods in analyzing the relative or absolute position and orientation of spatial ob-

---

Published as a conference paper at ICLR 2026
jects. With average score of 55.4, our Omni- View ranks first among existing spatial reasoning MLLMs. Compared to existing Spatial-MLLM (Wu et al., a) and VG-LLM (Zheng et al.), Omni- View improves Spatial-MLLM by 11.6, 9.6, and 24.6 in Absolute Distance (Abs. Dist.), Room Size, and Relative Distance (Rel. Dist.), respectively. Omni-View improves VG-LLM by 9.8, 13.4, 25.1, 11.0, and 9.5 in Abs. Dist., Object Size (Obj. Size), Rel. Dist., Relative Direction (Rel. Dir.), and Appearance Order (Appr. Order), respectively. These tasks necessitate the model to thoroughly predict the spatiotemporal state and measure the geometric properties of the scene it observes (Yang et al., 2025). This demonstrates that our proposed method substantially enhances the model's 's perti- nent capabilities.
4.2.3 3D SCENE GENERATION
Benchmarks. The model's generation capacity is validated under two tasks: novel view synthesis (NVS) from single view and scene generation. NVS from a single view necessitates that the model generate the subsequent 25 frames from the first image. 3D scene generation evaluates scenes that are reconstructed from the videos generated to 3DGS. The test scenes are randomly selected from the Re10k test following Chen et al. (a).
Comparison baselines. We compare with scene generation methods (Chung et al., 2025; Wang al.. 2024b; Ma al., 2025; Yu et al., 2025b; Chen al.. a; Huang et al., 2025) and a unified model (Deng et al., 2025). To evaluate the performance of scene generation, we use Dust3R to reconstruct the generated videos, ensuring fairness in the comparison, according to Chen et al. (a). The evaluation is only conducted on the first 25 frames generated by the model following Yu et al. (2025b); Zhai et al. (2025).
NVS from single view and scene generation. Omni-View achieved the highest PSNR and SSIM, and lowest LPIPS score, indicating that its image quality could surpass that of other methods. How- ever, in terms of pixel-level fidelity, Omni-Vie shows only slight improvements over popular scene generation models. This discrepancy may be attributed to Omni- View's 's challenges in being pre- cisely controlled via the camera pose. Visualization results are presented in the Appendix A.4.
NVS from Single View
Scene Generation
Methods
PSNR t SSIM f LPIPS & PSNR + SSIM f LPIPS
Task- specifie Models
LucidDreamer
22.27
0.766
0.204
21.98
0.698
0,290
MotionCtrl
15.86
0.520
0.431
15.33
0.479
0.590
Sce3D
22.37
0.781
0.199
21.60
0.744
0.238
ViewCrafter
22.60
0.754
0.195
22.25
0.709
0.204
Flex World-SB
23.05
0.788
0.182
22.86
0.756
0.198
Voyager- 13B
23.12
0.793
0.175
22.93
0.768
0.194
Unified Models
BAGEL-7B-FT 21.76
0.703
0.288
21.04
0.599
0.403
Omni- View-7B
23.22
0.817
0.114
23.12
0.801
0.146
Table 3: Evaluation of novel view synthesis from single view and scene generation on Rel0k. Bold and underline denote the best and second-best models, respectively.
4.3 ABLATION STUDIES
We perform ablation studies on the proposed architecture and training strategy to ascertain their ef- ficacy. The data used and the hyperparameters applied during the ablation studies remain consistent.
Effect of two modules in the generation model. Both the texture module and the geometry module can improve understanding performance, but they focus on different aspects. The results presented in Table 4 demonstrate that the integration of the texture module facilitates notable advancements in tasks dependent on spatiotemporal modeling, exemplified by an increase of 4. points in Appr. Or- der. Conversely, the introduction of the geometry module markedly enhances performance in tasks contingent upon relative position information, notably in Rel. Dist. However, because of the lacking absolute metric in the synthesized depth maps, improvements in tasks pertaining absolute metric comprehension, such as Abs. Dist., are constrained. Incorporating the task of accurately predicting camera pose mitigate the reduction resulting from the imprecise depth prediction. Furthermore,
8

---

Published as a conference paper at ICLR 2026
results corroborate that the segregation of the texture and geometry modules results in superior understanding performance compared to employing a unified architecture that concurrently learns both texture and geometry.
Geometry
SQA3D
ScanQA
ScanRefer
VSI-Bench (subset)
Texture
depth
camera
EM
C
B-4
Acc@0.25
Obj. Cnt.
Abs. Dist.
Obj. Size
Rel. Dist.
Appr. Order
x
x
57.2
95.5
14.7
(28.0)
62.8
36.3
56.4
46.1
43.1
x
x
58.3
97.4
14.7
48.8 (31.5)
67.7
44.6
59.0
53.2
47.2
58.9
100.5
48.2 (31.2)
44.9
69.7
63.0
47.9
58.7
99.2
15.0
(31.6)
69.5
45.9
67.8
63.8
48.2
59.2
103.0
16.2
50.8 (32.5)
70.3
46.4
68.6
65.9
49.0
Table Ablation on modules in the generation model. The gray row denotes that, in this experi- ment, both the texture module and the geometry module use the same architecture and parameters.
Effect of the autoregressive generation. Autoregressive generation significantly improves tasks requiring spatiotemporal modeling, like Appr. Order. Table compares the understanding perfor- mance with and without autoregressive generation. Although bidirectional generation offers some enhancement, autoregressive generation provides greater improvements. Specifically, it boosts per- formance by 5.8 and 4.4 points in Abs. Dist. and Appr. Order tasks, respectively, compared when not used. This demonstrates that autoregressive generation strengthens spatiotemporal modeling and enhances related scene understanding tasks.
SQA3D
ScanQA
ScanRefer
VSI-Bench (subset)
Autoregressiv
EM
B-4
EM
Acc@0.25
Obj. Cnt.
Abs. Dist.
Obj. Size
Rel. Dist.
Appr. Order
None
$7.2
95.5
14.7
46.9 (28.0)
62.8
36.3
56.4
46.1
43.1
57.2
98.8
15.0
47.0 (28.0)
68.3
40.6
69.0
60.6
44.6
59.2
103.0
16.2
5088 (32.5)
70.3
46.4
68.6
65.9
49.0
Table 5: Ablation on the autoregressive generation in stage "None" means only train under- standing model in this experiment.
Effeet of the D2S (dense-to-sparse The efficacy of D2S training is substantiated during stage 1. The results show that D2S significantly improves the model understanding performance, exceeding the visual reconstruction method advanced in Ross3D (Wang et al., 2025a) for scene understanding tasks necessitating spatiotemporal ordering. In this ablation, the geometry module is kept trainable.
SQA3D
ScanQA
ScanRefer
VSI-Bench (subset)
Condition in S1
B-4
EM
Acc@0.25
Obj. Cnt.
Abs, Dist.
Obj. Size
Rel. Dist.
Appr. Order
None
14.7
46.9 (28.0)
62.8
36.3
56.4
46.1
43.1
randon mask
58.9
98.9
15.5
49.4 (31.7)
65.7
38.9
55.5
$1.6
46.5
dense
57.3
94.7
15.0
47.2 (28.3)
67.7
42.3
60.1
65.6
46.6
sparse sparse (Ours)
57.9
97.3
15.7
50.1 (32.0)
69.0
44.9
65.5
63.1
48.3
dense
103.0
50.8 (32.5)
70.3
46.4
Table 6: Ablation on the D2S mechanism in stage 1 (S1). "None" means we don't train gener- ation model in this experiment. The "random mask" denotes the visual reconstruction method in Ross3D (Wang et al., 2025a).
Effect of training stage 2. The results in Table 7 show that stage 2 can significantly improve the scene generation performance of the model.
NVS from view
Scene Generation
Stage
PSNR
SSIM 1
I.PIPS
PSNR
SSIM
I.PIPS
21.90
0.705
0.365
21.44
0.683
0.307
23.22
0.817
0.114
23.93
0.768
0.194
Table 7: Ablation of the stage

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
We introduce Omni- View, a unified 3D) scene understanding and generation model. By decompos- ing the generation model into distinct texture and geometry components, establish the feasibility and efficacy of employing generation processes to enhance 3D scene understanding and spatial rea- soning. Our method illustrates that unified understanding and generation model is capable of achieving performance on par with the leading specialized understanding models. We posit that Omni-Vine View can serve as a foundational model across 3D and multiview domains, thereby advanc- ing the development of downstream applications such as spatial intelligence.
Ethics Statement. This paper aims to develop unified models to understand and generate 3D scenes. In light of the ongoing advancements in scene generation technology. we emphasize the importance of preventing its misapplication, such as the fabrication of deceptive scenes or the creation of scenes with nefarious intents.
Reproducibility Statement. We state that Omni-View highly reproducible. Implementation details main experiences are provided in Section 4.1 and Appendix B. It is anticipated that these descriptions can sufficiently demonstrate the reproducibility Omni- View. We plan to open- source the code and weight files after the paper passes peer review.
ACKNOWLEDGEMENT
This work was supported in part by National Key Research and Development Program of China C025YFA18057000 in part by National Natural Science Foundation China (82371112, 62501020); in part by the Science Foundation of Peking University Cancer Hospital (JC202505).
REFERENCES
Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question swering for spatial scene understanding. In Proceedings of the IEEE/CVF Conference on Com- puler Vision and Patlern Recognition (CVPR), pp. 19129-19139, 2022.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang. Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl techni- report. ArXiv, abs/2502. 13923, 2025. URL https: /ppi. sewanticschol.... org/ Corpus ID: 2764449776,
Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitz- mann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081-24125, 2024a.
Dave Zhenyu Chen, Angel X Chang, and Matthias Nießner. Scanrefer: 3d object localization in rgb-d scans using natural language. In European Conference on Computer Vision (ECCV), pp- 202-221. Springer, 2020.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal training and dataset. arXiv preprint arXiv:2505.0955 2025.
Luxi Chen, Zihan Zhou, Min Zhao, Yikai Wang, Ge Zhang, Wenhao Huang. Hao Sun, Ji--Rog Wen, and Chongxuan Li. Flexworld: Progressively expanding 3d scenes for dexible-view exploration. In The niith Annual Conference on Neural Information Processing Systems,
Yilun Chen, Shuai Yang, Haifeng Huang, Tai Wang, Runsen Xu, Ruiyuan Lyu, Dahua Lin, and Jiangmiao Pang. Grounded 3d-llm with referent tokens. arXiv preprint 2405 2024b.
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang. Zhijian Liu, et al. Longvila: Scaling cong-context visual language models for long videos. In The Thirteenth International Conference Learning Representations, b.
10

---

Published as a conference paper at ICLR 2026
Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free generation of 3d gaussian splatting scenes. IEEE Transactions on Visualization and Computer Graphics, (1(1)):00400-0600
2025. doi: 10.1 10.1099TVVC.22253361144
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Wei- hao Yu, Xiaoman Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.1468: 2025.
Zhiwen Fan, Jian Zhang. Renjie Li, Junge Zhang. Runjin Chen, Hezhen Hu, Kevin Wang. Huaizhi Qu, Dilin Wang, Zhicheng Yan, et al. VIm-3r: Vision-language models augmented with instruction-aligned reconstruction. arXiv preprint (rxii:2500520279 2025.
JiaKui Hu, Yuxiao Yang. Jialun Liu, Jinbo Wu, Chen Zhao, and Yanye Lu. Auto-regressively gen- erating multi-view consistent images. In Proceedings the IEEE/CVF International Conference on Computer Vision, pp. 2556-2566655 2566, 2025.
JiaKui Hu, Jialun Liu, Liying Yang. Xinliang Zhang, Kaiwen Li, Shuang Zeng, Yuanwei Li, Haibin Huang, Chi Zhang, and Yanye Lu. Geometry-as-conter. Modulating explicit 3d in scene- consistent video generation to geometry context. arXiv preprint arXiv:2602.21929 2026.
Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent 3d world. arXiv preprint arXiv:2311.128) 2023.
Tianyu Huang, Wangguandong Zheng, Tengfei Wang, Yuhao Liu, Zhenwei Wang, Junta Wu, Jie Jiang. Hui Li, Rynson Lau. Wangmeng Zuo, et al. Voyager: Long-range and consisconnittent video diffusion for explorable 3d scene generation. ACM Transactions Graphics (TOG), 44 (6):1-15, 2025.
Georg B Keller, Tobias Bonhoeffer, and Mark Hübener. Sensorimotor mismatch signals in primary visual cortex of the behaving mouse. Neuron, 7465):80--815, 2012.
Black Forest Labs. Flux. https://github/ http://ttuub.com/lllaak---eeeeee
2024.
Marcus Leinweber, Daniel Ward, Jan M Sobezak, Alexander Attinger, and Georg Keller. A sensorimotor circuit in mouse cortex for visual flow predictions. Neuron,
2017.
Junzhe Li, Sifan Zhou, Liya Guo, Xuerui Qiu, Linrui Xu, Delin Qu. Tingting Long, Chun Fan, Ming Li, Hehe Fan, et al. Uniface: A unified fine-grained face understanding and generation model. arXiv preprint 2rXiv:2503.08/22
2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow match- ing for generative modeling. In The Eleventh International Conference on Learning Representa- tions.
Baorui Ma, Huachen Gao, Haoge Deng, Zhengxiong Luo, Tiejun Huang, Lulu Tang, and Xinlong Wang. You see it, you got it: Learning 3d creation on pose-free videos at scale. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2016-2029, 2025.
Xiaojian Ma, Silong Yong, Zilong Zheng, Qing Li, Yitao Liang, Song-Chun Zhu, and Siyuan Huang. Squ3d: Situated question answering in 3d scenes. In International Conference Learning Rep- resentations (ICLR), 2023.
Gerrit W Maus, Jason Fischer, and David Whitney. Motion-dependent representation of space in area mt+. Neuron, 78(3):554 562, 2013.
Nora Nortmann, Sascha Rekauzke, Selim Onat, Peter König, and Dirk Jancke. Primary visual cortex represents the difference between past and present. Cerebral Cortex, 25(6):1427-1440, 2015.
OpenAl. Hello GPT-4o, 2024. URL https: /ettpss:/openai.... httppeaaa..cmmind////eeeeeeeeeee
Zhangyang Qi, Zhixiong Zhang, Ye Fang, Jiaqi Wang, and Hengshuang Zhao. Gpt4scene: Un- derstand 3d scenes from videos with vision-lan Lasionlanguuae models. arXiv preprint arX1v:2501.01428, 2025.
11

---

Published as a conference paper at ICLR 2026
Arijit Ray, Jiafei Duan, Ellis L Brown II, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsami, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, et al. Sat: Dynamic spatial aptitude training for multimodal language models. In Second Conference Language Modeling.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint
arXiv:2405.098/5 2024a.
Gemini Team, Petko Georgiev. Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context. arXiv preprint arXiv:2403.055.4 2024.
Hunyuan World Team, Zhenwei Wang. Yuhao Liu, Junta Wu, Zixiao Gu, Haoyuan Wang, Xuhui Zuo, Tianyu Huang, Wenhuan Li, Sheng Zhang, al. Hunyuanworld 1.0: Generating immersive, explorable, and interactive 3d worlds from words or pixels. arXiv preprint arXiv:2507.21809, 2025.
OpenGVLab Team. Intern Better than the Best-EEppaddig Performance Boundaries of Open-Source Multimodal Models with the Progressive Scaling Strategy, 2024b. URL https:
//internvl. github. io http//log//22/2 02-02- InternVL-2.0.
Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, and Zhuang Liu. Metamorph: Multimodal understanding and generation via instruction tuning. arXiv preprint arXiv:2412.14116 2024.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint WrXiv:2503.20314 2025.
Haochen Wang, Yucheng Zhao, Tiancai Wang, Haoqiang Fan, Xiangyu Zhang, and Zhaoxiang Zhang. Ross3d: Reconstructive visual instruction tuning with 3d-awareness. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9275-9286, 2025a.
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Puttern Recognition, 2025b.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12 191, 2024a.
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionetrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024b.
Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingehao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410. 13848, 2024a.
Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan, Spatial-mllm: Boosting milm capabilities in visual-based spatial intelligence. In The Thirty- Annual Conference on Neural Information Processing Systems, a.
Junfeng WU, Yi Jiang, Chuofan Ma, Yuliang Liu, Hengshuang Zhao, Zehuan Yuan, Song Bai, and Xiang Bai. Liquid: Language models are scalable and unified multi-modal generators. arXiv preprint arXiv:2412.04333 2024b.
Size WU, Wenwei Zhang, Lumin Xu, Sheng Jin, Zhonghua Wu, Qingyi Tao, Wentao Liu, Wei Li, and Chen Change Loy. Harmonizing visual representations for unified multimodal understanding and generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17739-17750, 2025.
12

---

Published as a conference paper at ICLR 2026
Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. In The Thirteenth International Conference Learning Repre- sentations, b.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In The Thirteenth International Conference on Learning Representations, a.
Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal models. In The Thirty-ninth Annual Conference Neural Information Processing Systems, b.
Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632-10643, 2025.
Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from any- where to everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Hong-Xing Yu, Haoyi Duan, Charles Hermmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from single image. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5916-5926, 2025a.
Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsif Wong, Ying Shan, and Yonghong Tian. Viewerafter: Taming video diffusion models for high- fidelity novel view synthesis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025b.
Shangjin Zhai, Zhichao Ye, Jialin Liu, Weijian Xie, Jiaqi Hu, Zhen Peng. Hua Xue, Danpeng Chen, Xiaomeng Wang, Lei Yang. et al. Stargen: A spatiotemporal autoregression framework with video diffusion model for scalable and controllable scene generation. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 26822-26833, 2025.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986. 2023.
Jiahui Zhang, Yurui Chen, Yueming Xu, Ze Huang. Jilin Mei, Junhui Chen, Yanpeng Zhou, Yu-Jic Yuan, Xinyue Cai, Guowei Huang, et al. From flatland to space: Teaching vision-language models to perceive and reason in 3d. In The Thitty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, a.
Jiawei Zhang, Chejian Xu, and Bo Li. Chatscene: Knowledge-enabled safety-critical scenario gener- ation for autonomous vehicles. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1p.15545 15459-15469, 2024a.
Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. Transactions Machine Learning Research, b.
Xinjie Zhang, Jintao Guo, Shanshan Zhao, Minghao Fu, Lunhao Duan, Jiakui Hu, Yong Xien Chng. Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, et al. Unified multimodal understanding and genera- tion models: Advances, challenges, and opportunities. arXiv preprint wrXiv:2505.02367. 2025.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data. arXiv preprint 2rXiv:2410.02711 2024b.
Duo Zheng, Shijia Huang, Yanyang Li, and Liwei Wang. Learning from videos for 3d world: Enhancing milms with 3d vision geometry priors. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
13

---

Published as a conference paper at ICLR 2026
Duo Zheng, Shijia Huang, and Liwei Wang. Video-3d Ilm: Learning position aware video repre- sentation for 3d scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8995-9006, 2025.
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: learning view synthesis using multiplane images. ACM Transactions on Graphics (TOG), 37(4): 1-12, 2018.
Xin Zhou, Dingkang Liang, Sifan Tu, Xiwu Chen, Yikang Ding, Dingyuan Zhang, Feiyang Tan, Hengshuang Zhao, and Xiang Bai. Hermes: A unified self-driving world model for simultaneous 3d scene understanding and generation. In Proceedings of the IEEE/CVE International Confer- ence on Computer Vision, pp. 27817-27827, 2025.
Chenming Zhu, Tai Wang. Wenwei Zhang, Jiangmiao Pang, and Xihui Liu. Llava-3d: A simple yet effective pathway to empowering Imms with 3d capabilities. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 4295-4305, 2025.
14

---

Published as a conference paper at ICLR 2026
Limitations and future work. Although Omni- View has demonstrated unified advance in the domains question answering, spatial reasoning, and novel view synthesis, its grounding capabili- ties remain to be substantiated. Additionally, its generation model currently lacks the capability for long-range world generation. Future efforts will be concentrated on using reinforcement learning augment Omni-View" performance in 3D visual grounding and long-range generation. At the same time, since the training data used by its geometry module a synthetic depth map, its actual geometry prediction ability may not be accurate enough.
A QUALITATIVE RESULTS
We show some qualitative results for 3D scene understanding and generation tasks in the appendix.
A.1
3D SCENE UNDERSTANDING
Multiview images
Question:
am standing by the ottoman on my right facing a couple toolboxes.
What instrume in fro of me is ebony and ivory?
Answer: piano
Multiview images
Question:
Detect the bounding boxes in the coordinate of the first frame.
Answer: ("label": "door", "bbox_360 3d": (-0.29, 0.21, 0.63, 0.07, 0.6, 1.69, 2.0, 1.14, -3.09)),
("label": "vanity", "bbox_ 3d" : [0.6, 0.47, 0.53, 0.59, 1.36, 0.7, 0.43, 1.14, -3099)).
("label": "towel", "bbox_3d" 3d" [0.31, 0.07, 0.79, 0.25, 0.35, 0.09, 2.0, 1.14, -3.09)),
("label": "sink", "bbox_3d" [0.57, 0.21, 0.53, 0.42, 0.63, 0.15, 0.43, 1.14, -3.09]),
("label": "doorframe", "bbox_ 3d": [-0.28, 0.19, 0.59, 0.2, 0.85, 1.63, 2.0, 1.14, -3.09)),
("label": "switch", "bbox_3d"" [0.16, -023, 0.66, 0.02, 0.08, 0.11, 2.0, 1.14, 33.9]],
("label": "mirror", "bbox_3d" 3d": [0.86, 0014, 0.44, 0.08, 1.06, 0.13, 0.43, 1.14, 3.09]]
Image
Image
Image
Image
15

---

Published as a conference paper at ICLR 2026
A.2 SPATIAL REASONING
Multiview images
Question:
What the this square meters)? multiple rooms are shown, estimate the size the combined space.
Answer: 10.3
Ground Truth: 10.5
(a) Results on room size estimation.
Multiview images
Question:
What will be the first-time appearance order of the following categories the video:
heater, ceiling light, printer? A, ceiling light, heater, tv, printer ceiling light, tv, heater, printer C. heater, tv, celling light, printer
Answer: ceiling light, heater, printer, tv
Results appearance order.
Multiview images
Question:
Measuring from the point of each object,
which of these objects (stove, tv. table, sofa) is the closest to stool?
Options:
table
sofa
Answer with the option's letter from the given choices directly.
Answer: D.
GT: D.
Object relative distance.
16

---

Published as a conference paper at ICLR 2026
Multiview images
Question:
How many chair(s) are in this room?
Please answer the question using single word phrase.
Answer: 4.
GT: 4.
(d) Object counting.
Multiview images
Question:
What is the length of the longest dimension (length, width, or height) of the sofa, measured centimeters?
Please answer the question using single word phrase.
Answer: 184.
GT: 173.
Object estimation.
Multiview images
Question: Measuring from the closest point of each object,
what is the distance between the and (in meters)? answer the question using single or phrase.
Answer: 1.0.
GT:
(f) Object absolute distance.
17

---

Published as a conference paper at ICLR 2026
Multiview images
Question:
If am standing the bathtub and facing the tollet, the table my front-left, front-right, back-left, back-right?
The directions refer the quadrants of a Cartesian plane standing origin and facing along the positive y-axis).
Options:
back-left
front-right
front-left
back-right
Answer with the option's letter from the choices directly,
Answer: B.
GT: B.
(8) Object relative direction.
Multiview images
Question:
You are robot beginning at the tv facing the bed.
You want navigate to the trash bin.
You will perform the following actions
(Note: for each (please fill in), choose either 'turn back,' 'turn left," or 'turn right/)):
1. [please fill in]
2. Go forward until the cabinet
3. [please fill in]
4. Go forward until the trash bin is your right.
You have reached the final destination.
Options:
A. Turn Left, Turn Left
B. Turn Right, Turn Left
C. Turn Back, Turn Left
D. Turn Right, Turn Right
Answer with the option's 's letter from the given choices directly.
Answer: B.
GT: B.
(h) Route planning.
18

---

Published as a conference paper at ICLR 2026
A.3 SCALE CONSISTENCY ACROSS SCENES
We analyze the scale consistency across different scenes. To this end, we selected some test samples from the SPAR-7M dataset and used Omni-View perform absolute distance prediction based multi-view images. This SPAR-7M consists of scenes from three datasets: ScanNet and ScanNet++, where the average scene depth ranges from and Structured3D, which features larger-scale environments with average depths between 4 and meters and maximum depths reaching up to 20 meters. Qualitative visualizations demonstrate that Omni- View can predict absolute depth in scenes with different metric scale.
meters with a maximum depth of 12 meters;
Multiview images
Question:
The Object63 (red point at Frame-0, [0.14, 0.67)) is positioned at 3.4 meters depth. Predict the depth of the Object22 (blue point at Frame-0, [0.23, 0.65)).
Calculate judge based on the 3D center points of these objects.
Answer: Object22's central depth estimated to be about 2.0 meters.
GT: With a central depth 2.2 meters, Object22 is referenced here.
(a] Absolute depth estimation structured3d, cene_03201,
Multiview images
W/
Question:
The Object26 (at Frame-1, J0.10, 0.39]] lies at 2.7 meters depth. Predict the depth Object30 (at Frame-1, [0.10, 0.70J). Calculate or judge based the 3D center points of these objects.
Answer: Object30 is centered at about 4.0 meters depth.
GT: At its center, object Object30 has depth of 4.3 meters.
(b) Absolute depth estimation structured3d, scene_ scene_02053
02053_25 25.
Multiview images
Question:
if the plant (Frame-1, J0.30, 0.41]] positioned meters, estimate the depth of the laptop (at Frame-1, [0.16, 0.63]). Calculate or judge based 3D center points of these objects. The answer should single number, nothing more..
Answer: 2.4
GT: 2.3
Absolute depth estimation scannetpp, 9d7e20fbec 6022.
19

---

Published as a conference paper at ICLR 2026
A.4 NVS FROM SINGLE VIEW
Indoor scenes. We present the two visual results of NVS in indoor scenes. Omni-Vies can reason- ably imagine unseen areas while maintaining the texture and structure of observed objects, like the followers on the table in the first row. Reference
View-12
View-24
View-36
View-48
We present the two visual results of NVS in outdoor scenes. When the camera movement is small, Omni- View can consistently generate new views.
Outdoor scenes.
Reference
View-8
View-16
View-24
View-32
Depth estimation. We present the depth prediction results of Omni- in indoor and outdoor scenes. In indoor scenes, the output of Omni- View's 's geometry module is more accurate.
Reference
View-10
View-20
View-30
View-40
Failure case. We claim that the existing Omni-Vise is inadequate for effectively processing outdoor scenes with substantial camera movement. As illustrated the figure below, the result images from Omni- View exhibit significant artifacts, highlighted within the red dashed boxes. Future work will focus on resolving the following challenges to enhance the handling of outdoor scenes with extensive camera motion: (1) the development of more precise camera control mechanisms, and (2) the improvement inter-frame texture consistency stability.
Reference
View-2
View-4
View-6
View-8
20

---

Published as a conference paper at ICLR 2026
VAE
Decoder
Denoised gnd image tolens
Texture Module
poses' placker in
Text Tokenizer
VAE Encoder
"A hed."
Captions (Ontional)
Reference imagos
Texture latent
Figure 2: Generation in grid.
A.5 EFFECT OF GENERATION IN GRID
Generation in grid. We explored small improvements to enhance inter-frame consistency and sup- port long-sequence scene generation: "generation in grid" Hu al. (2026). Specifically, we arrange 4 consecutive views into single frame by organizing them in grid layout, and perform autoregres- sive generation over the resulting sequence of grid-organized frames.
Results. As shown in the figures the next page, demonstrate that after using "generation in grid", Omni- View can generate more consistent novel view images.
21

---

Published as a conference paper at ICLR 2026
Reference
"Generation in grid"
View-4
View-8
View-12 12
View-16
View-20
View-24
Reference
"Generation in frame-by-frame
View-4
View-8
View-12
View-16
View-20
View-24
Figure 3: Different inference strategy in NVS (RealEstate e702004499/5ec737), The first three rows use the generation in grid strategy, achieving significant improvements in inter-frame consis- tency. The last three rows still use the frame-by-frame generation strategy, which has poor inter- frame consistency.
22

---

Published as a conference paper at ICLR 2026
B TECHNICAL DETAILS
B. DATASET
3D scene understanding and spatial reasoning. We curate filtered training dataset contain- ing 780k valid samples by consolidating data from multiple sources, including SQA3D, ScanQA, 3DOD, ScanRefer, VLM-3R, 234k subset of SPAR from VG-LLM, and 64k subset of Ilava- hound4 from VG-LLM. To ensure data quality, we perform two main filtering steps: (i) deduplica- tion across the combined dataset, and (ii) removal samples with invalid bounding box annotations. Specifically, we convert all bounding boxes to the (ar ;1, V2] format and exclude any instance height.
where #1 0, 91 < 0, 322 > width, or 1/2
Novel view synthesis. We select 61k video clips from RealE10K train set, excluding those with fewer than 32 frames or significant motion blur that could degrade training stability. Depth maps are synthesized using the Voyager data pipeline, while captions are generated via QwenVLMar
B.2 ARCHITECTURE.
Understanding model. The detailed architecture shown in Figure 4. The text tokenizer inher- its from the vocabulary used by Qwen2. The image tokenizer for understanding is SigLIP. The backbone has layers as 28, attention head as 28, and hidden size as 3584, totaling 7B parameters. Key architectural choices include SwiGLU the activation function, RMSNorm for normalization, Attention (via PyTorch) for efficient self-attention computation, and position encoding adapted from Qwen2.
VAE Decoder
Detokenizer
SwiGLU
Tmanned mad line iage
Understanding Model
RMSNorm
Texture Module
Uormpeee alacer riiing
Attentio
Text
SigLIP
Tokenizer
Tokenizer
RMSNom
Ui by Besik diaaggle.
Reforiee
Figure 4: Architecture of understanding model and texture module.
Texture module. This module uses FLUX-VAE as the image tokenizer, sharing the same backbone architecture as the understanding model.
VAE
VAE
Decoder depel
Decoder
Dieetor depth rakine
m m
Geometry Module
Geometry Module
Doxiiec uxfiiiie image Milso
Livenorr bacee 3200
Posz query
Damocrd medtenten
Eum bodaue medall
(1) Stage
(2) Stage
Figure 5: Architecture of geometry module in two stages.
Geometry module. The detailed architecture is shown in Figure 5. Depth maps are also encoded by FLUX-VAE. Its backbone is basically the same as the block architecture, but reduced 4 layers, containing about IB parameters. Unlike the others, the architecture of the geometry module changes across different training stages. In stage 1, it receives features from the understanding model for
23

---

Published as a conference paper at ICLR 2026
cross-attention. In stage 2, no longer receives outputs from the understanding model. The reason for this design can found in the discussion of stage 2 in Section 3.2.
B.3 TRAINING AND COMPUTATIONAL COST.
All training phases the AdamW optimizer with 31 0.9, B2 0.95, peak learning rate of 1 x 10-5, and linear warm-up schedule covering the first 5% of iterations.
Stage 1. We train on the 3D scene understanding dataset for 10,000 iterations with packed quence length of 50k, using 32 H100 GPUs, which takes approximately 160 hours.
Stage 2. For the generation task, we perform 20,000 iterations the novel view synthesis dataset with a packed sequence length of 32k, using 32 H100 GPUs, requiring approximately 40 hours in total.
Understanding inference. The model takes approximately 2.5 seconds on average to process 32-frame multi-view scene understanding query using single H100 GPU.
Generation inference. Generating single image at resolution 640 x 352 takes about 2.2 seconds on average with one H100 GPU.
B.4 CONVERGENCE BEHAVIOR.
Stage 1. Overall, most losses exhibit smooth and stable convergence. However, observe spikes in the camera pose loss during training, particularly later epochs. We hypothesize that this behavior may stem from instable optimizion by learnable queries in the camera pose estimation task. How- ever, these fluctuations do not prevent the model from converging to effective solutions, as evidenced by strong performance 3D scene understanding, spatial reasoning, and novel view generation.
24

---

Published as a conference paper at ICLR 2026
Understanding Loss
Undenstanding loss
Undenttadinng loss IEMA weightsD.
3.8
0.7
0.5
0.2
Step
Texture Loss Curve
0.32
0.30
0.24
Reciura Loss
Tecture Loos JIMA, we ghemo..c.
2000
Stepp
Depth Loss Curve
1.6
Depth
Dopth weight=0.8k
LoSs
0.8
2000
4000
5000
SODO
Stop
Camera Pose Loss Curve
2.00
Camera
Camera Pose loss JEWA, weight=0 91
1.75
1.00
0.75
0.50
2000
4000
B000
25

---

Published as a conference paper at ICLR 2026
Stage 2. Since used grid generation format in stage 2, the per-image prediction method in stage cannot be directly transferred to the "generation in grid" paradigm in stage 2. However, due to the generative prior learned in stage I, Omni- View quickly converge to lower loss in stage 2. However, it was observed that the geometry loss showed a slight increase at around 8000 iterations, followed by decrease, suggesting that it might be possible to further reduce Ageo improve the training stability.
Texture Loss Curve
weight-B.8)
1.2
3.0
0.6
0.4
0.2
2500
5000
7500
10000
12500
15000
17500
20000
Step
Depth Loss Curve
Dopth loss
Depth (EMA weight-n.at at
1.2
1.0
0.6
0.4
0.2
25:00
500D
7500
10000
12500
15000
17500
20000
Step
Camera Pose Loss Curve
Camrraa Pose losa
Cameraa Pose loss IeWA weight-D.at at
2.0
3.0
0.5
Step
B.5 ACTIVATION VISUALIZATION
We visualize the activation maps of Bagel and Omni- when they perform spatial reasoning tasks, as shown in the figure below. In this example, we want the model to locate how many cab- inets are in the current scene, using the prompt: "How many cabinet (s) are in this room?". It can be seen that Bagel mainly focuses the first two images that irrelevant to
26

---

Published as a conference paper at ICLR 2026
the question, whereas Omni- View is able to attend to each image as much as possible. The wider attention may be the reason why Omni- View performs better on 3D scene understanding tasks.
BLAEOMAIL
Bagel-FT
BLACKMMATE
BLACKMANT
Omni-Vies (Ours)
Figure 6: Activation map. Bagel-FT vs. Omni-View.
27

---

Published as a conference paper at ICLR 2026
C THE USE OF LARGE LANGUAGE MODELS (LLMS)
LLMss are used to correct potential grammatical inaccuracies in the manuscript. LLMs do not par- ticipate in research ideation.
28


Reviews:
[{"review_id": "0U5CuZzdRY", "reviewer": "Reviewer_Meww", "summary": "The paper presents Omni-View, a unified model for 3D scene understanding and generation from multiview images that tests the “generation facilitates understanding” hypothesis. Built on Bagel, it splits generation into a texture module (flow matching with Plücker pose encoding, autoregressive NVS) and a geometry module (depth and camera pose via flow matching with cross-attention to understanding features). A two-stage training recipe first jointly trains understanding/texture/geometry with a dense-to-sparse curriculum, then fine-tunes generation with RGB-Depth-Pose joint learning. Omni-View achieves SOTA on VSI-Bench (55.4), improves QA/localization versus unified baselines without 3D inputs, and delivers strong NVS/scene generation results on Re10k.", "questions": "- Clarify the novelty beyond architectural decoupling: In what ways is the texture/geometry split more than a clean engineering separation compared to prior “appearance vs. structure” decouplings (e.g., ViewCrafter, Voyager) and unified frameworks (BAGEL, VILA-U, BLIP3o, Harmon)? Could you provide a controlled comparison to a single-branch generator with two prediction heads (texture, geometry) at equal parameter count?\n- The ablations show AR improves spatiotemporal reasoning. Can you report exposure-bias analyses at inference time, e.g., teacher-forcing vs. free-running rollouts? Does diffusion forcing mitigate compounding errors, and how does performance vary with rollout length (8/16/32 frames)? Have you tested scheduled sampling or token-level AR only on camera poses while keeping texture bidirectional?\n- Provide qualitative and quantitative failure analyses: cases where geometry improves understanding but harms texture fidelity (and vice versa), per-category VSI error tied to pose/depth errors, and sensitivity to large viewpoint changes where you noted inconsistencies. Would integrating a small, explicit 3D proxy (e.g., Gaussians or sparse point clouds) at training but not inference close these gaps?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper presents a unified 3D understanding–generation framework that cleanly separates texture and geometry, a simple yet original design that operationalizes “generation facilitates understanding.”\n- The two-stage recipe with dense-to-sparse curriculum and autoregressive NVS is well-motivated, technically sound, and shows careful loss design and gradient routing to benefit the understanding model.\n- Writing is clear and structured, with concrete training details, datasets, metrics, and ablations that isolate the contribution of each module and training choice.\n- The empirical significance is strong, with SOTA results on VSI-Bench and competitive 3D QA/localization, plus solid NVS and scene generation metrics, demonstrating broad impact across 3D reasoning and generation.", "weaknesses": "- Limited novelty relative to prior unified frameworks (Bagel, VILA-U, BLIP3o, Harmon)  \nThe core idea of leveraging generation to aid understanding has precedents in 2D unified models and recent 3D works that inject reconstruction priors (e.g., Ross3D; VG-LLM/Spatial-MLLM via VGGT features). The split into texture vs. geometry resembles established “appearance vs. structure” decouplings in 3D pipelines (e.g., ViewCrafter, Voyager). Clarify what is fundamentally new beyond integrating these pieces within Bagel, and compare to a “single-branch with multi-heads” backbone.\n\n- Ambiguity in camera control and absolute metric grounding   \nThe paper reports strong perceptual metrics but acknowledges difficulty in precise camera control and absolute depth scale. Because the gains on VSI categories like Abs. Dist. hinge on metric grounding, add analyses: scale consistency across scenes, depth-scale calibration via known baselines, and camera-pose accuracy vs. ground truth under diverse motions.\n\n- Dataset overlap and generalization concerns  \nThough the authors state they avoid using understanding images for generation training, several datasets share scene domains with Re10k-like indoor content, risking leakage of priors. Please report cross-dataset generalization (e.g., ScanNet -> Replica, RealEstate10K -> ACID/CO3D subsets) to support robustness claims.\n\n- Incomplete ablations on design choices and routing  \nThe geometry module conditions only on the last-layer texture latent and uses cross-attention to the understanding model. Test alternatives: multi-scale latents, earlier-layer features, and gating that controls gradient flow to avoid potential interference. Provide compute/latency breakdowns for stage 1 vs. stage 2, and show sensitivity to $\\lambda_{geo}$, pose-query design, and Plücker vs. other pose encodings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ez231SW9PJ", "reviewer": "Reviewer_Lkur", "summary": "- This paper proposes Omni-View, a unified multimodal model that jointly performs 3D scene understanding, geometry estimation, and novel view synthesis from multiview images, based on the principle that generation facilitates understanding.\n\n- This paper introduces a dual-path architecture consisting of a texture module (appearance generation) and a geometry module (depth/pose estimation), enabling bidirectional synergy between generative and understanding tasks.\n\n- This paper uses a two-stage training strategy where joint training enhances understanding via generative signals, followed by refinement for high-quality 3D scene generation; achieves state-of-the-art performance on VSI-Bench and strong results in NVS and 3D Q&A.", "questions": "Mentioned in the weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This paper demonstrates that generative 3D tasks (novel view synthesis, geometry estimation) can actively enhance 3D scene understanding, rather than being separate objectives.\n\n- This paper has a unified architecture for 3D reasoning, with separate texture and geometry modules allow complementary learning of appearance and spatial structure, leading to better localization, spatial reasoning, and depth-aware Q&A.\n\n- This paper outperforms specialized models in 3D understanding benchmarks while maintaining competitive NVS and scene generation performance, closing the gap between multimodal understanding and 3D generative models.\n\n- A systematically organized evaluation and ablation study would strengthen the credibility of this paper.", "weaknesses": "- It would be beneficial to include a diagram that more precisely illustrates the functionality of each module and the architecture, compared to the current version.\n\n- Additionally, visualizing 3D scene understanding / spatial reasoning / NVS from a single view as a video could also be an effective way to present the capabilities of the system.\n\n- Is there a reason why you refer to Texture Module and Geometry Module in the equations (e.g., (eq. 1), (eq. 2)) without using italics? Also, I believe writing them as TextureModule and GeometryModule (without a space) would improve readability and be more suitable for mathematical notation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2fnzQztzsg", "reviewer": "Reviewer_U2JG", "summary": "This paper introduces Omni-View, a unified model for 3D scene understanding and generation that explicitly investigates the hypothesis that “generation facilitates understanding.” The contributions of the paper are:\n1. Unified architecture integrating 3D scene understanding and generation, composed of three main components:\n1.1 Understanding model (for spatial reasoning and QA)\n1.2 Texture module (for novel view synthesis)\n1.3 Geometry module (for depth and pose estimation)\n2. Proposed a novel two-stage training strategy:\n2.1 Jointly trains understanding and generation to encourage mutual benefits through geometry and spatiotemporal modeling.\n2.2 Fine-tunes generation with RGB-Depth-Pose joint learning for better geometric consistency.\n3. Empirical validation showing state-of-the-art (SOTA) performance on the VSI-Bench (score 55.4), outperforming both specialized and unified 3D models in reasoning tasks.\nOverall, Omni-View demonstrates how generative modeling (novel view synthesis, geometry estimation) can enhance 3D reasoning, localization, and understanding—a conceptually elegant and empirically supported contribution.", "questions": "1. Could the authors provide qualitative results of the geometry module’s depth estimation, camera pose estimation with metric-scale? Without any depth estimation results or broader test cases, it is difficult to assess whether the geometry module truly learns meaningful 3D structure rather than overfitting to training priors, without any generalizability.\n2. How well does Omni-View generalize to unseen or real-world multi-view scenes, rather than the well captured ones in the appendix? Have the authors tested its performance to verify the robustness and effectiveness of the learned spatial reasoning?\n3. Could the authors clarify key implementation and training details—such as dataset splits, optimizer configurations, training epochs, and computational cost—to ensure reproducibility? Including more specifics or releasing supplementary materials would improve transparency.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Clear intuition and solid empirical validation.\nThe paper builds upon a clear and intuitive idea — that generation can facilitate understanding — and the overall logic is easy to follow. Quantitative results across multiple benchmarks convincingly demonstrate the benefits of the proposed design, especially in spatial reasoning and novel view synthesis.\n2. Architectural innovation.\nBy decomposing the generation process into texture and geometry modules, the authors present a meaningful and modular architecture that captures both appearance and structure. This decomposition aligns well with human visual reasoning and can be viewed as an innovative contribution for the community.\n3. Comprehensive ablation studies.\nThe ablation results thoroughly verify the contributions of the proposed contributions. These analyses effectively demonstrate that each component contributes to the final understanding and reasoning performance.", "weaknesses": "1. The qualitative results in the appendix are sparse, and there are no depth estimation visualizations or broader test cases. This makes it difficult to verify the model’s generalization and effectiveness beyond the reported metrics. For instance, the quality and consistency of metric-scale prediction from the geometry module remain uncertain — the reported results could be influenced by selective visualization or data bias, since the paper lacks convincing examples that demonstrate accurate geometric reasoning across varied real-world scenes.\n2. The technical details provided for training, implementation, and comparison setups are relatively limited. Without clearer supplementary material (e.g., dataset statistics, architecture specifics, or convergence behavior), it is challenging to fully reproduce the reported results or assess robustness under different conditions.\n3. The absence of released code or live demonstrations restricts the ability of other researchers to validate or extend this work. Although acceptable for review, the paper would be strengthened by open-sourcing its checkpoints or providing additional evaluation on long-range world generation and 3D visual grounding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cwrqwh8XP2", "reviewer": "Reviewer_GSip", "summary": "This paper proposes Omni-View, a unified 3D scene understanding and generation model based on multi-view imagery, exploring the principle of \"generation promotes understanding.\" The model, composed of an understanding module, a texture module, and a geometry module, jointly models scene understanding, novel view synthesis, and geometry estimation. Using a two-stage training strategy, Omni-View achieved a state-of-the-art score of 55.4 on the VSI-Bench, surpassing existing dedicated 3D understanding models while also performing well in novel view synthesis and scene generation tasks.", "questions": "Strengthen theoretical analysis or visual explanation of the \"generation promotes understanding\" mechanism;\n\nCompare with state-of-the-art methods on more 3D grounding tasks and analyze the sources of the gap.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Unified modeling is novel: This is the first systematic exploration of the \"generation-driven understanding\" mechanism in 3D scenes, which is both inspiring and forward-looking.\n\nRational modular design: The generation module is split into texture and geometry components, modeling appearance and structure respectively, effectively improving understanding capabilities.\n\nEffective training strategy: The two-stage training (unified training + generation fine-tuning) balances understanding and generation performance, and the D2S mechanism improves robustness.\n\nComprehensive experiments: The effectiveness of the method is verified on multiple 3D understanding, spatial reasoning, and generation tasks, with results significantly outperforming existing unified models.\n\nNo 3D input required: Relying solely on multi-view images, this improves the model's practicality and generalization capabilities.", "weaknesses": "Weak theoretical analysis: Although \"generation promotes understanding\" has been proposed, there is a lack of theoretical or interpretable analysis of its underlying mechanisms.\n\nGeneration quality still has room for improvement: Despite leading in PSNR/SSIM, inter-frame consistency under large viewpoint variations remains suboptimal (see Appendix visualization).\n\nThe geometry module relies on synthetic data: The depth map is synthesized by Voyager, which may limit the realism and accuracy of geometry predictions.\n\nLimited long sequence generation capability: The model currently does not support long sequence scene generation, limiting its application in open-world scenarios.\n\nThere is still a gap compared to state-of-the-art dedicated models: In particular, in the 3D grounding task, there is still a significant gap compared to methods that rely on 3D input.Strengthen theoretical analysis or visual explanation of the \"generation promotes understanding\" mechanism;\n\nCompare with state-of-the-art methods on more 3D grounding tasks and analyze the sources of the gap.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)