Dynamic Focused Masking for Autoregressive Embodied Occupancy Prediction
Authors: Yuan Sun
Julio Contreras
Jorge Ortiz
Venue: NeurIPS 2025 poster ()
Keywords: 3D gaussian splatting
indoor Occupancy Prediciton
Primary area: deep_learning
TL;DR: An Autoregressive Gaussian Splatting Framework for Indoor 3D Occupancy Prediction

Abstract: Visual autoregressive modeling has recently demonstrated potential in image tasks by enabling coarse-to-fine, next-level prediction. Most indoor 3D occupancy prediction methods, however, continue to rely on dense voxel grids and convolution-heavy backbones, which incur high computational costs when applying such coarse-to-fine frameworks. In contrast, cost-efficient alternatives based on Gaussian representations—particularly in the context of multi-scale autoregression—remain underexplored. To bridge this gap, we propose DFGauss, a dynamic focused masking framework for multi-scale 3D Gaussian representation. Unlike conventional approaches that refine voxel volumes or 2D projections, DFGauss directly operates in the 3D Gaussian parameter space, progressively refining representations across resolutions under hierarchical supervision. Each finer-scale Gaussian is conditioned on its coarser-level counterpart, forming a scale-wise autoregressive process. To further enhance efficiency, we introduce an importance-guided refinement strategy that selectively propagates informative Gaussians across scales, enabling spatially adaptive detail modeling. Experiments on 3D occupancy benchmarks demonstrate that DFGauss achieves competitive performance, highlighting the promise of autoregressive modeling for scalable 3D occupancy prediction.

Dynamic Focused Masking for Autoregressive Embodied Occupancy Prediction
Yuan Sun' Julio Contreras' Jorge Ortiz!
Rutgers, The State University of New Jersey New Brunswick, NJ, USA 08901
ya8200aoe.rutgers rutgees..du.
Abstract
Visual autoregressive modeling has recently demonstrated potential in image tasks enabling coarse-to-fine, next-level prediction. Most indoor 3D occupancy predic- tion methods, however, continue to rely on dense voxel grids and convolution heavy backbones, which incur high computational costs when applying such coarse-to- fine frameworks. In contrast, cost-efficient alternatives based on Gaussian rep- resentations particully in the context of multi-scale intorggression-remiin underexplored. To bridge this gap, we propose DFGauss, a Dynamic Focused masking framework for multi-scale 3D Gaussian representation. Unlike conven- tional approaches that refine voxel volumes or 2D projections, DFGauss directly operates in the 3DD Gaussian parameter space, progressively refining representations across resolutions under hierarchical supervision. Fach finer-scale Gaussian is conditioned on its coarser-level counterpart, forming scale-wise autoregressive process. To further enhance efficiency, we introduce an finement strategy that selectively propagates informative Gaussians across scales, enabling spatially adaptive detail modeling. Experiments on 3D occupancy bench- marks demonstrate that DFGauss achieves competitive performance, highlighting the promise of autoregressive modeling for scalable 3D occupancy prediction.
re-
Introduction
With the accelerating progress in embodied intelligence and the deployment of active agents across domains such as robotics and autonomous navigation, spatial understanding has become a critical capability for intelligent systems [6, 40, 12]. To navigate indoor environments effectively, embodied agents must perform various perception tasks, among which occupancy prediction [38] plays a fundamental role in enabling agents interpret and interact with complex real-world spaces. A key challenge in occupancy prediction lies in the trade-off between resolution and completeness: Small voxel sizes cause holes and missing details, while larger voxels lead over-smoothed, inaccurate scosetry-issues that coarse-to-fint strategies address by progressively refining spatial resolution and structural fidelity [30]. Inspired by how humans perceive visual information from global context to local detail in a hierarchical manner [29]-recen] recent advances in autoregressive modeling [29, 28, 23, 20, 13] have shown great promise for addressing coarse-to-fine generation in 2D vision tasks, suggesting strong potential for enhancing 3D occupancy prediction.
While some efforts have attempted extend autoregressive modeling to 3D volumetric representa- tions using dense voxel grids [33, 1]. these approaches often incur substantial computational overhead, limiting their scalability and generalizability. In contrast, efficient Gaussian- -baee representations of- fer promising alternative, as they are both lightweight and capable of delivering strong performance. However, the autoregressive paradigm remains largely underexplored in the context of Gaussian- based methods, leaving gap in effectively leveraging hicrarchical modeling within computationally efficient 3D spatial frameworks.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ego-Centric Exploration
Multi-scale Gaussian Refinement
Multi-scale
Gaussian Autoregressive
Supervision
Hierarchy
P(g
Embodied Agent
Selected
Gaussian
Ego-centric Observation
Image to Gaussian
Gaussian Voxel Splatting
Figure 1: Overview of the proposed DFGauss framework. An embodied agent acquires egocentric RGB observations while navigating the environment, which are transformed into multi-scale Gaussian representations for autoregressive refinement. A selective masking strategy focuses computation on informative regions. In the illustrated example, the model identifies bed (shown in peach) and ceiling (shown in dark red) through hierarchical Gaussian updates.
In this paper, we propose DFGauss, a Dynamic Focused masking framework for multi-scale 3D Gaussian representation (Figure 1). Our novelty lies in multi-scale Gaussian Splatting approach for 3D scene understanding that explicitly addresses the above mentioned issue through dynamic, -rentioo--uuiedd
attention guided refinement. The core novelty of DFGauss lies in applying hierarchical autoregressive refinement directly to a sparse set of learnable 3D Gaussian primitives, avoiding reliance on dense volumetric or grid-based features. Unlike classical autoregressive models [29] that generate discrete tokens, DFGauss models spatial dependencies at each finer scale predicted based on coarser-level outputs and corresponding encoder features. This mirrors the coarse-to-fine "next-scale prediction" " strategy in Visual AutoRegressive (VAR) modeling [29], while diverging in its application to continuous Gaussian fields for structured 3D occupancy prediction. To improve efficiency, DFGauss incorporates a dynamic focused masking mechanism that selectively updates only the most informative Gaussians each level, reducing computational cost without compromising reconstruction quality of this paradigm. Together, these components form unified framework for efficient and expressive 3D spatial modeling, leading to the following contributions:
continuous space, where Gaussian parameters
A novel multi-scale autoregressive hierarchical 3D Gaussian Splatting framework that enhances 3D occupancy prediction for embodied agents.
A coarse-to- toarse-to-fine supervision strategy that progressively refines Gaussian parameters across scales using multi-resolution labels.
A dynamic focused masking mechanism that improves the efficiency of Gaussian refinement by selectively updating informative regions.
Extensive experiments on multiple datasets demonstrating state-of-the-art performance in 3D occupancy prediction.
Related Work
2.1 3D Occupancy Prediction for Embodied Agents
Among various 3D perception tasks, occupancy prediction has emerged as a compact and expres- sive representation for modeling spatial semantics. While outdoor occupancy prediction has been extensively studied in the context of autonomous driving [33, 41, 32, 21, 9, 31, 4, 16], indoor envi- ronments remain relatively underexplored despite their critical role embodied Al and robotics. MonoScene [3] proposes voxel-based framework that infers occupancy from a single RGB image using a 2D-to-3D U-Net architecture with contextual priors. To enable real-time exploration, Em- bodiedOcc [34] introduces Gaussian-based memory refinement scheme. However, most existing

---

approaches still rely on dense 3D voxel grids [33, 19, 18], and current Gaussian-based methods remain in early stages of development. In particular, the rich features generated during Gaussian splatting have not been fully exploited. In this work, we propose novel framework that refines sparse Gaussian features in multi-scale autoregressive manner, combining the efficiency of Gaussian-based representations with enhanced accuracy in occupaney prediction.
2.2 Multi-scale Autoregression
Recent advances in visual autoregressive modeling (VAR) introduce multi-scale next-resolution prediction strategy that amplifies supervision signals and enhances robustness, setting new bench- marks in 2D generation efficiency and scalability [29, 22, 11, 5]. Inspired by this, recent works have extended multi-scale modeling to 3D tasks such as occupancy prediction in autonomous driv- ing. SurrounOOce [33], for instance, employs multi scale 2D-3D attention to enable dense spatial reasoning and supervision. NOMAE [i] introduces a multi-scale scale self-supervised framework for LiDAR point clouds that focuses on localized occupancy reconstruction without modeling the full 3D volume. OctreeOcc [24] adopts an adaptive octree-based representation to support efficient and fine-grained occupancy prediction while reducing computational cost. Despite their differences, these methods rely on dense convolutional or transformer-buse backbones, In contrast, the recent rise of Gaussian Splatting offers a cost-effective and compact alternative for 3D scene representation via continuous 3D Gaussian primitives. However, its integration with multi-scale optimization remains underexplored. We posit that coarse-to-fine autoregressive modeling in the scale space-where iner-scale representations are conditioned on coarser ones provides an efficient and principled approach for structured refinement in sparse 3D representations.
2.3 Gaussian Splatting
3D Gaussian Splatting [14] has recently emerged a compelling alternative to traditional volumetric and mesh-based rendering methods, offering real-time and high-fidelity radiance field rendering via anisotropic Gaussian primitives. Building on this foundation, series of follow-up works aim to improve its efficiency and adaptability. Mip-Splatting [39] introduces a low-pass filtering mechanism to mitigate aliasing artifacts caused by the sampling-sensittee nature of splats. Multi-scale Gaussian Splatting [35] extends the framework to dynamic scenes by modeling object motion using MLPs. Other works focus on redundancy reduction: LightGaussian [7] and Compact3DGS [17] rank Gaussians by scale or opacity prune uninformative primitives, significantly reducing memory and computation costs. Motivated by this line of research, we propose a dynamic focus masking mechanism that adaptively selects informative Gaussians across scales, enabling more efficient and scalable optimization in multi-scale 3D spaces, particularly for structured scene understanding. temporal consistency, and downstream prediction tasks in complex indoor environments with diverse spatial layouts and semantic variations common in embodied Al applications.
3 Method
Our model converts 2D image features into 3D Gaussian representations via cross-attention and sparse convolution. As illustrated in Figure 2, we introduce a hierarchical refinement framework that jointly optimizes multi-scale image features and Gaussian parameters across spatial resolutions. Given a monocular RGB image, a Hierarchical Feature Generation module extracts a multi-scale feature pyramid. To provide geometric cues, incorporate a depth prediction network [36] to estimate depth maps at each level of the hierarchy. Each scale-specific feature map is then processed by a corresponding Gaussian Encoder, which predicts initial Gaussian parameters. Inspired by prior work [10, 34J, we design a multi-scale Gaussian encoder that produces 3D Gaussians at different resolutions directly from the image features. These parameters are progressively refined by a Multi- Scale Gaussian Refinement module, where each level is conditioned on the output of the coarser scale via attention- fusion, forming an autoregressive refinement process. For global Gaussian update, we adopt the same confidence confidence-guided -giided refinement strategy as in [34], selectively updating only the Gaussians corresponding to the original resolution. Finally, lightweight Gaussian-to- Voxel decoders apply 3D Gaussian splatting to project the refined Gaussians onto voxel grids, generating occupancy volumes at each resolution. Supervision provided across all scales using multi-resolution 3D) occupancy labels, facilitating effective learning of coarse-to-fime geometric structures. For training,
3

---

100%
Multi-scale Gaussian Encoder
Gaussian to Voxel Dş
Gaussian
Encoder
Hierarchical Feature
Eg
Multi-sed -es--call Gaussian
50%
Refinement
Gaussian to Voxel Ds-1
Generation
Gaussian
Encoder Es-1
30%
Gaussian Encoder E
Gaussian to Voxel
D
Figure 2: Overview of the DFGauss framework. Given an indoor monocular RGB image, a Hi- erarchical Feature Generation module extracts features multiple granularity. These multi-scale features are encoded into Gaussian parameters by an Multi-scale Gaussian Encoder. The Multi-Scale Gaussian Refinement Module further optimizes these Gaussians across the hierarchy. Finally, Gaussian-te to Voxel decoder predicts 3D occupancy from the multi-scale features. Supervision is provided by 3D occupancy labels multiple resolutions.
we adopt the loss formulation from [10], combining Focal Loss, Lovász Loss, Semantic- Loss, and Geometric-Scabe Loss to jointly address class imbalance, boundary precision, and multi-scale consistency. In summary, our framework establishes fully image-based Gaussian representation pipeline monocular 3D occupancy prediction across multiple spatial granularities.
3.1 Multi-Scale Gaussian Encoder
Inspired by recent works [10, 34], we propose a Gaussian encoder that operates at each hierarchical scale of the image feature pyramid. Given image features specific scale s, the encoder first initial- izes a sparse set of 3D Gaussian primitives within camera frustum. Each Gaussian is parameterized by tuple g; (*) {us, X; dis oi, 1. where μi RR is the mean position, A; R are scale factors, q: E R- is rotation quaternion, o, IR is the opacity, and RC are semantic logits over C' classes.
To incorporate visual cues, each Gaussian lifted into a high-dimensional feature space via feature alignment module, which integrates local image descriptors with geometric priors from depth-aware branch. These features are then refined through a combination of self-attention (among Gaussians) and cross-attention (with multi-scale image features), enabling context-aware adaptation to the observed scene geometry.
To extend this multi-scale design, we construct hierarchy of Gaussian encoders {E operating S scales of image features (F(*) ===1 ordered from coarse Each encoder (s) predicts of Gaussians (*) from its input features. Formally, define:
)==1
to fine (s S).
(x) N. (+) = E() D())), ( {(8
S 1=1
Vs E (1.
where (+)) is the predicted depth map at scale used for geometric alignment during Gaussian initialization. Each ( is defined in 3D space, enabling sparse yet expressive representation.
This multi-scale Gaussian encoding framework allows each level to specialize different spatial granularity, facilitating coarse-to-fine 3D understanding and efficient downstream refinement.
3.2 Hierarchical Multi-Scale Refinement in Gaussian Parameter Space
propose to perform hierarchical refinement directly on the parameters of 3D Gaussians. This representation offers three key advantages: (i) compactness Compaciness-Gaussinn primitives provide sparse,

---

continuous encoding of the scene without voxel quantization artifacts; (ii) expressiveness each Gaussian captures both spatial location and geometric shape via learnable scale and orientation; and (iii) hierarchical alignment-differen different scales of Gaussians naturally correspond to varying levels of scene abstraction. These properties make the Gaussian parameter space compelling domain for multi-resolution refinement, allowing the network to progressively increase spatial fidelity while preserving semantic structure. The core innovation of our approach lies in shifting the multi-scale refinement paradigm from dense voxel-based feature maps to a sparse, continuous set of learnable 3D Gaussians. By operating directly in the Gaussian parameter space, our method achieves finer geometric detail, efficient memory scaling, and hierarchical abstraction across spatial resolutions.
We explicitly formulate this process as autoregressive refinement hierarchy for 3D scene represen- tation, where the Gaussian set at each scale is conditioned on all coarser levels (1=8-1). Specifically, model the hierarchical dependency
pf (GiS) G(LS--) (
G(..)) = II p ((() s=1
G ((la-)),.(.))
(2)
Here, ((s) denotes the initial Gaussian predicted at scale 8, and G( represents its refined version obtained by updating ((+) with residual corrections guided by G1s--1), This autoregressive structure enables structured, scale-wise propagation of geometric and semantic information.
Given the coarse-to-fine Gaussian sets G (1:a-1) and the initial prediction (+)), we compute residual updates via cross-attention fusion block:
AG = p(5) (G(L:-1)),
(3)
where ( denotes a learned dieetton-bbsed MLP that integrates coarse geometric priors with initial scale-s predictions for residual parameter refinement.
The refined Gaussian parameters scale s are obtained by updating the initial predictions with learned residuals. For position, scale, opacity, and semantic logits, we have
((0) (Al + Aut", 2 (a) + +AX ((0) + Ao/
(()) + A
(4)
The rotation is updated separately by refining the initial quaternion prediction q (s) with a learned delta quaternion Aq (*):
Normalize (q Aq
(5)
where @ denotes quaternion multiplication and Normalize enforces the unit-norm constraint. Here, (*) is the initial rotation quaternion of the i-th Gaussian at scale 8, Aq (9) is the learned quaternion update, and g(*)) is the final normalized quaternion rotation after refinement.
This formulation enables structured multi-scale refinement while maintaining geometric and semantic coherence across scales and resolution levels.
3.3 Selective Gaussian Refinement Mask
To further enhance optimization efficiency, we introduce Selective Gaussian Refinement Mask. The core idea is to leverage coarse-scale importance cues to identify spatially relevant regions and selectively refine only a sparse subset of fine-scale Gaussian parameters (Figure 3). By concentrating computational resources on informative areas, this mask improves both efficiency and scalability for training high-resolution 3D Gaussian fields.
Let (+=11) N and (+) E ND denote the Gaussian parameters at scales s-1 and respectively. Rather than refining fine-scale Gaussians based on all N₂-1 coarse-scale anchors, we identify and propagate information from a smaller, semantically relevant subset.
(s-1) is assigned a scalar importance Bercentile-Based Importance Selection. Each coarse anchor g;
score -5 (s-1) via a learnable scoring function:
5

---

G select
G
fine
Figure 3: Illustration of the Selective Gaussian Refinement Mask. Coarse-scale Gaussians with high importance scores are first selected (left) and then used to guide the refinement of fine-scale Gaussians (right) through cross-scale attention.
(8-1) ScoreNet(g! (o-1)) mi ((-11) E R. T
(6)
We compute importance scores n; (a-1) for each coarse-scale anchor using lightweight MI.P-based scoring function. We then retain the top-p% of anchors ranked by I, (x-1) forming the index set:
Trelect = Topp, ([T)) Dprt)
((=11) select
{{ {g/ (-- Iseleca
Here, we apply a soft index set corresponding to the top-p% of Gaussians [26, 2]. Let T E RBxNxk be soft index set, where B is the batch size, N is the number of Gaussians, and K is the top-p% number of Gaussians. Each slice over the last dimension defines a distribution over the N elements, indicating their soft selection probability of being in the top-p%. This formulation identifies regions requiring further refinement, as Guelect E BB x is selected from the original matrix G E Bxxxx D where D is the feature dimension. During inference, replace this relaxation with a discrete Top, operator for both memory and computational efficiency.
This subset serves coarse-to-fine attention mask that identifies regions requiring further refinement in both geometry and semantics.
Cross-Scale A Attention for Refinement. Let Gl BBxMxD and Seelect E BxXXx denote the fine-scale and selected coarse-scale Gaussian sets, respectively, where the batch size, D is the feature dimension, and M, K are the number of queried and key-value Gaussians per sample.
((+-1)
We apply a cross-scale attention mechanism as follows:
GGlal (((=-1))T
G line
G Line
softmax
G
(s-1)) select
VD
Here, @ denotes the residual update operator in the Gaussian parameter space: quaternion components are composed, while position, scale, opacity, and semantic logits are updated through element wise addition, consistent with Eqs. (4)-(5).The resulting ((*)
iiee retains the same shape (B, M, D).
4 Experiment
We conduct experiments on three indoor occupancy prediction benchmarks: Occ-ScanNet [38], EmbodiedOce-Scanne [34], and their respective smaller variants, Dec-ScanNet-mini [34] and SawoooiiedOc--ScanNeet.ee [34]. Occ-ScanNet and its mini version provide monocular RGB inputs paired with voxel-level semantic labels within a crustum-aligned 3D space, supporting per-frame local occupancy prediction in a static setting. In contrast, LimbodiedOce-ScaNN.. introduces sequential and embodied formulation, where temporally continuous monocular observations enable iterative refinement of global scene understanding. Each frame is labeled with local occupancy volume projected from globally consistent ground-truth space, allowing both frame-wise supervision and memory-basee global prediction. Following standard protocols [38, 34, 33], we report Scene

---

Table 1: Local Prediction (Single- Viele Results the Oce-ScanNet dataset.
ceiling
window
furniture
objects
floo
bed
sofa
table
Method
Input
LoU
mloU
MonoScene [3]
41.60
15.17
44.71
22.41
12.55
26.11
27.03
35.91
28.32
6.57
32.16
19.84
24.62
ISO [38]
42.16
19.88
4188
22.37
16.98
29.09
42.43
42.00
29.60
10.62
36.36
24.61
28.71
Emb [34]
41.90
33.00
55.20
43.80
35.40
53.50
45.48
42.23
52.95
43.23
34.20
43.20
56.73
45.66
35.92
55.23
44.33
Table 2: Local Prediction Results on the Occ-ScanNet-mini dataset.
ceiling
window
furniture
objects
floo
bed
vofa
table
IVS
Method
Input
IoU
mloU
MonoScene [3]
1/
41.90
17.00
46.20
23.90
12.70
27.00
29.10
34.80
29.10
9.70
34.50
20.40
25.90
ISO [38]
42.90
21.10
42.70
24.60
15.10
30.80
41.00
43.30
32.20
12.10
35.90
25.10
29.40
EmbodiedOOe [34]
53.80
48.70
42.30
38.70
62.70
48.20
33.80
58.00
46.50
29.25
48.73
38.70
39.28
64.88
49.73
$7.29
48.07
Completion Intersection-onn ever-Union (loU) and mean Intersection-ove coriooo--vee-eeee Union (mloU) across semantic classes. Local metrics are computed within the camera frustum of each frame, while global metrics evaluate performance over the union of explored regions, capturing the model's ability to maintain spatial consistency over time. While our primary focus is indoor occupancy prediction, we further assess the generalization capability of our approach on outdoor scenarios using publicly available datasets. These additional experiments underscore the broader applicability of our method. Experimental details and additional evaluation results are provided in Appendix.
4.1
Main Results
Local Prediction "SingleViewi Results. In the single-view setting, DFGauss consistently outper- forms prior baselines across datasets of varying scales. On the Occ-ScanNet benchmark (Table 1), our method achieves IoU of 55. 55.28 and mloU of 47.03, surpassing the Gaussian- based baseline [34] by 1.33% and 1.55% respectively. This highlights the advantage of our multi-scale autoregressive Gaussian refinement. the smaller-scale Dcc-ScanNet-mini dataset (Table 2). DFGauss further improves performance with an IoU of 54.28 and mloU of 47.08, outperforming all voxel-based and Gaussian-based baselines. These consistent gains across both full and mini variants demonstrate the robustness and scalability of approach for local occupancy prediction. These results demonstrate that the hicrarchical multi-scale design of DFGauss enables robust generalization under limited training data and across diverse model architectures and input modalities.
Global Prediction (Continuous- View) Results. the continuous-vies setting, DFGauss also outperforms the Gaussian- based baseline (Table 3). The proposed multi-scale refinement enhances scene detail modeling and facilitates better alignment between the current view and previously observed frames. On the EmbodiedOcc and EmbodiedOcc-minn datasets, DFGauss surpasses the single-scale Gaussian baseline(Table 4), achieving 1.36% improvement embodied mloU and 1. 1.44% gain in mloU on the mini subset. These results highlight the robustness of DFGauss and the effectiveness of hierarchical multi-scale regression in capturing ine-grained temporal and spatial context through dynamic, agent-centric environments.
4.2 Ablation Study
Component- -iinee Ablation. In the component- wise ablation study (Table 5), DFGauss achieves the highest performance when both the multi-scale regression module and the dynamic masking strategy integrated. Compared to the vanilla model, introducing the multi-scale regression alone improves
7

---

Table 3: Global Prediction (Continuous-View) Results the EmbooiedOOcc-canNee Dataset.
celling
window
furniture
objects
wall
chair
bed
rable
1vs
Method
Input
IoU
mloU
SplicingOoe
49.01
31.60
38.80
35.50
36.30
47.10
54.50
57.20
34.40
32.50
51.20
29.10
40.74
EmbodiieOOdd
51.52
22.70
44.60
37.40
39.00
50.10
56.70
59.70
35.40
38.40
52.00
32.90
42.53
53.80
26.35
46.22
36.73
39.22
50.37
57.41
60.21
37.50
42.22
53.20
33.25
43.89
Table 4: Global Prediction Results the EmbodiedOOc--Senneeemiii dataset.
window
furniture
ceiling
objects
floor
bed
sofa
table
IVS
Method
Input
IoU
mloU
SplicingOoc
48.80
29.00
37.60
37.30
26.80
44.50
66.00
52.70
40.80
36.60
54.50
27.90
41.20
EmbodiedOce [34]
50.70
21.50
44.50
38.30
27.90
46.90
64.70
55.30
42.70
35.80
52.50
27.50
41.60
52.32
22.73
44.80
38.70
30.21
47.12
65.10
55.62
43.21
37.83
55.24
32.83
43.04
IoU by 1.85% and mloU by 0.78%. When the dynamic mask is further applied, the gains increase to 2.71% in IoU and 1.13% in mloU. These results indicate that the multi-scale hierarchy effectively refines the details of occupancy prediction, while the dynamic mask focuses computation on regions requiring the most refinement.
Mask Ratio Ablation. the mask ratio ablation study (Table 6). we observe that increasing the mask ratio up to 50% consistently improves both local and global performance metrics, indicating that denser refinement improves representational quality. However, beyond this point, further increases do not yield consistent gains and may slightly degrade performance, likely due overly sparse Gaussian selections limiting effective refinement. Additionally, lower mask ratios lead to reductions in memory usage, reflecting the efficiency benefits of selectively refining only the most informative regions.
Hierarchy Level Ablation. As shown in the hierarchy level ablation (Table 7), performance steadily improves with increasing refinement depth up to level 4, after which the gains plateau or slightly decline under fixed masking ratio. Notably, the latency increases gradually-from 165 to 287 ms-as the refinement depth grows, indicating that deeper hierarchies incur only moderate computational overhead. This suggests that our Gaussian-bass based framework enables scalable multi- scale refinement while maintaining reasonable inference efficiency.
Table 6:
Effect of mask ratio inference
Table 5: Ablation on the Components
memory and performance
Local Metric
Global Metric
Local Metric
Global Metric
Multi- Scale
Mask
loU
mloU
loU
mioU
Ratio%
Memory
lot:
mloU
loU
mloUi
5328 M
54.03
46.12
43.47
52,33
46.20
51.33
42.46
5524 M
55.82
46.54
53.72
43.63
54.32
46,65
53.04
43.21
5732 M
55.28
47.03
53.80
43.89
55.28
47.03
53.80
43.89
50
5998 M
55.68
46.93
53.76
43.58
6179 M
54.84
46.73
53.42
43.21
4.3 Qualitative Results
Figure presents qualitative results on the Occ-ScanNer dataset. Compared to the Gaussian-bace baseline without autoregressive hierarchy, DFGauss captures more fine- aeeegraaned structures in the target space, demonstrating the benefit of multi-scale refinement in improving spatial detail.

---

Table 7: Ablation on hierarchical depth; depth 1 the baseline without refinement.
Local Metric
Global Metric
Depth
Latency (ms)
IoU
mloU
IoU
mloU
165
52.33
46.20
51.33
42.46
193
54.67
46.88
52.25
42.95
3
232
55.28
47.03
53.8
43.89
4
268
55.89
46.78
53.65
43.72
5
287
55.13
46.83
53.83
43.62
scene0040_00
scene0031_00
a
b
a
b
d
Figure 4: Qualitative results Occ-ScanNet: (a) Input, (b) GT, (c) EmbodiedOcc, (d) DFGauss.
5 Limitation and Future Work
In this paper, we primarily focus on Gaussian- based model architecture for monocular indoor occupancy prediction. However, extending this framework to multi-view indoor perception [34] remains an open direction and is not explored in this work. Additionally, the proposed mechanism could be applied semantic segmentation [27], occupancy prediction [38], and 31) scene completion [15, 8]. In future work, we plan extend our experiments these 3D scene understanding tasks further investigate the potential of multi-scale Gaussian representations across diverse domains.
broader range of point cloud tasks, such panoptic segmentation [25, 37],
6 Conclusion
We present DFGauss, a multi-scale autoregressive Gaussian framework for 3D occupancy prediction. Unlike traditional approaches that rely dense volumetric multi-scale regression, our method focuses on learning in the Gaussian parameter space across scales- sparse and efficient representation that remains underexplored. To further enhance both accuracy and efficiency, introduce selective, dynamically focused refinement mask that prioritizes informative regions during hierarchical refinement. Extensive experiments demonstrate that approach improves occupancy prediction performance while also reducing computational overhead. We hope that this framework inspires future research efficient and expressive 3DD scene understanding.

---

References
[1] Mohamed Abdelsamad, Michael Ulrich, Claudius Gläser, and Abhinav Valada. Multi-scale neighborhood occupancy masked autoencoder for self-supervisee learning in LiDAR point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion (CVPR), 2025.
[2] Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference on Machine Learning, pages 950-959. PMLR, 2020.
[3] Anh-Quan Cao and Renaud de Charette. MonoScene: Monocular 3d semantic scene completion. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[4] Junliang Chen, Huaiyuan Xu, Yi Wang, and Lap-Pui Chau. OccProphet: Pushing the efficiency frontier of camera-only 4d occupancy forecasting with an observer-orrecatterreinne framework. In Proceedings of the Thirleenth International Conference on Learning Representations (ICLR),
2025.
[5] Liang Chen, Sinan Tan, Zefan Cai, Weichu Xie, Haozhe Zhao, Yichi Zhang, Junyang Lin, Jinze Bai, Tianyu Liu, and Baobao Chang. A spark of vision-language intelligence: 2-dimensional autoregressive transformer for efficient finegrained image generation. Proceedings of the Thirteenth International Conference Learning Representations (ICLR), 2025.
[6] Shizhe Chen, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. SUGAR Pre-training 3d visual representations for robotics. In Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18049-18060, 2024.
[7] Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. LightGaus- sian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Proceed- ings of the Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 140138-140158. Curran Associates, Inc., 2024.
[8] Martin Garbade, Yueh- Tueh-Tung Chen, Johann Sawatzky, and Juergen Gall. Two stream 3d semantic scene completion. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages
[9] Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. SelfOce: Self- supervised vision-based 3d occupancy prediction. In Proceedings of the 2024 IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 19946-19956, 2024.
[10] Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. GaussianFormerr: Scene as gaussians for vision-based 3d semantic occupancy prediction. In Proceedings of the European Conference on Computer Vision (ECCV), pages 376-393. Springer, 2024.
[11] Jiwan Hur, Dong-Jae Lee, Gyojin Han, Jaehyun Choi, Yunho Jeon, and Junmo Kim. Unlock- ing the capabilities of masked generative models for image synthesis via self-guidance. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 130977-130999, Curran Associates, Inc., 2024.
[12] Maximilian Jaritz, Jiayuan Gu, and Hao Su. Multi-view PointNet for 3d scene understanding. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pages 3995-4003, 2019.
[13] Tero Karras, Timo Aila, Samuli I Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In Proceedings of the Sixth International Conference on Learning Representations (ICLR), 2018.
[14] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Tronsactions on Gruphics (TOG), 42(4), 2023.
10

---

[15] Wesley Khademi and Fuxin Li. Point-based instance completion with scene constraints. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR), 2025.
[16] Tarasha Khurana, Peiyun Hu, Achal Dave, Jason Ziglar, David Held, and Deva Ramanan. Differentiable raycasting for self-supervised occupancy forecasting. In Proceedings of the European Conference on Computer Vision (ECCV), pages 353-369. Springer, 2022.
[17] Joo Chan Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, and Eunbyung Park. Compact 3d gaussian representation for radiance field. In Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21719-21723 2024.
[18) Jinke Li, Xiao He, Chonghua Zhou, Xiaoqiang Cheng, Yang Wen, and Dan Zhang. Viewformer: Exploring spatiotemppral modeling for multi-view 3d occupancy perception via view-guided transformers, 2024.
[19] Xiang Li, Pengfei Li, Yupeng Zheng, Wei Sun, Yan Wang, and yilun chen. Semi-supervised vision-centrie 3d occupancy world model for autonomous driving. In The Thirteenth Interna- tional Conference on Learning Representations, 2025.
[20] Tsung-Y Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 936-944, 2017.
[21] Haisong Liu, Yang Chen, Haiguang Wang, Zetong Yang, Tianyu Li, Jia Zeng, Li Chen, Hongyang Li, and Limin Wang. Fully sparse 3d occupancy prediction. In Proceedings of the European Conference on Computer Vision (ECCV), pages 54-71. Springer, 2024.
[22] Qihao Liu, Zhampeng Zeng, Ju He, Qihang Yu, Xiaohui Shen, and Liang-Chich Chen. Alleviat- ing distortion in image generation via multi-resolution diffusion models and time-dependent layer normalization. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 133879-133907. Curran Associates, Inc., 2024.
[23] D.G. Lowe. Object recognition from local invarinnv features. In Proceedings of the Seventh IEEE International Conference Computer Vision (ICCV), volume 2, pages 1150-1157, 1999.
[24] Yuhang Lu, Xinge Zhu, Tai Wang, and Yuexin Ma. OctreeOce: Efficient and multi- granularity occupancy prediction using octree queries. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Proceedings of the Advances in Neural Informa- tion Processing Systems (NeurIPS), volume 37, pages 19618-7964| Curran Associates, Inc., 2024.
[25] Gaku Narita, Takashi Seno, Tomoya Ishikawa, and Yohsuke Kaji. PanopticFusion: Online volumetric semantic mapping at the level stuff and things. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4205- 4212, 2019.
[26] Felix Petersen, Hilde Kuchne, Christian Borgelt, and Oliver Deussen. Differentiable top- k classification learning. In Proceedings of the 39th International Conference Machine Learning (ICML), 2022.
[27] Charles Ruizhongtai Qi, Yi, Hao Su, and Leonidas Guibas. PointNet++: Deep hierarchical feature learning point sets in metric space. I. Guyon, U. Von Luxburg, S. Bengio, H. Wal- lach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances Neural Information Processing Systems (NeurIPS), volume 30. Curran Associates, Inc., 2017.
[28] Keyu Tian, Yi Jiang, qishuai diao, Chen Lin, Liwei Wang, and Zehuan Yuan. Designing BERT for convolutional networks: Sparse and hierarchical masked modeling. The Eleventh International Conference Learning Representations, 2023.

---

[29] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. In A. Globerson. L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Proceedings of the Advances Neural Information Processing Systems (NeurIPS), volume 37, pages 84839 84865. Curran Associates, Inc., 2024.
[30] Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Oce3D: A large-scale 3d occupancy prediction benchmark for autonomous driving. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Proceedings of the Advances in Neural Information Processing Systems (NeurlPS), volume 36, pages 64318 64330. Curran Associates, Inc., 2023.
[31] Guoqing Wang, Zhongdao Wang, Pin Tang, Jilai Zheng, Xiangxuan Ren, Bailan Feng, and Chao Ma. OccGen: generative multi-modal 3d occupancy prediction for autonomous driving, In Proceedings of the European Conference on Computer Vision (ECCV), pages 95-112. Springer, 2024.
[32] JiaBao Wang. Zhaojiang Liu, Qiang Meng, Liujiang Yan, Ke Wang, JIE YANG, Wei Liu, Qibin Hou, and Ming-Ming Cheng. Opus: Occupancy prediction using a sparse set. In The ehirty-eighth Annual Conference on Neural Information Processing Systems.
[33] Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. SurroundOce: Multi-camert 3d occupancy prediction for autonomous driving. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 21729-21744 2023.
[34] Yugi Wu, Wenzhao Zheng. Sicheng Zuo, Yuanhui Huang, Jie Zhou, and Jiwen Lu. Embod- iedOce: Embodied 3d occupancy prediction for vision-based online scene understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[35] Zhiwen Yan, Weng Fei Low, Yu Chen, and Gim Hee Lee. Multi-scale 3d gaussian splatting for anti-aliases rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 20923-20931. 2024.
[36] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Anything V2. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tom- czak, and C. Zhang, editors, Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages
Curran Associates, Inc., 2024.
[37] Li Yi, Wang Zhao, He Wang. Minhyuk Sung, and Leonidas J. Guibas. GSPN:
Generative shape proposal network for 3d instance segmentation in point cloud. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3942-3951, 2019.
[38] Hongxiao Yu, Yuqi Wang, Yuntao Chen, and Zhaoxiang Zhang. Monocular occupancy predic- tion for scalable indoor scenes. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol, editors, Proceedings of the European Conference on Computer Vision (ECCV), pages 38-54, Cham, 2024. Springer Nature Switzerland.
[39] Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Sip-Splatting: Alias-free 3d gaussian splatting. In Proceedings of the 2024 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 19447-1944 2024.
[40] Dingyuan Zhang. Dingkang Liang, Hongcheng Yang. Zhikang Zou, Xiaoqing Ye, Zhe Liu, and Xiang Bai. SAM3D: zero-shot 3d object detection via the segment anything model. Science China Information Sciences, (7(4), 2024.
[41] Yunpeng Zhang, Zheng Zhu, and Dalong Du OccFormer: Dual-path transformer for vision- based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9433 14339943, 2023.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes. Our main contributions are introduced in Section 1 and elaborated in Section 3, primarily centered around novel multi-scale autoregressive Gaussian refinement framework for 3D occupancy prediction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Sec.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiaatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer:
[NA]
Justification:
We do not propose new theory in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Yes, we provide the methodological details of our approach in Section 3, and further elaborate the implementation specifics in Appendix. Our method developed us- ing publicly available datasets and can be implemented directly on top of standard baselines or off-the-shed models.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: The code is yet published, but the supplementary material provides all details needed to reproduce our model and results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /nips. httpppbbcc//uuiee......e...e guwdes/ /CodeSubmic wwwtebiissiooon wweeinsssoooooolooooo onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we provide details of the dataset splits and the hyperparameter settings in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: No, the baselines do not report error bars. Reproducing all baseline results and all experiments in our study with sufficient runs to report error bars is also impractical due computational constraints.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, report these details in Appendix, including the type of compute hard- ware used. We also provide memory consumption and runtime statisties related information in Tables 6, 7, and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //werrip/////....e/// wwtpeerww.....uuuleeeeeeeececeee cc/ppcccpccc////ccp.cccccccccclt wewhiiihieeeeeeeeeeeeeeeeee lines?
Answer: [Yes]
Justification: To the best of our knowledge, our work is conducted with NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[NA]
Justification:
To the best of our knowledge, we do not foresee societal impacts of work.
Guidelines:
16

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We properly cite the used existing datasets and models.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
17

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide detailed descriptions about our method.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not involve crowdsourcing research.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No human subject involved
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
18

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: the core method development this research does not involve LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: : /ttp///erssp/////ee cc/Confereecces/22555 /LLM) for what should or should be described.
19


Reviews:
[{"review_id": "qhOn1KEObn", "reviewer": "Reviewer_9Hse", "summary": "This paper presents DFGauss, a 3D occupancy prediction framework from single indoor images. Instead of using computationally intensive voxel grids, DFGauss operates on sparse 3D Gaussian representations with multi-scale autoregressive modeling to improve the efficiency. Moreover, a dynamic focused masking mechanism that improves efficiency by selectively updating only the most informative Gaussians, concentrating computational effort on important regions. Experiments show that DFGauss achieves state-of-the-art performance on indoor 3D occupancy prediction benchmarks.", "questions": "1. How do you use multi-scale autoregressive in your work?\n2. Could you provide the details about the Multi-Scale Gaussian Encoder and illustrate the network details?\n3. What kind of network is the first row in Table. 5? Please describe the baseline.", "limitations": "yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strength:\n1. The performance promotion is impressive.\n2. The proposed method is easy to understand.\n\nWeaknesses:\n1. The proposed Multi-Scale Gaussian Encoder should be introduced with more details, such as the number of layers of cross-attention and self-attention, and how to leverage the generated image features and depth map.\n2. There are many typos and inconsistencies in the paper: q in Eq. 5 is not defined in the manuscript. The notions of the Gaussian set are not consistent in Eq. 1 and Eq. 7.\n3. More visualizations should be provided, including comparisons on different datasets and abalations.\n4. The proposed method appears to be merely a multi-scale version of GaussianFormer , and the reviewer does not find any connection between the proposed work and visual autoregressive models.", "was_revised": true, "final_justification": "The authors did not address my concerns about the relationship between the proposed work and the previous GaussianFormer and VAR. The authors believe the proposed method is neither a generative model nor an autoregressive method.", "rebuttal": "Dear Reviewer 9Hse,\n\nThank you for acknowledging the method’s improved performance and intuitiveness. Below are our detailed responses to your feedback.\n\n## **Q1 Multi-scale autoregressive detail (weakness 4: the connection with visual autoregressive models)**\n\nAccording to the formal definition of autoregressive models [1], such models forecast the variable of interest using a combination of its **previous** values. Our proposed model follows this principle by refining the current fine-grained Gaussian parameters based on **previously** generated coarse-grained outputs, which aligns with the autoregressive modeling paradigm. This is formally expressed in Eq. (2) of the paper as $p\\left(g^{(s)} \\mid g^{(s-1)}, a^{(s)}\\right) = \\prod_i p\\left(g^{(s)}_i \\mid g^{(s-1)}_i, a^{(s)}_i\\right)$ , where the prediction of each finer-scale Gaussian is conditioned on its corresponding coarser-scale Gaussian and original fine-grained features from the input. Thus, our framework adheres to the broader definition of autoregression by modeling each finer-scale output as dependent on previously generated coarser-scale values, thereby aligning with the autoregressive paradigm.\n\nWhile Visual Autoregressive (VAR) models [2] demonstrate autoregressive modeling in the 2D image domain via pixel- or token-level next-resolution prediction, other areas have similarly extended the autoregressive principle to their own tasks, including normalizing flows [3], 3D object generation [4], monocular depth estimation [5], and offline policy learning [6]. Our work does **not** directly apply 2D VAR to the 3D setting; instead, we adopt the underlying autoregressive principle and reformulate it for spatial scale refinement in the 3D Gaussian parameter space. This forms a structured coarse-to-fine dependency unique to our sparse 3D Gaussian representation.\n\n## **Q2 Details on the Multi-Scale Gaussian Encoder architecture**\n\nAs mentioned in line 114, our framework builds upon the baselines [7, 8]. We focus primarily on describing our novel design components. For a fair comparison, modules that are identical to those in previous works reuse the same structure, including the number of layers and parameter settings.\n\n**In detail, our encoder includes the following components:**\n\n**Same** as in [7,8], our encoder includes an **image cross-attention and refinement module** to integrate Gaussian features with image features extracted by the backbone [9]. It estimates per-pixel depth and combines it with image features via a depth-aware MLP to initialize structured 3D semantic Gaussians in the camera coordinate system. Each Gaussian is represented by a vector composed of mean $\\mathbf{m} \\in \\mathbb{R}^3$ , scale $\\mathbf{s} \\in \\mathbb{R}^3$ , rotation quaternion $\\mathbf{r} \\in \\mathbb{R}^4$ , opacity $o \\in \\mathbb{R}$ , and semantic logits $\\mathbf{c} \\in \\mathbb{R}^C$ . The interactions between image features and Gaussians, as well as those among Gaussians, are all performed in the camera coordinate system. The initial Gaussian parameters are then sent to the self-encoding module, which uses 3D sparse convolution (rather than deformable attention) to enable interaction among the Gaussians. These parameters are subsequently voxelized to convert the sparse Gaussians into a format compatible with sparse 3D convolution, producing a sparse 3D grid of features, each centered on a Gaussian.\n\nThe **image cross-attention module** extracts visual information from multi-view images to enhance the 3D Gaussian queries. For each Gaussian, a set of reference points is sampled around its mean, with offsets determined by its covariance to reflect its spatial extent. These 3D reference points are projected into the image planes using the known camera intrinsics and extrinsics. The corresponding image features at these projected locations are then gathered via deformable attention[11]. The aggregated visual cues are used to update the high-dimensional Gaussian queries, which guide the subsequent refinement of the Gaussian properties.\n\nThe **refinement module** updates each 3D Gaussian $\\mathbf{G} = (\\mathbf{m}, \\mathbf{s}, \\mathbf{r}, \\mathbf{o}, \\mathbf{c})$ using its corresponding high-dimensional query vector $\\mathbf{Q} \\in \\mathbb{R}^m$ . Here, $\\mathbf{m}$ is the 3D center, $\\mathbf{s}$ is the scale (covariance), $\\mathbf{r}$ is the rotation quaternion, $\\mathbf{o}$ is the opacity, and $\\mathbf{c}$ denotes the semantic class logits. After aggregating contextual information through the self-encoding and image cross-attention modules, the model predicts the update amounts $\\Delta \\mathbf{G} = (\\Delta \\mathbf{m}, \\Delta \\mathbf{s}, \\Delta \\mathbf{r}, \\Delta \\mathbf{o}, \\Delta \\mathbf{c})$ , which are used to refine the Gaussian properties as $\\mathbf{G}_{\\text{new}} = (\\mathbf{m} + \\Delta \\mathbf{m},\\; \\mathbf{s} + \\Delta \\mathbf{s},\\; \\Delta \\mathbf{r} \\otimes \\mathbf{r},\\; \\mathbf{o} + \\Delta \\mathbf{o},\\; \\mathbf{c} + \\Delta \\mathbf{c})$ . Here, $\\otimes$ denotes a special quaternion composition operator for updating the rotation, enabling spatially coherent and stable refinement across blocks. These Gaussian parameters are subsequently passed into the refinement hierarchy described in Q1, where they are progressively optimized across scales via the autoregressive process.\n\nWe will address this in the appendix.\n\n## **Q3 Network in the first row of Table 5 ?**\n\nThe first row corresponds to the standard GaussianFormer model[7,8]. It first extracts image features using an image backbone [9], and simultaneously predicts depth features from[10], which are used to initialize the 3D Gaussian parameters in the camera coordinate system. These Gaussians are then processed by the Gaussian encoder [7,8] and projected into voxel space via Gaussian-to-voxel splatting [8] to produce the final occupancy prediction. Our method extends this baseline by introducing a multi-scale autoregressive refinement framework that progressively updates the Gaussian parameters. As shown in Table 5, removing this autoregressive structure leads to a noticeable drop in performance, highlighting its contribution.\n\n## **Q4 Inconsistent Gaussian set(weakness 2)**\n\nThe two Gaussian sets are **different**, so they are intentionally defined with different meanings. The symbol in Equation 1 represents the entire set of Gaussians at a given scale, while the symbol in Equation 7 refers to the subset of selected Gaussians identified by the importance-based masking mechanism.\n\n## **Q5 q in Eq. 5 is not defined(weakness 2)**\n\nAs noted in line 170 of the paper, Eq. 5 involves **quaternion multiplication** and is written as $ \\mathbf{q}_i^{(s)} = \\text{Normalize}(\\mathbf{q}_i^{(s-1)} \\otimes \\Delta \\mathbf{q}_i^{(s)}) $, where $ \\otimes $ denotes quaternion multiplication and $\\text{Normalize}$ ensures the result is a unit quaternion. The symbol $ \\mathbf{q}_i^{(s)} $ represents the rotation quaternion of the $i$-th Gaussian at scale $s$ , $ \\mathbf{q}_i^{(s-1)} $ is the previous rotation quaternion at the coarser scale, and $ \\Delta \\mathbf{q}_i^{(s)} $ is a learned rotation quaternion update (a delta quaternion). We apologize for the oversight and will clarify this definition in the final version to avoid confusion.\n\n\n[1] Hyndman, Rob J., and George Athanasopoulos. *Forecasting: Principles and Practice*. OTexts, 2018. Section 8.3: Autoregressive models.\n\n[2] Tian, Keyu, et al. \"Visual **autoregressive** modeling: Scalable image generation via **next-scale** prediction.\" Neurips 2024 \n\n[3] Bhattacharyya, Apratim, et al. \"Normalizing flows with **multi-scale autoregressive** priors.\" CVPR 2020\n\n[4] Chen, Yongwei, et al. \"SAR3D: **Autoregressive** 3D object generation and understanding via **multi-scale** 3D VQVAE.\" CVPR 2025\n\n[5] Wang, Jinhong, et al. \"Scalable autoregressive monocular depth estimation.\" CVPR 2025.\n\n[6] Zhang, Michael R., et al. \"Autoregressive dynamics models for offline policy evaluation and optimization.\" ICLR 2021\n\n[7] Wu, Yuqi, et al. \"Embodiedocc: Embodied 3d occupancy prediction for vision-based online scene understanding.\" ICCV 2025 \n\n[8] Huang, Yuanhui, et al. \"Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction.\" ECCV 2024\n\n[9] Tan, Mingxing, and Quoc Le. \"Efficientnet: Rethinking model scaling for convolutional neural networks.\" ICML 2019\n\n[10] Yang, Lihe, et al. \"Depth anything v2.\" Neurips 2024\n\n[11] Zhu et al., Deformable DETR: Deformable Transformers for End-to-End Object Detection ICLR 2021"}, {"review_id": "0pRaw9JsWD", "reviewer": "Reviewer_txXB", "summary": "This paper proposes a hierarchical Gaussian refinement strategy as an improvement over the conventional EmbodiedOcc framework. The proposed DFGauss mainly consists of a multi-scale Gaussian encoder, a hierarchical Gaussian refinement process, and a multi-scale supervision strategy. The paper conducts extensive experiments on both indoor and outdoor datasets, which demonstrates good performance compared with the baseline.", "questions": "1. How are the parameters used to compute the confidence score for Gaussians updated? Please refer to the weakness section for details.\n2. How is the GPU memory consumption of DFGauss compared with the baseline method EmbodiedOcc?", "limitations": "yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. Good motivation. The multi-scale Gaussian encoder and hierarchical refinement design stem from the coarse-to-fine nature of visual perception.\n2. Extensive experiments. The paper conducts experiments on two indoor datasets (excluding the mini versions) and one outdoor dataset to verify the effectiveness of the proposed method.\n\nWeaknesses:\n1. Potentially inappropriate expression. The \"Dynamic Focused\" expression in the title might be misleading because the paper actually focuses on static scenes and this phrase actually refers to adaptive selection of Gaussians requiring refinement. Also, the paper have potentially overclaimed itself to be autoregressive, I think \"hierarchical\" might be more suitable.\n2. Technical flaw. The Gaussian refinement module first computes a confidence score and select Gaussians to refine with a percentile-based strategy. The computation of confidence scores should introduce learnable parameters, but the percentile-based selection is not differentiable which prevents gradients from back-propagating to the score-related parameters. The paper does not discuss this.", "was_revised": true, "final_justification": "Post rebuttal, my concerns regarding the method details and efficiency have been addressed. Thus I am leaning towards accepting the paper.", "rebuttal": "Dear Reviewer txXB,\n\nThank you for recognizing the strengths of our work, particularly the multi-scale Gaussian design and extensive experimental evaluation. Below are our detailed responses to your feedback.\n\n## **Q1: Differentiable Scores (Weakness 2 flaw)**\n\nWe appreciate the reviewer’s observation regarding the differentiability of the subset selection. In our implementation, we adopt the off-the-shelf, training-free differentiable top‑$k$ operator [1]. The $I_{\\rho}$ in Equation (7) of the paper represents a **soft** index set corresponding to the top‑$\\rho$% of Gaussians, enabling end-to-end differentiability. This choice allows gradient flow from the final loss back through the selection mechanism.\n\nAfter computing the per-Gaussian importance scores $\\mathbf{q} \\in \\mathbb{R}^{B \\times N}$ using a lightweight scoring MLP, we apply the differentiable top-*k* operator to produce a soft selection matrix  $I_{\\text{select}} \\in \\mathbb{R}^{B \\times N \\times k}$ be a soft index set, where $B$ is the batch size, $N$ is the number of Gaussians, and $k$ is the top‑$\\rho$% number of Gaussians. Each slice over the last dimension defines a distribution over $N$ elements, indicating their soft selection probability of being in the top‑$\\rho$%.\nIt identifies regions requiring further refinement, as $\\mathbf{G}_{\\text{select}} \\in \\mathbb{R}^{B \\times K \\times D}$ is selected from the original matrix $\\mathcal{g} \\in \\mathbb{R}^{B \\times N \\times D}$ , where $D$ is the feature dimension.\n\nWe apologize for the omission and will add this clarification to the appendix.\n\n## **Q2 Memory consumption**\n\n| Method                    | Latency (ms) | Memory (MB) |\n|---------------------------|--------------|-------------|\n| ISO (ECCV 2024 )          | 650          | 10214       |\n| EmbodiedOcc (ICCV 2025)   | 114          | 3161        |\n| Ours                      | 232          | 5732        |\n\nAs shown in the table, compared to traditional convolution-based architectures such as ISO and EmbodiedOcc, our method remains efficient for occupancy prediction, achieving a favorable balance between latency and memory consumption.\n\n## **Q3 Potentially Inappropriate (Weaknesses 1)**\n\nWe acknowledge the reviewer’s concern regarding the use of **Dynamic** in our title. Here, we clarify that “Dynamic” refers to the adaptive, content-aware computation mechanism in our model rather than implying temporal variation or scene motion. Our design aligns with the notion of **dynamic neural networks**, as characterized in [8], where “dynamic” refers to models whose computational graphs or activation patterns change depending on input instances—enabling efficient, selective processing. Our model exhibits the same key characteristics: (1) per-sample adaptive behavior, (2) selective computation for efficiency, and (3) sparse activation within the computational structure. Thus, our use of the term “Dynamic Focused” aims to describe the model architecture type, consistent with the terminology adopted in this literature, rather than the specific task or input modality.\n\nAccording to the formal definition of autoregressive models[2], such models forecast the variable of interest using a combination of its **previous** values. Our proposed model follows this principle by refining the current fine-grained Gaussian features based on **previously** generated coarse-grained outputs, which aligns with the autoregressive modeling paradigm. Although Visual Autoregressive (VAR) models [7] exemplify this idea in the 2D image domain by performing next-resolution prediction over discrete tokens or pixels, the autoregressive principle has also been adapted across various other domains, including normalizing flows [3], 3D object generation [4], monocular depth estimation [5], and offline policy learning [6]. In the same spirit, our work applies autoregressive refinement across spatial scales in the 3D Gaussian parameter space, forming a structured coarse-to-fine **autoregressive dependency** that is unique to our Gaussian-based 3D scene representation.\n\n[1] Petersen, Felix, et al. Differentiable Top-k Classification Learning. ICML, 2022.\n\n[2] Hyndman, Rob J., and George Athanasopoulos. *Forecasting: Principles and Practice*. OTexts, 2018. Section 8.3: Autoregressive models.\n\n[3] Bhattacharyya, Apratim, et al. \"Normalizing flows with multi-scale autoregressive priors.\" CVPR 2020\n\n[4] Chen, Yongwei, et al. \"SAR3D: Autoregressive 3D object generation and understanding via multi-scale 3D VQVAE.\" CVPR 2025\n\n[5] Wang, Jinhong, et al. \"Scalable autoregressive monocular depth estimation.\" CVPR 2025\n\n[6] Zhang, Michael R., et al. \"Autoregressive dynamics models for offline policy evaluation and optimization.\" ICLR 2021\n\n[7] Tian, Keyu, et al. \"Visual autoregressive modeling: Scalable image generation via next-scale prediction.\" Neurips 2024\n\n[8]Dynamic Neural Networks: A Survey  IEEE TPAMI 2021"}, {"review_id": "61bBVaJYXo", "reviewer": "Reviewer_pKS8", "summary": "This paper proposes a multi-scale Gaussian representation framework for 3D occupancy prediction. It proposes a multi-scale Gaussian encoding method to convert image features into hierarchical Gaussian primitives (position, scale, opacity, semantics). Then It employs an autoregressive refinement to progressively refine Gaussian parameters from coarse to fine scales via cross-attention. To reduce computation, it proposes a dynamic focused masking, which selectively updates informative Gaussians (top-ρ% by learned scores).\nEvaluated on indoor (Occ-ScanNet, EmbodiedOcc) and outdoor (SSCBench-KITTI-360) benchmarks, DFGauss achieves SOTA results (e.g., 47.03 mIoU on Occ-ScanNet) with lower latency than voxel-based methods.", "questions": "1. In high-resolution settings, does the large number of Gaussian points lead to excessive memory consumption?\n\n2. Does the method rely too heavily on the effectiveness of monocular depth estimation?\n\n3. How robust is the model under large viewpoint changes?", "limitations": "1. The visual comparisons and results cannot demonstrate the robustness and superiority of the method.\n\n2. The paper does not discuss the effect of monocular depths.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed multi-scale Gaussian pipeline has lower latency than existing methods.\n2. In the benchmark, it performs better than existing SOTA methods. \n\nWeakness:\n\n1. Section 3.2 contains so many module design details, which are not clear. \n\n2. The paper uses DepthanythingV2 to predict multi-scale depths for geometric alignment during Gaussian initialization. However, the depths from DepthAnythingV2 are metric-unknown and temporally inconsistent. It may also contain many artifacts. I believe this will affect the Gaussian initialization. However, the paper does not discuss this. Why not use other metric depth models? In outdoor scenes, such as KITTI, DepthAnythingV2 works worse in far regions. Will this affect the Gaussian?\n\n3. The paper actually attempts to solve the occupancy prediction from a single view. It should compare with more occupancy methods. \n\n4. The paper only shows 2 visual results in the paper. It is hard to evaluate the performance and quality of predicted occupancy. Furthermore, the paper argues that it targets autonomous navigation for an embodied agent. The temporal consistency is important for robotics navigation. The paper does not present temporal results and evaluation. The single-frame evaluation cannot support the application scenarios. \n\n5. In indoor scenes, there are so many complex and fine-grained structures and objects. Could the method also work well on such objects?", "was_revised": true, "final_justification": "In the rebuttal, the paper provides more ablations on the DepthAnything module. The designed Gaussian Encoder can avoid the metric ambiguity issues and boost the performance. In summary, the paper provides a novel method for the 3D occupancy prediction. It ablates all designs in detail. I recommend accepting this paper.", "rebuttal": "Dear Reviewer pKS8,\n\nThank you for identifying the strengths of our work, including the low-latency design and competitive performance compared to existing methods. Below are our detailed responses to your feedback.\n\n## **Q1 High-resolution settings**\n\n| Method                    | # Gaussians | Latency (ms) | Memory (MB) | Grids Size       |\n|---------------------------|-------------|--------------|-------------|------------------|\n| MonoScene (CVPR 2022)     | -           | 870          | 20723.2     | 256×256×32       |\n| ISO (ECCV 2024 )          | -           | 650          | 10214       | 60×60×36         |\n| EmbodiedOcc (ICCV 2025)   | 12600       | 114          | 3161        | 60×60×36         |\n| Ours                      | 12600       | 232          | 5732        | 60×60×36         |\n| Ours                      | 25600       | 265          | 6531        | 256×256×32       |\n\nDespite increasing both the grid size from 60×60×36 to 256×256×32 and the number of Gaussians from 12.6K to 25.6K, the memory usage only grows from 5732 MB to 6531 MB — an increase of approximately 13.9%. This demonstrates that the proposed hierarchical Gaussian refinement and selective masking mechanism scale efficiently, avoiding excessive memory consumption even under high-resolution settings.\n\n## **Q2 Heavily rely on the monocular depth(weakness2)**\n\nWe apologize for the lack of clarity. Due to space limitations, we briefly mention the existing depth estimation module to focus on presenting our primary contribution in detail. We adopt DepthAnythingV2 because it is **also** used in our baselines, EmbodiedOcc (ICCV 2025) and ISO (ECCV 2024), ensuring a fair comparison under the same monocular depth setup. However, our method does not directly rely on raw depth outputs. Instead, we adopt the Gaussian encoder design from [1,2], which includes an internal learnable refinement multi-layer MLP module to enhance the depth features. These refined features are then used to initialize Gaussian parameters, which are further processed through an image cross-attention module[2] and a feature refinement module[1]. In this way, depth is treated as a geometric prior, while the final occupancy prediction is driven by adaptive learning across scales.\n\nWe also report results using depth predictions from DepthAnythingV2 directly, without the refinement module. As shown in the table below, using naive monocular depth features without further processing leads to a noticeable drop in performance. This highlights the importance of the refinement module in enhancing depth features and improving the overall occupancy prediction quality.\n\n#### *The results are presented in multiple tables due to Markdown formatting constraints that limit the width of single tables*\n\n| Method          | IoU   | mIoU  |\n|----------------|--------|--------|\n| Ours (refine)  | **39.89** | **14.58** |\n| Ours (naive)   | 34.26     | 11.95     |\n\n| Method         | car   | bicycle | motorcycle | truck | other-veh. | person | road  | parking |\n|----------------|-------|---------|------------|--------|-------------|--------|-------|---------|\n| Ours (refine)  | 22.21 | 1.85    | 4.88       | 14.78  | 5.97        | 2.03   | 54.23 | 15.78   |\n| Ours (naive)   | 18.56 | 0.21    | 1.92       | 13.26  | 3.32        | 0.35   | 52.22 | 13.21   |\n\n| Method         | sidewalk | other-grnd | building | fence | vegetation | terrain | pole | traf.-sign | other-struct. | other-object |\n|----------------|----------|-------------|----------|--------|------------|---------|------|-------------|----------------|---------------|\n| Ours (refine)  | 31.89    | 4.52        | 32.28    | 6.12   | 29.02      | 18.63   | 4.02 | 4.25        | 6.03           | 4.03          |\n| Ours (naive)   | 24.86    | 3.91        | 31.17    | 3.68   | 20.34      | 15.35   | 4.37 | 1.22        | 4.17           | 2.92          |\n\n## **Q3 Large viewpoint results**\n\nThe continuous-view experiment in Table 3 corresponds to a large-viewpoint setting, as it consists of 30 sequential frames with associated camera poses for each scene (line 464). This setup includes evaluating the union of frustums across 30 frames, thereby incorporating temporal consistency and naturally involving significant viewpoint variation and spatial transitions—making it a suitable benchmark for assessing robustness under large viewpoint changes.\n\n## **Q4 Single-view occupancy prediction (weakness3)**\n\nWe would like to clarify that our method is not limited to single-view prediction. We adopt the confidence-guided refinement strategy from[1](line 119), which enables temporal accumulation of information across frames. Our experiments in Table 3 demonstrate that DFGauss also improves performance in the continuous-view setting, where the model observes and integrates 30 sequential monocular frames with associated poses (line 464). This validates that our framework supports both single-view and continuous-view (embodied) occupancy prediction. Moreover, monocular occupancy prediction remains an important research problem due to its cost efficiency and relevance to real-world embodied agents, especially in camera-only systems[3,4]. Our contribution aims to advance this line of work by enhancing prediction quality in both single-frame and sequential-view settings.\n\n## **Q5 Insufficient Visual and Temporal Evaluation(weakness4)**\n\nThank you for the suggestion. Due to space limitations, we included only two qualitative examples in the main paper, but we provide additional visual results in the appendix. We will further expand this section in the camera-ready version to better illustrate the qualitative improvements of our method. While we cannot submit additional figures through the rebuttal system, we agree that visualizations are important for evaluating spatial and temporal consistency. Regarding temporal consistency, we note that Tables 3 and 4 present continuous-view results computed over the accumulated occupancy from 30 sequential monocular frames. These metrics **inherently** reflect the alignment and consistency of per-frame predictions over time. The observed improvements in global IoU and mIoU indicate that our model maintains robust predictions across diverse viewpoints, which is critical for navigation tasks in robotics.\n\n## **Q6 Complex structures performance (weakness5)**\n\nAs shown in Tables 1–4, DFGauss consistently improves the prediction of most object categories, particularly complex and fine-grained structures such as furniture and small objects. We observe that large planar regions like walls and ceilings show slightly lower improvements, which may indicate a trade-off: our dynamic refinement strategy tends to focus more on localized and semantically rich regions, potentially at the cost of global consistency in large homogeneous areas. This suggests an interesting direction for future work—developing strategies to balance detail-aware refinement with broader spatial regularization.\n\n[1] Wu, Yuqi, et al. \"Embodiedocc: Embodied 3d occupancy prediction for vision-based online scene understanding.\" ICCV 2025 \n\n[2] Huang, Yuanhui, et al. \"Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction.\" ECCV 2024\n\n[3]Yu, Hongxiao, et al. \"Monocular occupancy prediction for scalable indoor scenes.\" ECCV 2024\n\n[4]Zheng, Yupeng, et al. \"Monoocc: Digging into monocular semantic occupancy prediction.\" ICRA 2024"}, {"review_id": "HmMCH8ARVu", "reviewer": "Reviewer_SDTU", "summary": "This paper presents an autoregressive framework for efficient 3D occupancy prediction using multi-scale Gaussian representations. By progressively refining sparse 3D Gaussians through dynamic masking, the proposed DFGauss method aims to improve prediction accuracy while reducing computational cost, demonstrating promising results on standard benchmarks.", "questions": "* Provide a more detailed justification for the choice of occupancy grids in embodied scenarios, including a discussion of their benefits over depth-based or direct 3D Gaussian approaches. It would be better if the authors could show some real applications of the occupancy in embodied scenarios.\n* Include quantitative comparisons of inference speed and memory usage with other leading methods.\n* Clarify any methodological distinctions between occupancy prediction for autonomous driving and embodied perception tasks.\n\nIf these points are thoroughly addressed, I would be inclined to raise my rating.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strengths\n* **Methodological Design:** The DFGauss framework introduces a hierarchical, autoregressive process for 3D Gaussian representations, enabling coarse-to-fine occupancy prediction. The dynamic masking strategy selectively refines only the most informative Gaussians, which leads to more efficient training and inference.\n* **Clarity of Writing:** The paper is well-written, clearly articulating both the problem and the proposed methodology.\n\nWeaknesses\n* **Limited Novelty:** The use of self-attention and cross-attention mechanisms for generating 3D Gaussian representations has already been widely explored in the 3D AIGC field [1, 2]. Furthermore, as acknowledged by the authors, coarse-to-fine pipelines are standard practice in occupancy prediction, limiting the novelty of the approach.\n* **Inference Speed and Memory Consumption:** While the coarse-to-fine pipeline can boost accuracy, it may also result in slower inference, as noted in Table 7 in the paper. It would strengthen the work to include a comprehensive comparison of inference speed and memory consumption with other methods.\n* **Motivation for Embodied Occupancy:** The motivation for embodied occupancy prediction is not entirely convincing. In autonomous driving, occupancy grids are widely used because they provide basic geometric reconstruction from just visual input, with a typical voxel resolution of 200×200×16, which suffices for outdoor scenes. For embodied scenarios—***where finer geometric detail is crucial for manipulation and other tasks***—the paper adopts a lower voxel resolution (60×60×36), which may be inadequate. It remains unclear what advantages embodied occupancy offers over alternative perception methods such as direct depth estimation or 3DGS reconstruction. Additionally, the paper does not sufficiently discuss methodological differences between occupancy prediction for autonomous driving and embodied perception. Finally, it remains unclear why not use the reconstructed 3D Gaussian representations directly, as voxelization may discard valuable geometric information.\n\n[1] Chen et al., *3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion*, CVPR 2025.\n\n[2] Zhang et al., *GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation*, NeurIPS 2024.", "was_revised": true, "final_justification": "The author has addressed my problems during rebuttal and discussion, especially about the motivation for embodied occupancy.", "rebuttal": "Dear Reviewer SDTU,\n\nThank you for acknowledging the clarity of our writing and the design of our framework. Below are our detailed responses to your feedback.\n\n## **Q1 Detailed justification for the choice of occupancy grids.(Weakness 3 Motivation)**\n\nThe **choice of occupancy grids** is primarily determined by the baseline model and dataset specifications. It depends on the resolution provided by the original dataset, and all baselines adopt the same grid size to ensure fair comparison.\n\nThe key **advantage** of monocular semantic occupancy prediction is that it can implement **Occlusion reasoning**[7] to understand and interpret scenarios where objects are partially or fully obscured by others[9][10]. Additionally, its cost efficiency and rich visual information have garnered significant attention in the robotics community[7]. It enables the inference of complete 3D geometry and semantic understanding of scenes using only 2D images[8]. \n\nTraditional monocular **depth estimation** aims to infer the distance from the camera center to the 3D **surface** point corresponding to each pixel in a single RGB image, typically by predicting a depth map. However, depth maps only predict the nearest occupied point in each optical ray and are unable to recover the occluded parts of the 3D scene[18]. This depth information is often used as an **intermediate** representation for downstream tasks such as 3D occupancy prediction, as seen in our baselines in EmbodiedOcc (ICCV 2025) and ISO (ECCV 2024). \n\nDense representation is computationally expensive in robotics[13]. Occupancy prediction models often employ sparse formats, where only occupied voxels (or cells) are explicitly stored, reducing computational overhead and memory consumption[14][15]. For embodied perception, such as robotic navigation, a compact, discrete, memory-efficient, and uniform representation is essential[16]. In contrast, **Direct 3D Gaussian** approaches **can not naturally** provide these characteristics. Gaussian-based method can **contribute** to occupancy estimation in scale-aware training and faster rendering in the voxel space[11]. Importantly, predicting 3D semantic occupancy from Gaussian representations is **different** from 3D Gaussian splatting[12]. Thus, Gaussian-based occupancy prediction is a well-suited task format that can meet the above requirement. \n\nOccupancy grids are a common choice in robotics, primarily due to their explicit free-space definition and their ability to handle unobserved areas through inference and completion [6]. There is a wide range of **applications for occupancy prediction** in embodied scenarios, such as robotic navigation [1], planning [2], obstacle detection [5], and mapping [3,4].\n\n## **Q2 Quantitative comparisons...**\n\n| Method                    | Latency (ms) | Memory (MB) |\n|---------------------------|--------------|-------------|\n| ISO (ECCV 2024 )          |  650         | 10214       |\n| EmbodiedOcc (ICCV 2025)   | 114          | 3161        |\n| Ours                      | 232          | 5732        |\n\nAs shown in the table, all methods are evaluated under the same setup. Compared to traditional convolution-based architectures such as ISO and Gaussian-based models like EmbodiedOcc, our method achieves a favorable balance between efficiency and accuracy.\n\n## **Q3 Methodological distinctions**\n\nIndoor scenes typically exhibit more **diverse** and complex changes, such as variations in illumination, geometry, and appearance due to human interaction, making them harder to predict compared to outdoor scenes, which often have cyclic changes like day-night and seasonal variations[17]. In contrast, outdoor driving environments typically involve large-scale, **structured** spaces such as roads, intersections, and lanes, where objects like vehicles and pedestrians follow predictable spatial patterns[6]. Thus, feature refinement in the indoor scenario is important, while scalable training[11] is crucial for outdoor driving scenarios. \n\nFrom a methodological perspective, indoor scenes benefit more from voxel-based depth fusion due to the critical role of height information and complex object layouts in enclosed environments. In contrast, mainstream BEV-based depth fusion, which performs well in outdoor driving scenes, is less effective when directly applied to indoor scenarios, due to information loss from height compression[8]. Thus, a coarse-to-fine framework for the Gaussian representation can help delineate the overall structure at coarser levels and progressively focus on **finer details** at higher resolutions, enabling better modeling of complex and irregular shapes in indoor unstructured scenarios for occupancy prediction..\n\n## **Q4 Novelty (Weakness 1)**\n\nFrom the provided citation[19,20], 3D AIGC focuses on generating rich meshes, spatially varying textures (e.g., textured Gaussians), and editable 3D content, which are not primary concerns for embodied scenarios. These methods mainly focus on directly 3D Gaussian rendering, not Gaussian-based 3D semantic occupancy prediction. Such methods  **cannot** be directly applied to this different type of task.\n\nPrior coarse-to-fine pipelines typically rely on convolution-based architectures[18]. In contrast, our work demonstrates that direct optimization in the Gaussian parameter space provides a more efficient and robust alternative for embodied occupancy prediction. This is particularly important in complex indoor environments where fine-grained, occlusion-aware reasoning is needed.\n\nTo the best of our knowledge, no existing autoregressive pipeline operates directly on Gaussian representations for this task, primarily because conventional convolutional models are not well-suited to handle sparse and continuous Gaussian primitives. Our approach opens a new and promising direction for efficient, structured 3D scene understanding in embodied AI, particularly suited for modeling fine-grained geometric and semantic features in complex indoor environments.\n\n[1]Katyal, Kapil D., et al. \"High-speed robot navigation using predicted occupancy maps.\" ICRA 2021.\n\n[2]Kim, Jinkyu, et al. \"Stopnet: Scalable trajectory and occupancy prediction for urban autonomous driving.\" ICRA 2022\n\n[3]Asghar, Rabbia, et al. \"Vehicle motion forecasting using prior information and semantic-assisted occupancy grid maps.\" IROS 2023.\n\n[4]Wang, Xiaofeng, et al. \"Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception.\" ICCV 2023\n\n[5]Ess, Andreas, et al. \"Moving obstacle detection in highly dynamic scenes.\" ICRA 2009.\n\n[6]Tian, Xiaoyu, et al. \"Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.\" NeurIPS 2023\n\n[7]Zheng, Yupeng, et al. \"Monoocc: Digging into monocular semantic occupancy prediction.\" ICRA 2024.\n\n[8]Yu, Hongxiao, et al. \"Monocular occupancy prediction for scalable indoor scenes.\" ECCV 2024.\n\n[9]Jiang, Ziyu, et al. \"Peek-a-boo: Occlusion reasoning in indoor scenes with plane representations.\" CVPR 2020\n\n[10]Yuan, Xiaoding, et al. \"Robust instance segmentation through reasoning about multi-object occlusion.\" CVPR 2021\n\n[11]Gan, Wanshui, et al. \"Gaussianocc: Fully self-supervised and efficient 3d occupancy estimation with gaussian splatting.\" ICCV 25\n\n[12]Huang, Yuanhui, et al. \"Gaussianformer: Scene as gaussians for vision-based 3d semantic occupancy prediction.\" ECCV 24\n\n[13]Paria, Biswajit, et al. \"Minimizing flops to learn efficient sparse representations.\" ICLR 2020.\n\n[14]Wang, Jiabao, et al. \"Opus: occupancy prediction using a sparse set.\" NeurIPS 2024\n\n[15]Liu, Haisong, et al. \"Fully sparse 3d occupancy prediction.\" ECCV 2024\n\n[16]Lu, Yuhang, et al. \"Octreeocc: Efficient and multi-granularity occupancy prediction using octree queries.\"  NeurIPS 2024\n\n[17]Wald, Johanna, et al. \"Beyond controlled environments: 3d camera re-localization in changing indoor scenes.\" ECCV 2020\n\n[18]Wei, Yi, et al. \"Surroundocc: Multi-camera 3d occupancy prediction for autonomous driving.\" ICCV 2023.\n\n[19]Chen et al., 3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion, CVPR 2025.\n\n[20] Zhang et al., GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation, NeurIPS 2024."}]

Author rebuttal:

Meta-review: This submission receives 2 borderline accepts, 1 accept and 1 reject. The concerns of reviewer pKS8, SDTU, and txXB have been addressed successfully in the rebuttal discussion period. In terms of reviewer 9Hse, the remaining concern is the relationship between the proposed work and the previous GaussianFormer and VAR, which I believe is not a major issue and can be clarified in the revision. Given the strong performance of the proposed method as shown in the manuscript and responses of authors w.r.t. reviewers' concerns, I recommend acceptance of this submission. 

To improve submission quality, authors are required to include clarifications and additional ablation studies in the revision, including but not limted to:
1. the relationship between the proposed work and the previous GaussianFormer and VAR
2. modification of the title especially 'dynamic' and 'autoregressive' to avoid confusion
3. ablation study on grid resolution 
4. details and clarification of module design
Decision: Accept (poster)