Title: Dynamic Focused Masking for Autoregressive Embodied Occupancy Prediction
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D gaussian splatting

indoor Occupancy Prediciton
Abstract: Visual autoregressive modeling has recently demonstrated potential in image tasks by enabling coarse-to-fine, next-level prediction. Most indoor 3D occupancy prediction methods, however, continue to rely on dense voxel grids and convolution-heavy backbones, which incur high computational costs when applying such coarse-to-fine frameworks. In contrast, cost-efficient alternatives based on Gaussian representations—particularly in the context of multi-scale autoregression—remain underexplored. To bridge this gap, we propose DFGauss, a dynamic focused masking framework for multi-scale 3D Gaussian representation. Unlike conventional approaches that refine voxel volumes or 2D projections, DFGauss directly operates in the 3D Gaussian parameter space, progressively refining representations across resolutions under hierarchical supervision. Each finer-scale Gaussian is conditioned on its coarser-level counterpart, forming a scale-wise autoregressive process. To further enhance efficiency, we introduce an importance-guided refinement strategy that selectively propagates informative Gaussians across scales, enabling spatially adaptive detail modeling. Experiments on 3D occupancy benchmarks demonstrate that DFGauss achieves competitive performance, highlighting the promise of autoregressive modeling for scalable 3D occupancy prediction.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an autoregressive framework for efficient 3D occupancy prediction using multi-scale Gaussian representations. By progressively refining sparse 3D Gaussians through dynamic masking, the proposed DFGauss method aims to improve prediction accuracy while reducing computational cost, demonstrating promising results on standard benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n* **Methodological Design:** The DFGauss framework introduces a hierarchical, autoregressive process for 3D Gaussian representations, enabling coarse-to-fine occupancy prediction. The dynamic masking strategy selectively refines only the most informative Gaussians, which leads to more efficient training and inference.\n* **Clarity of Writing:** The paper is well-written, clearly articulating both the problem and the proposed methodology.\n\nWeaknesses\n* **Limited Novelty:** The use of self-attention and cross-attention mechanisms for generating 3D Gaussian representations has already been widely explored in the 3D AIGC field [1, 2]. Furthermore, as acknowledged by the authors, coarse-to-fine pipelines are standard practice in occupancy prediction, limiting the novelty of the approach.\n* **Inference Speed and Memory Consumption:** While the coarse-to-fine pipeline can boost accuracy, it may also result in slower inference, as noted in Table 7 in the paper. It would strengthen the work to include a comprehensive comparison of inference speed and memory consumption with other methods.\n* **Motivation for Embodied Occupancy:** The motivation for embodied occupancy prediction is not entirely convincing. In autonomous driving, occupancy grids are widely used because they provide basic geometric reconstruction from just visual input, with a typical voxel resolution of 200×200×16, which suffices for outdoor scenes. For embodied scenarios—***where finer geometric detail is crucial for manipulation and other tasks***—the paper adopts a lower voxel resolution (60×60×36), which may be inadequate. It remains unclear what advantages embodied occupancy offers over alternative perception methods such as direct depth estimation or 3DGS reconstruction. Additionally, the paper does not sufficiently discuss methodological differences between occupancy prediction for autonomous driving and embodied perception. Finally, it remains unclear why not use the reconstructed 3D Gaussian representations directly, as voxelization may discard valuable geometric information.\n\n[1] Chen et al., *3DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusion*, CVPR 2025.\n\n[2] Zhang et al., *GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation*, NeurIPS 2024.\n\nQUESTIONS:\n* Provide a more detailed justification for the choice of occupancy grids in embodied scenarios, including a discussion of their benefits over depth-based or direct 3D Gaussian approaches. It would be better if the authors could show some real applications of the occupancy in embodied scenarios.\n* Include quantitative comparisons of inference speed and memory usage with other leading methods.\n* Clarify any methodological distinctions between occupancy prediction for autonomous driving and embodied perception tasks.\n\nIf these points are thoroughly addressed, I would be inclined to raise my rating.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a multi-scale Gaussian representation framework for 3D occupancy prediction. It proposes a multi-scale Gaussian encoding method to convert image features into hierarchical Gaussian primitives (position, scale, opacity, semantics). Then It employs an autoregressive refinement to progressively refine Gaussian parameters from coarse to fine scales via cross-attention. To reduce computation, it proposes a dynamic focused masking, which selectively updates informative Gaussians (top-ρ% by learned scores).\nEvaluated on indoor (Occ-ScanNet, EmbodiedOcc) and outdoor (SSCBench-KITTI-360) benchmarks, DFGauss achieves SOTA results (e.g., 47.03 mIoU on Occ-ScanNet) with lower latency than voxel-based methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed multi-scale Gaussian pipeline has lower latency than existing methods.\n2. In the benchmark, it performs better than existing SOTA methods. \n\nWeakness:\n\n1. Section 3.2 contains so many module design details, which are not clear. \n\n2. The paper uses DepthanythingV2 to predict multi-scale depths for geometric alignment during Gaussian initialization. However, the depths from DepthAnythingV2 are metric-unknown and temporally inconsistent. It may also contain many artifacts. I believe this will affect the Gaussian initialization. However, the paper does not discuss this. Why not use other metric depth models? In outdoor scenes, such as KITTI, DepthAnythingV2 works worse in far regions. Will this affect the Gaussian?\n\n3. The paper actually attempts to solve the occupancy prediction from a single view. It should compare with more occupancy methods. \n\n4. The paper only shows 2 visual results in the paper. It is hard to evaluate the performance and quality of predicted occupancy. Furthermore, the paper argues that it targets autonomous navigation for an embodied agent. The temporal consistency is important for robotics navigation. The paper does not present temporal results and evaluation. The single-frame evaluation cannot support the application scenarios. \n\n5. In indoor scenes, there are so many complex and fine-grained structures and objects. Could the method also work well on such objects?\n\nQUESTIONS:\n1. In high-resolution settings, does the large number of Gaussian points lead to excessive memory consumption?\n\n2. Does the method rely too heavily on the effectiveness of monocular depth estimation?\n\n3. How robust is the model under large viewpoint changes?\n\nLIMITATIONS:\n1. The visual comparisons and results cannot demonstrate the robustness and superiority of the method.\n\n2. The paper does not discuss the effect of monocular depths.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a hierarchical Gaussian refinement strategy as an improvement over the conventional EmbodiedOcc framework. The proposed DFGauss mainly consists of a multi-scale Gaussian encoder, a hierarchical Gaussian refinement process, and a multi-scale supervision strategy. The paper conducts extensive experiments on both indoor and outdoor datasets, which demonstrates good performance compared with the baseline.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Good motivation. The multi-scale Gaussian encoder and hierarchical refinement design stem from the coarse-to-fine nature of visual perception.\n2. Extensive experiments. The paper conducts experiments on two indoor datasets (excluding the mini versions) and one outdoor dataset to verify the effectiveness of the proposed method.\n\nWeaknesses:\n1. Potentially inappropriate expression. The \"Dynamic Focused\" expression in the title might be misleading because the paper actually focuses on static scenes and this phrase actually refers to adaptive selection of Gaussians requiring refinement. Also, the paper have potentially overclaimed itself to be autoregressive, I think \"hierarchical\" might be more suitable.\n2. Technical flaw. The Gaussian refinement module first computes a confidence score and select Gaussians to refine with a percentile-based strategy. The computation of confidence scores should introduce learnable parameters, but the percentile-based selection is not differentiable which prevents gradients from back-propagating to the score-related parameters. The paper does not discuss this.\n\nQUESTIONS:\n1. How are the parameters used to compute the confidence score for Gaussians updated? Please refer to the weakness section for details.\n2. How is the GPU memory consumption of DFGauss compared with the baseline method EmbodiedOcc?\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper presents DFGauss, a 3D occupancy prediction framework from single indoor images. Instead of using computationally intensive voxel grids, DFGauss operates on sparse 3D Gaussian representations with multi-scale autoregressive modeling to improve the efficiency. Moreover, a dynamic focused masking mechanism that improves efficiency by selectively updating only the most informative Gaussians, concentrating computational effort on important regions. Experiments show that DFGauss achieves state-of-the-art performance on indoor 3D occupancy prediction benchmarks.

STRENGTHS AND WEAKNESSES:
Strength:
1. The performance promotion is impressive.
2. The proposed method is easy to understand.

Weaknesses:
1. The proposed Multi-Scale Gaussian Encoder should be introduced with more details, such as the number of layers of cross-attention and self-attention, and how to leverage the generated image features and depth map.
2. There are many typos and inconsistencies in the paper: q in Eq. 5 is not defined in the manuscript. The notions of the Gaussian set are not consistent in Eq. 1 and Eq. 7.
3. More visualizations should be provided, including comparisons on different datasets and abalations.
4. The proposed method appears to be merely a multi-scale version of GaussianFormer , and the reviewer does not find any connection between the proposed work and visual autoregressive models.

QUESTIONS:
1. How do you use multi-scale autoregressive in your work?
2. Could you provide the details about the Multi-Scale Gaussian Encoder and illustrate the network details?
3. What kind of network is the first row in Table. 5? Please describe the baseline.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 1
ORIGINALITY: 2
DATE: 2025-07-03