Progressive Gaussian Transformer with Anisotropy-aware Sampling for Open Vocabulary Occupancy Prediction
Authors: Chi Yan
Dan Xu
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Gaussian Spaltting
3D Occupancy Prediction
Open-vocabulary
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Progressive Gaussian Transformer for open-vocabulary 3D occupancy, adaptively expanding Gaussian representations to achieve detailed and scalable scene understanding.

Abstract: The 3D occupancy prediction task has witnessed remarkable progress in recent years, playing a crucial role in vision-based autonomous driving systems. While traditional methods are limited to fixed semantic categories, recent approaches have moved towards predicting text-aligned features to enable open-vocabulary text queries in real-world scenes. However, there exists a trade-off in text-aligned scene modeling: sparse Gaussian representation struggles to capture small objects in the scene, while dense representation incurs significant computational overhead. To address these limitations, we present **PG-Occ**, an innovative  **P**rogressive  **G**aussian Transformer Framework that enables open-vocabulary 3D occupancy prediction. Our framework employs progressive online densification, a feed-forward strategy that gradually enhances the 3D Gaussian representation to capture fine-grained scene details. By iteratively enhancing the representation, the framework achieves increasingly precise and detailed scene understanding. Another key contribution is the introduction of an anisotropy-aware sampling strategy with spatio-temporal fusion, which adaptively assigns receptive fields to Gaussians at different scales and stages, enabling more effective feature aggregation and richer scene information capture. Through extensive evaluations, we demonstrate that **PG-Occ** achieves *state-of-the-art* performance with a relative  **14.3\% mIoU improvement** over the previous best performing method. Code and pretrained models are available at: https://yanchi-3dv.github.io/PG-Occ.

PROGRESSIVE GAUSSIAN TRANSFORMER WITH ANISOTROPY-AWARE SAMPLING FOR OPEN VOCABULARY OCCUPANCY PREDICTION
Chi Yan and Dan Xui*
'he Hong Kong University of Science and Technology (HKUST)
ZZEKKR Automobile R&D Co., Ltd [cyanao, ease ust.hk
ABSTRACT
The 3D occupancy prediction task has witnessed remarkable progress in recent years, playing crucial role in vision- autonomous driving systems. While traditional methods are limited to fixed semantic categories, recent approaches have moved towards predicting text-aligned features enable open-vocabulary text queries in real-world scenes. However, there exists a trade-off text-aligned scene modeling: sparse Gaussian representation struggles to capture small objects in the scene, while dense representation incurs significant computational over- head. To address these limitations, we present PG-Oce, an innovative Progressive Gaussian Transformer Framework that enables open-vocabulary 3D occupancy prediction. Our framework employs progressive online densification, a feed- forward strategy that gradually enhances the 3D Gaussian representation to cap- ture fine-gran graiggr scene details. By iteratively enhancing the representation, the framework achieves increasingly precise and detailed scene understanding. An- other key contribution is the introduction of an anisotropy-aware sampling strat- egy with spatio-temporal fusion, which adaptively assigns receptive fields Gaussians at different scales and stages, enabling more effective feature aggre- gation and richer scene information capture. Through extensive evaluations, we demonstrate that PG-Oec achieves state-of-the-art performance with a relative 14.3% % mloU improvement over the previous best performing method. Code and pretrained models are available at: http://aanchii-d....ubb..oo.o
INTRODUCTION
3D Occupancy perception technology has emerged as pivotal trend in autonomous driving percep- tion systems, garnering substantial attention from both industry and academia due to its comprehen- sive perception capabilities (Xu al. 2025 Zhang et al.) 2024). Unlike previous BEV representa- tions (Li et al. 2022). 3D occupancy enriches scene understanding with crucial height information, enabling complete three-dimensional representation of the environment. Accurate prediction of 3D occupancy and semantic information serves as a cornerstone for robust scene understanding and reconstruction (Cao & de Charette [021 Ye & Xu 2022 Huang et al.) (003). While several benchmarks (Wang et 2023) Tian et al. 2024 Sun et al. (0200) have been established to provide semantic annotations for 3D occupancy superviston, they inherently constrain semantic information to predefined categories. This limitation severely hinders the system's 's ability to perceive general
objects beyond these predefined categories.
To enable semantic occupancy detection based on arbitrary user inputs. recent approaches (Tan al. 2023) Vobecky et al.) 2024, Boeder et al. 2024| Zheng al., 2024a) have shifted away from directly predicting predetined semantic categories of occupancy. Instead, they focus on predicting text-aligned features, which can then be used to calculate similarity scores with text queries to ob- tain semantic correspondences. This paradigm shift allows open-vocabulary detection capabilities, where the system can identify objects beyond predefined categories by leveraging the rich semantic space of text embeddings. By establishing this text feature alignment in 3D space, these methods effectively bridge the gap between language understanding and spatial perception, enabling more flexible and generalizable occupancy prediction systems. However, due to the high-dimensional na-

---

PG-Occ
PG Dez (Durs)
Progress ivve Goussian Modeling Anisotropy- Sampling
Open Vecabulary Retrieva
Uver paanpe
Stage
Stage
Stage
derk
Dicilhuut
GovsaTR
Largiez
burfo
Srage
Current view
Online Feed-forward Densification
Occupancy Prediction
Figure I: Overview of the proposed PG-Oce framework. The radar chart compares occupancy pre- diction accuracy across multiple methods, showing the superior performance of PG-Occ. The central panel highlights the key components: progressive Gaussian modeling with online feed-forward den- sification, anisotropy-awar sampling with adaptive receptive fields, and open-vocabulan retrieval conditioned on prompt inputs. The bottom row illustrates an example progression from the current input view through successive densification stages to the final occupancy prediction.
ture of text features, densely modeling the entire scene incurs substantial memory and computational overhead, severely impacting system efficiency.
Inspired by Gaussian representation (Kerbl et al. 2024 Huang et al., 2024b) Cao et al. (024)) recent work such as GaussTR (Jiang et al.) (024) leverages sparse Gaussian representation to achieve efficient scene perception. However, due to the inherent sparsity of Gaussians, these approaches often struggle to capture fine-grained details in complex scenes, limiting their effectiveness in comprehensive environmental understanding.
2023) and its applications perception tasks Gan
To overcome the aforementioned limitations, introduce PG-Oec, novel Progressive Gaussian Transformer framework for open-vocabulary 3D occupancy prediction. Our framework preserves the computational efficiency of sparse Gaussian representations while overcoming their limitation in modeling fine-grat aee-graiedd scene details through an iterative seed-forward densification strategy. Specif- ically, the framework first leverages coarse base Gaussians model the global scene structure. It then progressively refines regions with insufficient perception by performing feed-forward densifica- tion conditioned the current prediction. Furthermore, we propose anisotropy-awan sampling method that selects sample points according to each Gaussian's 's anisotropy and projects them onto feature planes with varying receptive fields, enabling more effective spatio-temporal feature fusion.
Specifically, we introduce:
A Progressive Gaussian Transformer framework for open-vocabulary 3D occupancy prediction, which iteratively enhances scene details through online progressive densification guided by per- ception errors from previous layers, significantly improving perception accuracy.
An anisotropy aware sampling method that adaptively adjusts the receptive fields of Gaussians according to their spatial distribution, enabling more effective integration of explicit Gaussian representations with spatio-temporal features.
Comprehensive experimental validations demonstrating that PG-Oec achieves state-of-the- art performance on the challenging Occ3D-nuScenes dataset, with remarkable relative 14.3% mloU improvement over the previous best results.
2 RELATED WORK
Close-set 3D Occupancy Perception. Using strong close-set 3D labels to supervise 3D oceu-
pancy networks is a straightforward idea, and most existing work (Wei et al.) 2023) Huang et al.)
2023
Zhang et al. 2023b Ma al. 2024 Hou et al.) 2024) is based on this training approach (Xu
al.
2025). Some improvements focus efficient spatial representation. SurroundOce (Wei al.)
2023
extends the BEV with height dimension through spatial cross-attention. TPVFormer (Huang
2023) divides the space into three perspective views, reducing the parameters and computa- tional costs. FastOce (Hou et al.) 2024) accelerates processing by replacing 3D convolutional net- works with lightweight 2D BEV convolutions, while GaussianFormer (Huang et al. 2024h) reduces
al.
2

---

computation in empty spaces by utilizing sparse Gaussian representations. Another line of research focuses on label efficiency (Pan et al. 2024 Zhang et al. 2023a Gan et al. 2024 Huang et al.)
2024at Diang et al. 2024). drawing inspiration from NeRF and 3D Gaussian splatting techniques. These approaches distill 3D occupancy information from Gaussians extracted by 2D foundation models, significantly redueing the need for extensive 3D annotations.
Open-vocabulary 3D Occupancy Perception. Current 3D occupancy benchmarks feature cat- egories with varying semantic clarity some, like "car", "pedestrian", and "truck", have explicit definitions, while others such as "manmade" and "vegetation", remain vague. These broader cat- egories contain numerous undefined semantics that would benefit from finer-grained subdivision to better characterize driving environments. Novel objects typically classified merely as gen- eral obstacles, lacking the flexibility to expand perception based on human prompts (Cao et al.) 2023 To address this challenge, OVO (Tan al. (023) pioneered framework that enables open- vocabulary 3D occupaney perception by distilling knowledge from a frozen 2D open-vocabulary segmenter and CLIP text encoder into the 3D model. Similarly, POP-3D (Vobecky al.) 2024) designed a semi-supervised framework incorporating three modalities to improve zero-shot open- vocabulary capabilities. VEON (Zheng et al. 2024a) further advanced the performance by assem- bling and adapting two complementary 2D foundation models. To reduce dependence on LiDAR sensors, LangOce (Boeder et al., 2024) integrated Neural Radiance Fields (NeRF), enabling purely vision-based perception approaches. To address the computational overhead of high-dimensional text-vision features, GaussTR (Jiang et al., 2024) models scenes as sparse unstructured Gaussian blobs, achieving reconstruction through a camera-win feed-forward approach.
Generalizable 3D Gaussian Splatting with Feed-forward Networks. A significant limitation of vanilla 3D Gaussian splatting (Kerbl et al., 2023) its requirement for offline scene-specific optimization rather than efficient foed-forward inference Wang et al. (025). The success of feed- forward approaches comes from the learning of powerful priors from large datasets. Recent works such as Splatter Image (Szymanowicz et al. (024) and pixelSplat (Charatan et al. 2024) have pro- posed novel approaches that enable direct prediction of 3D Gaussian Spaltting parameters from one or two input views, respectively. GPS-Gaussian (Zheng et al. (024b) utilizes feed-forward net- works for human reconstruction. DrivingForward (Tian et al. 2025) introduces this paradigm in sparse-view 3D reconstruction for autonomous driving scenarios. GaussTR (Jiang et al. (022) con- ceptualizes the perception of the autonomous driving scene as task to predict 3D Gaussians from six surround view cameras, while GaussianFlowOoc (Boeder al. 2025) captures dynamic scenes through Gaussian flow estimation. Unlike existing approaches, our work focuses on progressive Gaussian modeling, employing a coarse-to- fine approach to enhance scene perception capabilities.
METHODOLOGY
As illustrated in Fig. 2 PG-Oce introduces a novel open-vocabulary 3D occupancy prediction frame- work. Our approach represents scenes set of text-aligned feature Gaussian blobs in a progres- sive foed-forward manner. In Section 3. we first present 3D feature Gaussians as an effective scene representation for apen-vocabulary occupancy prediction. Subsequently, in Section 3.2 describe how spatio-temporal image features are progressively transformed into Gaussian represen- tations through an iterative architecture, consisting of a base layer followed by B progressive layers. Additionally, we propose an anisotropy-awart sampling mechanism for spatio-temporal feature fu- sion, enabling more precise and robust scene understanding. The loss functions used to train our model are detailed in Section 3.3] Finally, as detailed in Section 3.4 we convert the final Gaussian representations into a dense 3D occupancy field.
3.1 3D FEATURE GAUSSIAN SPLATTING
vec-bocayyl 3D occupancy prediction aims to identify and localize occupancy regions around a vehicle that correspond to arbitrary text prompts Ctert, given L spatio-temporal camera views = (4,, } at the current time step. Directly predicting 3D text-aligned voxel features is computationally and memory-intensive due to the high dimensionality of text features. Inspired by 3D Gaussian Splatting (Kerbl et al. 2023| Zhou et al., 2024), we model the driving scene as a set of sparse feature Gaussian blobs g. Vanilla 3D Gaussians typically encode color features; in contrast, we replace them with high- high-dimensional text-aligned features to better capture semantic information for ooen-loca occupancy prediction. Formally, each feature Gaussian blob G,
3

---

User Prompt:
object "trar" "road, "oan
General object: "Talffc ligltiligttt' "shop...
Spatio Temporal Feature
Multi camera Sequence
20 Labela
Base Layer
Shared Guussinn Decoder
Feature
Sampling
Asa egirion
Progressive Layer
Maoked
Voselizttoo
Aggrspation
Occupancy Figure 2: Architecture of the proposed PG-Oce framework. The scene represented as feature Gaussian blobs, starting from base layer and progressively refined and densified through B layers. Multi-camera inputs are processed to extract spatio-temporal features, which guide the update and refinement of the Gaussians, which are then voxelized to produce any-resolution 3D occupancy field, enabling both geometric reconstruction and open- apec-bocabylary semantic understanding.
Feature Gaussian
(hereafter simply called "Gaussian") is defined by its spatial position 14 E IR3 scale s; E RR3 rotation quaternion r, I opacity d; E IR, and a text-aligned feature f. E 12522;
G {G; (14, 94, Ti, Ta fi) i N},
where denotes the number of Gaussian blobs in the scene.
Gaussian representations, in addition to their inherent sparsity, enable efficient rendering and fa- cilitate training via 2D label supervision. Given the camera pose T₃ and intrinsic matrix K,, the Gaussians can be efficiently rasterized onto the 2D camera plane, producing per-pixel expected depth D and feature map F. For each pixel, these values are computed as:
ò EiEN daa I (1-ag) ΣiEN as II; (1 ag-
F= Σ fia (1- iEN
(2)
j=1
where d represents the depth value the i-th 3D Gaussian center point μ4 projected along the 2-axis in the camera coordinate system, and a denotes the blending weight of the i-th Gaussian.
3.2 PROGRESSIVE 3D GAUSSIAN MODELING
The core of PG-Occ is a set of learnable and adaptively expandable Gaussian queries G E ND with spatial positions μu, where the latent feature dimension D encodes Gaussian blob attributes, cluding scale rotation r. opacity and feature vector f. These Gaussian queries are processed by Progressive Gaussian Transformer, which consists of a base Gaussian layer that captures coarse scene geometry, followed by B progressive layers that iteratively refine the Gaussian representa- tions. illustrative example of this online progressive refinement shown Fig.
Each progressive layer b further incorporates three core components: Progressive Online Densifi- cation (POD), Asymmetric Self-Attention (ASA), and Anisotropy-aware Feature Sampling (AFS). Prior Gaussian-based occupancy estimation methods (Jiang et al. 2024 Boeder et al., 2025 use fixed number N queries, limiting their ability to model complex scenes. In contrast, our approach adaptively expands the queries in each layer by adding N queries via feed-forward module con- ditioned on the output of the preceding layer Gb-1, This progressive expansion allows adaptive, fine-grained modeling of intricate scene structures while maintaining computational efficiency.
3.2. PROGRESSIVE ONLINE DENSIFICATION (POD)
In contrast to vanilla 3D Gaussian Splatting (Kerbl et al., 2023). which initializes Gaussians from structure-from-mottin point clouds and relies on gradient- based densification, we propose an ficient feed-forward strategy for real-time Gaussian densification. Our approach comprises two stages: base initialization, which establishes initial Gaussian positions, and feed-forward densi- fication, which adaptively augments the Gaussian set in regions where the scene remains under- represented. This design enables online refinement of scene geometry without the computational overhead of gradient backpropagation.
4

---

Feature Progressive Online Densificatien (POD)
Updated
Anisotropy-aware Feature Sampling (AFS)
Pooudo Depth
Food ferdofrr
Rastoctration
Decoder
densificttio
Previess Casariane
Ankoorapy sware receptive
1
Prograshe Gensinas
Updated
Fubtr Aggregation
Gaussian
3D Gaussian g
Ferture Sempiing Paint
Gaussian G
Iggooeed Sampling Point
Figure 3: Illustration of the Progressive Online Densification (POD) and Anisotropy Sampling (AFS) modules. POD) leverages depth-aware densification to progressively add and refine 3D Gaussians. AFS exploits the anisotropic properties of Gaussians, sampling feature points within anisotropy-aware receptive fields to enable more effective spatio-temporal feature extraction.
geometry, utilize pseudo depth maps from Metric3D V2 (Hu et al.) 2024). The downsampled depth map D is back poojeproeetted into the ego-vehicle frame using the camera-to-ego transformation T and camera intrinsics K, yielding pseudo point clouds P:
Base Initialization. To capture coarse
P U Ti (KT Di)
1997) is applied to select N representative points as initial Gaussian positions p.. Each selected point is paired with a trainable query feature, while the Gaussian scale a is fixed and the rotation r is initialized as unit quaternion.
Farthest Point Sampling (FPS) (Eldar et al.
Feed-forward Densification. After obtaining the intermediate Gaussian representation G from the (b 1)-th layer (details in Section 3.2.3), render an expected depth map D using Eq. ( By
comparing D with the reference depth D, we identify undre--epeesented regions:
Usetet = {u, u) E Q D(u, v) - D(u, v) > >?},
(4)
where y is half the final occupancy grid resolution. This module relies solely on Gaussian rendering, avoiding gradient computation and maintaining efficiency. For each under-represented region, we generate point set pe and sample n' new points via FPS. The new Gaussians uddd and
their query features gatd are concatenated with the previous layer's optimized positions pa-1 and queries g form the input for the b-th transformer layer:
n. 11 b-1
( Padd-
-0 ( Lodd
(5)
3.2.2 ASYMMETRIC SELF-ATTENTIO (ASA)
In progressive Gaussian modeling, newly added Gaussians from online densification are initially under-optimized. Self-attention (Vaswani al. 2017) is widely employed model relationships be- tween Gaussians, enhancing overall scene representation. However, applying standard self-attention in this setting risks allowing these under-optimized Gaussians interfere with the well-trained ones from earlier stages, potentially causing training instability.
To address this, we introduce an Asymmetric Self-Attention (ASA) mechanism that enforces asym- metric interactions: newly added Gaussians cannot influence the existing, well-optimized Gaussians, while they can attend to and leverage the features of the existing ones to refine their own under- optimized representations. This design ensures that previously learned Gaussians remain stable, while the newly extended Gaussians progressively improve by utilizing existing information.
Formally, let I4 denote the number of Gaussian queries in layer b, with the first -;-1 inherited from the previous layer and the remaining 24 - -t-1 newly added. Given Gaussian queries q² and their positional encodings PE(μ"), the ASA operation is defined
Gasa AS + +P(()"), d² PE(μ"), g². M). where PE denotes the positional encoding. The attention mask M R x= is constructed
(6)
Mr -00, i < 2h-1 and j T6-1
0,
otherwise

---

By restricting the influence of new Gaussians, ASA stabilizes progressive Gaussian modeling while enabling effective Matr-Gaassiinn feature propagation.
3.2.3 ANISOTROPY-AWARE FEATURE SAMPLING (AFS)
Treating each Gaussian G3 solely as a point at its center (4; oversimplifies feature sampling and ig- nores the anisotropic properties encoded by its scale and rotation quaternion r, which significantly affect the Gaussian's effective receptive field 2D feature space. This simplification reduces sam- pling accuracy, as the anisotropic geometry strongly influences the effective receptive field of each Gaussian the 2D feature space, as illustrated Fig. 3 To exploit anisotropy, the query feature Jasa is passed through an MLP to generate unit offsets fa8 (with n 16 in practice), which are scaled and rotated to lie within the Gaussian ellipsoid:
sample P: + R(r:) (s, ② P
where R(r) the rotation matrix derived from the quaternion r, s the Gaussian scale vector, and C denotes element-wise multiplication.
The resulting 3D sampling points Psample are projected onto 2D feature planes using known camera intrinsics and extrinsics. To aggregate the corresponding features across views and timestamps, we adopt a simple yet effective feature aggregation module from prior work (Liu al. 2023):
fa Aggregation (Interp(() f-j sammle f)}}=1)
where Interp(-) denotes bilinear interpolation at the projected 2D locations, and { indicates the collection of n sampled features corresponding to the i-th Gaussian, which are subsequently aggregated form single representative feature.
Finally, the aggregated feature fa is fed into two lightweight MLPs to decode geometric properties and text-aligned features separately:
f MLP MLPPfeat (ea) 84, 0.) MLP geo (fa): 14 14-1 + Ay4,
(10)
where Aμ, represents learned displacement, allowing the model to refine Gaussian positions in- crementally rather than predicting absolute coordinates directly.
3.3 EFFICIENT TRAINING VIA RASTERIZATION
Due to the lack of labeled open-vocabulary 3D occupancy data, our model is trained using only 2D supervision. Specifically,
leverage text-aligned features F and pseudo depth maps D, both of which are extracted directly from images. This approach climinates the need for LiDAR scans or any explicit 3D point cloud data. To ensure stable training, we rasterize the 3D Gaussians Go from each layer onto the 2D image plane and supervise the model using the losses described below.
Depth Rendering Loss. The depth rendering loss combines SILog, L1, and temporal photometric consistency losses (Godard al. 2019 Yao al. 2024) for geometric consistency:
Cdepth LLa (D, D, D) + CoILog (D, D) + Atemp (D, D).
(11)
where ASTT.og. and Atemp are weighting coefficients to balance different depth loss terms.
Feature Rendering Loss. The feature rendering loss combines mean squared error (MSE) loss and cosine similarity loss achieve the feature alignment:
Ljeat = Leos (F, F) + Amsse F, F):
(12)
where Amse represents weighting coefficient to balance the cosine similarity and MSE losses. Final Objective. The final loss combines the depth and feature rendering losses as follows:
Listal Adepth deepth + A feat,
(13)
where Àdepth and Afeat weighting coefficients to balance the two losses.
3.4 TEST-TIME INFERENCE VIA VOXELIZATION
After obtaining the progressive 3D Gaussian representation of the scene based on the current cam- input, the decoded text-aligned feature Gaussian blobs of the final transformer layer for

---

Input Image
Render Depth
Prompt Query
Semantic Occupancy
# Road Siga
Front Right
# Treee
Back
Figure 4: Illustration of PG-Oec predictions. Given camera inputs and text prompts, the method predicts depth (column 2), produces open-vocabulary semantic labels (column 3), and generates the final semantic occupancy map (column 4). Additional visualizations are provided in appendix C.
evaluation. We convert feature Gaussians into semantic occupancy via two-step process. First, arbitrary text prompts Ctent are encoded using the CLIP text encoder to obtain feature embeddings Steats which are then matched against the Gaussian features assign semantic labels 3D Gaus- sians. Second, Gaussian-to-vont dense occupancy representation. In particular, since our method does require dense occupancy labels during training. this conversion is applied only at inference. Further technical details on the text prompt and the cqussian-to-voxe module are provided in appendix D.I
post-processing step transforms the labeled 3D Gaussians into
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and Metries. We conduct comprehensive experiments two benchmarks: Occ3D- nuScenes and nuScenes retrieval. Open-vocabulary occupancy prediction is evaluated on the Occ3D-nuSeenes dataset (Tian al. 2024). which contains 1,000 scenes captured by surround- view cameras and LiDAR sensors. Semantic labels are assigned to each voxel based on predefined text text queries (see appendix Performance is measured using IoU, mean IoU (mloU), and ray-IoU metrics. Open-vocabulary occupancy retrieval is conducted on the nuScenes retrieval benchmark proposed by POP-3D (Vobecky et al. 2024)), which contains 105 samples. Each LiDAR point cloud is paired with text query, and retrivval performance is evaluated using mean average precision (mAP) over all LiDAR points, as well as mAP (v), which considers only points visible in at least one camera. Depth estimation follows the GaussianOce (Gan et al. (024) setting, where the ground truth depth maps are obtained by projecting LiDAR point clouds. Standard error metrics are used for evaluation, including absolute relative error (Abs Rel), square relative error (Sq Rel), root mean square error (RMSE), and RMSE log.
Implementation Details. We adopt ResNet-50 (He al.. 2016) our image feature extraction backbone, utilizing the previous seven frames to capture spatio -emporal information. Our Progres- sive Gaussian Transformer comprises one base layer and two progressive layers. All experiments are run on 8 x A800 GPUs, with 8 epochs of training (approximately 9 hours). To improve compu- tational efficiency, we use a resolution of 180 x 320 for depth and feature rasterization, as well as Gaussian point initialization. More implementation details can be found in appendix D
4.2 MAIN EXPERIMENT RESULTS
Semantic Occupancy Prediction Results. We report open-vocabulan, occupancy prediction re- sults on the Occ3D-nuSeenes dataset (Tian et al. 2024) in Table [ and group methods based on the sensor modalities employed during training (Camera, LiDAR, and Text). To ensure a fair com- parison and emphasize the benefits of our approach, LangOce, GaussTR, and our PG-Occ all use MaskCLIP [Zhou et al. 2022) for text supervision. Our method achieves SOTA performance with an mloU of 15.15. corresponding to 14.3% relative improvement over previous best methods. Remarkably, despite not using LiDAR data during training, PG-Occ outperforms VEON and other competitors. As shown in the LiDAR chart in Fig. [ our approach excels detecting medium- sized objects. The slightly lower performance on small objects can be attributed to the coarse voxel resolution, with voxel size of 0.4 m, which limits the contribution of finely optimized Gaussians.
7

---

Quantitative performance of 3D occupancy methods Oec3D-nuSeenes dataset. The Mod. column specifies the sensor/modalities used for training: C for Camera, L for LiDAR, and T for Text. IoU scores for "others" and "other flat" classes are consistently zero and thus omitted. "Cons veh." " means construction vehicles, and "drive. surf." means drivable surfaces. The best and second-best results are denoted in bolded and underlined, respectively.
Table I:
conn. veh.
mosonsycle
pedestria
traffic coDe
drive sur.
Vogetation
munnaade
bunier
bicyele
trailer
tomain
bus
cur
Method
Mod.
mloU
SelOOoe (Huang ot al.
21/24
C
10.54
15
0.66
5.46
12.54
0.00
0.80
2. 10
0.001
0.00
8.25
55.49
26
31
26
54
14.22
5,60
OccNeRI Zhang
10.81
5.13
12.49
3.50
1.84
3.90
52.62
24.75
GaussianGce Kuan ct 3/
2024
11:26
1.79
5.82
14.58
13.55
1.30
2.82
7.95
9.76
0.56
9.61
44.59
20.
17.58
8.61
10.29
GoussianHowwGg [Secaer
2025
14.07
6.27
12.38
VEON Zheng et al
2024u
C+L.TT
13.95|4.0
4,80
2,70
14.70
10.90
11.00
3,80
4.70
4.00
5.30
9.60
46.50
21
22.
12
24,80
23.70
DistillNeRI Wang
2024
C+L+T
10:05
1.35
2.08
10.21
10.09
2.56
1,98
5.54
4.62
1443
7.90
43.02
16.86
15.02
14,06
15.06
LangOce Border et al]
20241
C+T 12.04
2.20
7.20
5.80
13:90
0.50
10.8
6.40
8.70
3.20
11.00
42.10
12.50
27.20
14.10 14.50
GausTRR Daang al
C+T
13:25
2.09
5.22
14.07
20.43
5.70
7.08
5.12
3.95
0.92
15.36
39.44
15.68
22.89
21,17
21.87
PG--0e (OurS)
C+T 15.15
4.33
7.31
23.63 26.42
11.38
6.33
2.74
5.79
3.0Z
17.29
37.81
19.29
20.85
19.02
21.92
Table 2: Depth estimation error metrics on the nuScenes validation set. The best results denoted in bold. Abs Rel is used as the primary evaluation metric.
55000 mAFFo
18.4
Method
Rel
5y Rel
RMSF
RMSF log
SelfOce (Huane al. 20226
0.215
2.743
6.205
0.316
OcoNeRF (Zhane
0.302
0.319
GonosianCee Ocs Caincl
0.197
1846
6.733
0.312
GoussanFlowoee Boceke atal. 2025
0.278
2.522
5.232
0.389
Mas
LangDes
P6
Figure 5: SOTA comparison on the
Metric3D V2
al. 2024
0.170
4.016
6.153
0.291
PG.Occ (Ous)
5,466
nuScenes retrieval dataset.
nuScenes Retrieval Dataset Results. As shown in Fig. 5 our method achieves a visible mean Average Precision (mAP(v)) of 21.2 on the nuScenes retrieval dataset. This outperforms the isting vision-based method, LangOec, which achieves 18.2. This improvement demonstrates the effectiveness of progressive Gaussian modeling in enhancing both accuracy and robustness for 3D open-vocabulary retrieval tasks. Note that GaussTR does not report results on the nuScenes retrieval dataset and cannot be evaluated using LiDAR points. Qualitative results in Fig. [ further illustrate our system's 's language- bansuage-based based retrieval capabilities. Queries such as "Locate the cars" and "Locate the garbage bin" are precisely grounded within the predicted 3D occupancy grid.
Depth Estimation Results. We evaluate the geometric accuracy of our Gaussian scene represen- accompanied by depth visualizations in Fig. Remarkably, although supervision is derived from depth maps of Metric3D V2 (Hu al., 2024). our model produces depth estimates that exceed the original labels, achieving an error of 0.139, corresponding
tation via quantitative depth estimation, presented in Table 2
+1..2% boost. This improvement results from geometric constraints imposed by multi-view depth consistency and feature coherence, which help maintain smooth and accurate scene geometry even in challenging regions.
Additional Corner-case Examples. As shown in Fig. we present additional examples of open- vocabulary retrieval results for uncommon categories, such as mailboxes and warning signs. Al- though these categories rarely appear in the dataset, our method is still able to detect them reliably. These examples further demonstrate PG-Oce's ability to generalize to -ocabuacyyy scenarios, accurately capturing semantic occupancy even for rarely seen or atypical objects.
Efficiency Comparison. We compare mloU, training time, and inference speed (frames per second) provide comprehensive assessment of the efficiency of our method. As shown in Table 4 our approach achieves significant relative improvements on the Occ3D-nuScenes dataset (Tian et al. 2024). with + 14.3% in- crease in mloU, a +41.1% boost in FPS, and 25% reduction in training time.
Table 3: Effects of the proposed key model com-
ponents (i.e., POD, AFS, ASA).
Method
mloU
RayloU
mAP (v)
w/o POD
14.84
12.58
19.21
w/o AFS
15.03
13.56
20.12
w/o SA
11.14
10.44
15.60
w/o ASA Full model
14.85
12.76
19.41
15.15
13.92
21.20
Zero-shot Generalization to Unseen Domains. As shown in Fig. 13 to further validate the gen- eralization capability of our method, we include additional qualitative results on the Lyft Level-5 dataset (Christy et al.] 2019). Importantly, we do not retrain or fine-tune model on the Lyft Level-
8

---

PG-Occ (Ours) Figure 6: Qualitative comparison of 3D occupancy prediction using PG-Occ and prior methods. This figure presents visual comparison between PG-Oce (ours), GaussTR, and Ground Truth data in reconstructing urban scenes. PG-Oee achieves more accurate and perceptually coherent 3D oc- cupancy predictions, capturing finer structural details and producing thicker, more realistic surfaces. The red and blue bounding boxes highlight regions where PG-Oee notably outperforms previous SOTA fixed-query methods, demonstrating improved fidelity and spatial consistency.
GaussTR
Ground Truth
3D Oecupancy Image put
Foant Right
Bath Gaft
Bock
PG-Oce
PG-Oce (w/o AFS)
PG-Oec
PG-Oec (w/o AFS)
Figure 7: Qualitative comparison illustrating the effect of Anisotropy Feature Sampling (AFS). The red boxes in the left-side example show that AFS captures finer, semantically-ave aware details, leading to clearer and more accurate occupancy predictions. The right-side examples further demonstrate that treating anisotropic Gaussian features as identical points (w/o AFS) may result in oversized Gaussians near the camera, which destabilize the overall occupancy estimation. In con- trast, our AFS-enhanced model maintains stable and detailed geometric structures.
5 dataset. Instead, we directly perform inference using the model trained solely on nuScenes. This setting represents challenging domain shift: image resolution, camera positions, camera intrinsies, and the overall environmental domain all differ significantly from those in nuScenes. Despite these differences, our method consistently produces reliable open-vocabulary occupancy predictions, and notably, it is still able to detect and recover small or rare objects in these unseen scenes.
Robustness Evaluation with Different Pretrained Depth Models. To further evaluate the robust- ness of our approach under different depth estimators, we train and test PG-Oce using UniDepth V2 (Piccinelli et al. 2025). Table 13 reports both the depth estimation errors and the open- vocabulary semantic occupancy performance the nuScenes validation set. The results of ad- ditional analyses can be found in appendix B.4
4.3 MAIN ABLATION STUDY
Effeet of Progressive Online Densification (POD), visualized in Fig. the Progressive Online Densification module iteratively expands Gaussian queries during inference, allowing the model to refine complex scene geometries progressively. This adaptive densification mechanism dynamically allocates computational resources to regions with insufficient reconstruction, thereby improving the fidelity of recovered details. Compared to prior methods like GaussTR (Jiang et al.] 2024). which rely fixed number of queries, reconstructs objects with greater precision, as further demonstrated in Fig. 7 The impact of POD quantitatively validated in the ablation study reported in Table 3 where removing POD leads to substantial performance degradation across all key metrics. These results highlight the crucial role of POD in achieving high-accuracy 3D perception by adaptively focusing model capacity on challenging regions of the scene.
approach more effectively captures thick scene surfaces and
Effeet of Anisotropy-awand Feature Sampling (AFS). To evaluate the impact of the AFS design, disable Gaussian anisotropy and treat Gaussians simple point clouds during sampling. As

---

Table 4: Efficiency comparison of methods on mloU, training time, and inference FPS.
Table 5: Ablation study on the number of extended queries in the progressive layer.
Method
mioU
Training
FPS
Queries
mloU
RayloU
mAP
LongOoc
Boeder
2024
248
14.84
500
14.95
13.27
20.42
GausTRR
Dang et al.
2024
13.25
12 hours
1.04
15.15
21.20
PG-Ocz (6000 Queries)
15.15
hours
2.40
2000
14.79
13.21
21.29
Query: "mailbox"
Query: "warning
Figure 8: Additional corner-case examples illustrating the capability PG-Oce for open open-vocabulary occupancy retrieval.
shown in Table 3 this results in a 0.12% drop in mloU. The performance difference stems from the anisotropy strategy, which enables more effective multi-view feature capture, improving both semantic occupancy prediction (mloU, RayloU) and open-vocabulary retrieval (mAP(v)).
Effect of Asymmetric Sell-attention Module (ASA). As shown in Table removing all self- attention ("w/o SA") leads to substantial performance collapse (mloU drops from 15.15 to 1 11. 11.14), indicating that attention-basee interactions are essential for maintaining coherent Gaussian features. Reintroducing standard symmetric self-attention ("w/o ASA") significantly alleviates this issue, yet our asymmetric design still delivers the best results, further improving mloU from 14.85 to 15.15. This improvement suggests that ASA not only enables cross- Goss-Gaus communication but also more effectively preserves historical features while reducing interference from newly added Gaussians during densification, ultimately leading to more stable and accurate feature aggregation.
Effect of the Number of Extended Gaussian Queries. We evaluate the impact of varying the number of extended Gaussian queries, keeping the base queries fixed at 4000. As shown in Table increasing extended queries gradually improves the mloU from 14.84 to 15.15, although slight drop occurs at 2000 queries. We attribute this to the fact that both the mloU and RayloU metrics are evaluated voxel size of 0.4, where further Gaussian refinement provides limited gains for scene representation. However, the mAP(v) metric evaluated on LiDAR data, not constrained by voxel resolution, continues to improve at 2000 queries, achieving increase of 21.29, indicating that larger number of extended Gaussian queries still benefits scene optimization.
Additional Ablation Studies. We provide further analyses in appendix B.I covering model design (base Gaussian queries, densification threshold, and number of sampling points), training supervi- sion (loss terms and photometric cues), and efficiency/robustness (layer-wise time and pose noise).
5 CONCLUSION AND LIMITATIONS
In this paper, we propose progressive Gaussian transformer framework for the open-vocabulary occupancy prediction task. Our method models driving scenes as extendable feature Gaussian blobs purely feed-forward manner, achieving state-of-the-art results with high efficiency. The anisotropy-awan sampling also further improves detail capture. However, due to sparse viewpoints in driving scenarios, constraining the Gaussian scale in depth is challenging, which can cause pop- ping artifacts. Additionally, as Gaussians increase during modeling, memory and computation costs grow, potentially affecting real-time performance. For future work, we will explore 4D Gaussian approaches and multi-view constraints to address these issues.
10

---

Acknowledgements. The research is supported in part by Early Career Scheme of the Research Grants Council (RGC) of the Hong Kong SAR under grant No. 26202321, Department of Science Technology of Shandong Province under grant No. SDST26EG01, SAIL Research Project, HKUST-Zeekr Coolaborative Research Fund, HKUST-WeBank Joint Lab Project, and Tencent Rhino-Bird Focused Research Program.
REFERENCES
Simon Boeder, Fabian Gigengack, and Benjamin Risse. Langoec: Self-supervised open vocabulary occupancy estimation via volume rendering. arXiv, 2024.
Simon Boeder, Fabian Gigengack, and Benjamin Risse. Gaussianflowoce: Sparse and weakly pervised occupancy estimation using gaussian splatting and temporal flow. ICCV, 2025.
Anh-Quan Cao and Raoul de Charette. Monoscene: CVPR, 2021.
Monocular 3d semantic scene completion.
Yang Cao, Yihan Zeng, Hang Xu, and Dan Xu. Coda: Collaborative novel box discovery and
cross-modal alignment for open- vocabulcyyyl 3d object detection. In NewrlPS, 2023.
Yang Cao, Yuanliang Ju, and Dan Xu. 3dgs-det: Empower 3d gaussian splatting with boundary guidance and box-focused sampling for 3d object detection, 2024.
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaus- sian splats from image pairs for scalable generalizable 3d reconstruction. In CVPR, 2024.
Christy, Maggie, NikiNikatos, Phil Culliton, Vinay Shet, and Vladimir Iglovikov. Lyft 3d
object detection for autonomous vehicles. https:/ /aass://kaggle. com/ competitions/ 3d dooject-deecctinn
for autonomous -vehicles 2019. Kaggle.
Y. Eldar, M. Lindenbaum, M. Porat, and Y.Y. Zeevi. The farthest point strategy for progressive image sampling. TIP, 609):1305-, 1997.
Wanshui Gan, Fang Liu, Hongbin Xu, Ningkai Mo, and Naoto Yokoya. Gaussianoce: Fully self- supervised and efficient 3d occupancy estimation with gaussian splatting, arXiv, 2024.
Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel Brostow. Digging into self- supervised monocular depth estimation. In /CCV, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Jiawei Hou, Xiaoyan Li, Wenhao Guan, Gang Zhang, Di Feng, Yuheng Du Xiangyang Xue, and Jian Pu. Fastoce: Accelcrating 3d occupancy prediction by fusing the 2d bird's-eye s-eye view and perspective view. arviv, 2024.
Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. PAMI, 2024.
Yuanhui Huang, Wenzhao Zheng, Yumpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. CVPR, 2023.
Yuanhui Huang, Wenzhao Zheng. Borui Zhang, Jie Zhou, and Jiwen Lu. Selfoce: Self-supervised vision-based 3d occupancy prediction. In CVPR, 2024a.
Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianformer: Scene as gaussians for vision-based semantic occupancy prediction. arXiv, 2024b.
Haoyi Jiang, Liu Liu, Tianheng Cheng, Xinjie Wang, Tianwei Lin, Zhizhong Su, Wenyu Liu, and Xinggang Wang. Gausstr: Foundation model-aligned gaussian transformer for self-supervised 3d spatial understanding. arXiv, 2024.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. TOG, 42(4): 2023.
11

---

Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird's s-eye-view representation from multi callii images via spa- tiotemporal transformers. In ECCV, 2022.
Haisong Liu, Yao Teng, Tao Lu, Haiguang Wang, and Limin Wang. Sparsebev: Hiigh-peffrmance sparse 3d object detection from multi camera videos. In ICCV, 2023.
Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction. In CVPR, 2024.
Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang, Li Liu, and Shanghang Zhang. Renderoce: Vision-centric 3d occupancy prediction with 2d ren- dering supervision. In ICRA, 2024.
Luigi Piccinelli, Christos Sakaridis, Yung- Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, and Lue Van Gool. Unidepthv2: Universal monocular metric depth estimation made simpler. arXiv, 2025.
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for au- tonomous driving: Waymo open dataset. In CVPR, 2020.
Stanislaw Szymanowiez, Chrisitian Rupprecht, and Andrea Vedaldi. Splatter image: Ultra-fast single-view 3d reconstruction. In CVPR, 2024.
Zhiyu Tan, Zichao Dong, Cheng Zhang, Weikun Zhang, Hang Ji, and Hao Li. Ovo: Open-vocabulary occupancy. arXiv, 2023.
Qijian Tian, Xin Tan, Yuan Xie, and Lizhuang Ma. Drivingforward: Feed-forward 3d gaussian splatting for driving scene reconstruction from flexible surround-view input. In AAAI, 2025.
Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang. Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. NewrlPS, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurlPS, 2017.
Antonin Vobecky, Oriane Siméoni, David Hurych, Spyridon Gidaris, Andrei Bursue, Patrick Pérez, and Josef Sivic. Pop-3d: Open-vocabulary 3d occupancy prediction from images. NeurIPS, 2024.
Letian Wang, Seung Wook Kim, Jiawei Yang, Cunjun Yu, Boris Ivanovic, Steven Waslander, Yue Wang, Sanja Fidler, Marco Pavone, and Peter Karkus. Distillnerf: Perceiving 3d scenes from single-glance images by distilling neural fields and foundation model features. NeurIPS, 2024.
Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Ji- wen Lu, and Xingang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. In CVPR, 2023.
Yuxin Wang, Qianyi Wu, and Dan Xu. F3d-gaus: Feed-forward 3d-aware generation on imagenet with aggle-aggee gaussian splatting. arXiv preprint arXiv:2501 067 14, 2025.
Yi Wei, Linging Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundoce: Multi- camera 3d occupancy prediction for autonomous driving. In ICCV, 2023.
Huaiyuan Xu, Junliang Chen, Shiyu Meng. Yi Wang, and Lap-Pui Chau. A survey on occupancy perception for autonomous driving: The information fusion perspective. Information Fusion, 114: 102671, 2025.
Yuanqi Yao, Gang Wu, Kui Jiang, Siao Liu, Jian Kuai, Xianming Liu, and Junjun Jiang. Improving domain generalization in self-supervised monocular depth estimation via stabilized adversarial training. In ECCV, 2024.
Hanrong Ye and Dan Xu. Taskprompter: Spatial-channe multi-task prompting for dense scene understanding. In ICLR, 2022.
12

---

Chubin Zhang, Juncheng Yan, Yi Wei, Jiaxin Li, Li Liu, Yansong Tang, Yueqi Duan, and Jiwen Lu. Occnerf: Self-supervised multi-camera occupancy prediction with neural radiance fields. arXiv, 2023a.
Yanan Zhang, Jinqing Zhang, Zengran Wang, Junhao Xu, and Di Huang. Vision-based 3d occupancy prediction in autonomous driving: a review and outlook. arXiv, 2024.
Yunpeng Zhang, Zheng Zhu, and Dalong Du. Oceformer: Dual-path transformer for vision-based 3d semantic occupancy prediction. In ICCV, 2023b.
Jilai Zheng, Pin Tang, Zhongdao Wang. Guoqing Wang, Xiangxuan Ren, Bailan Feng, and Chao Ma. Veon: Vocabulary-enhanced occupancy prediction. In ECCV, 2024a.
Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, and Yebin Liu. Gps-gaussiang Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis. In CVPR, 2024b.
Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. In ECCV, 2022.
Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. In CVPR, 2024.
13

---

PROGRESSIVE GAUSSIAN TRANSFORMER WITH ANISOTROPY-AWARE SAMPLING FOR OPEN VOCABULARY OCCUPANCY PREDICTION
SUPPLEMENTARY MATERIAL
This supplementary material offers more detailed descriptions to ensure reproducibility. along with extensive evaluations and diverse qualitative results, which collectively highlight the effectiveness, robustness, and efficiency of our proposed method, PG-Oec.
& appendix A Video demonstrations comparing the open-vocabulary occupancy inference results of PG-Occ, the previous state-of-the-art method, and the Ground Truth on the Occ3D-nuScenes validation and test sets.
B Additional experimental results, including extended quantitative comparisons and additional ablation studies.
appendix
appendix More qualitative visualization results of PG-Oec.
appendix Additional details on implementation.
A VIDEO DEMONSTRATION
We provide video demonstrations of our open-vocabulary occupancy inference results on the Occ3D-nuSeenes (Tian al. (2224) validation and test sets. For better visualization quality, please refer to our project page http:aannciiiiadd.iiiibb.ioooe
B ADDITIONAL EXPERIMENT RESULTS
B. ADDITIONAL ABLATION STUDY
Impact of the Number of Base Gaussian Queries. We evaluate the effect of varying the number of base Gaussian queries while keeping the number of ex- tended Gaussian queries fixed at 1000. As shown in Table G increasing the number of base layer queries from 1000 to 4000 consistently improves the mloU from 13.17 to 15.15, indicating enhanced per- ception accuracy. However, further increasing the queries to 8000 results in a slight drop in mloU. This decline is due to the excessive number of Gaussian queries overwhelming the self-attention mechanism, therehy weakening the model's ability capture the critical spatial interactions between queries, similar the observations reported in Jiang et (2024).
Table 6: Ablation study on the number of ini- tial queries in the base layer. The best and the second-best performances of each metric are
highlighted with bold and underlined in the table.
Queries
mloU
RayloU
mAP (v)
1000
13.17
10.33
17.25
2000
14.54
12.84
18.98
4000
15.15
13.92
21.20
8000
14.99
13.52
21.07
Ablation Study of Feed-forward Densification Module Threshold. We investigate the impact of the threshold in the feed-forward densification mod- ule, which determines the minimum distance from points outside to the center of an occupancy cell, on both computational cost and prediction accuracy. Gaussian points are selected according to different thresholds, and the subsequent farthest point sam- pling (FPS) time, mloU are measured. The results are summarized Table 7 As shown in the table, decreasing the threshold selects more points, increasing the subsequent FPS time from 30 ms to 50 ms, which results in higher computational overhead. Meanwhile, a threshold of 0.2 achieves the highest mloU of 15.15, slightly outperform- ing thresholds 0.0 and 0.4 (15.11 and 15.13, respectively). This indicates that the chosen threshold
Table 7: Impact of the feed-for beed-forward densifi- cation on farthest point sampling (FPS) time and occupancy prediction.
Threshold
Total FPS Time (ms)
mloU
0.0
50
15.11
0.2
34
15.15
0.4
30
15.13
14

---

Table 9: Ablation on the loss function L. The best performances of each metric are highlighted with bold the table.
LL1
LssLog
Lromp
Emne
Leos
mloU
RayloU
mAP
13.51
12.30
18.13
15.12
13.81
20.52
14.47
13.01
19.14
15.10
13.89
20.41
14.69
13.23
19.95
15.15
13.92
21.20
of 0.2 effectively balances point selection for Gaussian densification, maintaining high prediction accuracy while controlling computational cost.
Ablation Study of the Number of Sampling Offsets. We evaluate the impact of the number of sampling offsets per Gaussian on occupancy prediction. As shown in Table increasing the number of offsets from to 32 consistently improves performance, with mloU rising from 15.05 to 15.46. This demonstrates that a larger number of sampling points allows the network to capture more fine- grained scene details, enhancing occupancy prediction accuracy. However, the increase in offsets also leads to longer training times, growing from hours for 8 offsets to 11.2 hours for offsets on an 8xA800 GPU setup. These results highlight trade-off between accuracy and computational cost, indicating that 16 offsets provide balanced choice, achieving strong performance with moderate training time.
Ablation Study of Loss. We systematically evaluate the impact of different loss function combinations, including LLI, Log Leemp. Lmse: and Leos. The results are summarized in Table We observe that using all five loss functions consistently yields the best performance, achieving mloU of 15.15%, RayloU of 13.92%, and an mAP of 21 21.20% Omit- ting any individual loss results in slight drop across these metrics, indicating that each component con- tributes to both geometric accuracy and feature alignment. These findings confirm that the combina- tion of complementary losses enables PG-Occ to more effectively capture fine-grained scene details and improve overall 3D occupancy prediction.
Table 8: Impact of the number of sam- pling offsets per Gaussian on performance and training time.
Sampling Points
mloU
Training Time
15.05
hours
16
15.15
hours
32
15.46
11.2 hours
Ablation Study of Photometric Supervision. Our occupancy prediction aims recover both geomet- ric and semantic components. Due to the challenges of large-scale outdoor scenes and limited view su- pervision, photometric information often fails to provide effective geometric supervision. Moreover, color features do not reliably correspond to semantic categories in such scenes, so we exclude them dur- ing training. We performed an ablation study by adding a photometric prediction head regress 3D
Table 10: Ablation study on photometric su- pervision. The best performances of each metric are highlighted with bold.
Color Supervision
mloU
RayloU
w/o color supervision
15.15
13.92
w/ color supervision
14.96
13.89
color values for supervision. The quantitative results are summarized in Table 10
Ablation Study of Pose Noise. We investigate the robustness of PG-Occ to pose noise during temporal- spatial feature fusion by adding Gaussian perturba- tions with different standard deviations to the histor- ical ego poses during inference, summarized in Table iT] The results show that slight amount of pose noise leads to minor improvement mloU, while larger noise levels cause a small decrease followed by stabilization, demonstrating that PG-Oce is robust to pose errors. We attribute this ro- bustness to two factors: first, the nuScenes dataset poses are not perfectly accurate and contain small misalignments, which naturally provide tolerance to minor noise; second, significant pose inaccu- racies result in feature sampling failures on the camera plane, preventing unreliable features from degrading system performance and thereby maintaining relatively high perception accuracy.
Table 11: Impact of Gaussian pose noise on occupancy prediction.
Standard Deviation
0.01
0.1
0.5
mloU
15.15
15.19
15.12
15.12
15

---

B.2 LAYER-WISE TIME CONSUMPTION
The Table reports the inference time of each Gaussian transformer layer milliseconds. The base layer, using the fewest Gaussians, achieves the fastest speed at 27.4 ms. As more Gaussians are
Table 12: Inference time of different layers.
Component
Time (ms)
Base Layer
27.4
added in the First and Second Progressive Layers, the inference time correspondingly increases to 58.3 ms and 60.6 ms, reflecting the higher computational cost of processing denser representations.
First Progressive Layer
58.3
Second Progressive Layer
60.6
B.3 ROBUSTNESS EVALUATION WITH DIFFERENT PRETRAINED DEPTH MODELS
In the main paper, we adopt Metric3D V2 etu al. 2024 as our default depth estimator. To further examine the robustness of PG-Occ under different sources depth supervision, we additionally train and evaluate our model using UniDepth V2 (Piccinelli et al. (2025) pseudo-depth. The Table [3] reports both the depth estimation errors and open-vocabulary semantic occupancy performance on the nuScenes validation set.
Similar to the setting with Metric3D V2 pseudo-depth, using UniDepth V2 also enables PG-Occ to recover depth estimates that outperform the pseudo-labels. For example, while UniDepth V2 provides an Abs Rel of 0.158, PG-Occ improves it to 0.137, showing that PG-Occ can consistently refine imperfect depth supervision regardless of the depth model used.
In terms the open- vece-vocabulary occupancy prediction task, the performance remains stable across depth models. When switching from Metric3I V2 to UniDepth V2 pseudo-depth, the mloU only changes slightly from 15.15 to 15.08, confirming that PG-Oce is largely insensitive to the specific choice of depth estimator.
These findings highlight two key properties of PG-Occ. (i) PG-Occ does not depend on any specific depth architecture; it only requires coarse geometric cues for initialization and supervision, making it naturally compatible with diverse metric depth models. (ii) PG-Occ consistently refines these cues and maintains robust occupancy performance even the upstream depth model changes.
Table 13:
Robustness evaluation across different depth models. The best performances of each metric are highlighted with bold.
Method
Abs Rel
Sq Rel
RMSE
RMSE log
mloU
Metrie3D V2 (Hu et al. 2024
0.170
4.016
6.453
0.291
UniDepth V2 (Piccinelli al. 2025
0.158
2.232
5.491
0.259
PG-Oce (Metri3DD V2)
0.139
1.159
5.466
0,269
15.15
PG-Oce (UniDepth V2)
0.131
1.129
5.049
0.248
15.08
B.4 EFFECTIVENESS IN MULTIMODAL. SETTINGS WITH LIDAR AND CAMERAS
While our approach targets image-based occupancy prediction, which is important direction in the field, it is also effective in multimodal settings combining LiDAR and cameras. To validate this capability, we performed additional experiments on the nuScenes dataset, substituting pseudo- depth inputs with ground-truth -ruth sparse LiDAR point clouds. The results demonstrate that PG-Occ effectively leverages multimodal inputs, maintaining robust semantic occupancy prediction even for challenging scenes.
As summarized in 14 even a direct, naive applacement-withowe any tailored optimiza- tion-significantly boosts 3D occupancy metrics: mloU rises from 15.15 (Depth) to 18.98 (Sparse LiDAR), RayloU from 13.92 to 15.22, and mAP(v) from 21.20 to 29.53. This validates the pipeline's capability to generalize beyond pseudo-depth and effectively absorb multimodal spatial cues. Notably, approach yields robust improvements with
Table 14: 3D occupancy performance with pseudo-depth vs. LiDAR inputs.
Method
mIoU
RayloU
mAP (v)
PG-Oce (Depth)
15.15
13.92
21.20
PG-Occ (LiDAR)
18.98
15.22
29.53
16

---

basic point cloud substitution, indicating that further gains remain achievable via more sophisticated fusion techniques.
C ADDITIONAL VISUALIZATION RESULTS
C.I PG-OcC CAPABILITIES
We present more qualitative results in Fig. demonstrating that, using single-frame multi-view inputs and feed-forward inference, PG-Occ accurately estimates scene depth and generates open- vocabulary feature renderings capturing semanties beyond fixed categories. supports zero-shot semantic 3D occupancy prediction and enables flexible open-vocabulary text queries for object trieval and localization.
C.2 BEV VISUALIZATION
In this subsection, we present BEV (bird's-eye view) occupancy visualizations produced by PG-Oce. This perspective provides a comprehensive overview of the scene layout, allowing clear observation of spatial relationships among various objects. As illustrated in Fig. TO our method accurately reconstructs both large and small scene elements, including vehicles, pedestrians, and barriers, while maintaining sharp and consistent occupancy boundaries. We select variety of diverse scenes to demonstrate the robustness and generalization capability of our approach, highlighting its ability to handle complex environments effectively.
C.3 EGO-CENTRIC PERSPECTIVE OCCUPANCY VISUALIZATION WITH PREVIOUS SOTA METHOD
In this subsection, we visualize occupancy from the vehicle's perspective and compare our results with the previous state-of-the-ar method, GaussTR (Jiang et al.) 2024).. This comparison aims to highlight the strengths and improvements of our approach in estimating occupancy within the scene. As illustrated in Fig. II method demonstrates superior detection results for small objects compared to GaussTR (Jiang et al. 2024), particularly for car, bicycle, bus, truck, and barrier. Interestingly, our approach capable of detecting elements that are not well annotated in the Ground Truth, such as the pedestrians and bicycles shown in the second visualization of the figure.
C.4 THIRD PERSPECTIVE OCCUPANCY VISUALIZATION
In this subsection, we present the visualization of our method from two different third sard-perroon per- spectives. As illustrated in Fig. 12 we compare the zero-shot semantic occupancy estimations gen- erated by our approach with the Ground Truth. The visualizations illustrate the effectiveness of our method in accurately capturing the spatial occupancy of various objects within the scene. The re- sults underscore our model's s ability to perform zero-shot semantic occupancy estimation, enabling it to infer the occupancy of objects it has not encountered during training. However, it is important to note that due to occlusion issues present in the scene, our self-supervised method may face chal- lenges in making accurate predictions in areas lacking visual observations. Nevertheless, it can still yield reasonable inferences to a certain extent.
D ADDITIONAL IMPLEMENTATION DETAILS
D.1 VOXELIZATION
As described in section 3.4, after obtaining the progressive 3D Gaussian representation that models the scene based on the current camera input. We convert the 3D feature Gaussian blobs output to a semantic occupancy field. To begin with, we take n arbitrary text prompts Ctext and encode them using the CLIP text encoder to obtain their corresponding feature embeddings fient- And then compute the similarity between these text embeddings and the text-aligned features f, of Gaussian subsequently. The text probability for each 3D feature Gaussian blob G under Geet be calculated
17

---

Table 15: Text prompts used for zero-shot semantic oceupancy estimation on the Oce3nD-nuSeenes dataset
(Tian et al. 2024). indicates that no prompts were made for this class.
nuScenes Class
Prompts
others
barrier
barrier
bicycle
bicycle
bus
hus
car
car
construction vehicle motorcycle
construction vehicle motorcycle
pedestrian
person
traffic cone
cone
trailer
trailer
truck
truck
driv. surface
road
other flat
sidewalk
sidewalk
terrain
terrain, grass building, wall, fence, pole, sign vegetation
manmade
vegetation
empty
sky
as follow equation:
P: = off sten)
(14)
where p; represents the text probability the i-th Gaussian blob under Cexts and a denotes the softmax operation.
After that, we define a voxel grid within the region of interest (ROI) occupancy range and then calculate the influence of each Gaussian each voxel, accumulating the results. This process affected by the anisotropy parameter 8, r of the Gaussians, their opacity o, and assigned text probability p. The formulation for this voxelization be written as:
N
N
V. =
Σ
St,
Σ
exp(
z (x m1) E.) - -ma))os
(15)
E1
1=1
N
N Σ expl - (i) 1. (x--)))p:
V3 = Σ 1=1
(16)
where Vo. Vp denote the final occupancy probability and semantic 3D occupancy field, denotes the voxel grid position of occupancy. Σ the Gaussian covariance matrix of each Gaussian, revived from its scale 8; and rotation quaternion
In the evaluation of the nuScenes retrieval dataset experiment in Section since the ground truth consists of text annotations for sparse LiDAR points P, we treat each LiDAR point p as the center of voxel. This allows to obtain the corresponding final occupancy probability and text feature, as shown in the following formula.
P. = Σ G.lp
Σ i=1
exp( (p ((P--)) -,) 1p -u))on,
(17)
N
P = Σ G. = Σ
exp( マ (p - 14) " -)))f
(18)
where P.. Py denote the final occupancy probability and the corresponding text feature of the Li- DAR point cloud P.
18

---

D.2 TEXT PROMPT
Due to the imprecise semantics in the Occ3D-nuScenes (Tian et al. 2024) dataset, we made some minor adjustments to the prompts used in PG-Occ, shown in Table [5 detect the categories 'others' or 'other flat,' as they can lead to ambiguities. Note that further fine- tuning of these anbiguous prompts could enhance performance.
Specifically, do
For the retrieval task in Section 4.2 we directly use the prompt provided by the dataset.
D.3 ADDITIONAL MODEL AND TRAINING DETAILS
D.3.1 SUPERVISION STRATEGY
Metric3D V2 (Hu et al. 2024) and MaskCLIP (Zhou et al. 2022) are utilized for depth and feature supervision. The loss weight parameters are set follows: AşrLog 0.15, Atemp 10, and Amee 10. The learning rate is initialized at 2c-4 with a weight decay of 0.01, using the AdamW optimizer.
D.3.2 MODEL ARCHITECTURE AND TRAINING SETUP
We adopt ResNet-50 (He al. 2016) the image feature backbone, utilizing the previous seven frames to capture spatio-temport information. PG-Occ is initialized with 4,000 Gaussian queries in the base layer and progressively adds 1,000 queries per layer, resulting in one base and two progressive layers with an embedding dimension of 256. All training experiments are conducted on A800 GPUs for 8 epochs, while inference is performed on single A800 GPU. To improve computational efficiency, we use a resolution of 180 x 320 for depth and feature rasterization, as well as for Gaussian point initialization.
19

---

Ieput Image
Frand
Rrael
Back
Hender Fouture
Ocoupancy
Semantic
Prompt: Road Sign
Frompt
Figure 9: PG-Oce capabilities. Given only single-frame frgme-frame multi-view inputs and using only feed- forward passes, PG-Oce can: (1) estimate depth (row 2); (2) render open-vocabulan model features (row 3); (3) predict 3D occupancy in zero-shot manner (rows 4); (4) predict semantic 3D oceu- pancy in a zero-shot manner (rows 5) (5) support open-vocabulary text queries (rows 6).
FrEE
Back
Bruck
Blckk Rinhs
Ground Truth
PG-Oce (Ours)
Sceac
Tlack Tufft
Black
Ground Truth
PG-Oce (Ours)
Secoo-k273 9373
Ground Truth
PG-Oce (Ours)
Figure 10: BEV visualization of open- vocabulary scene perception by PG-Oce. The figure illustrates predicted occupancy and semantic structures from bird's-eye s-eye perspective, emphasizing the model's ability to capture spatial relationships and overall scene layout.
20

---

aisso
TAANE COFFEE
Pedestrian Human
Car
Gauss TR
Ground Truth
PG-Oce (Ours)
Pedestrian
Sidewalk Bicycle
Human
Gauss'TR
Ground Truth
PG-Oce (Ours)
Bock
Manmade Traffic light
Pedestrian
Human
GaussTR
Ground Truth
PG-Oce (Ours)
barrier
bicycle
bus
car
cons. veh. sidewalk
motorcycle
pedestrian
traffic cone vegetation
trailer
truck
drive. surf.
terrain
manmade
Figure 11: Qualitative comparisons of zero-shot semantic occupancy estimation from ego-centric multi-camera perspective. Each row shows input images from multiple viewpoints (top), corre- sponding occupancy predictions by GaussTR (left bottom), the ground truth occupancy (middle bottom), and our PG-Occ method (right bottom). Dashed boxes and lines highlight specific ob- jects such pedestrians, cars, bicycles, and traffic lights-thhr that have been successfully detected and reconstructed. Our approach demonstrates superior detection and reconstruction of small or distant objects, better preserves spatial relationships, and provides more accurate object shapes compared with GaussTR. Colors indicate semantic categories as defined in the legend. For best inspection of fine details, recommend viewing the color version and zooming in.
21

---

Scene 0770
Seene 0565
Right
Wark Lait
Back Right
Back Lefi
Back
Ground Truth (View 1)
PG-Oee (View
Ground Truth (View 1)
PG-Oee (View
Ground Truth (View 2)
PG-Oee (View
Ground Truth (View
PG-Oce (View
Scene 0345
Scene 0344
Right Front
Front Hight
Ground Truth (View 1)
PG-Oce (View 1)
Ground Truth (View 1)
PG-Oce (View 1)
Ground Truth (View 2)
PG-Oce (View 2)
Ground Truth (View 2)
PG-Oce (View 2)
barrier bicycle
bus truck
veh. sidewalk
motorcycle termin
pedestrian manmade
traffic vegetation
trailer
drive. surf.
Figure 12: Qualitative zero-shot semantic occupancy results on the third perspective for two views. For each view (View 1 and View 2), show the predictions of method (PG-Occ) alongside the Ground Truth. The results demonstrate that PG-Occ accurately captures semantic occupancy patterns across different perspectives.
22

---

Occupancy Prediction
light
Occupancy Prediction
Unotkel
Raj
Occupancy Prediction
Occupancy Prediction
Occupancy Prediction
Figure 13: Zero-shot generalization on the Lyft Level-5 dataset (Christy et al. 2019). Our model is not retrained or fine-tuned on the Lyft Level-5 dataset but used directly after training on nuScenes. This scenario involves a substantial domain shift, including differences in image resolution, camera intrinsics, viewpoints, and overall scene distribution. Despite these challenges, method main- tains strong zero-shot generalization, accurately predicting occupancy and successfully recovering small rarely seen objects in completely unseen scenes.
23


Reviews:
[{"review_id": "UsQ1gKPWJ1", "reviewer": "Reviewer_BKpf", "summary": "This paper introduces PG-Occ, a framework for open-vocabulary 3D occupancy prediction in autonomous driving. The central challenge it addresses is the trade-off between sparse Gaussian representations, which are efficient but miss fine-grained details, and dense representations, which incur high computational costs. It resolves this with two key contributions: \n1. Progressive Online Densification (POD): A feed-forward strategy that iteratively refines the 3D Gaussian scene representation. It starts with a coarse model and progressively adds detail to regions with higher perception errors, efficiently capturing fine-grained objects without modeling the entire scene densely.\n2. Anisotropy-aware Sampling (AFS): A sampling method that adaptively assigns receptive fields to Gaussians based on their specific scale and rotation (anisotropy). This allows for more effective spatio-temporal feature aggregation.\n\nThe model is trained using only 2D supervision (pseudo-depth maps and text-aligned features) without requiring 3D LiDAR data. Experiments show PG-Occ achieves state-of-the-art performance, demonstrating a significant 14.3% relative mIoU improvement over the previous best method on the Occ3D-nuScenes dataset.", "questions": "1. Table 4 reports the final method's speed as 2.40 FPS. However, Table 12 reports the inference time of the final progressive layer as only 60.6 ms, and the sum of all three layers as 146.3 ms (27.4 + 58.3 + 60.6). This sum suggests a throughput of ~6.8 FPS. What component is responsible for the major bottleneck that reduces the final speed to 2.40 FPS? Is it the ResNet-50 spatio-temporal backbone, the final Gaussian-to-voxel post-processing, or another part not listed in Table 12?\n2. You acknowledge a key limitation: \"constraining the Gaussian scale in depth is challenging, which can cause popping artifacts\". Could you elaborate on the severity and frequency of these artifacts? For example, do they primarily affect distant/occluded objects, or is it a general instability? How much do you believe this instability impacts the method's reliability for downstream tasks like motion planning, which require temporally stable geometric representations?\n3. The model is initialized and supervised by pseudo-depth. Have you investigated the model's robustness under different depth models?\n4. The ablation study shows that removing ASA (\"w/o ASA\") degrades performance. Did you also experiment with using standard, symmetric self-attention instead of just removing it?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. PG-Occ introduces an efficient, online method that adaptively adds Gaussians to \"under-represented regions\" identified by depth errors. This allows the model to start coarse and progressively capture fine-grained details.\n2. The paper originally identifies a weakness in prior methods that treat Gaussians as simple points, ignoring their shape. The AFS module is a novel solution that samples features based on the Gaussian's specific scale and rotation. This allows for adaptive receptive fields and more effective spatio-temporal feature aggregation.\n3. ASA ensures training stability by allowing new, under-optimized Gaussians to learn from established ones, but not vice versa.\n4. The method achieves SOTA results on the challenging Occ3D-nuScenes dataset, with a 15.15 mIoU score.", "weaknesses": "1. The method is trained using only 2D supervision from sparse-view cameras. This setup creates inherent ambiguity. A small, nearby object can project to a 2D feature patch similar to a large, distant object. While multi-view consistency and pseudo-depth supervision  help, they don't fully resolve this.\n2. The model is initialized using pseudo-depth maps from Metric3D V2 and also supervised using them. The authors should present experimental results using other depth prediction methods to demonstrate the robustness of the proposed method.\n3. The number of total queries should be provided in Table 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YpjovZXddx", "reviewer": "Reviewer_pxcU", "summary": "The paper proposes PG-Occ, a progressive Gaussian Transformer framework for open-vocabulary 3D occupancy prediction. The method couples (i) Progressive densification of Gaussians through an iterative feed-forward densification strategy, (ii) Anisotropy-aware feature sampling that selects sample points and projects them onto feature planes with varying receptive fields. Extensive experimental results demonstrate that PG-Occ achieves SOTA performance on Occ3D-nuScenes.", "questions": "1. Could you show more corner-case examples, such as uncommon categories?\n2. Could you provide a deeper intuition or theoretical analysis of the motivation to justify the necessity of the method?\n2. Is it also effective in a multimodal setting with LiDAR and cameras? If experiments are not feasible due to dataset constraints, an analysis of the underlying rationale would suffice.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The strengths of this paper lie in its clear motivation and cohesive design: it addresses the trade-off in Gaussian representations via progressive densification, stabilizes the training dynamics through asymmetric attention, and improves feature alignment via anisotropy that matches each Gaussian. Empirically, main results, ablations, and efficiency comparisons reinforce one another, showing that under fixed or limited compute, the approach delivers a better balance of performance and speed than previous baselines.", "weaknesses": "However, I also have some concerns:\n(1) The robustness of corner cases is underexplored: In autonomous driving, identifying corner cases is critical. Common classes like cars, trucks, and pedestrians are already well recognized by supervised learning, whereas less common categories, such as plastic bags or trash bins, are much harder to detect. Can PG-Occ recognize such corner cases (not limited to these two examples)?\n(2) From the paper, the motivation seems somewhat trivial, more like a data augmentation extension of GaussTR, so it should explicitly articulate the deeper thought behind this motivation.\n(3) The method shows strong performance in a camera-based pipeline. Does the method work in a multimodal setting?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SyRdAlkbTP", "reviewer": "Reviewer_TDbw", "summary": "The paper introduces PG-Occ, a progressive Gaussian transformer for open-vocabulary 3D occupancy. It starts from a coarse set of 3D Gaussians and, in a feed-forward way, adds new Gaussians only where depth reveals under-modeled regions, so the scene becomes denser exactly where it needs more detail. It further makes each Gaussian anisotropy-aware and keeps the model stable with asymmetric self-attention so newly added Gaussians do not disturb existing ones.", "questions": "- i) The current densification is driven by a depth discrepancy threshold. Have you tried combining this with a text-feature uncertainty signal, so that regions that are geometrically covered but semantically ambiguous can also trigger new Gaussians?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- i) The paper targets a clear gap in current Gaussian-based open-vocabulary occupancy. The proposed progressive, depth-guided densification is a straightforward and well-motivated way to increase capacity only where the scene is under-modeled.\n\n- ii) The anisotropy-aware feature sampling makes good use of the Gaussian’s scale and orientation, which is sensible for driving scenes where many structures are not isotropic. \n\n- iii) The paper is well organized and clearly written.", "weaknesses": "- i) The whole progressive pipeline relies on the quality of the pseudo depth.  A short discussion on robustness to worse depth (or to LiDAR-sparse depth) would make the claim stronger.  \n\n- ii) Because the Gaussian set can only grow, not shrink, the last layers still have to process the largest token set. This is fine at Occ3D-nuScenes resolution, but may need pruning for HD-maps or city-scale scenes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "40RrFhpLYB", "reviewer": "Reviewer_qE6H", "summary": "This paper presents PG-Occ, a Progressive Gaussian Transformer for open-vocabulary 3D occupancy prediction. The method progressively densifies 3D Gaussian representations in a feed-forward manner to capture fine scene details while maintaining efficiency. An additional anisotropy-aware sampling strategy adaptively adjusts receptive fields across scales and time for better spatio-temporal feature aggregation. Integrating language-aligned features enables text-driven 3D reasoning. Experiments show that PG-Occ achieves improvement over previous state-of-the-art methods, delivering more detailed and scalable scene understanding.", "questions": "1. Could the authors provide a detailed analysis or profiling of memory consumption across different densification stages?\n2. Have the authors tested the method on additional datasets or unseen domains to further validate its generalization capability?\n3. While AFS shows limited quantitative improvement in the ablation, could the authors provide qualitative examples or visualizations that better illustrate its specific benefits or effects?\n4. Could the authors include comparisons with more recent methods (2024-2025)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a novel Progressive Gaussian Transformer that progressively refines 3D Gaussian representations through a feed-forward densification process, balancing detail capture and computational efficiency.\n2. The proposed anisotropy-aware sampling adaptively adjusts receptive fields across directions and scales, enabling more accurate feature aggregation and fine-grained geometric modeling.", "weaknesses": "1. Although progressive densification effectively enhances scene details, the increasing number of Gaussians at higher stages inevitably raises inference time and memory consumption. While the paper acknowledges this issue and plans to optimize it in future work, no quantitative memory analysis or profiling results are provided, leaving the practical cost-performance trade-off unclear.\n2. Experiments are conducted on two benchmark datasets, which is relatively limited. The evidence for cross-domain robustness and generalization remains insufficient.\n3. One of the main contributions, the anisotropy-aware sampling, provides quantitative gains in the ablation study, but the improvement is relatively small.\n4. The comparisons mainly cover works published up to 2024. Recent approaches from the past year in similar or related directions are missing from the comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)