S2GO: Streaming Sparse Gaussian Occupancy
Authors: Jinhyung Park
Chensheng Peng
yihan hu
Wenzhao Zheng
Kris Kitani
Wei Zhan
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Gaussian Splatting
3D Occupancy Estimation
Autonomous Driving
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Despite the efficiency and performance of sparse query-based representations for detection, state-of-the-art 3D occupancy estimation methods still rely on voxel-based or dense Gaussian-based 3D representations. However, dense representations are slow, and they lack flexibility in capturing the temporal dynamics of driving scenes. Distinct from prior work, we instead summarize the scene into a compact set of 3D queries which are propagated through time in an online, streaming fashion. These queries are then decoded into semantic Gaussians at each timestep. We couple our framework with a denoising rendering objective to guide the queries and their constituent Gaussians in effectively capturing scene geometry. Due to its efficient, query-based representation, S2GO achieves state-of-the-art performance on the nuScenes and KITTI occupancy benchmarks, outperforming prior art (e.g., GaussianWorld) by 2.7 IoU with 4.5x faster inference.

Published as a conference paper at ICLR 2026
S2GO: STREAMING SPARSE GAUSSIAN OCCUPANCY
Jinhyung Park
Chensheng Pengl.a 1.3
Yihan Hu! Wenzhao Zheng*
Kris Kitani² Wei 1.3
Applied Intuition
CCarnegie Mellon University
3 University of California, Berkeley
ABSTRACT
Despite the efficiency and performance of sparse query- based representations for detection, state-of-the-art 3D occupancy estimation methods still rely on voxel- based or dense Gaussian- -aaed 3D representations. However, dense representa- tions are slow, and they lack flexibility in capturing the temporal dynamics of driving scenes. Distinct from prior work, we instead summarize the scene into compact set of 3D queries which are propagated through time in an online, streaming fashion. These queries are then decoded into semantic Gaussians at each timestep. We couple framework with denoising rendering objective guide the queries and their constituent Gaussians in effectively capturing scene geometry. Due its efficient, query-based representation, S2GO achieves state- of of-the-art art performance on the nuScenes and KITTI occupancy benchmarks, out- performing prior art (e.g., Gaussian World) by 2.7 IoU with 4.5x faster inference.
INTRODUCTION
Vision-centric autonomous systems provide a more cost-effective and scalable alternative to LiDAR-
based solutions (Wayvel 2024 Mobileye 2024 Tesla 2022 Zhang al. 2024), but they struggle with the absence of dense 3D geometry priors, an obstacle to achieving beyond Level 3 autonomy. To address this gap. 3D occupancy semantic estimation has emerged as a powerful complement to conventional sparse 3D perception tasks like bounding box detection (Philion & Fidler 2020 Yin
al.
2021 Liu et al. 2022 Wang et al. 2023 Li et al. 2022b a) or vectorized mapping (Li al, Liao al. 2022a |
2021
Chen et al. 2024 Yuan al. 2024). because it captures a richer and more comprehensive view of unknown and arbitrarily shaped objects, thereby improving safety.
Recent 3D occupancy methods often rely on regular grids (Cao & de Charette [022 Li et al. 2022b
Zhang et al. 2023; Huang et al. 2023; Liu et al., (2024) or dense Gaussians (Huang et al. 2024] Zuo al. 2025b Zheng et al.) (202b)) Although these methods capture high-fidelity details, they
slow and inflexible when integrating long-term historical context, limiting both static infrastructure localization well dynamic actor modeling. Existing grid-based approaches reduce redundancy by warping or projecting features from previous frames (Li et al.) 2022b [Huang & Huang. 2021 Li
al. | [023b Liu et al. 20233 (024)) but suffer from unnecessary computation unoccupied regions and artifacts introduced by dense grids. Meanwhile, recent Gaussian based techniques (Huang et al., 2024 2025 Zheng et al., 2024b; Zuo al., 2025b) show promise by focusing computation on occupied regions. However, they rely on tens of thousands of Gaussians (25.6k ~ 144k) and use local sparse convolutions because global modeling becomes computationally prohibitive.
To address the inefficiencies of voxel-based and dense Gaussian based methods in streaming per- ception, we propose to use sparse 3D queries to summarize and propagate the dense 3D world over time. More specifically, our method (S2GO) maintains a queue of past sparse 3D queries, refines the current set of queries using both previous queries and current image observations, and then pre- dicts 3D occupancy by decoding the current queries into a denser set of semantic Gaussians. This online framework enables efficient propagation and global feature interaction among a sparser set of 3D queries (~1k) while retaining the high fidelity of Gaussian- based representations.
Query- based perception has proven effective for sparse object detection (Carion et al,, 2020 Wang al. 2020 2023), but employing sparse queries for dense, high-fidelity occupancy estimation presents several challenges. First, object detectors typically employ hundreds of queries, which far
Work done during internship at Applied Intuition 1 "Corrsponndenc: wei.zhan @applied.co

---

Published as a conference paper at ICLR 2026
outnumber the target objects (approximately 30 per scene), allowing for direct Hungarian Matching. In contrast, 3D occupancy estimation must cover the entire scene, making the mapping from sparse queries to dense semantic Gaussians inherently ambiguous. Second, in voxel-aligned occupancy estimation, fixed voxels simply perform classification at their predetermined locations. By compari- son, query-based approaches require that queries first move to regions of interest before classifying- This creates a chicken-and -aiicenn-andegg problem: for instance, if query lies between a car and the road, it is unclear whether it should shift toward the car or the road, as the correct target location depends on the query's s intended class. Third, while dense Gaussian methods mitigate this ambiguity through extensive spatial coverage, increasing the sparsity of the representation for efficiency exacerbates the difficulty of aligning queries accurately with occupied regions.
To fully unlock the streaming potential of query-based occupancy estimation, we introduce pre- training phase that first trains the network to capture 3D scene geometry. During pretraining, query locations are initialized noised LiDAR points, and the network is trained to recover 3D geometry through a denoising objective. To capture fine-grained local shape, decoded Gaussians are rendered from the current and neighboring views and supervised accordingly. The network also predicts a velocity for each query to model dynamic objects. This pretraining addresses the aforementioned challenges of using sparse queries by 1) supervising queries and their decoded Gaussians to model local scene structure, 2) training queries to self-organize to evenly cover the scene, and 3) supervis- ing queries explicitly to move from empty space to occupied regions. Then, during the following semantic occupancy estimation stage when LiDAR no longer used and queries are randomly ini- tialized throughout the 3D scene - the network uses its pretrained knowledge to preciscly reposition the queries and decode to Gaussians to capture dense 3D structure.
Our contributions are summarized as follows.
We present S2GO, a streaming framework for semantic occupancy where persistent, sparse queries carry long-horizon context and are decoded into dense occupancy via Gaussians.
To address the challenge of making dense estimations from a sparse representation, we introduce a novel geometry denoising pretraining phase. This enables sparse queries to traverse empty space and self-organize onto occupied regions, yielding dense 3D structure.
We further address key limitations in the oft-used Gaussian formulation and voxel splatting algorithm and halve training time with better performance.
We evaluate S2GO on nuScenes and KITTI and achieve state-of-the--ne performance and inference speed. Notably, our lightweight model improves over prior art (e.g. Gaussian- World) with 5.9 x faster inference, achieving real-time inference on a single 4090 (26 FPS).
2 RELATED WORK
3D Occupancy Estimation is increasingly crucial for vision-centric systems due to limited geomet- ric priors inherent in purely vision-based methods. This task provides dense, volumetric represen- tations of the environment, significantly enhancing semantic understanding and improving safety in decision-making, effectively complementing LiDAR. Recent camera-based benchmarks (Wei et al., 2023 Tian al., 2023 Liao al. 2022b), featuring detailed annotations created through ofiboard techniques, have driven substantial progress in vision-based occupancy modeling research.
Building on these benchmarks, existing methods (Cao & de Charette, 2022: Li al. 2023b Huang ctt al. 2023: Zhang et al., [2023] Ye et al.) 2024 Zheng et al. [022a Tong el al. 2023) typically employ dense BEV or voxel-based representations, but such structures hinder real-time process-
ing efficiency and scalability. Sparse voxel approaches (Liu et al.) 2024 Li et al., 2023a Wang
al. (024b) enhance efficiency but encounter challenges such as complex temporal modeling and
increased overhead in temporal integration due to their grid-based nature. One line of work He al.) (2025): : Kim et al.) (2025) leverage separate processing of voxels and BEV grids to capture both detail and context, and ProtoOcc Oh al.] (2025) lifts 2D prototypes to 3D and optimizes via contrastive loss. Recently, StreamOcc Moon et al. (025)) leverage both efficient 3D voxels and explicit instance queries with bi-directional aggregation. While achieving strong performance, the voxel-query interaction remains computationally expensive.
Recently, Gaussian-basse based representations (Kerbl et al. 2023 Peng et al. 2024 Yang et al. 2024 Xu et al. (024) have emerged due to their strong 3D and semantic representational capabilities. Methods such Huang al. (2024): Zheng et al. (2024b): Zuo al. (2025b) exploit probabilistic
2

---

Published as a conference paper at ICLR 2026
semantic Gaussians for 3D occupancy modeling but require large numbers of Gaussians, posing challenges for real-time performance and efficient temporal fusion. Also related is OSP (Shi et al, 2024), which represents the scene set of points. While flexible, sparse points cover narrower region of the scene compared to Gaussians, and OSP requires grid-aligned point sampling to make voxel-aligned predictions. The recent work QuadricFormer (Zuo et al. 2025a) makes efficient use of quadrics, but it still requires up to 12.8k independent primitives for competitive performance.
Query-based Representations. Since DETR (Carion et al. 2020), query-based methods have rapidly advanced, demonstrating effectiveness in tasks like detection, mapping, and tracking. DETR3D (Luo et al., 2022) efficiently extends 2D queries into 3D for detection, while Stream- (Wang et al. 2023) fuses temporal information
streaming fashion. MapTR (Liao et al.) 2024) leverages structured Transformers for HD map generation, and MapTTracker (Chen 2024; Yuan et al. 2024) reframes the mapping task with object tracking. Sparse4D (Lin et al.) integrates detection and tracking into unified, end-to-end framework. However, object- centric query methods remain underutilized for dense reconstruction tasks like occupancy estima- tion. We bridge this gap by introducing Gaussian queries, establishing streamlined, query- massd framework for efficient 3D semantic occupancy estimation.
PETR
2022a
ct al.
2023
S2GO
3.1 PRELIMINARY: GAUSSIAN OCCUPANCY ESTIMATION
GaussianFormer (Huang et al. 2024) and follow-up work (Huang al., 2025 Zheng al.) 2024bb
Zuo et al., (025b) propose to represent the driving scene set of 3D Gaussian primitives G [G.]-= with each semantic Gaussian G; specified by its position x; E TR&, rotation r; E RM,, scale 8, E opacity a, IR and class distribution C; RC where C is the number of foreground classes. Given this set, GaussianFormer-22 (Huang et al. 2025) predicts the semantic occupancy of a voxel coordinate x R&3 by first predicting binary occupancy probability and then expressing the class distribution of occupied regions as a mixture of nearby Gaussians. More specifically, the occupancy probability (x) E IR is modeled as the probability that x occupied by at least one of P nearby Gaussians:
P
a(x) = II i=1
(1 G();
where (x;; G+) is the probability that æ is occupied by G₁:
axx; G) = exp(
2 (x - m) N x (x m))
(2)
Σ = RSS RD S = diag(s), R = q2r(r)
(3)
Further, the foreground class distribution e(x: E RC is expressed as mixture of Gaussians weighted by opacity a:
e(x:9)= = Σ x) = E G(x
(4)
Σ₁ p(x
Gi) (2
2 | exp( (x m) x (x-
(5)
Finally, the joint semantic occupancy distribution over foreground classes and the empty back- ground written as (a(x) e(x; axx) e RC+1), We refer the reader to prior work (Huang
et al. 2024 2025) for additional details.
3.2 ARCHITECTURE
Our framework shown in Figure is inspired by streaming query-based object detection methods (Carion et al. 2020; Wang et al. 2020 2023 Lin al., 20222 Yuan et al.] 2024). We keep a queue of past sparse 3D queries, update the current queries based on historical queries and current images, and predict a detailed set of 3D Gaussians. The temporal transformer design follows PETR (Liu
et al. 2022 and StreamPETR (Wang al. 2023
More specifically, at each timestep t. we represent the scene with a set of sparse 3D queries Q₂ = {q() with associated 3D locations {pijk queries are refined using a queue of past queries Q, and the 2D features F, = CNN(I.) from the RGB images of that timestep J E RR N
R3, where K is the number of queries. These
where N is the number of cameras.
3

---

Published as a conference paper at ICLR 2026
Past 3D Queries
Stage Denoising & Rendering Pretralning
Current Images
Ege-Transfomm
Image Encoder
Query Denoising
Depth Map Rendering
Temporal Transtormer
Stage 2: 3D Occupancy Estimation
Queries
Splatting
Future SD Queries
Semantic Gausslans
Semantic Occupancy
Figure 1: Overall framework of S2GO for streaming perception. At each timestep, our method refines new 3D queries using current image observations and a queue of past queries. These queries are decoded into a set of fine-gran fine-grained Gaussians, and a portion of the queries are propagated to future timesteps in streaming fashion. In Stage 1, this query refinement and Gaussian estimation pipeline is pretrained to effectively model the 3D scene using query denoising and rendering pretraining. In Stage 2, the predicted Gaussians are splatted to voxels for training 3D occupancy estimation.
Each query predicts position offset o', opacity a', and velocity vt, alongside attributes for set of finer Gaussians. Relaxing the timestep t subscript on Gaussians for clarity, the derived Gaussians written as:
G: = (((p' + +o ooj, v sis,,, a² -
a)))
where is the number of Gaussians per query. Each Gaussian has a 3D position p¹ + of + of combining the query position, query offset, and its own offset of. velocity v inherited from its parent query, rotation r/, scale s), and opacity a' af where the query opacity modulates the -aussian-specific opacity a}. This hierarchical decomposition allows each query to anchor spatial region while the finer Gaussians capture local structure within that region.
Our framework for efficiently extracting 3D Gaussians from image observations is consistent across both the denoising pretraining and occupancy estimation tasks. The primary distinction lies in the additional attributes predicted by each Gaussian: during pretraining, each Gaussian independently predicts its own color, whereas, during occupancy estimation. Gaussians derived from the same query collectively share a semantic class label. This shared semantic class ensures consistency among Gaussians originating from single query.
3.3 STAGE 3D GEOMETRY DENOISING
3.3. MOTIVATION
While S2GO can directly be trained for occupancy estimation, the resulting performance is subopti- mal. The queries and their Gaussians are unable to move effectively to occupied locations to capture fine details they instead coarsely model nearby regions shown in Figure This stems from the weak and ambiguous supervision that queries and Gaussians receive from occupancy labels.
This limitation arises from two interconnected factors: First, unlike in GaussianFormer where each Gaussian is refined individually, in our sparse query-based framework, each query moves J Gaus- sians as a group before individual Gaussians locally branch out. As any perturbations to query location propagate to its constituent Gaussians, aligning the query precisely with scene geometry before predicting Gaussian offsets is critical. However, 3D occupancy estimation lacks a clear as- signment between parts of the scene and individual queries with multiple nearby scene elements, the lack of clear-cut supervision causes query refinements to be noisy. Second, this ambiguity is exacerbated by the inherent locality of the Gaussian-to-vooxt splatting operation in Section 3.1 As

---

Published as a conference paper at ICLR 2026
LIDAR (only for visualization)
Query Offsets (No Pretraining)
Gauss. Centers (No Pretraining)
Occ. Pred. (No Pretraining)
3D Occupancy
Query Offsets (Pretrained)
Gauss, Centers (Pretrained)
Occ. Pred. (Pretrained)
Figure 2: Impact of denoising pretraining on occupancy estimation. We visualize query offsets (column 2). Gaussian centers (column 3) colored by opacity, and occupancy estimation (column 4) for S2GO without and with pretraining. Without pre-training, queries remain largely stagnant, and Gaussians fail to capture 3D structures. In contrast, pretraining with rendering and denoising allows queries to move towards occupied cadions-particulaan visible for walls and cars-while Gaussians self-organize to better represent the scene, significantly improving occupancy estimation.
Gaussians are each locally pulled to different scene elements - resulting in suboptimal local min- ima (Kerbl al. 2023) Charatan et 2024) their corresponding queries are similarly stuck in suboptimal locations, unable to properly cover the scene.
3.3.2 DENOISING AND RENDERING FRAMEWORK
To explicitly supervise query movement and train Gaussians to model 3D geometry around their queries, we introduce denoising and rendering framework for pretraining S2GO. The model func- tions as described Section 3.2 but in this stage, we initialize current query locations p at noised LiDAR points at that timestep. Relaxing the subscript, given 3D points pts RMX3 we set
(p') FPSx (pts) +
where is the # of LiDAR points, ~ e(-e,e) e FPS applies Furthest Seeett-Poott---pppppee (FPS) to yield K points, and U(-e, E) is the uniform distribution with e as a hyperparameter. Starting at these noised positions, the model predicts query offsets {o and derived Gaussians G,-
3.3.3 TRAINING OBJECTIVES
We then supervise these outputs with the loss function:
L C=
||PSx (pts:) (pi + oi)ll + Xa + Xa 1)
The first term is the denoising objective, training the network to self-organize the queries to cover 3D structure. Then, Ldepth and Lrgb render depth maps and RGB images from the Gaussians and supervise them with LiDAR projected depth maps De and image observations. This explicitly trains Gaussians represent detailed scene structure around the aligned queries. Notably, the rendering supervision is done on current and neighboring keyframes (+/- 0.5s) by moving the Gaussians with predicted velocities v and accounting for ego-motion. This further improves final 3D occupancy performance. Altogether, this denoising and rendering stage provides S2GO with strong prior for sparse queries and Gaussians to effectively model the 3D scene geometry.
3.4
STAGE 2: 3D SEMANTIC OCCUPANCY ESTIMATION
3.4.1
OCCUPANCY ESTIMATION FRAMEWORK
Equipped with the pretraining prior, S2GO is then trained for 3D semantic occupancy estimation. The model processes image observations, predicts set of Gaussians G. each timestep, which now
5

---

Published as a conference paper at ICLR 2026
also include semantic class predictions, and "splats" Gaussians to nearby voxels as in Section 3.1 Notably, unlike the pre-training phase, query positions are initialized at learnable 3D locations. As such, S2GO only uses RGB images during inference. The "splatted" voxel predictions using ground truth semantic occupancy, and we additionally supervise neighboring frames similar to Stage 1. In this section, we propose crucial improvements to further strengthen this pipeline.
trained
3.4.2 OPACITY- SISTry-EEIGIIIE WEIGITED GEOMETRY ESTIMATION
The Gaussian-tor vo-xelll splatting framework in GaussianFormer-2 handles foreground classes as a mixture of Gaussians, and opacity is only used for weighting Gaussians inside the mixture. As such, opacity has no bearing on determining binary occupancy of a location, in contrast to Gaussians in rendering (Kerbl al., 2023) where opacity acts proxy for density. This leads unexpected behavior: Gaussians in unoccupied regions decrease their scale s and position themselves between voxel centers to minimize their occupancy contribution (Eq. 2 all while maintaining significant opacity. This unnatural representation for Gaussians conflicts with the rendering initialization and hurts performance. To address this issue, we additionally weight the oceupancy probability a(x; G) with the opacity estimation yielding:
| (x m) T - m)) 2
axx; G) = a exp( -
Our change significantly improves performance by encouraging Gaussians in unoccupied regions to simply predict lower opacity and stabilizing scale supervision to be more consistent.
3.4.3 EFFICIENT GAUSSIAN-TO- VOXEL SPLATTING
To implement Gaussian-to-voxel splatting, GaussianFormer (Huang et al. 2024) first determines pairs of interacting Gaussians and voxels, then parallelizes over voxels in the forward pass and over Gaussians in the backward pass. However, this formulation does not account for the inherent locality of the splatting operation: neighboring voxels process a similar set of Gaussians and vice- versa. Such voxels and Gaussians should be processed together in a CUDA block for optimized L1 cache usage. This is especially problem for the backward pass since naively parallelizing over Gaussians incurs random access costs on large number of voxels (640k).
To address this problem in the forward pass, we block voxels into 4x4x4 grids and have threads tied to each voxel collaboratively load nearby Gaussians onto memory before splatting them, similar to 3DGS (Kerbl et al.] 2023). In the backward pass, adopt similar approach but addition- ally take care to tie threads to individual Gaussians to avoid atomic operations on the gradients (Mallick al., 2024). Our efficient Gaussian-to-vee and 640k voxels on an A100, speeds up the forward pass by 1.5x (1.29ms to 0.87ms) and the backward pass by 20.4x (116ms to 5.7ms), substantially reducing the wall-clock time required for training. Furthermore, it decreases the GPU memory cost of the forward/backwardt pass by 3.3x from 2013MBB/2079MB MB1633B33
splatting implementation, with 9k Gaussians
3.4.4 QUERY PROPAGATION
A key point in our streaming 3D occupancy pipeline is query propagation. More specifically, we need determine the optimal subset of current queries push onto the queue for future timesteps. While a straightforward selection of top-k largest query opacities works, propagating the most confi- dently occupied regions of the scene, we find that queries end up highly overlapping over time, with insufficient coverage over the entire scene. To mitigate this, we choose the highest opacity queries that are pairwise separated by a distance ö, where is hyperparameter. This maintains an effective balance between maintaining high-opacity regions and distributing queries across the scene.
4 EXPERIMENTS
We perform extensive experiments three benchmarks derived from the nuScenes and KITTI datasets. S2GO uses the ResNet50 (He et al. 2016) backbone, S2GO-Small uses 900 queries with 10 Gaussians each, and S2GO-Base uses 1800 queries with 20 Gaussians. Additional details about the experiment setup are in Supplementary A and more implementation details are in Supple- mentary B
4.1 QUANTITATIVE RESULTS
We first evaluate S2GO on the nuScenes dataset. On the muSceney-SSrroundO.e benchmark in Ta- ble I S2GO-Small improves over the previous Gaussian-based ofth--ff-the-at method Gaussian-

---

Published as a conference paper at ICLR 2026
Table 1: 3D occupancy estimation results the nušeenes-Surrounddocc validation set (Wei et al. All baselines leverage 900 1600 resolution while S2GO uses 256 x 704 resolution images for efficiency. All methods are benchmarked on the 4090 GPU. *Gaussian paper results over-weight intermediate frames during evaluation. Wc re-evaluate relcased checkpoints under the standard setting.
2023).
burrier
bicycle
COOM. voh.
motorsylle
paletrian
uallic coDe
traler
drive. suf.
other flat
sidewalk
tarrain
vegoration
track
bus
CUP
Method
Ioll
mlol
FPS
MonoScene (Cao & de Charette 2022
24.0
7.3
4.0
0.4
8.0
8.0
2,9
0.3
1.2
0.7
4.0
44
27.7
5.2
15.1
11.3
9.0
14.9
Atlas Marez
2020
28.7
15.0
10.6
5.7
19.7
24,9
6.5
3.3
10.4
162
34,9
15.5
21,9
21.0
20.5
BEVForee
40226
30.5
16.8
14.2
6.6
23.5
28.3
8.7
10.8
6.6
4.1
112
17.8
37.3
18.0
22.9
22.2
13.
22.2
3.3
TPVFormer
Jaang al M024
30.9
17.1
16.0
5.3
23,9
27.3
9.8
8.7
7.1
5.2
11.
192
38
21.3
24.3
23.2
11.7
20.8
2.9
OocFormer
Thane al:
31.4
19.0
18.7
10.4
23
30.3
10.3
14.2
13.6
10.1
12.5
20.8
38.8
19.8
24.2
22.2
13.5
21.4
SurrondDee
e al 2015
31.5
20.3
20.6
11.7
28.1
30,9
10.7
15.1
14.1
12
14.
22.3
37
23.7
24.5
22.8
14.9
21,9
3.3
GaussiunFummer
Huang
2024
29.8
19.1
19.5
26
29.8
10.5
13.
12.6
8.7
12.7
39.6
23.3
24.5
23.0
19.1
2.7
Gamssinn
HHanng ct ml 2025
31.7
20.8
214
13.4
28,5
30.8
10.9
15.8
136
10.5
14.0
22.9
40.6
24
26
24
13.
22.0
2.8
QuadricFommer
A0250
20.1
27.3
9.2
12.5
24.3
6.2
GaussanWarld*
Zuo et al 20256
32.8
21.8
21.6
13.3
27.3
31,2
13.9
16.9
13.3
11.8
14.8
23.7
41.9
24
28
26.3
15.7
24,5
4.4
ALOce-mini-GF Chic
23.1
27.9
32.7
15.3
14.5
29.0
26.6
5.4
ALOcc-GE GF (Chen el ar 20250 0
38.2
25.5
24.3
18.8
29.8
34.3
17.9
19.6
17.5
17.2
15.5
26.5
47.6
29.9
31.2
29.2
20.0
29.0
0.9
SmallSmall
34.3
22.1
20.8
13.1
27.5
30.3
14.5
16.5
11.7
10.9
13.5
23.3
46.3
29.2
29.7
28.4
13.0
25.
26.1
S2GO-Base
35,5
22.7
21.9
13.4
27.5
32.1
14,9
15.3
12.9
11.8 13.4
24.0
46.9
29.1
30.3
29.1
14.7
26.4
19.6
Table 2: Results on the SSCBench-KITTI.
test set Geiger et al. 2012) with a monocular camera.
S2GO achieves new state-of-the-ar achieving strong performance in all categories.
motoreycle
other-vch.
other gmd
vegetation
other struct.
other objecc
bicyole
person
sidewalk road
parking
building
fenoe
terrain
pole
rig.-
Input
Method
IoU mIoU
L.MSNNet Roldao et al. 2020
1. 47.5
13.7
20.9
0.0
0.0
0.3
0.0
0.0
63.0
13.5
33.5
0.2
43,7
0.3
40.0
26.8
0.0
0.0
3,6
0.0
SSCNet (Song cl al. 2017
1. 53.6
17.0
32.0
0.0
0.2
10.3
0.6
0.1
65.7
17.3
41.2
3.2
44.4
6.8
43.7
28.9
0.8
0.8
8.6
0.7
MonoScene
Charette 2022
C
137.9
19.3
0.6
2.0
11.4
3.2
32,9
3.1
Voxformer
el al 20ZM0
C
38.8
11.9
17.8
1.2
0.9
4.6
2.1
1.6
47.0
9.7
27.2
2.9
31.2
5.0
29.0
14.7
6.5
6.9
3.8
2.4
TPVForer
Hoang el al. 2023
C
40.2
13.6
21.6
1.1
14
8.
2.6
2.4
53.0
12.0
31.
3.8
34,8
4.8
20.1
17.5
7.5
5.5
2.7
OceFomner Zhang
C
40.3
13.8
22.6
0.3
3.8
54.
13.4
31.5
36
36.4
31.0
19.5
7.8
7.0
4.6
GaussianFormer
(Huang 2l a.
2024
C
35.4
12.9
18.9
1.0
4.6
18.1
7.6
3.4
45.5
10.9
25.0
5.3
28.4
5.7
29.5
8.6
3.0
2.3
9.5
5.1
Gaussian Fommen
[Tuang cl
38.4
21.1
32,0
3.5
S2GO-Base (ours)
C 40.8
15.1
22.7
1.7
15.9
5.1
2.1
53.8
13.3
33.4
3.8
35.3
7.2
31.2 21,1
6.4
6.0 4.2
World (Zoo et al. 2025b) by 1.5 IoU while substantially improving inference speed by 5.9x. The larger $2GO-BBase further improves IoU by 2.7 and retains 4.5x speed advantage, demonstrating the efficacy of our sparse, query-based framework. We note that S2GO achieves these performance improvements while using a smaller image size (900x 1600 256x704) Additional experiments with equivalent image size are in Appendix H
Compared with the state-of-the-art grid-aligned method Chen et al. (0025)) S2GO-Base achieves comparable results to ALOcc- Gini-GF while being 3.6x faster in FPS. While ALOcc-GF achieves the highest mIoU on SurroundOce, its throughput is 0.9 FPS, placing outside the real-time regime. In contrast, our S2GO-Small and S2GO-Base variants run at 19.0 and 23.6 FPS, respec- tively, and are specifically designed for high-throughput deployment, offering more favorable ac- curacy-efficiency trade-off in this real-time regime. We also provide results on nuScenes-Occ3D in Supplementary]] J
where we achieve competitive results with high FPS.
We also evaluate our approach on the KITTI-360 dataset (Geiger et al. 2012) in Table 2 In this monocular 3D semantic occupancy estimation setting, S2GO again achieves state-of-the-art perfor- mance, substantially improving over GaussianFormer-2 (Huang al. 2025).
4.2 QUALITATIVE ANALYSIS
In Fig. 3. we qualitatively compare Gaussian and our method, visualizing different timesteps from two example driving sequences in nuScenes. After several seconds, after both the ego vehicle and surrounding vehicles move, Gaussian World struggles to maintain independent representations of distinct objects and incorrectly merges multiple instances. This limitation occurs because Gaus- sian World, despite its streaming nature, directly operates on low-level Gaussian primitives without object-level representations. Consequently, its local convolutions merge nearby objects. In contrast,
7

---

Published as a conference paper at ICLR 2026
t= 0s
[ 2.5s
t t= Os
t t= 5.0s
GaussianWorld
GausslanWorld
S2GO (Ours)
S2GO (Ours)
Ground Truth
Ground Truth
Figure 3: Qualitative comparison of occupancy estimation. We compare S2GO with Gaussian- World (Zuo et al., 2025b) by visualizing different timesteps from two example driving sequences. Gaussian World incorrectly merges different instances over time, while S2GO effectively preserves distinct object identities by operating at a higher semantic level with sparse queries.
S2GO hierarchically decomposes the scene into sparse set of queries with constituent Gaussians, enabling to operate higher semantic level and effectively preserve distinct object identities. We provide qualitative results on -CCBenc--KITTI-660 in Supplementary
4.3 ABLATIONS
In this section, we verify the effectiveness of our proposed components. By default, models are trained for 12 epochs during both stages. All ablations
the SurroundOcc-uuSeen dataset.
Pretraining. Table 3 we ablate the impact of pretraining. First, directly training occupancy estimation for 12 epochs (a) or 24 epochs (a)+ yields poor results due to ambiguous supervision.
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation pretraining. We ablate pretraining query ini- tialization and the depth, RGB, and query denoising loss terms. Li- DAR + indicates queries are initialized from noised LiDAR. Pre- training with all objectives is essential. TTrained for 24 epochs.
Table 4: Ablation query propaga- tion strategies. "None" indicates no temporal information is used.
Propagation Type
mloU
loU
Query Init
Depth
RGB
Denoise
mloU
loU
None
17.92
29.24
top-k opacity
19.94
32.03
13.02
25.73
8-dist top-k opacity
20.51
32.51
(a)
X
x
15.83
28.35
Table 5: Ablation on using velocity
(b)
Learnable
12.42
26.64
modeling in each stage.
(c)
LiDAR
7
/
13.62
27.08
Pretrain.
Occ. Est.
mloU
loU
(d)
LiDAR+E
V
20.55
32.68
x
20.07
31.87
(e)
LiDAR+
x
20.25
32.44
x
20.15
31.94
(d)
LiDAR+c
20.55
32.68
/
20.50
32.62
(1)
LiDAR+
21.60
33.91
20.55
32.68
t Os
t = 1s
t 2s
Figure 4: Visualization of future occupancy predictions. We use the self-supervised velocity pre- diction for each query roll out future occupancy predictions. Our streaming query-based frame- work well-decouples motion of individual objects.
Next, pretrain with depth and RGB supervision and ablate query initialization. Learnable ini- tialization during pretraining - which what S2GO uses in the second stage - is worse than not pretraining. This occurs because such queries randomly distributed over 3D space, with most queries far from occupied geometry and hence unable to get adequate supervision. On the other hand, initializing query locations precisely at LiDAR points is only slightly better than pretrain- ing this baseline supervises Gaussians to capture local geometry, but the queries themselves are not supervised to move. Finally, adding noise LiDAR before initializing achieves remarkable performance, providing meaningful supervision to both queries and Gaussians. We emphasize that this is the only initialization method that substantially improves over not pretraining with the same compute budget (24 epochs of occupancy in (a)t 12+ 12 epochs with pretraining).
Next, we ablate each pretraining loss function. Depth supervision alone enough to achieve good performance. Adding RGB loss slightly boosts results as RGB supervises finer details, and denoising supervision gives substantial final boost. This confirms that our proposed pretraining essential for our streaming, sparse-query framework to reach its potential and achieve state-of-the-art.
Query Propagation. Query selection for future frames is critical for streaming perception. In Table we ablate propagation strategies. Compared to the single-frame baseline without propagation, selecting top-k queries by opacity already provides a substantial performance gain. However, this leads to excessive overlap between queries over time, wasting capacity in the model. Enforcing minimum distance between queries enables wider query coverage, further improving performance. Supplementary [ further examines S2GO performance with varying history propagation horizon.
Velocity Modeling, S2GO predicts a velocity for each query, which is used in both stages to move dynamic regions before applying RGBD or occupancy supervision in neighboring frames. While this module is useful on its own for future occupancy prediction as shown in Fig. 4 we ablate its Velocity modeling improves performance in both stages, with
impact on performance in Table 5 motion modeling during pretraining proving particularly important.
Gaussian to Voxel Splatting. In Table 6 we ablate our inclusion of opacity in occupancy probability and efficient Gaussian-to- -o-oelll------lll--ooooool splatting algorithm. First, including opacity substantially improves performance

---

Published as a conference paper at ICLR 2026
Table 6: Ablation Gaussian-to- Voxel Splatting (G2V). GPU Training hours calculated for training 12
epochs on single GPU. benchmarks are calculated on a single 4090 GPU.
Opacity in
Efficient G2V
mioU
loU
Train GPU hours
Infer GPU Mem.
Infer. FPS
x
X
16.97
28.75
55h
2436 MB
25.2
X
20.13
32.28
129h
7043 MB
20.4
20.55
32.68
28h
2448 MB
25.3
(+3.16 mloU), but doubles the training time as Gaussians opt to reduce occupancy probability by lowering opacity instead of scale, thus increasing the number of voxels each Gaussian affects. We resolve this by using optimized CUDA kernels, which further improves performance while substantially lowering training cost, halving the original training time. Furthermore, our optimized kernels also substantially reduce the required GPU memory single-hatch inference from 7099 MB to 2511 MB, and also improve inference throughput from 24.1 frames/s to 31.5 frames/s.
Number of Gaussians. In Table 7 we ablate the # of queries and Gaussians. We observe that even just 900 sparse queries and 10 Gaussians per query is enable capture the overall scene and achieve high mloU with a real-time 26.1 FPS on 4090. With more queries Gaussians, performance steadily improves, the cost. longer runtime.
Pretraining with Various Sources of Depth. While our use of LiDAR dur- ing pretraining does not add data collec- tion overhead, since LiDAR already re- quired to generate occupancy labels, we also investigate pretraining on diffcrent sources of depth in Table / In Li-
Table 7: Ablation on the number of queries and Gaussians. FPS is measured on 4090 GPU.
# Query
# Gauss /Query
# Gauss.
mloU
IoU
FPS
900
10
9000
21.60
33,91
26.1
1260
14
17640
21.78
34.15
22.7
1800
20
36000
21.84
34.51
19.6
DAR is not available during training, placing LiDAR with occupied voxel lo- cations yields almost identical results. Replacing the nuScenes' line LiDAR with a cheaper 16-line sensor largely pre- serves performance, DAR entirely with zero-shot monocular depth predictions from Metric3D on RGB images also achieves good results. This demonstrates the general applicability of our pretraining pipeline.
Table 8: Ablation on pretraining with various depth sources.
Pretraining Query Initialization
mloU
loU
replacing
Occupied voxel locations LiDAR (32-line)
21.61
33.75
21.60
33.91
LiDAR (16-line)
21.16
33.39
Zero-shot RGB depth pred. (Yin et al. 2023
20.99
33.57
CONCLUSION
We presented novel framework for 3D semantic occupancy estimation that leverages sparse 3D queries efficiently capture and propagate scene information over time. Our method replaces traditional dense, grid- aligned Gaussian representations with a compact, streaming set of semantic queries. A geometry denoising pre-training phase ensures effective alignment of sparse queries with dense occupancy targets, accurately mod- nuScenes and KITTI benchmarks
eling both static and dynamic scene elements. Extensive evaluations demonstrate state-of-the-art performance while operating 5.9x faster than previous methods. Our work demon- strates a query-based approach can effectively bridge the gap between efficiency and high-fidelity scene representation. In the future, we plan to explore multitask, end-to-end learning and large-scale pretraining using unlabeled data to further enhance model performance and generalization.
Reproducibility Statement. To aid reproducibility. we have provided details throughout the paper regarding S2GO framework as well as efficient Gaussian-to-voot splatting algorithm. Section 3.2 provides gen- eral details regarding the architecture, and Supplementary [ provides low-level details including image size, learning rate, channel dimension, and other hyperparameters. Dataset and evaluation details are in Supplemen- tary A and details regarding the splatting algorithm are in Section 3,4.3
REFERENCES
Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan. Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pp. 11621-11631,
2020.
10

---

Published as a conference paper at ICLR 2026
Anh-Quan Cao and Raoul de Charette. Monoscene: Monocular 3d semantic scene completion. In CVPR, pp- 3991-4001, 2022.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve. Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference computer vision, pp. 213-229. Springer, 2020.
David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19457-19467, 2024.
Dubing Chen, Jin Fang, Wencheng Han, Xinjing Cheng, Junbo Yin, Chengzhong Xu, Fahad Shahbaz Khan, and Jianbing Shen. Aloce: Adaptive lifting-based 3d semantic occupancy and cost volume-base flow predictions. In Proceedings of the IEEF/CVF International Conference Computer Vision, pp. 4156-4166, 2025a.
Dubing Chen, Huan Zheng. Jin Fang, Xingping Dong, Xianfei Li, Wenlong Liao, Tao He, Pai Peng, and Jianbing Shen. Rethinking temporal fusion with a unified gradient descent view for 3d semantic occupancy prediction. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1505-1515, 2025h.
Jiacheng Chen, Yuefan Wu, Jiaqi Tan, Hang Ma, and Yasutaka Furukawa. Maptracker: Tracking with strided memory fusion for consistent vector mapping. European Conference on Computer Vision, pp. 90-107. Springer, 2024.
Tri Dao, Daniel Y Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. arXiv preprint arXiv:2205.14135, 2022.
Wanshui Gan, Ningkai Mo, Hongbin Xu, and Naoto Yokoya. A comprehensive framework for 3d occupancy estimation autonomous driving. IEEE Transactions Intelligent Vehicles, 2024.
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEFE conference on computer vision and pattern recognition, pp. 3354-3361. TEEF, 2012.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Yulin Hc, Wei Chen, Siqi Wang. Tianci Xun, and Yusong Tan. Achieving speed-accuracy balance in vision- based 3d occupancy prediction via semantic--emannic disentanglement. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 39, pp. 3455-3463, 2025.
Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues in multi-camera 3d object detection. arXiv preprint arXiv:/2203. 170544 2021.
Yuanhui Huang, Wenzhao Zheng, Yumpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision- based semantic occupancy prediction. CVPR, pp. 9223-9232, 2023.
Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Gaussianforner: Scene as gaus- sians for vision-based 3d semantic occupancy prediction. In European Conference on Computer Vision, pp. 376-393. Springer, 2024.
Yuanhui Huang, Amonnut Thammatadatrakoon, Wenzhao Zheng, Yunpeng Zhang. Dalong Du, and Jiwen Lu. Probabilistic gaussian superposition for efficient 3d occupancy prediction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis, 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139 2023.
Jungho Kim, Changwon Kang, Dongyoung Lee, Schwan Choi, and Jun Won Choi. Protooce: Accurate efficient 3d occupancy prediction using dual branch encoder- prototype query decoder. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 4284-4292. 2025.
Qi L.,, Yue Wang, Yilun Wang, and Hang Zhao. Hdmapnet: A local semantic map learning and evaluation framework. arXiv preprint 200.006000000000 2021.
Yiming Li. Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anandkumar. Voxformer: Sparse voxel transformer for camera based 3d semantic scene completion. In Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pp. 9087-9098, 2023a.
11

---

Published as a conference paper at ICLR 2026
Yiming Li, Sihang Li, Xinhao Liu, Moonjun Gong. Kenan Li, Nuo Chen, Zijun Wang. Zhiheng Li, Tao Jiang, Fisher Yu, Sscbench: large-scale 3d semantic scene completion benchmark for autonomous driving. In 2024 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 13333-13340) IEEE, 2024.
Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang. Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detection, 2022a.
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Qiao, and Jifeng Dai, Bev- former: Learning bird's s-cye-vviww representation from multi-camera images via spatiotemporal transformers. In European conference computer vision, pp. 1-18. Springer. 2022b.
Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang. Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-oce: 3d occupancy prediction based on backard view transformation. arXiv preprint 0rXiv:2307.014999 2023b.
Bencheng Liao, Shaoyu Chen. Xinggang Wang, Tianheng Cheng, Qian Zhang. Wenyu Liu, and Chang Huang. Maptr: Structured modeling and learning for online vectorized hd map construction. arXiv preprint arXiv:2208. 0122078.4437, /4437, 2022a.
Bencheng Liao, Shaoyu Chen, Yunchi Zhang, Bo Jiang, Qian Zhang, Wenyu Liu, Chang Huang, and Xinggang Wang. Maptrv2: An end-to-end framework for online vectorized hd map construction. International Journal of Computer Vision, pp. 1-23, 2024.
Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: novel dataset and benchmarks for urban scene under- standing in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):322-33110, 2022h.
Zhimin Liao, Ping Wei, Shuaijia Chen, Haoxuan Wang, and Ziyang Ren. Steoee: cascade renovation for 3d occupancy and scene flow prediction. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 1516-1526, 2025.
Sparse spatial-temporal
Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detection with sparse spatial-temporal fusion, 2022.
Xuewu Lin. Zixiang Pei, Tianwei Lin, Lichao Huang, and Zhizhong Su. Sparse4d v3: Advancing end-to-end 3d detection and tracking. arXiv preprint arXiv:2311.1122. 2023.
Haisong Liu, Yao Teng, Tao Lu, Haiguang Wang, and Limin Wang. Sparsebev: High-performance sparse 3d object detection from multi-camera vidcos. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18580-18590, 2023.
Haisong Liu, Yang Chen, Haiguang Wang, Zetong Yang. Tianyu Li, Jia Zeng, Chen, Hongyang and Limin Wang. Fully sparse 3d occupancy prediction. In European Conference on Computer Vision, pp- 54-71. Springer, 2024.
Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi- view 3d object detection. Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXVII. pp. 531-548. Springer. 2022.
Zhipeng Luo, Changqing Zhou, Gongjie Zhang, and Shijian Lu. Detr4d: Direct multi-view 3d object detection with sparse attention. arXiv preprint arXhv:2212.0784 2022.
Saswat Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Markus Steinberger. Francisco Vicente Carrasco, and Fernando De La Torre. Taming 3dgs: High-quality radiance fields with limited resources. SIGGRAPH Asia 2024 Conference Pupers, pp. 1-11, 2024.
Mobileye. Mobileye under the hood. https: wwwww/wwwwwww///////////////// mob wew.bbileee.... com/ ces ces-2024/ 2024.
Scokha Moon, Janghyun Back, Giscop Kim, Jinkyu Kim, and Sunwook Choi. Mitigating trade-off: Stream and query-guided aggregation for efficient and effective occupancy prediction. arXiv preprint arXiv:2503.2208) 2025.
Zak Murez, Tarrnce Van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End-to-end 3d scene reconstruction from posed images. In European conference on computer vision, pp. 414 414-431. Springer, 2020.
12

---

Published as a conference paper at ICLR 2026
Gyeongrok Oh, Sungjune Kim, Heeju Ko, Hyung gun Chi. Jinkyu Kim, Dongwook Lee, Dachyun Ji, Sungjoon Choi, Sujin Jang, and Sangpil Kim. 3d occupancy prediction with low-resolution queries via prototype- aware view transformation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp.
17144,17144, 2025.
Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xie, Bing Wang. Li Liu, and Shanghang Zhang. Renderoce: Vision-centric 3d occupancy prediction with 2d rendering supervision. In 2024 IEEE International Conference Robotics and Automation (ICRA), pp. 12404-12411. IEEE, 2024.
Chensheng Peng, Chengwei Zhang. Yixiao Wang, Chenfeng Xu, Yichen Xie, Wenzhao Zheng, Kurt Keutzer, Masayoshi Tomizuka, and Wei Zhan. Desire-gs: 4d street gaussians for static-dynamic decomposition and surface reconstruction for urban driving scenes. arXiv preprint 1xXi::4411.11922 2024.
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In European Conference on Computer Vision, pp. 194-210. Springer. 2020.
Luis Roldao, Raoul De Charette, and Anne Verroust-Blondet. Lmsenet: Lightweight multiscale 3d semantic completion. In 2020 International Conference on 3D Vision (3DV), pp. 111-119, IEEE, 2020.
Yiang Shi, Tianheng Cheng, Qian Zhang, Wenyu Liu, and Xinggang Wang. Occupancy as set of points, 2024.
Shuran Song, Fisher Yu, Andy Zeng, Angel X Chang, Manolis Savva, and Thomas Funkhouser. Semantic scene completion from single depth image. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1754,1154 2017.
Tesla. Tesla AT Day. https: httpp:///www.youuub.... wctmm//atccuo-0SSsssaa.a
2022.
Xiaoyu Tian, Tao Jiang, Longfei Yum, Yucheng Mao, Huitong Yang. Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Neural Information Processing Systems, 36:64318 18-63338,,,3,, 2023.
Wenwen Tong, Chonghao Sima, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, al. Scene as occupancy. In ICCV, pp. 8406 8415, 2023.
Jiabao Wang, Zhaojiang Liu, Qiang Meng, Liujiang Yan, Ke Wang, Jie Yang, Wei Liu, Qibin Hou, and Ming- Ming Cheng. Opus: occupancy prediction using a sparse set. Advances in Neural Information Processing Systems, 37:119861-119588 2024a.
Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection. In ICCV, pp. 3621-3631, 2023.
Yue Wang. Vitor Guizilini. Tianyuan Zhang. Yilun Wang, Hang Zhao, and Justin Solomon. Detr3d: 3d object detection from multi view images via 3d-to-2d queries. arXiv preprint arXiv:21 2020.
Yuqi Wang. Yuntao Chen. Xingyu Liao, Lue Fan, Zhaoxiang Zhang. Panooce: Unified occupancy rep- resentation for camera-based 3d panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17158-17168, 2024b.
Wayve End-to-End Autonomy:
A New
Era of Self-Driving.
http://wayve.a11 ai/
cvpr- w-eead-tutorinn
2024.
Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen La. Surroundoce: Multi-camera 3d occupancy prediction for autonomous driving. ICCV, pp. 21729-21740, 2023.
Shaoqing Xu, Fang Li, Shengyin Jiang, Ziying Song, Liu, and Zhi-xin Yang. Gaussianpretrain: A simple unified 3d gaussian representation for visual pre-training in autonomous driving. arXiv preprint arXiv:2411.12452 2024.
Honghui Yang. Sha Zhang, Di Huang, Xiaoyang Wu, Haoyi Zhu, Tong He. Shixiang Tang, Hengshuang Zhao, Qibo Qiu. Binbin Lin. et al. Unipad: universal pre- train-iiiing paradigm for autonomous driving. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15238-15250, 2024.
Zhangchen Ye, Tao Jiang, Chenfeng Xu, Yiming Li, and Hang Zhao Cyt-occ: Cost volume temporal fusion for 3d occupancy prediction. In European Conference Computer Vision, pp. 381-397. Springer, 2024.
Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-hased 3d object detection and tracking. In Pro- ceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 11784-11793, 2021.
13

---

Published as a conference paper at ICLR 2026
Wei Yin, Chi Zhang. Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from single image. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9009-9000 2023.
Tianyuan Yuan, Yicheng Liu, Wang. Yilun Wang. and Hang Zhao. Streammapnet: Streaming mapping network vectorized online hd map construction. Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 7356-7365, 2024.
Yanan Zhang, Jinqing Zhang. Zengran Wang, Junhao Xu, and Di Huang. Vision based 3d occupancy prediction in autonomous driving: a review and outlook. arXiv preprint Sikiv:2400.0259955 2024.
Yunpeng Zhang, Zheng Zhu, and Dalong Du. Oceformer; Dual-path transformer for vision-based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 9433-9443 2023.
Wenzhao Zheng, Weiliang Chen, Yuanhui Huang, Borui Zhang. Yueqi Duan, and Jiwen Lu. Occworld: Learn- ing 3d occupancy world model for autonomous driving. In European conference computer vision, pp- 55-72. Springer, 2024a.
Wenzhao Zheng, Junjie Wu, Yao Zheng, Sicheng Zuo, Zixun Xie, Longchao Yang. Yong Pan, Zhihui Hao, Peng Jia, Xianpeng Lang, al. Gaussianad: Gaussian-centric end-to-end autonomous driving. arXiv preprint arXiv:2412. 10371, 2024b.
Xizhou Zhu, Weijic Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end end-to-end object detection. arXiv preprint arXiv:2010.04159 2020.
Sicheng Zuo, Wenzhao Zheng, Xiaoyong Han, Longchao Yang, Yong Pan, and Jiwen Lu. Quadricformer:
Scene as superquadrics Systems, 2025a.
semantic occupancy prediction. Advances in Neural Information Processing
Sicheng Zuo, Wenzhao Zheng. Yuanhui Huang. Jie Zhou, and Jiwen Lu. Gaussianworld: Gaussian world model for streaming 3d occupancy prediction. 2025 IEEE/CVF Conference on Computer Vision and Pottern Recognition (CVPR), pp. 6772-6781, 2025b.
14

---

Published as a conference paper at ICLR 2026
A ADDITIONAL DETAILS ON THE EXPERIMENT SETUP
Datasets. We conducted comprehensive experiments on three henchmarks derived from nuScenes and KITTI. The nuScenes dataset (Caesar et al. 2020) provides 1000 scenes of surround-view driving scenes, We evaluate method on both the Surround One (Wei al. 2023) and Oce3D (Tian et al. 2023) benchmarks. Sur- roundOce provides voxel-based annotations in
100 x m² range around the car with a 200 x 200 16 resolution, classifying voxels into 18 classes (16 semantic, I empty, and noise). Occ3D offers voxelized semantic occupancy in a 80 80 x 6.4 m² range with a 200 x 200 x 16 resolution, derived from an auto-labeling pipeline. The K/TTI dataset [Geiger et al. 2012) comprises over 320k images and 80k laser scans from subur- ban driving scenes. We adopt the dense semantic annotations from SSCBench- 3SCBench-KIITT1-36l KITTI-360 (Li at 2024 Liao ict al. (2022b). The official split consists of 7/1/1 sequences for training, validation, and testing, respectively. The voxel grid spans an area of 51.2 x 51.2 x 6,4 m² in front of the ego car, with a resolution of 256 x 256 x 32. Each voxel is classified into one of 19 classes (18 semantic categories and I empty).
Evaluation Metries. Following MonoScene (Cao & de Charette 2022), use IoU and mloU evaluation metrics. For the Oce3D dataset, we adopt RayloU as our primary metric following SparscOce (Liu et al. 2024). RayloU extends mloU by evaluating occupancy estimation at the ray level rather than voxel level. simulates LiDAR rays and assesses predictions based on both depth accuracy and class correctness. RayloU ensures balanced evaluation by resampling rays across distances and incorporating temporal casting from past, present, future viewpoints assess scene completion. By preventing inflated IoU scores caused by thick surface predictions and applying a depth threshold for true positive classification, RayloU provides a more robust evaluation. Metrics are defined as:
TP.
mloU/RaylaU
(10)
C
TR + FPP
TP TP4c + P + FN
IoU
(11)
where TP.. FP., and FN, are the number of true positive, false positive, and false negative predictions for class i, C' is the set of semantic classes, and co is the nonempty class. For RayloU, query ray is classified as true positive (TP) if the predicted class matches the ground truth and the LI error between the predicted and ground-truth depth is within certain threshold (Im, 2m, 4m)
Baselines We evaluate S2GO against representative approaches spanning diverse 3D representation paradigms. Specifically, we compare with voxel-based methods. including MonoScene (Cao de Charette 2022). At- las (Murez et al. 2020), SurroundOce (Wei et al. 2023). which employ dense 3D voxel grids for occu- pancy reconstruction. We further benchmark against BEV-based methods like BEVFormer (Li et al. 2022b).
In addition, we consider the triplane-based TPVFormer (Huang al. 2023), which decomposes 3D space into orthogonal 2D plancs, facilitating efficient feature aggregation. Lastly, we include Gaussian-based ap- proaches-Giin Gaassian (Huang et al.] 2024). GaussianFormer-2 (Huang ct al.) 2025). and Gaussian- World (Zuo et 2025b) which employ 3D Gaussians to model 3D occupancy and semantics.
B ADDITIONAL IMPLEMENTATION DETAILS
On nuScenes, S2GO uses a 256x704 resolution image and is pre-trained on denoising and rendering for 12 epochs without semantic annotations, and then trained for 24 epochs for 3D semantic occupancy estimation. S2GO-Small uses an ImageNet1k backhone, while S2GO-Base leverages nulmages pre-training. On KITTI, we use 256x 1408 resolution image and an ImageNet backbone. The model is pre-trained for epochs, then trained for occupancy for another 12 epochs.
The temporal transformer closely follows the design from PETR (Liu et al., 2022 and StreamPETR (Wang) ct al.] 2023), with a 4-frame (2 second) queue. Each transformer block consists of a sclf-attention layer across all the queries followed by pross-attentiin layer to refine queries based on image features, and a feedforward layer. The self attention layer includes kcys and queries derived from past queries as in StreamPETR, and we use deformable attention for the cross attention layer for efficiency. All models are trained with a 4c-4 lcarning rate with a batch size of 16, with the cosine annealing schedule and the AdamW optimizer with weight decay 0.01. We use gradient clipping with max norm 35, and scale the learning rate for the backbone by factor 0.25. trained with mixed precision. On P.Scenee-Surronndoe
All of models set to meter. During training, the pairwise query distance for query propagation is randomly sampled between to 3 meters, and during inference, it is set to 1.6m. For nuScenes-Oan Occ3D and KITTI, all distances are scaled according to the smaller extent of the 3D scene. The embedding dimension of the temporal transformer is 768, and we leverage Flash Attention (Dao et al. 2022) for efficient self-attention between queries. Queries
the LiDAR nosing factor
interact with the image through Deformable Attention (7hu al. 2020 Lin
2022
Wang al. 2023
15

---

Published as a conference paper at ICLR 2026
Query lifetime distribution per semantic class
10a
All (avg 13.1 frames) barrier frames)
Number queries (log scale)
bicycle (avg frames)
bus lavg 9.1 frames)
10"
13.2 construction, vehide (avg frames)
metorcydle lavg 14.3 frames)
pedestrian lavg traffic_cone cone lavg 12:1 framest
10a
trailer lavg 12.0 frames)
truck lavg frames)
citveable surface lavg 12.7 framesi
other f lavg 13.4 framess)
(avg frames))
terrain (ovg 13.3 frames)
10a
vegetation (aug 13.3 frames]
10
15
20
25
30
35
40
Query (timesteps)
Figure 5: Distribution query lifetimes per semantic class. Most queries continue propagating over significant number of frames, demonstrating that our queries are persistent over time.
# of History Frames vs mioU
# of History Frames vs IoU
35
mV-u
33
20
25
History
History
Figure 6: Impact of history length on occupancy performance. A longer history consistently improves performance, showcasing the advantage of our streaming approach over prior projection- based methods.
C ANALYSIS ON QUERY PERSISTENCE
In Figure 5 we analyze query persistence over time. For cach query, measure how long it remains in the active query set after first being spawned, and plot the resulting lifetime distribution as well per- class averages. Following tracking works, prune short-lived tracklets during this analysis. We find that substantial fraction queries persist for tens of steps (>55 seconds), demonstrating that once query explains region or object, it remains active throughout much of the sequence. Taken together with Figure 4 this analysis demonstrates that S2GO can effectively handle static and dynamic elements of the scene with persistent queries.
D ABLATION ON THE NUMBER OF HISTORY FRAMES
To further evaluate S2GO. we plot occupancy performance over different streaming history lengths in Figure 6 With longer history, performance steadily improves, demonstrating the efficacy of our streaming framework. We emphasize that unlike prior projection- based works, S2GO incurs additional cost from longer history.
E ANALYSIS ON THE QUEUE LENGTH VS QUERY DISTANCE 8
In Figure 7 we examine how queue length and query distance affect the mloU and IoU of S2GO-Bas -asee First, notice that performance steadily improves with longer history, also seen in Figure [ Next, find that query distance 5 of around 1.5m best for most history lengths.
16

---

Published as a conference paper at ICLR 2026
micU length/2
IoU
22.5
18.43
18.93
18.63
20.46
21.76
21.05
28.56
23.35
20.52
32.07
24,42
33.74
22.0
34
10.41
19.01
19.00
20.
22:22
21.5
28.55
35.02
Qaery daterce (m))
Q (m))
33
21.0
20.5
20.0
18.43
19.12
18.92
20.83
22.19
22.48
0.7
28.56
29.54
30.25
32.29
34.58
34.81
19.5
30
19.0
21.
29
18.5
Queue length (frames)
Queue llnng (frmme))
Figure 7: Analysis of the relationship between the queue length and query distance 8. A queue length of indicates no history is used.
Notably, we with shorter history, the query distance 8 matters less. instance, the difference in ToU between the best and worst 5 for long history of 40 frames 1.66 loU. However, for history length of 4, the difference is just 0.3 IoU. This indicates that maintaining diverse set of spread-out queries is increasingly important the longer the sequence proceeds. This is because unlike other methods that repeatedly sample past image features, S2GO, the query is the only source of history information passed onto the future. queries are poorly selected, the past information is lost.
F ANALYSIS ON THE SCALE s vs OPACITY a FOR OCCUPIED & UNOCCUPIED VOXELS
In this section, thoroughly investigate the impact of including opacity into the occupancy probability in the Gaussian-to-voxel splatting formulation. In Section 3.4.2 proposed that GaussianFormer2's [Huang al. (2025) the exclusion opacity in the occupancy probability is unnatural, then opacity no bearing on determining binary occupancy of a location.
In Figure (top), we visualize the 2D heatmap of learned pairs of 3D Gaussian sizes and opacities a for Formera We use the 1a 3D Gaussian ellipsoid volume computed from the scales the 3D Gaussian expected, unoccupied regions. This is because Former. opacity is only used as the weight in the Eexture-of-Gaussiin for determining foreground classes. As result, Gaussians in unoccupied regions are forced to learn to position themselves between voxel centers instead of simply lowering their opacity.
size. We find that while GaussianFormer22 uses broad range Gaussian sizes to represent the the opacities for occupied regions are lower than opacities
We visualize this clearly in Figure where plot the distance between predicted 3D Gaussian centers and the closest voxel center for both occupied (blue) and unoccupied (orange) regions. clearly see huge dis- tribution difference in distances for GaussianFormedt confirming our hypothesis. This Gaussian formulation destabilizes training and is suboptimal: slight shifts in the Gaussian locations causes unwanted foreground artefacts to appear the background.
To address this, propose in S2GO weight the occupancy probability by opacity This change results more natural formulation occupancy, where opacity dynamically adapts explain regions foreground, as shown in Figure 6 (bottom-left). Furthermore, the network easily learns to predict low opacity for background regions (Figure 6, bottom- right) and is not forced to learn to position Gaussians away from voxel centers for unoccupied regions (Figure 7, left).
G DETAILED LATENCY BREAKDOWN
We henchmark our 9000 Gaussian model an A100 GPU. The backbone, temporal transformer, gaussian estimation, and propagation take 11 11.54ms, 22.79ms, 2.22ms, and 1.45ms, respectively.
17

---

Published as conference paper at ICLR 2026
GaussianFormer2 Occupied Scale Opacity
GaussianFormer2 Unoccupied Scale Opacity
40
25
1.0
0.0
0.7
0.D
35
Goussian Volume
3D Gaussian Ellipsoid Volume
30
17.3
3.1
1.2
0.1
25
Percentage (%6
26.5
8.6
5.6
0.8
Percentage (36
15
20
29.2
-5
41.4
1.9
0.6
0.1
25.6
2.4
1.2
0.1
Opacity
Opacity
S2GO Occupied Scale Opacity
S2G0 Unoccupied Scale Opacity
10.9
2.4
2.6
35.0
1.6
0.6
30
Gaussian Ellisooid Volume
3D Gaussion Ellipsoid Volume
11.9
2.6
2.9
6.1
Percentage ge (%6
24.6
2.2
1.6
1.1
Percentage (%6
20
15
1.1
10
12.8
4.1
5.3
14.1
-2
9.8
1.1
1.0
0.9
0.1
Opacity
Opacity
Figure 8: Analysis learned Gaussian size vs opacity a over occupied and unoccupied voxels for FaussianFormer2 vs S2GO.
Table 9:
Quantitative comparisons with QuadricFormer Zuo et al.
2025a with 256x704 resolution. All meth-
are benchmarked on an A6000 GPU.
const. vel.. car
motorryele
pedestrian
tratii cone
drive. suf.
sidewalk
manmade
vegetation
barrier
bicycle
trailer
track
other
terrain
hus
Method
QuadricFormer-3.
3.2k
28.9
18.5
18.0
11.0
25.7
28.0
11.0
13.3
11.7
9.1
11.5
19.2 39.0
23.1
24.5
22.9
9.4
19.2
22.3
QuudricFormeet Fo.
30.3
18.9
18.2
10.6
25.3
28.4
11.0
126
11.6
9.4
12.3
19.8
39.9
22.6
25.6
23.7
10.8
19.8
13.9
$2GO Small
34.3
22.1
20.8
13.1
27.5
30.3
14.5
16.5
11.7
10,9
13.5
23.3
46.3
29.2
29.7
28.4
13,0
25.1
32.2
S2GO-Base
35.5 22.7
21.9
13.4
27.5
32.1
14.9
15.3
12.9
11.8
13.4
24.0
46.9
29.1
30.3
29.1
14.7
26.4
23.6
H ADDITIONAL COMPARISONS ON NUSCENES
We additionally compare with the recent method QuadricFormer Zuo et al. 2025a with the same 256x704 resolution image input into the same 50 backbone in Table We find that S2GO consistently demon- strates stronger performance at the same, faster, FPS. For instance, QuadricFormer-3.22 3.2k and S2GO-Base have similar FPS, but S2GO-Bass delivers 6.6 higher loU and 4.2 higher mloU.
18

---

Published as a conference paper at ICLR 2026
Histogram Distances Between 3D Gaussian Meeen Closest 30 Voxel Center
Gomssienfommrr2
52G0
Eatusiaans derupled Regiors
Gacoslar Decupled Regans
Unotcopled
2.5
2.5
2.0
2.0
1.5
1.-
0.5
6.0
0.4
2.6
0.8
0.5
0.2
0.4
0.5
0.8
Vocel Distance
Voxel Distance
Figure 9: Histogram of distances between 3D Gaussian centers and the nearest voxel center.
Further, while we were unable to develop a high-resolution (900 x 1600) version of the model due to compu- tational limitations, we have benchmarked its FPS for reference. The S2GO- --Be model with high resolution settings in Former achieves 130.2ms, reaching 7.7 FPS. We note that in such model, the backbone alone occupies most of the computation, and the difference efficiency enabled by our sparse query framework most apparent when the backbone comparatively lightweight in real-time inference settings.
I COMPARISON OF GROUND TRUTH SURROUNDOCC AND Occ3D
In Figure 10 we present side-by-side comparisons of ground truth in SurroundOce-nuSSenes and Oce3D- nuScenes. While both accurately capture the underlying scene. find that SurroundOce often generates more complete ground truth labels along with larger seene extent (100m x 100m vs 80m 80m). While both datascts are strong testbeds semantic occupancy development, this work, focus SurroundOcc- nuScenes benchmark as enables longer-range range perception and serves as the main testbed for Gaussian-focuse methods.
ADDITIONAL JUSCENES-OCC] EVALUATION RESULTS
While our experiments are focused also provide extensive comparisons with existing methods the nuScenes-Occah benchmark using detailed metrics, shown Tah. [0] Our method achieves strong RayloU while maintaining fast inference speed, demonstrating the potential of our sparse, query-basee occupancy framework.
Surenns-SurrouudOec. which provides more complete ground truth, we
K QUALITATIVE RESULTS ON SSCBENCH-KITTI-36G KITTI-360
In Fig. | we visualize example predictions and ground truth from the SSCBench-KITTI-366 dataset. Our framework flexible adapts monocular setting and precisely predicts the semantic occupancy of the driving scene.
L LLM USAGE.
An L.LM lightly used the end for checking grammar and re-wording small parts of the manuscript.
19

---

Published as a conference paper at ICLR 2026
Sur GT
Occ3D GT
Figure 10: Comparison of 3D GT generated by SurroundOOce-nuSeenes and OcSDesuS.eness
20

---

Published as a conference paper at ICLR 2026
Table 10: 3D occupancy performance the Occ3D-nuScenes validation set (Tian al. 2023
FPS
benchmarked on both A100 and on a 4090. A100 numbers prior work are sourced Irom Sparse LLu et al. (2024) and original papers, while 4090 numbers are sourced from ALOcc (Chen et al. 2025a ProtoOcc benchmarks 3090 GPU.
Method
Backhone
Mask
Input Size
Epoch
RayloU
RayloUn in.
mloU
FPSAHO
1P5400
Hi-/oommer
R101
1600=
24
52.4
38.0
RenderOoe
KU54
Swin-B
140sx512
12
19.5
19.6
25.5
24.4
SimpleOce
Gan elal: - 20
R101
672x
12
22.5
17.0
22.7
27.9
31.8
9.7
RF-Der-Oot
2021
R50
754
29.6
35.1
36.1
BEVDxt-Ore-ae
oit Hiaggg Huniig 2021
R50
704x384
90
32.6
33.1
38.2
39.3
FB-0xc [ et al 21736
R50
704x256
90
33.5
26.7
34.1
39.7
39.1
10.3
(Kin
ProtoOce
et al. 2125
R50
800x
12
39.0
HisVoormer
20226
R101
1600x
24
33.7
23.7
3.0
4.4
SparseOee
Liu cial 2024
R50
256
36.1
30.2
36.8
41.2
30.9
12.5
GSD-Oee
R50
704 256
24
38.9
20.0
SmeamOee
2025
R50
704x
24
41.1
34.2
41.9
47.1
12.0
OPUS-L
Wain 21 17/4
R50
254
100
41.2
42.1
46.7
STCOce
Ligo el
R50
704
36
42.1
36.9
42.8
46.7
ALOoc-3D7 Ce
20125a
R50
754x256
54
43.7
37.8
44.7
48.8
38.0
AL.
20.36
R30
704
44.1
38.2
45.0
49.2
S2GO-Sman SaGO-Small (ours)
R50
704
24
37.2
31.3
38.1
42.2
30.8
20.8
25.3
$20O-Base
R50
254
24
39.1
40.0
44.1
14.5
21

---

Published as a conference paper at ICLR 2026
Pred. Occupancy
Ground Truth
Pred. Occupancy
Ground Truth
STYLEI
Pred. Occupancy
Ground Truth
Pred. Occupancy
Ground Truth
Pred. Ocoupancy
Ground Truth
Pred. Occupancy
Ground Truth
Figure 11: Qualitative Results on the SSCBench-KITT11366 dataset. S2GO well-captures oceu- pancy details even in a monocular setting.
22


Reviews:
[{"review_id": "XYDRBrUPp7", "reviewer": "Reviewer_GjMC", "summary": "This paper presents a new method for sparse occupancy prediction. It proposes several methodological and engineering improvements over prior Gaussian-based approaches, including a refined pretraining strategy and an innovative Gaussian-to-Voxel Splatting operator. These contributions demonstrate both conceptual insight and practical significance. The proposed method achieves superior performance compared to previous Gaussian-based methods.", "questions": "1. The method achieves excellent results using lower-resolution inputs. Could the authors report performance using 900×1600 image size to compare under identical conditions with previous Gaussian-based methods?\n\n2. How is the velocity of Gaussians predicted? Is there explicit velocity supervision during training?\n\n3. In the ablation tables (Tab. 3–5), the best-performing configurations achieve different absolute scores. Could the authors clarify the cause of these variations?\n\n4. Could sparse-based methods potentially outperform BEV-encoding methods? For example, ALOcc [4] achieves 39.3 RayIoU and 30.5 FPS on Occ3D-nuScenes, appearing more efficient.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-organized and written with clear logical flow.\n\n2. It conducts an in-depth exploration of sparse occupancy prediction, proposing improvements such as pretraining strategies and the Gaussian-to-Voxel Splatting operator that are both insightful and practically valuable.\n\n3. The method achieves better performance than prior Gaussian-based approaches, demonstrating tangible benefits of the proposed design.", "weaknesses": "1. Inference speed comparison (Tab. 1) may be unfair: the proposed method uses 256×704 input images, while other methods use 900×1600. Since the image backbone usually dominates inference time, comparison should be made at the same resolution or explicitly clarified in the paper/table.\n\n2. Methods such as [2] achieve 25.5 mIoU on the SurroundOcc dataset but are not included in the comparison table.\n\n3. While reporting results on Occ3D is commendable, the paper omits several recent competitive baselines with higher RayIoU (e.g., [1], [3], [4]).\n\n4. Minor: L242 — “where M is the # of LiDAR points” appears to contain a small typo.\n\n[1] OPUS: Occupancy Prediction Using a Sparse Set, NeurIPS 2024\n\n[2] Rethinking Temporal Fusion with a Unified Gradient Descent View for 3D Semantic Occupancy Prediction, CVPR 2025\n\n[3] STCOcc: Sparse Spatial-Temporal Cascade Renovation for 3D Occupancy and Scene Flow Prediction, CVPR 2025\n\n[4] ALOcc: Adaptive Lifting-based 3D Semantic Occupancy and Cost Volume-based Flow Prediction, ICCV 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L0XoRVLc7j", "reviewer": "Reviewer_Hic9", "summary": "This paper introduces S2GO, a novel streaming framework for 3D occupancy estimation that utilizes 'sparse 3D queries' instead of conventional dense representations. The core contributions are threefold: (1) A LiDAR-based 'geometry denoising' pretraining stage resolves the spatial learning ambiguity of sparse queries. (2) 'Opacity-weighted occupancy estimation' is introduced to improve accuracy. (3) 'Efficient splatting' via CUDA optimization (20x faster backpropagation) halves the training time. As a result, S2GO demonstrates a +2.7 IoU improvement over the SOTA (Gaussian World) on the nuScenes benchmark and achieves 4.5x faster inference, enabling real-time performance (26 FPS).", "questions": "See Weaknesses", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel and Well-Motivated Approach: The paper introduces S2GO, a novel streaming framework for 3D semantic occupancy prediction that leverages sparse 3D queries instead of voxels or Gaussian-based 3D representations. This represents a clear conceptual advancement, successfully extending Gaussian splatting to the occupancy domain while addressing the long-standing trade-off between accuracy and computational efficiency.\n- Core Contributions Address Key Challenges: The paper's main contributions effectively solve the primary challenges of a sparse-query approach. (1) The 'geometry denoising pretraining' directly tackles the sparse-to-dense mapping ambiguity by teaching queries 3D structural priors, as validated by strong ablation results (Table 3). (2) The 'Opacity-weighted occupancy estimation' improves prediction fidelity and stability. (3) The 'Efficient Gaussian-to-Voxel Splating' CUDA optimization (20x faster backpropagation) overcomes the training bottleneck, halving training time and making the high-accuracy Gaussian representation computationally feasible.\n- Strong Empirical Results and Efficiency: S2GO achieves new state-of-the-art performance on both nuScenes and KITTI, outperforming Gaussian World by +2.7 IoU while being 4.5–5.9× faster in inference. Notably, the lightweight S2GO-Small runs at 26 FPS on a single RTX 4090, demonstrating strong potential for real-time deployment in autonomous driving systems. The strong performance is well-supported by a comprehensive analysis, including thorough ablations on all key components. Furthermore, qualitative results (Fig. 3,4) confirm strong temporal consistency and object identity preservation, outperforming prior methods.", "weaknesses": "- Missing Comparisons to Voxel-based Efficient Methods: The paper lacks comparisons to recent efficiency-focused voxel methods (e.g., GSD-Occ (AAAI’25)[1], ProtoOcc (AAAI’25 / CVPR’25)[2,3], and StreamOcc (arXiv’25)[4]). Adding direct comparisons (IoU, FPS, memory) and discussing the design trade-offs (sparse query based GS vs. structured voxel) would strengthen the paper.\n- Robustness of Geometric Prior: The pretraining's reliance on high-quality geometric data (LiDAR or strong depth models like Metric3D) is a concern. The analysis would be more robust with sensitivity studies on (a) varying LiDAR sparsity (e.g., 16 vs. 64-beam) and (b) the accuracy of the depth estimation input.\n\n[1] He, Y., Chen, W., Wang, S., Xun, T., & Tan, Y. (2025). Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3), 3455-3463.\n[2] KIM, Jungho, et al. Protoocc: Accurate, efficient 3d occupancy prediction using dual branch encoder-prototype query decoder. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2025. p. 4284-4292.\n[3] OH, Gyeongrok, et al. 3d occupancy prediction with low-resolution queries via prototype-aware view transformation. In: Proceedings of the Computer Vision and Pattern Recognition Conference. 2025. p. 17134-17144.\n[4] MOON, Seokha, et al. Mitigating trade-off: Stream and query-guided aggregation for efficient and effective 3d occupancy prediction. arXiv preprint arXiv:2503.22087, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vrMFntZJ5Z", "reviewer": "Reviewer_RwQB", "summary": "The paper proposes S2GO, a streaming, query-based 3D semantic occupancy method. Instead of dense voxels or many Gaussians, S2GO maintains a small set of persistent 3D queries that are propagated through time and decoded into semantic Gaussians each step. A two-stage training is used: (1) geometry denoising & rendering pretraining with noised LiDAR anchors and RGB/depth supervision; (2) camera-only semantic occupancy with improved Gaussian-to-voxel splatting (opacity-weighted occupancy and custom CUDA kernels). On nuScenes SurroundOcc and KITTI-360 SSCBench, S2GO reports SOTA IoU/mIoU with much higher FPS vs prior art (e.g., GaussianWorld).", "questions": "1. I am curious how sensitive performance is to δ and history length in the propagation queue; can you plot mIoU vs δ and vs queue length to find a sweet spot?\n2. How about failure cases under heavy rain/night where monocular depth priors are less reliable: does zero-shot pretraining still hold up, and how does it affect query placement stability?\n3. Whether the shared-class assumption within a query ever blends adjacent classes (curb/road/sidewalk). Could per-Gaussian class residuals recover edges without exploding compute? and whether velocity is actor-consistent: do queries attached to a moving car maintain identity over 5–10s, or do they hop? Any quantitative ID-switch metric?\n4. If opacity-in-α encourages degenerate small-scale/high-opacity vs large-scale/low-opacity solutions. Did you inspect learned (s, a) distributions across classes and empty space?\n5. The paper fails to evaluate model performance when pre-training and fine-tuning datasets differ (e.g., pre-trained on KITTI, fine-tuned on nuScenes). No experiments address whether the pre-training strategy generalizes across domain shifts, undermining the method’s practical applicability", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Pros:\n1. Clear motivation for sparsity + streaming. The paper articulates the inefficiency of dense grids and dense Gaussian fields for long-horizon temporal fusion and proposes queries (~K) that each spawn a handful (J) of Gaussians to cover local structure—bridging object-centric queries and dense occupancy.\n2. Well-designed pretraining. Initializing queries at noised LiDAR FPS points and supervising with denoising + RGB/depth rendering directly teaches queries to “move onto” geometry and Gaussians to model fine shape. Ablations show large gains vs training occupancy from scratch.\n3. Principled fix to splatting. Making binary occupancy opacity-weighted (α(x;G) := a · exp(…)) aligns semantics with density in rendering and improves stability/performance; optimized CUDA “blocked” splatting cuts backward time ~20× on A100 (116ms→5.7ms).", "weaknesses": "Cons:\n1. Pretraining dependence & potential dataset bias. Stage-1 uses LiDAR-based supervision (depth and LiDAR-anchored queries). Although authors argue LiDAR already exists for label generation, the camera-only promise hinges on quality of this pretraining. The Metric3D zero-shot depth alternative is encouraging but still trails LiDAR-based pretrain. Quantify generalization gap more broadly. Futhermore, the model achieves SOTA performance only with LiDAR-based pre-training (\"LiDAR+ϵ\" in Table 3). Without this step, performance degrades significantly (as evidenced by the absence of a \"no pre-training\" ablation in Table 3). This contradicts the paper's claim of a \"monocular\" method, as LiDAR is required for pre-training a mama critical barrier for pure-vision systems.\n2. Ambiguity around semantic consistency. During occupancy training, Gaussians derived from one query share a class—good for local consistency, but might limit fine-grained multi-class boundaries inside a query’s spatial span (e.g., curb vs asphalt vs pole tightly adjacent). Some discussion or per-Gaussian residual semantics would help. \n3. Query identity & drift. The paper relies on velocity-aided propagation and δ-separation but doesn’t quantify identity preservation (e.g., re-association failure under occlusion) beyond qualitative visuals. A per-query lifetime/coverage analysis would strengthen claims.\n4. Opacity-weighted α trade-offs. Authors note opacity in α improved accuracy but increased training cost (more affected voxels) before CUDA optimizations; inference-time trade-offs (e.g., memory traffic, batch size limits) aren’t reported. A wall-clock throughput table (train/infer) vs baselines would be useful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K50AUW92yq", "reviewer": "Reviewer_m6uz", "summary": "In this paper, the authors propose S2GO, a streaming sparse query framework for 3D semantic occupancy estimation. This method efficiently summarizes and propagates scene information through sparse 3D queries and decodes it into a semantic Gaussian distribution. Core contributions include a geometric denoising pre-training stage and an efficient Gaussian-voxel transformation implementation. The paper achieves state-of-the-art performance and significant inference speed advantages on nuScenes and KITTI benchmarks. The method is novel, the experimental design is comprehensive, and the results are convincing.", "questions": "It is essential to address the above comments in weakness.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. A streaming occupancy prediction with Gaussians is an insightful, useful and unexplored problem.\n\n2. To address the above problem, the authors propose a simple but in-depth architecture S2GO. Their solution is elegant. \n\n3. Extensive experiments on the multiple datasets show the effectiveness of the proposed method.", "weaknesses": "1. It is recommended to enhance the display of module details in Figure 1.\n\n2. The paper focuses on engineering and experimentation in its methodological description, and lacks theoretical analysis or discussion on the representation capabilities of sparse queries or the convergence of the model.\n\n3. The efficient Gaussian-to-voxel splatting proposed in the paper significantly improves the forward/backward pass speed, but lacks comparative experiments on memory usage. It is suggested that the authors supplement this with a comparison of memory consumption.\n\n4. In Figure 3, the small subfigures partially obscure the main visualization. I suggest adjusting the layout so that the smaller insets do not overlap with the main figure, ensuring all visual details are clearly visible.\n\n5. The paper would benefit from a clearer description of the experimental parameter settings (e.g., learning rate, batch size, number of epochs, optimizer, etc.). Providing these details would enhance reproducibility and allow for a more precise evaluation of the experimental results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)