VoxDet: Rethinking 3D Semantic Scene Completion as Dense Object Detection
Authors: Wuyang Li
Zhu Yu
Alexandre Alahi
Venue: NeurIPS 2025 spotlight ()
Keywords: 3D Semantic Scene Completion
Free-lunch
Voxel-to-Instance (VoxNT) Trick
Dense Object Detection
Primary area: applications
TL;DR: Based on a newly discovered "free lunch" in voxel labels, VoxDet reformulates 3D semantic scene completion as dense object detection using a VoxNT trick.

Abstract: Semantic Scene Completion (SSC) aims to reconstruct the 3D geometry and semantics of the surrounding environment. With dense voxel labels, prior works typically formulate SSC as a *dense segmentation task*, independently classifying each voxel. However, this paradigm neglects critical instance-centric discriminability, leading to instance-level incompleteness and adjacent ambiguities. To address this, we highlight a "free lunch" of SSC labels: the voxel-level class label has implicitly told the instance-level insight, which is ever-overlooked by the community. Motivated by this observation, we first introduce a training-free **Voxel-to-Instance (VoxNT) trick**: a simple yet effective method that freely converts voxel-level class labels into instance-level offset labels. Building on this, we further propose **VoxDet**, an instance-centric framework that reformulates the voxel-level SSC as *dense object detection* by decoupling it into two sub-tasks: offset regression and semantic prediction. Specifically, based on the lifted 3D volume, VoxDet first uses (a) Spatially-decoupled Voxel Encoder to generate disentangled feature volumes for the two sub-tasks, which learn task-specific spatial deformation in the densely projected tri-perceptive space. Then, we deploy (b) Task-decoupled Dense Predictor to address SSC via dense detection. Here, we first regress a 4D offset field to estimate distances (6 directions) between voxels and the corresponding object boundaries in the voxel space. The regressed offsets are then used to guide the instance-level aggregation in the classification branch, achieving instance-aware scene completion. VoxDet can be deployed on both camera and LiDAR input and jointly achieves state-of-the-art results on both benchmarks, which gives 63.0 IoU on the SemanticKITTI test set, **ranking 1$^{st}$** on the online leaderboard.

VoxDet: Rethinking 3D Semantic Scene Completion as Dense Object Detection
Wuyang Li!
Zhu Yu²
Alexandre Alahi!
"Zhejiang University
*cooe Polytechnique Fédérale de Lausanne (EPFL)
wuyang. 1i@epf1.ch
yu ru_zzuuzju. cet.uuzju.euuu.coo edu. cn
alexandre. alahi@epfl.cm ch
Project Page: https://iilb-ppllliiiuuboooooDD
Abstract
Semantic Scene Completion (SSC) aims to reconstruct the 3D geometry and seman- ties of the surrounding environment. With dense voxel labels, prior works typically formulate SSC as a dense segmentation task, independently classifying each voxel. However, this paradigm neglects critical instance-centri discriminability, leading to instance-level incompleteness and adjacent ambiguities. To address this, we highlight "free lunch" of SSC labels: the voxel-level class label has implicitly told the instance-level insight, which is ever-overlooked by the community. Mo- tivated by this observation, (VoxNT) trick: a simple yet effective method that freely converts voxel-level class labels into instance-level offset labels. Building this, further propose VoxDet, an instance-centric framework that reformulates the voxel-level SSC as dense object detection by decoupling into two sub-tasks: offset regression and semantic prediction. Specifically, based on the lifted 3D volume, VoxDet first uses (a) Spatially-decoupler Voxel Encoder generate disentangled feature volumes for the two sub-tasks, which learn task-specific spatial deformation in the densely projected tri rrrpeeceet space. Then, we deploy (b) Task desk-lec Dense Pre- dictor to address SSC via dense detection. Here, we first regress 4D offset field estimate distances (6 directions) between voxels and the corresponding object boundaries in the voxel space. The regressed offsets are then used to guide the instance-level aggregation in the classification branch, achieving instacce-aware scene completion. VoxDet can be deployed on both camera and LiDAR input and jointly achieves state-of-the-ar results on both benchmarks, which gives 63.0 IoU the SemanticKITTI test set, ranking 1 st on the online leaderboard.
first introduce training- free Moxel-to-Instanee
Introduction
Spatial AI [12] is crucial for autonomous systems perceive and interpret the physical world. As a critical step, precise reconstruction of geometric structures and semantics lays the foundation for scene understanding (27 41 3T, underpinning the downstream forecasting and planning [[ [4]
This capability is indispensable for applications such as autonomous driving and robotic navigation.
To this end, Semantic Scene Completion (SSC) has attracted significant attention by simultaneously inferring complete 3D) geometry and semantics via occupancy representation. Prior SSC works can be broadly categorized into LiDAR-based [55] 50 (6) and camera-based approaches [
R4 79] 22
The former uses sparse 3D inputs (e.g., point clouds) for SSC to provide precise geometric information. In contrast, camera-based methods have recently demonstrated promising potential due to their flexibility and computational efficiency. They employ dedicated vision-centric algorithms to lift 2D imagery into 3D space, enabling SSC, including Features Line-of-Sight Projection (FLoSP) 6),
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Previous works: Instance-agnostic segmentation
Our VoxDet: Instance-centric dense object detection
Inclanea Vakume
Semancc Suprrason
Classificction
Classiiiccion Branch
Hscance Appregation
Labels
SemurGe
Spatiel Decourlig
Ro Dffiet Field
instance Tric
Offset (Fgg 4
2D Image
Valume
Volume
Orss
20 image
20 Volume
Haad
Regression
4D Offict Fleid
Supenssiot
Semantic Encoder
Semantic Predictor
[a] Spatially-decoupled Voxel Encoder
[b] Tosk-deecouped Dense Predictor
Figure I: Schematic comparison of previous SSC paradigm 6 79 and the proposed Vet Left:
Existing works formulate SSC as 3D semantic segmentation, directly predicting voxel labels agnostic to object instances. Right: VoxDet reformulates SSC as dense object detection to explicitly learn with instances, which decouples it into instance-awarr regression and classification. This is achieved via a Voxel-to-Instance (VoxNT) trick (Fig. inspired by our observed free-lunch in labels (Sec.
depth-driven back projection 34 22 78
and Lift-Splat-Shot (LSS) 446 based voxels 77 24 2
These methods offer unique advantages in resource- conutrccconttr settings such as robotics, owing to their low cost, high flexibility, and real-time processing capabilities.
While achieving great progress, existing methods uniformly formulate SSC as a semantic segmen- tation task" on dense voxels, which fail to understand the instance-level semantics and geometry. This paradigm typically employs 3D encoders [83] 779 to encode semantic patterns from the lifted volumes (see Fig. [ Left), subsequently classifying each voxel independently. In SSC, due to the absent instance labels, this voxel-centric paradigm may seem viable, but it has to face severe instance-level incompleteness and adjacent ambiguities (see Fig. 7). Although prior works, such as Symphonize [22] 58, 40). have noticed this issue and attempted to mitigate it with object queries from 2D images, they still optimize the 3D space via dense segmentation on each voxel. The gap between the image and voxel domains prevents effective instance-drive driven learning, making the segmentation- based formulation fail to infer the complex environment with dynamic agents.
To address this issue, we first highlight a "free-lunch" of SSC labels: the voxel-level class label has implicitly told the instance-level insight, which is ever-overlooked by the community. First, unlike in 2D images where occlusion causes overlapping objects to conflate into a single entity, instances in 3D voxels are inherently separable. Every voxel is deterministically assigned single class without rellusooninduceee ambiguity, making the instance-level discovery essentially realistic. Second, instance boundaries in 3D voxel space are naturally tractable and regressible, ensuring clear distinctions between instances. More details can be found in Sec. 3
Inspired by these observations, we first develop Voxel-to-Instance (VoxNT) Trick (Fig. 4 that can freely convert the voxel-level class labels to the instance- offset labels, fully harnessing the free-lunch (Sec. 3 Here, the offset is defined as the Euclidean distance between each voxel and its associated instance borders. Then, these free offset labels prompt us to rethink the segmentation- SSC formulation. In response to this, we propose VoxDet to reformulate Voxel-level SSC instance- centric object Detection (see Fig. | Right). Unlike prior works, VoxDet decouples SSC into two sub-tasks: offset regression and semantic prediction. We achieve this with (a) Spatially-decoupleer Voxel Encoder (SVE) and (b) Task-decoupled Dense Predictor (TDP) at the representation and prediction levels, respectively. Specifically, given 3D volumes lifted from 2D images (46), SVE first learns task-specific features by spatially deforming in the tri-perceptive space, which are sent to the two branches of TDP, respectively. Within TDP, we regress each voxel to the associated instance boundary by predicting a 4D offset field, which guides an instance-leve aggregation for SSC.
Hence, with our new detection-based formulation, VoxDet the first work achieve true instance- centric perception solely using voxel-level labels, which is not only highly efficient but also achieves state-of-the-an performance benchmarks, setting new records of 18.47/ 2..40 mloU and 47.27/ 48.59 loU on the SemanticKIITT and SSCBench-KII K3TT-3360 datasets, respectively.
In summary, contributions lie in the following aspects:
By analyzing the difference of 2D pixels and 3D voxels (Sec. 3) we reveal the overlooked free lunch of SSC labels for instance- learning. With this observation, we propose a
For simplicity, in SSC, uniformly refer voxel recognition and completion o dense classification.

---

VoxNT trick to freely convert voxel-level labels to instance-level offsets. As a byproduct, this trick can also identify wrong labels of dynamic objects (see Appx. Baand Fig. 77.
We reformulate SSC as a dense object detection task by advancing VoxDet, decoupling SSC into two sub-tasks: offset regression and semantic prediction, for instance-level SSC.
We design a ppatially-decoulee Voxel Encoder (SVE) to decouple 3D volumes for our new SSC formulation, which learns task-specifie spatial deformation in the densely projected tri pereeeeeee space for the two sub-tasks, avoiding the misalignment between them.
Dense Predictor (TDP) to enable instance-driver SSC. This comprises regression branch that predicts 4D offset field, delineating instance boundaries, and classification branch using learned offsets for instance-aware aggregation.
We propose Taas-decouppe
2 Related Work
3D Semantic Scene Completion [55 aims to jointly reconstruct the semantics and geometry of surrounding environment with voxelization. Existing studies can be generally divided into LiDARR- based and camera- based methods [51]. The former utilizes point clouds to achieve high accuracy with precise depth completion [70-72) which is limited by the computational cost. Camera-basee methods rely solely on 2D visual inputs to generate 3D scene understanding. With the advancement of monocular vision like LSS [46), these approaches offer great advantages in efficiency. MonoScene [6] pioneered the field of camera-bass based SSC by connecting 2D images with the 3D voxel space via the FLoSP. VoxFormer (34] uses 3D-to-2D back pook-projcttion and disseminates the semantics of the visible queries across the entire 3D volume via MAE [19]. Subsequent works have further enhanced voxel representations using techniques such as tri tri-perceptive enhancement (79, 2], diffusion models (35), vision-language models [58 and extra modalities [18 60) However, existing methods uniformly treat SSC as voxel segmentation, which lacks instance-level perception. contrast, we reformulate SSC as a dense object detection task with explicit instance-level awareness.
Dense Object Detection, such as point- FCOS [56] 62 is a fully convolutional paradigm known for its lightweight design, efficiency, and performance, which garnered significant attention prior to the DETR series 88 63
86 443. The core insight is the notion of "densely detecting like segmentation" every pixel within a bounding box regresses its distances in four directions (up, down, left, right) while simultaneously predicting the instance-level class label, which is a dense process like the per- pixel segmentation. The following works focus improving this paradigm from different aspects, including considering better label assignment [8] 80 like ATSS [8]] architecture search [59]. spatially task decoupling (10. border enhancement
dense feature distillation
84 30 29]
optimization signals [68] 33). Besides, dense detection has been extended to 3D vision. For instance, FCOS-3D 62) predicts 3D targets in the 2D space by regressing 3D attributes for each pixel, and UVTR 332 predicts 3D position for each pixel to enhance instance-level localization. Additionally, some instance detection works propose 3D-inspired paradigm for geometrically reliable detection in 2D views 226 53 Unlike previous works, we reverse the philosophy via a "segmenting like dense detection" framework that endows voxel-based segmentation with instance-level awareness, which requires no additional manually annotated instance- labels for free instance-level learning.
3 Preliminaries and Motivation
We start by analyzing the differences between 2D pixel and 3D voxel space with respect to semantic- level classification and instance-level regression, from the perspective of dense perception [56). We then clarify our observations and insights on the ever-overlooked "frec-lunch" in voxel-level SSC labels and explain the associated motivation for the following methodology.
Semantic- based Dense Classification. In Fig. (a.c), can be observed that occlusion in the 2D space often leads to the merging of distinct object instances (e.g.. the highlighll cars) into single entity (Fig. 2). Due to the perceptive projection [3. multiple 3D points at varying depths in the world coordinate system converge into single pixel of the image. In contrast, the 3D voxel space inherently avoids most of such occlusion, as each voxel is uniquely assigned class label, including the empty class (Fig- (p) Consequently, the occlusion-free property of 3D voxels facilitates the natural separation of object instances, although minor ambiguities may arise in cases such as the densely overlapping foliage of adjacent trees, which typically do not affect the scene understanding.
3

---

Classification on
(e) Dense Classification on Vozels
Figure 2: Conceptual comparison be- tween 2D pixels and 3D voxels re- garding semantic-level dense clas- sification and instance-level regres- sion. In (a,b), 2D occlusion makes instance perception fail with seman- tic labels. In (c,d), the occlusion- free nature of 3D space enables instances separable only using semantic labels.
makia
Diclavin--fee makea
(b) Dense Regnession on Pixels
[d] Dense Regression Voxels
Spacee
20 Space
1D Scare
Instance- Dense Regression. In Fig. (b.d), each pixel and voxel aims to regress the distance to the instance border [56]. In the 2D domain (Fig. ()), such regression supports instance-level perception via bounding box labels; however, it fails to infer instance cues with semantic labels. Conversely, in the 3D voxel space. the natural separability of instances makes it possible to regress instances without instance labels (Fig. (Fig.2)). For example, consider the red voxel inside the highlighted car in the figure. As the voxel space occlusion-fire free, we can regress from this voxel to its instance boundary by identifying adjacent voxels with differing semantic labels (see Fig. 4. This property enables us to infer instances using only semantic annotations, a valuable yet overlooked free lunch.
Motivation and Insight. Inspired by these observations, we aim to rethink Voxel-level SSC as object Detection (56). termed VoxDet, which adopts novel "segmenting like dense detection" philosophy. Instead of directly recognizing each independent voxel [83] 77 2
[4) VoxDet decouples SSC into regression and classification sub-tasks the 3D voxel space, where, in particular, the core innovative regression (or offset) branch enables explicit instance-level 3D regression. The learned offsets are subsequently used for instance-level aggregation, facilitating scene understanding. Note that this fundamentally differs from and advances beyond the traditional segmentation paradigm by explicitly lifting voxels instances, with potential applicability to the broader occupancy community.
4 Methodology
Overview. Fig. shows the overall workflow of our VoxDet. Given RGB input, we follow previous works (79 to conduct (a) 2D-to-3D lifting to generate 3D feature volumes V, which uses the estimated depth Z given by the arbitrary depth estimator 5 52 Then, we send V to (b) Spatially- decoupled Voxel Encoder to generate the disentangled feature for dense classification Vas and regression veg. Here, we encourage the two tasks to learn spatially decoupled features to avoid task- misslainingg (68], which is achieved in a densely projected tri-perceptive (TPV) space. Next, the decoupled volumes are sent to (c) Task-decoupled Dense Predictor. In this part, we first regress a 4D offset field A to estimate the distance between each voxel j. to the instance boundary Aij. E R in six directions (see Fig. 4. The learned offset A is subsequently sent to the classification branch to guide instance-level aggregation, thereby achieving instance-awer aware semantic scene completion.
4.1 2D-to-3D Lifting
We follow previous SSC works [79 34] to conduct the same 2D-to-3D lifting (Fig. 3R), outputting the 3D feature volume V E X-XxXx xxx
where X, Y, Z, and the depth, width, height, and channel respectively. The process is briefly described as follows. More details are in Appx.
Given the RGB image the depth map Z with the frozen depth estimator [5] 52) following the unified SSC practice [79 34 2
extract the 2D image feature F using the image encoder, and estimate
Then, estimate the depth probability D using LSS 46). Based these, we can establish the 3D feature F3D using the fused depth probability D and pre-extracted 2D feature F2D (79), which is
subsequently projected onto the voxel grid to generate the 3D volume V. In this procedure, each
voxel in V is able to query the information from 3D features F3D via deformable cross-attention [
4.2 Spatially-decouplee Voxel Encoder
Given the 3D volume V, we aim to extract task-specific representations for the two tasks. Prior encoders are designed for voxel segmentation, which lacks the spatial context for regression with

---

Chussication
Instance-ivvel
Vineo Trann Arrannoommmnn Aoneoommiin
Diso Lobill
Spatial Decouplting
Da
kinen son iirtareee
lrick Iig.
Regieso ossio
Wil
Projection
liaad
Difoit
Seción
(A. 2.4)
2D-to 3D Litting (Baseline)
[b) Spatially-decoupeed Vowell Encoder (SVE)
(c) Taskkddecoupee Dense Predictor (TDP)
Figure 3: Overview of our Vex After 2D-to-3D lifting, VoxDet spatially decouples 3D volumes V into two task-specific branches, learning different spatial deformations the densely projected tri-perceptive space. Then, VoxDet regresses a 4D offset field A towards instance boundaries with
V reg. serving for the instance-level aggregation with Vels in the classification branch.
mask-misalignment [54 22, [0]. Hence, we spatially decouple V into Vels for classification and Vreg for regression (Fig. 3)) in a densely projected TPV space. Fig. shows the decoupling effect.
Dense Projection. Given the 3D volume V, we use a shared encoder to extract L-level task-shared feature volumes {V ) Due to the task-misaligned feature preference [68]. directly using the shared volume for the two tasks will severely influence the instance-level perception. To resolve this, we spatially decouple task-specific features in a densely projected tri tri-perceptive (TPV) space (21] This can avoid the large computational cost of decoupling in 3D, effectively reducing the dimension to 2D. To this end, a dense projection Pa(-) is proposed to generate 2D feature planes for our detection-based formulation. Specifically, given V(I) layer 7- W Wa(V!!),, 7) E XX-XXx22 xx33, which guides the adaptive pooling for the three TPV planes. enc), Then, we conduct pooling across each axis (X, Y, Z) to generate projected 2D feature planes:
enie we learn per-voxel dense weight with a linear
2
Y
P2 (V) = Σ S2 V (LAe))
Σ y=1
Sy (7),)))) V V..e. Σ Sx [mnd
Here, pla) indicates the (A)-axis projection, S(A) is softmax on the (A) axis, and o is Hadamard
product. Next, we deploy 2D refinement layer to generate TPV feature XY, X 2, Y planes:
V-XY = Convor cls (P) (V())) )), vXz = -Conve X2
cls (P) (V(l)) )), VYZ
Conv Y (P)) (Vel))) cl
cls
cls
V-XY Conv rog
XY (P& (V(l) ene)),
X2 V-22 -onvx veg (P) (V ene)), VYZ Conv Y2 (PX (V()) reg
reg
reg
reg
Here, ( 2D convolution module for plane (P) and task (T). Compared with the conven-
tional TPV pooling (2I). our dense projection can adaptively discover the essential voxels during the 3D-to-2D dimensional reduction, thereby mitigating the information loss for the dense perception.
Spatial Decoupling. With projected 2D features, we spatially decouple it in each TPV plane with different spatial offsets, encouraging two tasks to focus on task task-specific regions. Then, we use Conv layer to fuse the decoupled features with the same expanded size, which is carried out as follows:
(1)
= Conv (DefConv
els (V (V-XY, + DefConv
XY
X cls
x (V
DefCony Y
(V=2))
V (4) Conv Jusee (DefConv reg
XY
(V XY DefCony (V
X2 reg
X reg
+ DefCony YZ (V Y2
reg
Here, DefConv(-) is the 2D deformable convolution module, which spatially decouples the dense voxel features for the two tasks. This decoupling reduces the computational burden and preserves task-driven spatial context, effectively addressing the misaligned feature preference of the two tasks. Finally, we send the decoupled multi-level features to the lightweight FPN (38] branches, respectively, to improve the task-specific learning, and collect the last-layer output with the same resolution as V
for the two tasks, which is denoted as: Vel FPN(1 V (1)
(1) reg 1m)
ets ( and V reg
FPN(
4.3 Task-decoupled Dense Predictor
Then, the decoupled features Val and Vrey are sent to the classification and regression branches (see Fig. 3b) Here, we first densely regress the instance boundary to identify 3D objects, then aggregate instance-level semantics in the classification branch, achieving instance-centrie SSC prediction.
5

---

(o.g
6
Werkaaree
4
ViJA
3.5
1.5
2.5
1.5
0.5
1.5
save the scanning distance as the offset Plane View VouNT Trlcc
XZ
VoxNT
V
Figure 4: Illustration of our regression objective. For each voxel V Vij.k. we scan in 6 directions and calculate the distance the instance boundary to generate labels Áij using the free-lunch (Sec. 3
Regression Branch. Given the volume for regression V reg E IXXXXZZC we aim to regress the distance to the instance boundary for each voxel Vi,j. E reg (see Fig. 4 in six directions, which form a 4D offset field A XxXXX22 x2x6 We use 6 directions as it is the minimum number to determine a 3D bounding box, similar to the 2D bounding boxes determined with directions [56]. Thus, considering the voxel in Vegg
reg at position (i, k), the spatially associated element A.j. in the offset field represents 6-channel vector for the specific offset 5 along the X, Y. and Z axes, which is divided into positive (+) and negative (-) directions. This can be denoted as follows,
A RegressionHead (Vig), where A.,. (8 o.j, k.j.k' L.j.k- Oij.A Kf,,k) R. (4)
6y 6"
Here, RegressionHead(-) a lightweight head predict the offiet field, which is simply deployed as a two-convolution module to enable non-linearity with 6-channel output, followed by Sigmoid for normalization. Compared with the anchor-basee based design 48), this design is highly computationally efficient due to its fully convolutional nature when deployed in the dense 3D voxel space.
Regression Loss. In SSC, only have dense class labels for the voxel grid Y E NXxYx2 without instance-level labels for regression, such as bounding boxes. Hence, to break through this, we propose (VoxNT) Trick freely transform the class labels the instance-level offsets, fully using the free lunch in Sec.
3 The algorithm details are in the Appendix.
Fig. 4 shows the VoxNT trick. In brief, for each voxel A.j.k. e.g., the blue example, in labels Y, and stop when the class of next scanned voxel changes, indicating approaching the border. Then, labels: Å X XxxxXxx For stability, round up Á to the integer and normalize into [o, 1] via the volume size. Finally, we deploy L1 loss to optimize the regression with enough gradient:
across
x directions d (xt,2* .V" "" ",, =
Ceg
X.Y.Z.. kij.k,d A..j.k.d - Aij.k.al
(5) by-product, the offset field
where A E R-XYX2X66 is the predicted 4D offset field. Surprisingly, can identify wrongly labeled dynamic objects (like cars) for more accurate training (see Appx. B.3)
Classification Branch. Based on the regressed offset A, we aim to enhance instance-level per- ception by aggregating the semantics within instances. A natural idea [17 is crop 3D bounding boxes, which is extremely computationally expensive. Hence, we propose an alternative solution by adaptively aggregating the voxels regressed positions, considering the informative nature of borders [23 (85). For each voxel Vi,,A E Vels. we have the predicted offsets Ai.j.M associating with 6 voxels, which are extracted as instance-level voxel set Vs, where E Aij,e and |Aij.e/=d Then, each voxel Vi,j. will query the semantics from the associated voxel set V attentively:
exp(W, ViJ.s) (W& Vo /de) Vij.A Norm( Σ Exes.,, exp (W, V...AK) (W& V /vde) dEAig.
W. V. V.J.)).
(6)
Here, &. E Val is refined feature, each W (-) is linear layer, da is the channel number, and Norm Group Normalizat ization. With NV = 4 aggregation layers, each voxel is able to gain sufficient perception of the whole instance, enhancing instance-level semantics. This design is justified in Fig. 5 Finally, Val is sent simple classification head to generate class predictions.
Classification Loss. dense detection (56). classification is optimized as segmentation in a per-pixel dense manner. Hence, to optimize the classification branch in our VoxDet, we naturally deploy the with binary and semantic settings. We set the consistent loss weight as 1.0 for convenience.
dense semantic prediction loss following the previous SSC arts [83 79 82
(7) where fue is the cross-entropy loss weighted by class frequencies, and Lbi and Lam are affinity loss
Las Lee + Cbin Csem

---

Table 1: Results of SSC on SemanticKITTI [3] hidden test set. * is from 82 T is using extra 21
temporal information. The best and the second best results are in bold and underlined, respectively.
Adeewalk
parking
other grd..
building
heyyde
mobeeryll
other- webor-sehh
vegetiation
bicy bicyolist
sign
road
mukk
Eonco
pole
Method
MonoSomee
6
34.16
11.08
54.70
27.1
10
24.80
5.70
14.40
18.80
3.30
0.50
0.70
4.40
14,90
2,40
19.50
1,00
1.40
0:40
11,10
3.30
2.10
TPVFormer
34.25
55.10
14.80
1.00
2.30
13:90
20.40
1,30
0.30
1.50
SuroundOee
34.72
56.90
10.20
18.90
OseFormer
34.53
55.90
31.50
6.50
1.50
3.30
3.90
2.20
0.20
11.90
3.80
3.70
JAMSSC
67
43.24
12.57
54.00
25.50
24.70
0.90
19:20
21.30
3.80
1.10
0.00
350
22.70
8.00
19.40
1.50
2.90
0.50
11.90
5.30
4.10
VoxFormer
43.05
12.20
53.90
25.30
21.10
5.60
19.80
20.80
3.50
1.00
0.70
5.70
22
40
7.50
21
1
2.60
0.20
11.10
5.10
4.90
VoxFarmer
E4
43.21
13.43
54.10
26.90
25.1
7.30
23.
50
21.70
3.60
1.90
1400
4.10
24
40
8.10
24
24.20
140
1.10
0:00
13.10
6.60
5.70
DepbhSCC
76
44.58
13.11
55.64
27.25
25.72
5.78
46
21.94
3.74
1.35
0.88
4.17
23.37
7.64
21
56
1.34
2.79
0.28
12.94
5.87
6.23
Symphoeiz:
42.19
8.00
54.60
23.80
21.10
5.50
HASSC 61
42.87
14.38
55.30
19.60
25.90
11.30
23.
23.00
2.90
990
1.30
4.90
24
30
9.80
26, 50
1,40
3.00
0.00
14.30
7,00
7.10
StreesSeee
43.34
15.36
61 90
31.20
10.70
10.70
24.
22,80
2.80
3:40
2400
6.10
23 50
8400
27.00
2.90
2.20
0.50
16.50
7.00
7.20
F12
bd
44.20
13.72
56.40
28.00
26.50
4.90
80
23.40
4.50
0.80
0,90
4.30
24.50
9.10
23
1.20
2.50
0.10
13.30
6.40
6.30
112GFomer
43.52
14,50
57.90
30.40
30.00
24.00
23.70
5.20
0.60
20
5:00
25.20
10.70
25 80
1
0:00
14.60
7.50
9.30
Monotica
83
40
2.20
25.00
0.20
6.40
CGFormer
[v
64.30
24.20
10
26.10
3.70
2.70
3.60
9.30
HTC1.
44.23
17.02
12:40
27.90
70
5.40
25
10.50
1.10
3.10
0.90
21.10
9.00
8.30
Voxtbet (Ours)
47.27
18.47
64.70
35,50
34,80
14.40
28.10 26.90
5.90
28.70
13,00
31.70
3.10
4,00
1,30
21.50
10.10
10.30
Table 2: Results of SSC on C-CBench-KKTTISSE test set." is the corrected results using the consistent
dataset version
18
The best and second best results are are in bold and underlined, respectively.
bigelee
motoreyele
track
other veh.
Road
parking
sitewalk
ofheeggdd.
building
fence
Vegebation
femain
pole
traf. traf-sign
other
other
Method
IoU
mloU
LiDAR-bared mechous
SSCNet 65]
553.58
16.95
31.95
0.00
0.17
10.29
0.00
0.07
65.70
17.33
41.24
3.22
44.41
6.77
43.72
28.87
0.78
0.75
8.69
0.67
LMSCNei
47:35
13.65
20.91
0.00
0.00
0.26
0.58
62.95
13.51
33.51
0.20
43.67
0.33
40.01
26.80
0.00
3.63
0.00
Comen--buved methools
MonoSeene
0
137.87
12.31
19.34
0.43
0.58
8.02
2.03
0.36
48.35
11.38
25
3.32
32.89
3.53
26.15
16.75
6.92
5.67
4.20
3.09
TPVFamer
28
40.22
13.64
21.56
1.09
1.37
2.57
2.38
52.99
11.99
31.07
3.78
34,83
4.80
30.08
17.52
7.46
5.86
5.48
2.50
OceFomer
40.27
13.81
22.58
0.26
54,30
13.44
4.50
31.00
19.51
6.95
VoxFommr
14
38.76
11.91
17.84
1.16
0.89
4.56
2.06
1.63
47.01
9.67
21.21
2.89
31.18
4.97
28.99
14.69
6.51
6.92
3.79
2.43
JAMSSC 67
41.80
12.97
18.53
2.45
1.76
5.12
3.92
47:55
10.56
28
4.12
31,53
6.28
29.17
15.24
8.29
7.01
6.35
4.19
DepthSSC
40.85
14.28
21.90
50.88
37.33
5:22
7.71
Symphonics
13.41
17.82
26.36
1.21
1.90
14,20
7.76
6.57
57
13.58
35.24
4.57
39.20
7.95
34.33
19.19
11.01
15.78
8.23
6.01
SGN-S 4
46.22 17.71
28.20
2.09
3,02
11.95
3.68
4.20
59.49
14.50
36.53
4.24
39.79
7.14
36.61
23.10
14.66
16.14
8.24
4.95
SGN-T 44
47.06
18:25
3.43
2:00
5,20
4.49
9.93
CGFormer
45.07
20.05
29,85
3.96
6.79
63,85
17.15
42.73
8.22
80
24.94
17
10.18
6.77
stiFormer
[X
46.35
18.
10.30-10
27.00
0.91
2.55
10.73
5.67
4.28
61.04
04
13
37.00
5.07
43.05
7.46
38.98
24.87
15.75
16.90
8.85
5.35
VoxDet
48.59
21.40
29.92
5.13
8.36
19,13
8.04
7.84
62.83
18.99
10.10
5.58
44.47
10.62
39.03
26.16
18.19
20.78
11.66
8.34
4.4
Optimization
To train the proposed VoxDet, we implement the whole optimization objective written as follows,
LNxxDet = Lets + Lreg + X
(8)
where Lots is dense classification loss (Eq. 7 and Leeg is dense regression loss (Eq. 5). Following the baseline [83], we retain the voxel-centric segmentation loss (cross-entropy and affinity terms) applied to V as an auxiliary prior constraint (Caur) before instance-centric learning. which can stabilize the optimization. This also enhances the multiple-run robustness, which is justified in the Appendix. The weighting factor 0.2 empirically adjusted balance the model learning at the instance level.
Experiments
5.1 Comparison with State-of-the-art Methods
Camaera-based SSC. Tab. D shows the comparison on the hidden test set of SemanticKITTI [
VoxDet achieves new records with an IoU of 47.27 and mIoU of 18.47. Compared with the methods using additional temporal labels like VoxFormer-T 34,, HASSC-T 611, H2GFormer-T (64]. and
HTCL-T 224) VoxDet comprehensively surpasses them and gives noticeable 3.04 and 1.38 gains
on IoU and mloU over the previous best entry 244). We further list the comparison on SSCBench-
KITTI-360 (36] in Tab. 2 VoxDet achieves an IoU of 48.59 and mloU of 2 21.40, outperforming all
7

---

Table Results of LiDAR-base based SSC on SemanticKITTI hidden test set with single frame (no extra temporal information) for fair comparison. Our VoxDet only uses point cloud input. The best and the second best results are bold and underlined, respectively. Previous SoTA
57
motoreyelist
road
sidewalk
purking
other-gma
bailding
tucck
bicyole
motorcyek
other-veh.
vegetation
tormin
person
bieyelstt
fenoe
pole
traf. Aigni
Method
IoU
mloU
SSCNel
29.8
9.5
27.6
17.0
15.6
6.0
20.9
10.4
1.8
0.0
0.0
25.8
18.2
0.0
14.4
7.9
3.7
SSCNet-full
50.0
16.1
51.2
30.8
27.1
34.5
24.3
1.2
0.5
0.8
4.3
35.3
18.2
29.0
0.3
0.3
19.9
13.1
6.7
TS3D 15
29.8
9.5
28.0
17.0
15.7
4.9
23.2
10.7
2.4
0.0
0.0
0.2
24.7
12.5
18.3
0.0
0.1
0.0
13.2
7.0
3.5
TS3D/DNet
25.0
10.2
27.5
18.5
18.9
6.6
22.1
8.0
2.2
0.1
0.0
4.0
19.5
12.9
20.2
2.3
0.6
0.0
15.8
7.6
7.0
LMSCNet 50
55.3
17.0
64.0
33.1
24,9
3.2
38.7
29.5
2.5
0.0
0.0
0.1
40.5
19.0
30.8
0.0
0.0
0.0
20.5
15.7
0.5
LMSCNat-SS
56.7
17.6
64.8
34.7
29.0
4.6
38.1
30.9
1.5
0.0
0.0
0.8
41.3
19.9
32.1
U.D
0.0
0.0
20.5
15.7
0.8
Local-DIFs
49
57.7
22.7
67.9
42.9
40.1
11.4
40.4
34,8
4,4
3.6
2.4
4.8
42.2
26.5
39.1
2.5
1.1
0.0
29.0
21.3
17.5
JS3C-Net 69
56.6
23.8
64.7
39.9
34,9
14.1
39.4
33.3
72
14.4
88
12.7
43.1
19.6
40.5
8.0
5.1
30.4
18.9
15.9
SSA-SC 77
58.8
23.5
72.2
43.7
37.4
10.9
43.6
36.5
5.7
13.9
4.6
7.4
43.5
25.6
41.8
4.4
2.6
0.7
30.7
14.5
6.9
L2COcc-D 60
45.3
18.1
68
36.9
34.6
25.8
28.3
4.5
4.9
3,3
7.2
26.2
32
2.4
21.6
9.6
9.5
L2COce-L
60
60.3
23.3
68.5
40.6
33.2
6.1
41.5
36.8
5.4
8.7
4.1
9.0
42.6
28.7
36,9
2.9
10
27.7
27.0
21.9
VPNet 577
60.4
25.0
72.4
44.3
40.5
44.0
37.2
4.3
14.0
9.8
8.2
45.3
2.0
32.7
17.1
8.8
VoxDet-L. (Ours)
63.0
26.0
73.0
43.6
37.5
10.3
44.5
37.7
6.6
9.9
6.2
11.
45.9
30.7
43.5
2.7
3.2
13
34,0
27.8
23.7
Table 4: Ablation study each key module.
Table 5: Further analysis of varied designs.
TDP REG
loU (%%
mloUi (%)
Naarum (M)
Detailed designs
IoU (56)
mloU
CLS
REG
CLSS
A
guidance + Self attention
46.82
18.15
TDP
42.71
Eq,
47.01
45.42
16.42
48.9
bq
T+ Average pooling
47.18
18.49
46.79
17.85
49.4
DelCouv + Coav
46.92
18.08
TPV ResBlock
47.02
18.32
(c)
47.08
18.27
51.1
Task Task-deoupled + Task-sbared
46.81
17.88
/
47.36
18.73
5288
model
47.36
18.73
Figure Justifying the aggre- gation design using voxels at re- gressed positions Va. By mod- ulating regressed offsets A with scaling factor a for each voxel V Vij.. we select the vox- els V, based on modulated off- sets A = Aijjk, where in- creasing a will select voxels at larger distances and vice versa.
10.0
minii
18.7
10.0
D<a<l
18.5
[a] Within the local
neighborhoood
28.4
18.3
18.2
18.
18.0
1000
existing camera-based methods. Notably, VoxDet achieves the best on all instance-related classes, showing an impressive capacity in understanding outdoor agents.
5.2 Quantitative Study
Extension to LiDAR-base SSC. In Tab. we deploy VoxDet with LiDAR input and compare it with the hidden test set SemanticKITTI. Our VoxDet-L. achieves the new record with 63.0 loU and 26.0 mloU, significantly surpassing state-of-the-art methods (60 [57] Our method achieves the best on large-scale objects like building, vegetation, middle-scale objects like car, and small-scale objects such pole, traffic-sign. Notably, with effective regression-inclus VoxDet-L achieves a significantly higher loU of 63.0 without using extra labels/datw/temporal information/models. ranking 1st on the CodaLab leaderboard (until May 22). This capability is crucial for autonomous navigation to prevent collisions with obstacles.
SSC formulation,
Ablation Study. Tab, presents the ablation study, highlighting the following insightful observations. Line (b): Compared with the baseline, introducing the REG branch in TDP gives a significant 2.71 IoU gains owing to the better instance-level perception. Interestingly, the limited 0. 0.14 mloU shows that the REG does not contribute to the semantic discrimability, aligning with our motivation. Line (c): After deploying the CLS with instance-level aggregation, significant 1.43 mloU gain can be

---

Table 6: Efficiency comparison with SoTA methods.
Table 7: Results with monocular depth.
Method
Npaan
Tur
(S)
miool (5)
Method
JoU (%)
mioU (5)
OceFormer
(ccov12s]
214
36.42
13.50
VoxFomer
38.68
10.67
StereoScene
25 [UCAA194
117
258
43.85
15.43
VoxFormer-T
58.08
11.27
CGFommer
122
205
45,99
16.89
Symphonize
22
38.37
12.20
SGFommer
R CVVPR22
126
16.68
OceFormer
X2
36.50
13.46
ScanSSC
CVVW222
145
261
45,95
17.12
CGFormer 29
41.82
14.06
VoxDet (Ours)
53
18.73
VoxDet (Ours)
43.92
16.35
REG
23G
415
Figure 6: Visualization of the decoupled feature for classification (CLS) and regression (REG).
observed, revealing the importance of instance guidance in voxel prediction. Line (d-f: By gradually decoupling the volume, both metrics improve progressively, eliminating the task misalignment.
Different SSC Formulations. We provide detailed analysis of each SSC formulation and make a comparison in Tab. 8 (a) Segmentation (previous SSC works): Each voxel only predicts its class. This paradigm uniformly treats all vox- els of the same class as single class group and entirely agnostic instance-level geometry,
(a) Seg.
(b) FCOS Det.
(c) VoxDet.
IoU mloU
42.71
45.42
47.36
16.28
16.42
18.73
Table 8: Different SSC formulations.
such as length. Hence, it risks incorrectly estimating object instance scales, such as extremely long cars. (b) Naive FCOS- FCOS-based detection: Each voxel regresses the borders of objects and predicts the class simultancously [56]. By capturing instance- geometry, this can use the regressed object instances to represent the scene with multiple instance groups. (c) Our VoxDer: Each voxel first regresses the borders of object instances, aggregates instance-level context, and predicts the class. Compared with (b), we use regressed offsets to enhance class prediction with instance aggregation.
Design Variations. In Tab. 5 we delve into each module with different design variants. For TDP, replacing the A-guided instance-level aggregation (Eq. 66) with learned deformable self-attention and soft-weights both decreases the performance, revealing the necessity of explicit instance guidance. The A guidance and attentive design may relieve the negative influence of regression outliers far from instances, consistent with Fig. [ In SVE, we observe a consistent decline by replacing dense projection, DefConv, TPV, and decoupling with average pooling, Conv, ResBlock, and task-shared designs, respectively. This can verify that our spatial decouulcu in the TPV space is optimal.
Model Efficiency. In Tab. 6 compare with state-of-the- works in parameters (Nparam) ) and inference time (nn Vox uses significantly fewer parameters (M) and less inference time (ms), setting new records all IoU metrics, highlighting the effectiveness of our new SSC formulation.
Comparison with Monocular Depth. In Tab. 7 we make a comparison with state-of-the-art methods by using monocular depth (5). Our VoxDet also achieves the best results on both metrics, surpassing the previous best entry with 2.10 IoU and 2.29 mloU, showing significant robustness on depth.
Exploring Instance-level Aggregation. Fig. 5 carefully analyzes our aggregation designs (Eq, 6). Instead of directly aggregating regressed voxels Va with A. we modulate by scaling factor a E R. and aggregate at the scaled offset A aA, revealing three observations. (1) Local aggregation does not work (a 0). When a approaches zero, performance degrades noticeably. Naively aggregating neighborhoods yields no benefit, as it degenerates to convolutional kernels with local receptive fields. (2) Wear-boundary aggregation helps (a 1). As grows toward 1.0, performance steadily improves, indicating that the boundary contributes more informative signals for dense perception. Notably, 1.0, likely by suppressing outliers outside instances. (3) Outside-instance aggregation hurts (a > 1). When a exceeds 1.0 substantially, performance declines sharply, revealing the negative impact of aggregating voxels beyond the instance extent.
= 0.8 slightly outperforms our default of

---

RGB frcus
Ground
Cofoener (S07A))
Vastbe (Dhnn)
Figure 7: Qualitative comparison with state-of-the-ard method
77]. Zoom in for better view.
5.3 Qualitative Results
Disentangled Volume. In Fig. 6 we visualize the decoupled feature volume (Eq.3) at Bird's Eye View (BEV) view for the classification (CLS) and regression (REG), revealing the following three insightful observations. (1) Different from CLS, focusing semantically informative local regions, REG naturally seeks discover the more complete boundary of instances. This capacity critical in SSC, as completing the scene is the key focus. (2) Although REG has not been trained with semantic supervision, can discover informative cues of potential objects (4th sample) with orthogonal effects with CLS. (3) We can see the four borders of the BEV map are activated, as the 3D volume borders are unified boundaries of adjacent large-scale instances, like vegetation and buildings.
Semantic Scene Completion. Fig. visuallly compares our VoxDet with saae--of-the-ne method [79]. VoxDet shows noticeable gains geometric completion, such as complete car borders sample better infers instance-level semantics, such the truck in sample 2, and can correctly detect the challenging yet important traffic sign (3nd sample). This mainly owing to our instance-centric formulation, which enhances spatial perception and semantic understanding of object instances. Notably, in the 4th sample, VoxDet has built-in safeguards (size-based filtering) that mitigate the wrong-label concern, making SSC supervision more informative and safer (see Appx. B).
6 Conclusion
In this work, we first reveal an ever-overlooked free lunch SSC labels: the voxel-level class label has implicitly provided the instance-level insight thanks to its occlusion-free nature. Then, propose VoxNT, a simple yet effective algorithm that freely converts class labels to the instance-level offsets. Based on this, we propose VoxDet, new detection-basse SSC. Vox first spatially decouples 3D volumes to generate task-specific features, learning spatial deformations in the tri-perceptive space. Next, it adopts task-decoupled predictor to generate instance-aware predictions guided by the regressed 4D offset field. Extensive experiments reveal lot of insightful phenomena for the following works and verify the state-of-the-a role of VoxDet.
formulation for instance-centn
7 Discussion
Safety Clarification. While our method effectively discovers informative geometric cues for naturally separated instances (e.g., traffic signs, poles, and trunks), acknowledge that our assumption may not hold in crowded or highly dense scenes. In cases where instances are naturally merged in ground -ruth labels (such as overlapping tree crowns), our method may fall back to conventional SSC methods by treating these single class, without instance-level distinction. This may limit our performance gains complex perception scenarios and could affect downstream tasks that heavily rely on precise object separation. For such challenging cases, we recommend incorporating more reliable instance-level labels (either from human annotation or large foundational models) to provide more accurate supervision and further enhance the safety and reliability of downstream applications.
10

---

Acknowledgment
We would like to express our gratitude to Wentao Pan, Reyhaneh Hosseininejad, Yasamin Borhani, Yuanfan Zheng, and Hengyu Liu for their insightful discussions, which contributed to the improvement of this work. Additionally, gratefully acknowledge the financial support provided by Valeo.
References
[1] Alexandre Alahi, Kratarth Goel, Vignesh Ramanathan, Alexandre Robiequet, Li Fei-Fei, and Silvio Savarese. Social Istm: Human trajectory prediction in crowded spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 961-971, 2016.
[2] Jongseong Bae, Junwoo Ha, and Ha Young Kim. Three cars approaching within 100m! enhanc- ing distant geometry by tri-axis voxel scanning for camera-baue maased semantic scene completion. arXiv preprint arXivv2411. 16129, 2024.
[3] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Jürgen Gall. Semantickitti: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9297-9307,
2019.
[4] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and
Jurgen Gall. Semantickitti: A dataset for semantic scene understanding of lidar sequences, In Proceedings of the IEEE/CVF international conference computer vision, pages 9297-9307,
2019.
[5] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
4018, 2021.
[6] Anh-Quan Cao and Raoul de Charette. Monoscene: Monocular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3981-3991 2022.
[7] Anh-Quan Cao, Angela Dai, and Raoul De Charette. Pasco: Urban 3d panoptic scene completion with uncertainty awareness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 14554-14564, 2024.
[8] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Proceedings of the European Conference on Computer Vision, pages 213-229, 2020.
[9] Yuhua Chen. Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. Domain adaptive faster r-cnn for object detection in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3339-3348, 2018.
[10] Zehui Chen, Chenhongyi Yang. Qiaofei Li, Feng Zhao, Zheng-Jun Zha, and Feng Wu. Disen- tangle your dense object detector. In Proceedings of the 29th ACM international conference on multimedia, pages 4939 4948, 2021.
[11] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In Proceedings of the IEEE international conference on 2017
computer vision, pages 764-773, 2017.
[12] Haoyi Duan, Hong-Xing Yu, Sirui Chen, Li Fei-Fei, and Jiajun Wu. Worldscore: A unified evaluation benchmark for world generation. arXiv preprint arXiv:2504.0098. 2025.
[13] Olivier Faugeras. Three-dimensional computer vision: a geometric viewpoint. MIT press, 1993. [14] Lan Feng. Yang Gao, Eloi Zablocki, Quanyi Li, Wuyang Li, Sichao Liu, Matthieu Cord, and Alexandre Alahi. Rap: 3d rasterization augmented end-to-end planning. arXiv preprint arXiv:2510.044 043333, 2025.
[15] Martin Garbade, Yuch-Tung Chen, Johann Sawatzky, and Juergen Gall. Two stream 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0-0, 2019.
[16] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. 2012 IEEE conference on computer vision and puttern recognition, pages 1354-3361. IEEE, 2012.
[17] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440-1448, 2015.
[18] Xiyue Guo, Jiarui Hu, Junjie Hu, Hujun Bao, and Guofeng Zhang. Sgformer: Satellite- ground fusion for 3d semantic scene completion. In Proceedings of the IEFF/CVF international conference computer vision, 2025.
11

---

[19] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVP Conference on Computer Vision and Puttern Recognition, pages 16000-16009, 2022.
[20]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 770-778, 2016.
[21] Yuanhui Huang. Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective
view for vision-based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9223-9232, 2023.
[22] Haoyi Jiang. Tianheng Cheng. Naiyu Gao, Haoyang Zhang, Wenyu Liu, and Xinggang Wang. Symphonize 3d semantic scene completion with contextual instance queries. Proceedings of the IEEE/CVF' Conference on Computer Vision and Pattern Recognition, 2024.
[23] Hei Law and Jia Deng. Cornemet: Detecting objects as paired keypoints. In Proceedings of the European conference on computer vision (ECCV), pages 734-750, 2018.
[24] Bohan Li, Jiajun Deng, Wenyao Zhang, Zhujin Liang, Dalong Du, Xin Jin, and Wenjun Zeng. Hierarchical temporal context learning for camera-based semantic scene completion. In European Conference on Computer Vision, 2024.
[25] Bohan Li, Yasheng Sun, Xin Jin. Wenjun Zeng, Zheng Zhu, Xiaoefeng Wang, Yunpeng Zhang, James Okae, Hang Xiao, and Dalong Du Stereoscene: Bev-assisted stereo matching empowers
3d semantic scene completion. arXiv preprint arXiv:2303. 13959, 2023.
[26] Bowen Li. Jiashun Wang, Yaoyu Hu, Chen Wang, and Sebastian Scherer. Voxdet: Voxel learning for novel instance detection. Advances in Neural Information Processing Systems, 16:10604-1062,, 2023.
[27] Pengteng Li, Pinhao Song, Wuyang Li, Weiyu Guo. Huizai Yao, Yijie Xu, Dugang Liu, and Hui Xiong. See&trek: Training-fre spatial prompting for multimodal large language model. arXiv preprint arXiv:2500. 02.50966087... /6087, 2025.
[28] Wuyang Li. Zhen Chen, Baopu Li, Dingwen Zhang, and Yixuan Yuan. Htd: Heterogencous task decoupling for two-stage object detection. IEEE Transactions on Image Processing, 30:9456-9469, 2021.
[29] Wuyang Li, Xinyu Liu, Xiwen Yao, and Yixuan Yuan. Scan: Cross domain object detection with semantic conditioned adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022.
[30] Wuyang Li, Xinyu Liu, and Yixuan Yuan. Sigma: Semantic-complete graph matching for domain adaptive object detection. In Proceedings of the IEEE/CVÉ conference on computer pages [31] Wuyang Li, Wentao Pan, Po-Chien Luan, Yang Gao, and Alexandre Alahi. Stable video infinity: Infinite-length video generation with error recycling. arXiv preprint (rXiv:2510.092)
vision and pattern recognition, pages 5291-5300, 2022.
2025. [32] Yanwei Li. Yilun Chen, Xiaojuan Qi, Zeming Li, Jian Sun, and Jiaya Jia. Unifying voxel- based representation with transformer for 3d object detection. Advances in Neural Information Processing Systems, 35:1 15:18442-188 18455, 2022.
[33] Yiming Li, Sihang Li, Xinhao Liu, Moonjun Gong, Kenan Li, Nuo Chen, Zijun Wang. Zhiheng
Li, Tao Jiang, Fisher Yu, Yue Wang, Hang Zhao, Zhiding Yu, and Chen Feng. Sscbench: A large-scale 3d semantic scene completion benchmark for autonomous driving. 2024.
[34] Yiming Li, Zhiding Yu, Christopher B. Choy, Chaowei Xiao, José M. Álvarez, Sanja Fidler, Chen Feng, and Anima Anandkumar. Voxformer: Sparse voxel transformer for camera-based 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9087-9098, 2023.
[35] Li Liang, Naveed Akhtar, Jordan Vice, Xiangrui Kong, and Ajmal Saeed Mian. Skip mamba diffusion for monocular 3d semantic scene completion. arXiv preprint arXiv:2501. 5006000000....... 2025. [36] Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 3292-3310, 2022. [37] Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 329-33332.-3300 2022.
[38] Tsung- Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 2117-2125, 2017.
[39] Tsung-Yt Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980-2988, 2017.
12

---

[40] Enyu Liu, En Yu, Sijia Chen, and Wenbing Tao. Disentangling instance and scene contexts for
3d semantic scene completion. arXiv preprint arXiv:2507.0853 2025.
[41] Parker Liu, Chenxin Li, Zhengxin Li, Yipeng Wu, Wuyang Li, Zhiqin Yang. Zhenyuan Zhang, Yunlong Lin, Sirui Han, and Brandon Y Feng. Ir3d-bench: Evaluating rision-language model scene understanding as agentic inverse rendering. arXiv preprint arXiv:2506.22 23329, 2025.
[42] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint
arXiv:1711.05101. 2017.
[43] Wenyu Lv. Shangliang Xu, Yian Zhao, Guanzhong Wang, Jinman Wei, Cheng Cui, Yuning
Du, Qingqing Dang, and Yi Liu. Detrs beat yolos on real-time object detection. arXiv preprint arXiv:2304. 08069, 2023.
[44] Jianbiao Mci. Yu Yang, Mengmeng Wang, Junyu Zhu, Xiangrui Zhao, Jongwon Ra, Laijian
Li. and Yong Liu. Camera-based 3d semantic scene completion with sparse guidance network. arXiv preprint mrxiv:23/2.057555 2023.
[45] Zhenxing Ming, Julie Stephany Berrio, Mao Shan, and Stewart Worrall, Inverse++: Vision- centric 3d semantic occupancy prediction assisted with 3d object detection. arXiv preprint SSt::2504.04332 2025.
[46] Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Proceedings of the European Conference on Computer Vision, pages 194-210. 210. 2020.
[47] Han Qiu, Yuchen Ma, Zeming Li, Songtao Liu, and Jian Sun. Borderdet: Border feature for dense object detection. In European Conference on Computer Vision, pages 549-564, Springer, 2020.
[48] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-ting time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6); 137-1149, 2016.
[49] Christoph B Rist, David Emmerichs, Markus Enzweiler, and Dariu M Gavrila. Semantic scene completion using local deep implicit functions on lidar data. IEEE transactions on pattern analysis and machine intelligence, 44( (4(10):2005-72
2021.
[50] Luis Roldão, Raoul de Charette, and Anne Cerroust-Blondee Lmsenet: Lightweight multiscale
3d semantic completion. In Proceedings of the International Conference on 3D Vision, pages 111-119, 2020.
[51] Luis Roldao, Raoul De Charette, and Anne Verroust- Blonndtt 3d semantic scene completion: A
survey. International Journal of Computer Vision, 130(8) 1978-90099 2022.
[52] Faranak Shamsafar, Samuel Woerz, Ratia Rahim, and Andreas Zell. Mobilestereonet: Towards lightweight deep networks for stereo matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2417-2426, 2022.
[53] Qianqian Shen, Yunhan Zhao, Nahyun Kwon, Jeeeun Kim, Yanan Li, and Shu Kong. Solving instance detection from open-world perspective. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 9901-9910, 2025.
[54] Guanglu Song, Yu Liu, and Xiaogang Wang. Revisiting the sibling head in object detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11572, 2020.
[55] Shuran Song, Fisher Yu, Andy Zeng, Angel X. Chang, Manolis Savva, and Thomas A. Funkhouser. Semantic scene completion from a single depth image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 190-198, 2017.
[56] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Feos: Fully convolutional one-stage object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9627-9636, 2019.
[57] Lubo Wang. Di Lin, Kairui Yang, Ruonan Liu, Qing Guo, Wuyuan Xie, Miaohui Wang. Lingyu Liang, Yi Wang, and Ping Li. Voxel proposal network via multi-frane knowledge distillation for semantic scene completion. Advances in Neural Information Processing Systems, 37:101096-1011 115, 2024.
[58] Meng Wang, Huilong Pi. Ruihui Li. Yunchuan Qin, Zhuo Tang, and Kenli Li. Vlscene: Vision- language guidance distillation for camera-based based 3d semantic scene completion. In Proceedings the AAA/ Conference on Artificial Intelligence, volume 39, pages 7808-7816, 2025.
[59] Ning Wang. Yang Gao, Hao Chen, Peng Wang. Zhi Tian, Chunhua Shen, and Yanning Zhang. Nas-fcos: Fast neural architecture search for object detection. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1 11943-11 11951, 2020.
[60] Ruoyu Wang, Yukai Ma, Yi Yao, Sheng Tao, Haoang Li, Zongzhi Zhu, Yong Liu, and Xingxing Zuo. L2coce: Lightweight camera-centric semantic scene completion via distillation of lidar model. arXiv preprint arXiv: 2025.
13

---

[61] Song Wang, Jiawei Yu, Wentong Li, Wenyu Liu, Xiaolu Liu, Junbo Chen, and Jianke Zhu. Not all voxels are equal: Hardness-aware semantic scene completion with self-distillation. arXiv preprini arXiv:2404. 1958, 2024.
[62] Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. Fcos3d: Fully convolutional one-stage monocular 3d object detection. In Proceedings of the IEEF/CVF international conference on computer vision, pages 913-922, 2021.
[63] Yue Wang. Vitor Guizilini, Tianyuan Zhang, Yilun Wang, Hang Zhao., and Justin M. Solomon. Detrôd: 3d object detection from multi-view images via 3d-to-2d queries. In Conference on
Robot Learning, pages 180-191, 2021.
[64] Yu Wang and Chao Tong. H2gformer: Horizontal-to-glaan gloobl voxel transformer for 3d semantic scene completion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages
5722-5730, 2024.
[65] Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundoce:
Multi-camera 3d occupancy prediction for autonomous driving. In Proceedings of IEEF/CVF
International Conference on Computer Vision, pages 1729-21740, 2023.
[66] Zhaoyang Xia, Youquan Liu, Xin Li, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, and Yu
Qiao. Sepnet: Semantic scene completion on point cloud. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17642-17651, 2023.
[67]
Haihong Xiao, Hongbin Xu, Wenxiong Kang, and Yuqiong Li. Instance-aware monocular 3d
semantic scene completion. IEEE Transactions on Intelligent Transportation Systems, 2024.
[68] Dongli Xu, Jinhong Deng, and Wen Li. Revisiting ap loss for dense object detection: Adaptive ranking pair selection. In Proceedings of the IEEE/CVF conference on computer vision and pattem recognition. pages 14187-14196, 2022.
[69] Xu Yan, Jiantao Gao, Jie Li, Ruimao Zhang, Zhen Li, Rui Huang, and Shuguang Cui. Sparse single sweep lidar point cloud segmentation via learning contextual shape priors from scene completion. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 3101-3109, 2021.
[70] Zhiqiang Yan. Jianhao Jiao, Zhengxue Wang, and Gim Hee Lee. Event-driven dynamic scene depth completion. arXiv preprint arXiv:2505. 13279,, 2025.
[71] Zhiqiang Yan, Xiang Li. Le Hui, Zhenyu Zhang, Jun Li, and Jian Yang. Rignet++: Semantic assisted repetitive image guided network for depth completion. International Journal of Computer Vision, pages 1-23, 2025.
[72] Zhiqiang Yan, Kun Wang, Xiang Li, Zhenyu Zhang, Jun Li, and Jian Yang. Rignet: Repetitive image guided network for depth completion. In European Conference on Computer Vision, pages 214-230. Springer, 2022.
[73] Chenhongyi Yang. Mateusz Ochal, Amos Storkey, and Elliot J Crowley. rediittionguided distillation for dense object detection. In European conference on computer vision, pages 123-138. Springer, 2022.
[74] Xuemeng Yang, Hao Zou, Xin Kong, Tianxin Huang, Yong Liu, Wanlong Li, Feng Wen, and Hongbo Zhang. Semantic segmentation-assinn scene completion for lidar point clouds. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages
3555-3562. IBBE, 2021.
[75] Zhen Yang. Yanpeng Dong. Heng Wang, Lichao Ma, Zijian Cui, Qi Liu, and Haoran Pei. Daoce:
3d object detection assisted multi-sensor fusion for 3d occupancy prediction. arXiv preprint
24Xi 19972, 2024.
[76] Jiawei Yao and Jusheng Zhang. Depthssc: Depth-spatial alignment and dynamic voxel resolution
for monocular 3d semantic scene completion. arXiv preprint 2r 17044, 2023.
[77] Zhu Yu, Bowen Pang, Lizhe Liu, Runmin Zhang, Qihao Peng, Maochun Luo, Sheng Yang. Mingxia Chen, Si- Yuan Cao, and Hui-Liang Shen. Language driven occupancy prediction.
arXiv preprint arXiv:24/1. 166622,,.. 2024.
[78] Zhu Yu, Zehua Sheng, Zili Zhou, Lun Luo, Yuaa Cao, Hong Gu, Huaqi Zhang, and Hui- Lui-Liang Shen. Aggregating feature point cloud for depth completion. In Proceedings of the IEFF/CVF international conference on computer vision, pages 8732-8743, 2023.
[79] Zhu Yu, Runmin Zhang, Jiacheng Ying, Junchen Yu, Xiaohai Hu, Lun Luo, Si- Yuan Cao, and Hui-liang Shen. Context and geometry aware voxel transformer for semantic scene completion. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[80] Haoyang Zhang. Ying Wang. Feras Dayoub, and Niko Sunderhauf. Varifocalnet: An iou-aware dense object detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8514 8523, 2021.
[81] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Li. Bridging the gap between mnchor-based and anchor-free detection via adaptive training sample selection. In Proceedings
14

---

of the IEEE/CVF conference computer vision and pattern recognition, pages 9759-9768, 2020.
[82] Yunpeng Zhang, Zheng Zhu, and Dalong Du Oceformer: Dual-path transformer for vision- based 3d semantic occupancy prediction. Proceedings of the IEEE/CVF Conference Com- puter Vision and Pattern Recognition, pages 9433-9443, 2023.
[83] Yupeng Zheng, Xiang Li, Pengfei Li, Yuhang Zheng, Bu Jin, Chengliang Zhong, Xiaoxiao Long, Hao Zhao, and Qichao Zhang. Monooce: Digging into monocular semantic occupancy prediction. arXiv preprint arXiv:2403.0877
2024.
[84] Zhaohui Zheng. Rongguang Ye, Ping Wang. Dongwei Ren, Wangmeng Zuo, Qibin Hou, and
Ming-Ming Cheng. Localization distillation for dense object detection. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9407-9416, 2022.
[85] Xingyi Zhou, Jiacheng Zhuo, and Philipp Krahenbuhl. Bottom-up object detection by grouping extreme and center points. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 850-859, 2019.
[86] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXi::0010004159,
2020.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction sections offer comprehensive discussion the context, intuition, insight, ambitions, and contributions of the manuscript.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes. The limitations of this work have been comprehensively discussed in the Appendix due to the page limit of the main paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: For each theoretical result, the paper provides the full set of assumptions and a complete (and correct) justification.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: In the Appendix, have presented the implementation details and the experi- mental setups with corresponding reproducible credentials. Besides, we have submitted the raw evaluation results from the CodaLab online platform of the hidden set. We ensure the code and models will be open-source.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
17

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: All data used in this study are drawn from publicly accessible platforms and employed following existing works. The cleaned codebase and trained models, including a comprehensive README, have been released publicly.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
hengliee/guides//ddeeubbmm...s Sot.eesuuubsssioooooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ctspps.ccpuulli//uuiee guwdes/ CodeeSumiiiseeeeee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: the Appendix, we have presented the implementation and experimental details with corresponding reprodueible credentials. Besides, have submitted the raw evaluation results from the Codal.ab online platform of the hidden test set.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The results contain the standard deviation of the results over several random runs. We have presented the experiments in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The details credentials.
of experiments are presented with corresponding reproducible
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttps///eeerrsss..//bbbbb/iiiia
Answer: [Yes]
Justification:
The research conducted in the paper conforms with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper has discussed the border impacts in the Appendix.
19

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This study does not involve any generative models that pose a risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets, including data, code, and models, are credited to their original owners, with their licenses and usage terms clearly specified and fully honored.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
20

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All newly introduced assets are comprehensively documented and supplied alongside the existing resources.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This study does include human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
21

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurips hett::///eerrrps.cccc/ooeeereeee cc///onfereeeee//222////... for what should or should not be described.
22

---

This Appendix aims to clarify novelty, practical applications, border impacts, experimental justifica- tions, and future directions, which are organized follows.
Appendix A Additional quamtitative results
More comprehensive comparison of camera-based SSC
Robust analysis with multiple runs
Qualified as new, powerful, robust, lightweight, and efficient baseline for SSC
Results on SemanticKI1TI validation set
How to define object instances
A generalized definition works best, which is different from the 2D intuition
Sensitivity analysis on hyperparameters
More comparison model efficiency
Appendix B
More insights and practical usages of VoxNT trick
Freely understand instance scales
Ability to freely identify wrong labels
Freely eliminate wrong labels in training
Rethink the evaluation on dynamic objects
Appendix C
Detailed experimental setups
Datasets and evaluation metrics
Implementation details
Algorithmic details the VoxNT trick
Appendix D
Discussion
Difference with detection-assisted SSC works
Broader impacts
Limitations and future works
Ethical claims
Appendix Additional qualitative results
Failure case analysis
More qualitative comparison
A Additional Quantitative Results
A.1 More Comparison of Camera-based SSC
In Tab. 9 we make a more comprehensive comparison on the SemanticKITTI hidden test set. For fair comparison, also report the performance using the same weight (3.0) of cross-entropy loss with the latest concurrent work [60) replacing our default (1.0), denoted as Vox Based on ResNet-50, method surpasses all the methods using our method achieves at least the best or the second best in all classes, showing superior effectiveness.
advanced EfficientNet backbone. Additionally,
A.2 Robustness Analysis of Multiple Runs
In Fig. g we report the results of multiple runs (5 times independent experiments) to justify the robustness of our VoxDet. The curves summarize the IoU and mloU results for each epoch on the KITT validation set. To better illustrate the difference, also demonstrate the results of the previous state-of-the-ar method, CGFormer [79], using the officially released training log.
It can be observed that our VoxDet achieves impressive robustness with multiple runs, giving very similar performance on the IoU and mloU in different runs. Note that there some tradeoff between IoU and mloU metrics, i.e., some runs achieve slightly higher IoU while sacrificing a little mloU, Additionally, our method achieves visually significant gains over CGFormer on both the robustness and loU/mloU metrics, highlighting the strength Vox Hence, due to the superior effectiveness, robustness, and efficiency, we believe that VoxDet qualified to serve as a powerful, lightweight, and efficient baseline model for the following works.
23

---

Table 9: Results of Camera-based SSC on SemanticKITTI [ hidden test set. T is using extra temporal information. The best and the second best results are in bold and underlined, respectively. R-50 indicates ResNet-50, and Eff-B7 is the stronger EfficientNet-B7 BB backbone with more parameters. f indicates setting the same weight (3.0) on the cross-entropy loss as the recent work [60].
obhergmml
Porkin
beilding
mnck
1800
1eace
Method
Arch.
JoU
MonuSceze
16
34.16
11.08
56.70
10
25.3
3.30
5:10
14.40
15.80
3:30
0.50
0.70
4.40
14.90
2.40
19.50
1.00
1.40
0.40
11.10
3.30
2.30
TPVForme
31
ER--87
34.25
11:26
55.10
27 20
27.40
6.50
14.30
19.20
3.70
1.00
0.50
2.30
13.90
2.601
20.40
1.10
2.40
0.30
11.00
2,50
1.50
Surowadtive
[33
E1:87
34.72
11.86
56.50
28.30
30.20
6.80
15.20
20.60
1.40
1.50
1.20
4.401
14.90
3.401
19.30
4.10
2,00
0.30
11.30
3,50
2.40
Oc:Fomez
82
EH--8
54.53
12.92
55.00
90.3 50
31.5
50
6.50
15.70
21.60
1.20
1.50
1.20
4.20
16.80
3.90
21.30
2.20
1.10
0.20
11:5
3,80
3.70
AMSSSC 6:
R:50
43.74
12.37
54.00
25.5 50
24.70
5.90
19.20
21.30
3.30
1
0
3.90
22
70
5.80
19.00
1.50
2.50
0.50
11.90
5.30
4.10
VexFomer
[3
R:30
12.95
12
20
53.90
25.30
21.10
5.60
19.50
20,80
3.50
1.00
0.70
3.70
22
40
7.50
21.30
1.40
2.60
0.30
11.10
5.10
4,90
VodFomerr
33
R-50
43.21
13.41
54.30
26.90
25.
7.30
23.50
21.70
3
1.90
L.0
4.10
24.40
8.10
24.20
1.60
1.10
0.00
13.30
6.60
5.70
DepeSSC
[6
R-50
44.58
11
55.64
27.35
25.72
5.78
20.46
21.04
3.74
3.35
0.95
17
23
7.64
21.56
34
2.79
0.28
12.04
5.87
6.33
Symphonine
R-50
42.19
15.04
40
29 30
26.00
11.70
24.70
23.60
3,30
3:60
2.60
5.00
24 20
10.00
23.10
3
1.90
2800
16.10
7.70
8.00
[ASSC 6]
R::0
43.60
13.34
36.00
27.70
23
6.20
22:80
1.50
L.00
23.
23.30
1.60
4.00
0.30
13.10
5.80
5.50
HASSC
25.90
14.30
7,00
SterecSeene
43.34
31.20
16.50
7,00
HDGiKamerr
44.20
13.50
43.52
14:00
MenaDee
CGFommer
EI:-87
44.41
3.70
70
29.30
3.60
18.70
8,70
1.20ss5C
RI--7
25.10
3.50
30.10
2.40
0.30
9.10
HTCI. (4]
RI--87
44.23
17.09
64.40
27.30
5.70
1.90
2.20
25
10.80
31.20
1.10
3.10
0.50
31.10
9.00
8.30
VorDet (Ours)
R-50
47.22
18:47
64.70
35.40
24 50
14.40
28.10
26.90 90
6.00
5,90
5.10
5.00
26 70
13,00
30
1.10
6.00
1.00
50
10.10
10:30
VorDet' (Ours)
5.40
29.50
IoU Comparison on Semantic KITTI Vol set
mioui Comparison on Semantic KITTI Val set
0.16
3
Det Irun]]
Voalee (un))
Det trun])
0.12
(unk)
Veabee juav
Dea frund)
Vedbee (uun))
CCPorner
Cofarmer
10000
35000
20000
25236
20020
Figure 8: Robustness analysis of our VoxDet with multiple runs. We report the per-epoch validation results. The results from the official training log (not our reproduced results) given by the previous state-of-the-ar method, CGFomrer (79), are also listed for comparison. We can observe visually significant improvements in both performance and robustness of our method.
A.3 How to Define Object Instances?
Table 10: Analysis of object definitions by remov- ing instance-level regression for specific classes.
In Tab. T0 we explore different definitions of ob- jects by removing the instance-level regression for specific classes, including (1) background, including road, sidewalk, etc (2) empty class. We empirically find that a generalized definition of objects, i.e., all the semantic classes, performs best. This is different from the intuition in the 2D image domain, which only considers objects with well-defined shapes.
Setting
loU
mIoU
w/o, regress background
46.78
18.19
w/o. regress empty
47.08
18.05
Full model
47.36
18.73
A.4 Results on SemantickITT Validation Set
Tab. II presents the comparison on the SemanticKITTI validation set. Our VoxDet achieves the best results of 47.36 IoU and 18.73 mloU, surpassing the second-best counterpart (4 with a noticeable 1.60 mloU gains. Compared with the previous state-of-the-ee 16.87 mloU, our VoxDet gives a 1.37 IoU and 1. 1.86 mloU gains, verifying the superior effectiveness. Besides, VoxDet achieves the best and second-best results in 17 of the 19 classes, which shows its effectiveness. Specifically, our method performs well on object instances, achieving the best on building, car, truck, motorcycle, traffic sign, etc, revealing the superior instance-centrie learning.
method (79 with 45.99 IoU and
24

---

Table 11: Quantitative results SemanticKITTI B
validation set. T indicates using extra temporal information. The best and the second best results are in bold and underlined, respectively.
59300
Mestrol
151:41
Scone
16.96
16.90
76-10
14.35
0.46
14.00
25-26
6:95
1.46
17.85
19.64
1.86
1.50
0.00
5.64
4.14
2:35
TPPTomm
15.01
15.00
30.50
0.55
13.88
73.81
8.08
11.66
0.05
6.15
16.93
7.76
33.38
0.51
0.88
0.00
5.64
114
1.57
Occfeemee
13:00
15.
76.18
19:51
0.31
01.11
8:53
19.03
3.95
13.03
0.24
3.87
0.00
561
Wechemer
46.00
54:75
5.
17.05
0199
5.06
75.85
3.31
0.00
766
1:18
Wochemer
13
66.3 15
13:55
53.57
70-17
19.59
0:43
19.50
76
1.36
1.78
0.56
7:81
76.10
6.10
11.05
199
140
0.00
7-1
6.15
6:04
Syrphuninc
56.17
37.18
15.78
1.58
20.04
11:99
15.23
6.00
13.87
1-5
1.34
0.00
46.37
16.04
19.12
17.55
19.75
10.00
54.43
1.51
1.85
0.00
733
4.6
1DGhemme
A1
45.00
18:29
$1.00
19:17
21.14
0.34
2051
78-11
6.90
1196
8.9
9.2
17.44
7.80
36.25
1.15
6.10
0.00
748
083
5.1
SGN
43.00
16:12
1846
B44
0.8
6.12
34:1
123
1/8
0.00
576
13.09
16.07
12:11
30.92
271
26.93
K35
12.34
035
418
0.00
9.30
24
15.51
12.13
53.70
12:45
15.17
25.15
50.30
20.72
3.9
2.00
11.0
35.95
8.19
31.73
355
1.30
0.00
11:22
1140
8.55
Table 12: Analysis on instance- level aggregation layers N.
Table 13: Analysis on the loss Table 14: Analysis on the loss
weight X deployed on Laux
weight Aeg deployed on LReg
N
loU
mloU
A
IoU
mioU
Avea
loU
mloU
1
47.06
18.2
0.1
47.15
18.53
47.19
46
47.38
18.32
0.2
47.36
18.73
1.0
47.36
18.73
47.34
18.54
0.4
47.43
18.69
1.5
47.48
18.66 68.68
4
47.36
18.73
0.8
47.28
18.59
47.55
5
47.39
18.79
1.0
47.39
18.42
2.5
47.68
18.55
A.5
Sensitivity Analysis
In this section, we further give a detailed analysis of the hyperparameters used in VoxDet. The experiments conducted with the same random seed ensure fair comparison. The results reported on the SemanticKITTI validation set as the unified SSC practice.
Number of Instance-level Aggregation Layers. Tab. 12 reports the performance changes the number of aggregation layers N. Performance improves steadily with increasing N, confirming the benefit of more effective aggregation. Although extending to five layers (one more than our default N = 4) yields a slight additional gain, the marginal improvement will lead to extra computational overhead. We therefore adopt N 4
vary
default setting.
Weight of Auxiliary Loss. Tab. 13 presents the sensitivity of our model to the auxiliary segmentation loss weight X in Caux. which governs the strength of voxel-level supervision. Overall, mloU remains relatively stable across wide range of values: as we increase X, performance steadily improves, peaking at À = 0.2, and then gradually declines for larger weights. This trend indicates that moderate auxiliary loss provides beneficial guidance for voxel-wise feature learning, while an overly large weight interferes with the subsequent instance-centric optimization. Consequently, adopt 0.2 as our default setting, as it yields the best mloU.
Loss weight of Regression. In Tab. 14 we add a loss weight on the regression Areg and explore the effect of varying the loss weight. moderate increase in Xreg yields further IoU gains, likely due to improved fitting of instance contours. However, excessively large weights cause slight mloU drop, probably from conflicts among the combined loss terms. We therefore set Areg. 1.0 to balance the effect and eliminate the term in the main paper for convenience.
A.6 Model Efficiency
Tab 15
presents a comprehensive comparison between our VoxDet and current state-of-the-ard SSC methods on the SemanticKITTI test set, evaluating model size, inference speed, and SSC performance. VoxDet requires the fewest learnable parameters, achieves the fastest inference time, and attains the highest per-class IoU as well as the best overall mloU. Compared to its strongest competitor (2) our approach slashes the parameter count, accelerates inference to real-time performance, and delivers a notable gain in mloU. These results underscore the strength of our effective SSC formulation, giving a lightweight yet powerful framework that simultancously advances speed, efficiency, and accuracy.
25

---

Table 15: Comparison of model efficiency and accuracy with SoTA on SemanticKITTI test set.
Venue Method
CVPR'
CVPR'S 23 VexFormer
CVPR'
ICCV
DCAT
NeurlPS' CGFammer
CVPR'
TPVFomerr
21
3+
Symphomie
22
Octivemer
Sterechouse
25
P0
ScanSSC
VoxDet
Param (M) 1
107
59
59
214
117
122
145
53
Inf. Time (mg)
207
214
216
199
258
205
361
159
1oU 1
34.25
42.95
42.19
34,53
43.34
44.41
44.54
47.27
12.20
16.63
18.47
X-axis scale distribution (log scale)
normalized aleedd density
Small width (K-axis] for pole
Large width [Xaxis) road
num, vosels in direction linstance length)
Y-axis scale distribution (log scale)
norma lized denuity
Large length (Y-axis) for road
leneth (x-sxis for pole
num. voxels in direction linstance length)
Z-axis scale distribution (log scale)
norme ized derutt
Small height (-axes) for road
Large height [2 for pale
num, vaxels in direction (instance length)
Figure 9: Instance-level scale distribution in X, Y, Z axis given by our voxel-to-instance trick. We randomly sample voxels in all classes and calculate the instance scale (I) with coupled offset terms (positive and negative directions), e.g., 1" = 8"" + 8" in X X-xxis. The horizontal axis represents the scale (the number of voxels) respectively. The vertical represents the number of samples, which is peak normalized to [0, 1] by dividing the maximum number of each class for better view. Zoom in for better view.
log scale. The scale of the whole scene in X, Y, Z is 256, 256, 32
B
Voxel-to-Instance (VoxNT) Trick
B.1
VoxNT Trick Can Freely Understand Instance Scales
In Fig. 9 conduct statistical analysis on the instance scale along different axes based on the proposed Voxel-to-Instanee (VoxNT) Trick. To better visualize the seale patterns, we visualize the distribution for different classes (in different colors), highlighting their specific scale patterns. The resolution of the whole scene is 256 x 256 x 32. To better explore the instance-level geometries, we sum up the positive and negative directions of the 4D offset field A represent the scale information in different axes. This process is denoted as follows,
Fs 8
8"
f 89
+ 8
(9)
8z*
+ 8z
26

---

X-axis scale distribution
Y-axis scale distribution
Z-axis scale distribution
Twe Scale
Scoie Absonoaliiss
Abnormaliies
Daraaly
Lsngthie
Loorto C
LandloTd
Longthes
Num vovels (lenghth)
Num vosels (lenght))
Num vowels (lenghen)
The distribution of dynamic object given VoxNT trick.
LengehoIcn
Lengthed
Zoom in
Zoom
Lengthes
Groundicruth
(b) Illustration of the two scale abnormalities of dynamic object Car.
Figure 10: More observations from VoxNT. (a) Scale distribution (same as App. B of the car along the X, Y, axis. We can see two types of abnormality: minimal and large lengths. (b) Visualizing the abnormality with ground-truth. The left shows the isolated voxels leading to the minimal-scale abnormality. The right shows the long-length abnormality caused by the traces of moving objects.
Here {8** .8 .8u+ 8" ,8**, _5 indicates the offset in the six directions. Surprisingly, we find that A is able to freely give sufficient instance-level cues beyond the spatially agnostic semantic labels: the essential scale information. Besides, the geometric scale distribution different classes also demonstrates different unique patterns, greatly aligning with the real-world environment. The following are intuitive observations.
(1) For tall instances, such as pole in the light yellow color, the scales tend to give small values in X and Y axes (usually lower than six voxel width/length). and provide large scale on the axis (e.g.. larger than 10-voxel height). This is aligned with the comment scene as the width and length of these things are small, while the height is significant.
(2) For the flat items, such as mad the pink color, we can see a significant value in X and Y axes (usually larger than 30 voxel width/length). while giving small scale the Z axis (smaller than five voxels), which also aligns with our intuition.
(3) For some large objects, such as vegetation in the green color, we can see that the scale in all three axes can be relatively significant, aligning with the real-world environment.
Hence, these observations further highlight the value of the proposed VoxNT trick, which can convey extensive valuable information in geometry the instance level. Note that this information is not available in the original voxel-level class labels.
B.2 VoxNT Trick Can Freely Identify Wrong Labels
Observe the Scale Abnormality of Dynamic Objects with VoxNT Trick, We first study the car class the most prevalent and dynamic object category autonomous driving. Fig. 10 (a)
Cars the key target and dominate the evaluation of detection benchmarks
9 62
27

---

demonstrates the distribution of the instance scale along different axes, generated by our VoxNT trick. Given the whole scale of 256 x 256 x 32 in the scene, we find that the scale of the car is problematic: In the X and Y axis, there are lots of samples with tiny scale with length less than 5 and huge scale with larger than 100. In the Z axis, many samples have length less than 5. Note that the typical scale of a car is less than 30 x 20 x 10, indicating that the ground-truth has significant abnormality.
Delve into the Abnormality with Label Visualization. To study the essential reason, we visualize the ground-truth voxels in Fig. (b). We can ground the observed issues mentioned above according to the visualized ground truth, which is voxelized from point clouds. In the left sub- nub-figure, we find that the reason for the samples with abnormally small length is isolated voxels. In the right sub-figure, we can see that the failed filter object dynamies lead abnormally large lengths.
B.3 VoxNT Trick Can Freely Eliminate the Influence of Wrong Labels in Training
We can see that the scale information in our 4D offset field has valuable property for solving this issue, which, for the first time, makes the abnormality identification realistic. In this section, we discuss some tricky uses of the proposed VoxNT trick to inspire the follow-up works. Note that these operations improve a more reasonable prediction but cannot improve the mIoU evaluation, because the ground-truth truth is noisy (see App. B.4.. In brief, the key idea to use scale information to identify the voxel with an offset that is too large or too small. Specifically, to identify the impact of the isolated voxels, we can use a binary voxel-Ievel mask M {0, 13 XXXxXxx x2 to identifyy the voxels that have abnormally small offsets in all axes:
App.B.4..
1100,
(Ei <min) n (1i..A otherwise,
K min) n (....A
K Kmin)
(10)
1.0,
where Kiin E N2t is a scale threshold for the three axes for measuring the removed minimal scale and can empirically set to Similarly, to filter out the huge-scale samples, e.g., the car in Fig. TO similar mask can be deployed M E {0, 1]xxxxxx x2 to identify the voxels with abnormally large offsets appearing in one of the axes. This can be written as follows,
1.0, 0.0,
(7 2 Km U (P 2 KH U ( 2 K otherwise,
M
(11)
where Kmn is the threshold filtering out the large lengths. Another class-based mask can also be adopted Mmax, with class labels Y.,j,k and desired class K. for the wrong label filtering, such as car. b.j.k
By using these two types of masks, we can clearly identify and localize those wrong voxels with the specific positions (i. k) in the volumes. To remove their influence on training, it is possible to directly refine the ground truth labels by ignoring these voxels with a straightforward yet effective label transformation. In practice, it can be implemented as the following equations,
255,
(M = 1) U (M min = 1) n (Yij,. = Kear); A.j.A
Yreffind
Keary
(Miix = 0) n (Main = 0) n (Yi,j. = Kear); );
(12)
Visjks
otherwise.
Here, the key idea is to set the label as ignored (255) if the scale of the car voxel is too large or too small. This can effectively remove lots of wrong predictions in the dynamic car.
B.4 Rethink the Evaluation on Dynamic Objects
Based on the observation about wrong labels, we further delve into the SSC evaluation on the dynamic object car. In Fig. [1] we visualize the ground-truth of the sample from SemanticKITTI (3] validation (Left), the corresponding prediction from the faate-o--th-aae method [79 (Middle), and the prediction from our method (Right). We can see that previous works are prone to overfitting these wrong labels while generating higher results on the loU metric for the car. Differently, our method gives a more reasonable prediction. However, this advantage cannot be demonstrated by using the conventional IoU metric due to the wrong labels. We hope this observation can inspire the following works to notice and address this issue, thereby pushing forward the community.
28

---

Wrong labels
Wrong predictions with high IoU
Correct predictions with a low Iol
Ground-truth
CGFormer
Illustration of the problems in the existing evaluation metrics, Left: The ground-truth wrongly labels the car with sequential afterimage. Middle: Previous works wrongly these errors by predicting long car. Right: Differently, our VoxDet can give more reasonable predictions without fitting the error, but the car IoU in this sample is worse due the wrong labels.
Figure 11:
C Experimental Settings
C.1 Datasets and Metries
Benchmark Setting. Following the unified practice in SSC, we evaluate our approach on two benchmarks: SemanticKITTI and SSCBench-KII -ITTCch300 33 which are derived from the original KITTI Odometry [16 and KITTI-360 [37 datasets, respectively. In all experiments, we follow the unified setup [34 79 22 restricting to the frustum of size 51.2 m in the forward direction, 25. laterally, and 6.4 m vertically above the sensor. This volume is voxelized into grid of size 256 x 256 x 32, with each voxel measuring 0.2 m on a side. The dataset details are as follows.
(1) SemanticKITTTi consists of 22 sequences (00-21) of LiDAR scans and synchronized stereo images. We adopt the standard split of 10 sequences for training (00-07, 09-10), one sequence for validation (08), and 11 sequences for online evaluation on the hidden test server (1 (112)).. Input images are provided at 1226 370 resolution, and ground-truth occupancy grids are annotated with 20 labels (19 semantic classes + empty class).
(2) SSCBench-KII -ITTCch- is recently released extension that re-labels subset of the KITTI-360 sequences for SSC. It comprises sequences in total, of which 7 are used for training, for validation, and held out for final testing. RGB images captured 1408 x 376 resolution, and each voxel is annotated with one of 19 labels (18 semantic categories + free-space class).
Evaluation Metrics. We assess performance along two complementary axes, i.c., geometry com- pletion and semantic completion, using the Intersection over Union (IoU) and mean Intersection over Union (mloU) metrics, for occupied voxel grids and voxel-wise semantic predictions. All reported results follow the evaluation protocols SSC, including the online test-server evaluations
for SemanticKITTI [4) ensuring fair comparison with prior methods 34 21 79).
C.2 Implementation Details
Backbone Network. Following the main stream of SSC works 33
83
we use ResNet-50 (0 as
the 2D image feature extractor. We follow the unified SSC settings in the recent 2-year publications for the depth estimation and use MobileStereoNet (52) and Adabins [5] as the depth estimators. Note that the compared state-of-the-an methods use the same depth images for the fair comparison.
View Transformation. In the main paper, the 2D-to-3D view transformation follows [79] 2]. Specifically, given the input image we first extract 2D image feature F2D and depth map Z. Then, we adopt a depth refinement module that takes the 2D feature F2D and depth as input. first estimates the depth distribution via LSS [46), and then generates voxel queries Vq XxXx2xx Here, (X, Y, Z) is spatial resolution 128 x 128 x 16, and = 128 is the channel.
Based on this, the pixel (u, v) can be transformed to the 3D point (x. y, 2) using the camera intrinsic matrix (E TR. 4) and extrinsic matrix (E 1x44), termed the query proposals Q in the projected voxel space. Finally, the 3D deformable cross-attention is deployed to query the information from 2D
29

---

image to the 3D voxel space. The number of deformable cross attention layers is 3, and the number of sampling points around each reference point is set 8. This can generate the 3D feature volume with dimensions of 128 x 128 x 16 and 128 channels, which is the V in the main paper. For fair comparison, these operations view transformation are the same as previous works [79] 2 60
Kindly refer to [79] for more details.
VoxDet. The number of instance-driven aggregation layers N is set 4. The loss weight terms of and 8 are empirically set to 1.0 and 0.2, respectively. For the task-shared voxel encoder in our spatially- decoupled voxel encoder, we directly use the encoder part of the conventional 3D UNet in other SSC works [79]. The regression branch is simply deployed as Conv + GroupNorm - ReLU - Conv, with the last convolution outputting 6 channels. Code and models will be released.
Model Training. We train our VexDet with a batch size of 4 using AdamW 442 optimizer. Follow- ing 79 the cosine annealing schedule is adopted, with the first 5% iterations of warm-up, maximum learning rate of 3 x 10-4, weight decay of 0.01 and B1 = 0.9, 32 = 0.99, The experiments are conducted on 2 NVIDIA A100 GPUs (40G) with two samples for each GPU. The final prediction has dimensions of 128 x 128 x 16, which is upsampled to 256 x 256 x 32 through trilinear interpolation to align with the ground truth. We use the VoxNT trick to remove the wrong labels as Eq. 12 with the scale threshold set to 30 for each axis. The efficiency experiments are conducted on a NVIDIA 4090 (commercial GPU). considering the more practical deployment property. The training requires around 19 GB of memory per sample, with about 9.0 training hours for SemanticKITTI and 18 hours for BSCBeench-KITTISE which very friendly for the research groups with commercial GPUs.
Implementing VoxDet with LiDAR Input. Vex is designed on the lifted 3D volumes, which can be effortlessly transferred to LiDAR-based pipeline. To further analyze this flexibility, we deploy a LiDAR-based VoxDet, denoted as VoxDet-L. We achieve this by replacing the 2D-to-3D lifting with a simple point cloud encoder. This can directly generate the 3D feature volume V with 3D ResNet-50, using LiDAR point cloud as input. The 2D image encoder, depth estimator, and view transformation are removed. All the implementations only use a single model without multi-frame distillation and only single frame input without temporal information, which will be open-sourced.
C.3 Algorithmic details of the VoxNT Trick
We present the detailed process of our VoxNT Trick in [ with Py PyTorch-style pseudo code. By calling the function compute all direction distancess))))) this algorithm aims to generate the ground-truth of 4D offset field Á only using the vor-voxel class labels Y E xxxxx2 (gt_occ). In Y, each item satisfies 0- Y.j,A < K with K representing the number of classes.
In brief, for every scanning direction d E "=".""".. "=" zero-valued matrix R E N x Then, as we iteratively traverse the voxels along the chosen direction d, we compare consecutive voxels. Given the two adjacent voxels V; and Vi+ suppose the class label of them matches with each other: Y, = Y++1 considering the scanning direction, the corresponding entry in R is incremented by one; otherwise, the accumulation halts once a class change is detected, revealing approaching the instance boundary.
we begin by initializing a
This trick efficiently captures the spatial continuity of object instances and provides reliable ground truth offsets for regression. Considering the symmetrical property between the two scanning di- rections (forward and backward), we deploy a flip operation along the selected axis for efficient implementation. Based on this, we can generate the 4D tensor saving these relative distances in 6 directions, where each dimension is then normalized into [0, 1] divided by the volume size X, Y. Z accordingly, Thus, we can obtain a normalized offset field Á ground-truit trunt from semantic labels.
D Discussion
D.1 Difference with Related Works
Occupancy prediction assisted with 3D object detection. Some existing works have attempted to assist the voxel prediction with 3D object detection (5). Although they consider the instance-level representation, our Vox is essentially different from these works in the following aspects.
30

---

RGB Input
Ground-truth
(Ours)
Figure 12: Failure cases of the our VoxDet. is difficult for our method detect objects satisfactorily and correctly at extremely far distances due to limited semantic cues on the RGB input images.
(1) Extra-label Free. Different from these works that require extra instance-level bounding box labels, VoxDet does not need any handerafted bounding box labels for the object instances thanks to the proposed VoxNT trick, which significantly reduces the labeling labor with more practical usage.
(2) Extension to Image and LiDAR-base Pipelines. Only using SSC labels, VoxDet can be extended to both camera-based and LiDAR A--AR-sasee settings in unified pipeline, achieving state-of-the-are results on both benchmarks. This differs from most existing works, which are tailored only for a single type of input modality. Hence, our method has more substantial flexibility and transferability.
(3) Flexible Definition of Object Instances. These works 44 75 highly rely on the specific definition of instances in the detection datasets, which usually only considers some primary objects like cars and humans. However, extending to more profound class set, such as buildings, is challenging and almost impossible, making this instance-level perception un-extensible. Differently, our VoxNT can handle the instances in arbitrary classes, providing an extremely flexible definition for object instances. For example, we can discover the length, width, and height of buildings, vegetation, etc (see Fig. 99, which is highly valuable for autonomous navigation.
(4) Technical Difference. Existing works 45 75 introduce a separate 3D-object-detection branch to support occupancy prediction. In contrast, we consolidate both tasks into a single, detection- driven formulation with ultra-lightweight design, which can avoid the potential conflict between occupancy prediction and 3D object detection. Our feature decoupled designs also avoided the task- misalignment issue, which ignored in existing works. The proposed regsessegresioon design, i.e.. each voxel regresses the distance to the instance boundary, is a different paradigm for instance- perception, which implemented purely with convolutional layers, eliminating the complexity and computational overhead of abject-query-basse attention.
D.2 Broader Impacts
Impacts to the Broader Occupancy Community. Our VoxNT and VoxDet can be directly used in any voxel-included settings, such as multi-view occupancy prediction, as it is based similar voxel representation. By freely transferring semantic voxel labels into instance- offset labels, this work may also inspire the occupancy community reconsider the instance-level perception (e.g.. instance and panoptic segmentation) achieved using only semantic labels, potentially contributing to training scale-up with practical label usage.
Impacts to the 3D Point Cloud Community. As voxelization is key procedure in point-cloud understanding, our method, fully based on voxel representation, has great potential for the point cloud community. We achieve state-of-the-art results on the LiDAR-based SemanticKITTI benchmark. which justifies our potential impact on the point cloud community.
Impacts to the 3D Object Detection Community, Our VoxNT bridges the gap between dense occupancy and sparse 3D object detection. It converts semantically rich voxel-wise labels (e.g.. car, building, vegetation) into instance-level offset labels that capture precise object extents and class-aware geometric priors. By integrating these instance labels into the occupancy, 3D detectors can revisit their image-driven origins, i.e., detecting objects directly from 2D inputs, while evolving toward voxel-based instance detection paradigm that more faithfully reflects the physical 3D world.
31

---

D.3 Limitations and Future Work
Generate Sparse 3D Bounding Box Visualization. The current Vex uses instance-level supervi- sion to guide the dense SSC tasks, which can not directly generate spare 3D bounding boxes. The reason that dense detection [56 39 23]. the Mon-Maximum Suppression (NMS) required to remove low-quality boxes for final visualizations. However, there is a gap between the 2D pixels and 3D voxels. To solve this, we will develop a NMS algorithm tailored for voxels, bridging the gap between the voxel-level occupancy prediction and instance-level 3D object detection.
Lack the Usage of Extra Temporal Information. The current VoxDet based on single-frame input. While state-of-the-ars the performance can be further improved with extra temporal information [24] which can correct the cross-frame inconsistency. This will be our future work.
Implementation on Multi- View Pipelines. The current VoxDet evaluated with a single camera input, which may limit its application scope. Our method can be transferred to the multi-vies settings effortlessly, as is deployed on the 3D feature volume, which is also the representation for multi-view occupancy prediction. This will be our future work.
Rely on the Depth Prediction Accuracy. Although VexDet achieves state-of-the-art performance, empirically find that it will also suffer from some false-negative predictions of the objects at long distance (see App. D The sub-optimal depth estimation may be the reason. In the main paper, we also find that replacing the stereo depth with monocular depth leads to some performance drop, highlighting the reliance on depth accuracy. Therefore, improving the depth models in the future may further enhance the performance and serve our future work.
D.4
Ethical Claims
Our VoxDet uses only publicly available datasets and produces 3D semantic volumes without retaining any personally identifiable information. We do not foresee any significant negative impacts: the method does not enable individual tracking or intrusive surveillance, poses no additional privacy or safety risks beyond standard camera perception systems, and does not rely sensitive demographic attributes. By focusing on high-level scene understanding for benign applications such as autonomous navigation, approach raises no known ethical, fairness, or regulatory concerns.
E
Additional Qualitative Results
E.1
Failure Cases
we present some failure cases generated by the proposed VoxDet. We observe that method fails to give correct detection for the object at extremely far distances (top sample), especially when objects of similar color are highly overlapped in the 21) images (bottom sample). This may be caused by the minimal visual cues on the 2D images, limiting the capacity. Some potential solutions may be (1) deploying
In Fig. 12
powerful visual feature extractors tailored for high-quality dense perception; (2) introducing extra modality, such as LiDAR, to enhance the information in the far distance; (3) developing tailored algorithms refining the features of objects in the far distance.
E.2 More Comparison with Other Methods
In Fig. 13 and 14 we provide additional qualitative comparisons against the state-of-the-ar meth- ods CGFormer [79 and OccFormer 82
Our approach exhibits visually superior instance-level completeness, greater environmental consistency, and enhanced semantic perception.
32

---

Algorithm PyTorch Style Pseudocode toxcl-to-Instance (VoxNT) Trick.
def compute_a _atputteellll__ae all_ddrection, direaceion_isstnnces (gt_occ):
B, X, Y, z "F_occ.shapp
dist_x_pos run_length_alen
weiggghhhllnngeeeeeeeeee alwoaaldooonaeeee dim(gt_oce, 1, "positive")
dist_x_neg
1, "negative")
dist___pos run_length_ anngg
dim(gt_oca 2, "positive")
dist_y_neg run_length__nd alla__llonnndddal aaloon_di((g")) dim(gt_oce, 3, "positive")
lit_gtocc, 2, "negative")
dist___pos run
dist_z_neg run runglength___ along dim dim(gt_occ, 3, "negative")
Stack into shape (B, 6, X, Y, 2)
distances = torch.stack([
dist_x_pos, dist_x_neg,
dist_y-pos, dist.y_neg,
dist_z.pos, dist_z_neg dim=1)
return distances
def www.lennthhalog..iiiie dim, direction):
if direction == "positive"; return www_length_ppesiiveel dim)
else:
# flip, compute positive, then flip back tf torch. torch. flip(t, dims=(dim,)) of run_ length.posittveettf. dim) return forch.flip(ee dims- (dim,))
def run welennth_positvvette dim): shape shape shape [dim]
torch. emptee dtype=torch. int32)
Initialize the last slice along dim to 1 last [slice(None)) len(shpe))
idx last [dim] = out (tdxle(idx_lastt
torch. tensor (1, dtype=torch.int.2. devicest.eevice
for range(L 2, 1, 1):
idx = [None] len (shape); idx [dim] idx_next = [slice (None)] len(shape); idx_next [dim] i + 1
t[tuple(idx)] nxt [ttpleeiix
# Check whether the next voxel is in the same class same (current == axt)
next out [utplel (tdxxuup((ddx_eext))
out val torch. where(
same, out_next 1,
torch.tensor tensor dtype=torch.int32) int32, devicest.device)
out [tuple(idx)] out_val
return out
33

---

Ground -truhh
OccFormer
CGFormer
VoxDet (Ours)
Figure 13: More qualitative comparisons on SemanticKITTI validation set.
34

---

Ground-truth
OccFormer
CGFormer
VoxDet (Ours)
Figure 14: More qualitative comparisons on SemanticKITTI validation set.
35


Reviews:
[{"review_id": "XrF5bZnlGX", "reviewer": "Reviewer_ai3H", "summary": "This paper formulates semantic scene completion (SSC) as a dense object detection task, incorporating offset regression and semantic prediction to address 3D semantic scene completion. The spatially-decoupled voxel encoder and task-decoupled dense predictor are introduced to effectively model task-specific spatial deformation and estimate the 4D offset field. Experimental comparisons demonstrate that the proposed method achieves superior performance compared to other recent SSC methods.", "questions": "Tiny weaknesses:\n\n1.\tThe authors should provide more references to support the assumption “Although prior works, such as Symphonize [16]”\n2.\tUsing (a), (b), (c), and (d) to distinguish the four examples in Figure 2, rather than simply 'left' or 'right,' may be better.\n3.\tConfusing underlines. For example, “Voxel-level SSC as instance-centric object Detection” in Lines 62-63 of the main manuscript.\n4.\tThe authors should increase the font size of the text in the figures.\n\n\nThe paper reformulates the SSC as a dense object detection task, without sufficient analyses and explanations on the motivations. The missing technical details make the main manuscript hard to follow. Additionally, the authors provide more comparisons and deeper analyses to differentiate their proposed method from previous approaches, which formulate SSC as a dense segmentation task. Therefore, I vote for borderline reject in this review round. I would like to raise the rating if the authors can address my concerns.", "limitations": "Yes. The authors have discussed limitations in the supplemental material.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Paper Strengths:\n\nThe main insight of this work is the reformulation of SSC. The authors utilize two modules (e.g., spatially-decoupled voxel encoder and task-decoupled dense predictor) to achieve effective semantic scene completion, leading to improved SSC performance.\n\n\nPaper Weaknesses:\n\n1.\tMy first concern is the motivation and high-level summary of the reformulation of SSC. The authors reformulate the SSC as a dense object detection, which is different from previous works. However, as demonstrated in Figure 1(b) and technical details (e.g., classification loss) in Section 4.3 Task-decoupled Dense Predictor, the proposed method appears to be similar to previous segmentation-based methods. The authors should provide more analysis to differentiate between these two types of approaches.\n\n2.\tMy second concern is the technical details. Conv_{(T)}^{(P)} is a 2D convolution module for place (P) and task (T). How many convolutional layers are involved in Conv_{(T)}^{(P)}? Do the different convolution modules share weights or not? It is confusing how the authors utilize these convolution modules to enable effective decoupling. Additionally, the authors utilize FPN to generate V_{cls} and V_{reg}, do the FPNs for these two different tasks have the same network structure and share weights?", "was_revised": true, "final_justification": "The author has addressed my concerns and concerns of other reviewers, and I have updated my score.", "rebuttal": "We sincerely thank you for the careful review, and sorry for the confusion. We would be grateful if these clarifications prompt a reconsideration of our score, and we are actively available for any follow-up concerns during the author-reviewer discussion period.\n\n## [W1] Clarify the motivation and reformulation\n\n**Motivation.** VoxDet reformulates SSC from per-voxel segmentation (where each voxel only predicts its class based on itself) into an improved dense object detection paradigm, i.e., each voxel first regresses object borders, then uses this information to guide instance-level aggregation, and finally predicts class labels. The fundamental strength of our method is that each voxel can perceive whole object instances and aggregate instance-level geometry (context) to make a reliable class prediction, instead of relying solely on limited, independent local voxels.\n\n**Reformulation.** We provide a detailed analysis of each formulation to clarify its differences:\n\n- **[Formulation 1] Segmentation (previous SSC works): Each voxel only predicts its class.** This paradigm uniformly treats all voxels of the same class as a **single class group**, and is **entirely agnostic to instance-level geometry**, such as length, scale, and height. Hence, it risks incorrectly estimating object instance scales, such as extremely long cars, as evidenced by the qualitative results in Figs. 6 and 7 of the Appendix.\n\n- **[Formulation 2] Our vanilla dense object detection: Each voxel regresses the borders of object instances and predicts the class simultaneously [#1].** Using the regressed object instances, this paradigm can represent the scene with **multiple instance groups** by capturing instance-level geometry. This differs from the segmentation formulation, which treats all voxels of a class as a **single class group** and is agnostic to instance geometry. Although the classification branch is optimized in a segmentation-like manner [#1], this formulation is fundamentally different due to the unique instance-level regression.\n\n- **[Formulation 3] Our improved dense object detection: Each voxel first regresses the borders of object instances, then aggregates instance-level context, and finally predicts the class.** Building on vanilla dense detection, we introduce a technical improvement, i.e., using the regressed offsets to enhance class prediction with instance-level aggregation, making the formulation more principled and practical.\n\nWe further present a detailed ablation study comparing these formulations. **To verify Formulation 1 (segmentation),** we deploy a 3D UNet following the common practice of existing SSC works on the 3D feature volume, then apply a classification head and optimize it with the standard SSC classification loss. **To verify Formulation 2 (our vanilla dense detection),** we follow [#1] by introducing an additional regression head to predict offsets, optimized with an L1 loss, consistent with our main paper. **To evaluate Formulation 3 (our improved dense detection),** we implement the instance-level aggregation in the classification branch to enhance instance-level semantics. We also report the **Full Model (VoxDet),** which further adds a Spatially-decoupled Voxel Encoder to improve task-specific representation learning.\n\nWe have the following observations and conclusions. **(1)** Compared to the segmentation (Formulation 1), introducing regression (Formulation 2) yields a substantial 2.71-point increase in IoU, demonstrating the fundamental strength of the detection-based paradigm in geometry, consistent with our motivation and prior work [#1]. **(2)** Adding instance-level aggregation (Formulation 3) further boosts mIoU by 1.57 points, showing that our design meaningfully enhances the conventional detection framework. **(3)** The full VoxDet achieves the highest performance by improving task-specific representation learning for both classification and regression. **This evidence fully justifies that our detection-based formulation yields optimal SSC performance.**\n\n|           | Formulation 1            | Formulation 2              | Formulation 3              | Full Version        |\n|:---------:|:------------------------:|:--------------------------:|:--------------------------:|:-------------------:|\n| IoU   | 42.71                    | 45.42 (+2.71)              | 46.79 (+4.08)              | 47.36 (+4.65)       |\n| mIoU  | 16.28                    | 16.42 (+0.14)              | 17.85 (+1.57)              | 18.73 (+2.45)       |\n\n[#1] FCOS: Fully Convolutional One-Stage Object Detection (ICCV 2019)\n\n## [W2] Clarify technical details\n\nSorry for the confusion. We show the core code for this part below (`TPVDecoupleModule`). For classification and regression, we use **unshared** `TPVDecoupleModule` (one per feature level) to extract task-specific features, which are then sent to two unshared task-specific FPNs, respectively. This module is parameter-efficient, as justified in Tab. 5 of the main paper.\n\n- **Details about $Conv_{(T)}^{(P)}$.** We use a single convolution layer for each $Conv_{(T)}^{(P)}$. For different feature levels and tasks, these convolutions do not share weights, encouraging fully decoupled representations. \n\n- **Effective Decoupling.** In the below source code, the main decoupling effect is achieved by unshared `DeformConv2dTPV` instead of just simple convolution, i.e., **the classification and regression branches learn different spatial offsets** (`offset_xy/yz/zx`), guided independently by their respective task losses. The effectiveness of this spatial decoupling has been well-justified in [#2, #3]. Moreover, **in Fig. 6 of the main paper, our qualitative results show that the two branches learn largely different features, justifying our decoupling design**. In Tab. 4 of the main paper, we also show that replacing DeformConv with a standard convolution causes noticeable performance drops, highlighting the **essential role of our spatial decoupling**.\n\n- **FPN for classification and regression share the network architecture but do not share weights.** We have conducted a detailed ablation study in Tab. 4 of the main paper, finding that using unshared FPN weights for the two tasks benefits task-specific representations, consistent with the observations in [#2, #3].\n\n[#2] Revisiting the Sibling Head in Object Detector (CVPR 2020)  \n[#3] Disentangle Your Dense Object Detector (ACM MM 2021)\n```\nclass TPVDecoupleModule(nn.Module):\n    def __init__(self, in_channels, mid_channels=None, kernel_size=1, padding=1,\n                 gn_groups=32, use_bias=False, adaptive_pooling=True):\n        super().__init__()\n        mid_channels = mid_channels or in_channels\n        self.adaptive_pooling = adaptive_pooling\n\n        if adaptive_pooling:\n            self.weights_conv = nn.Conv3d(in_channels, 3, 1, bias=use_bias)\n            self.pool_xy = DenseProjection('xy')\n            self.pool_yz = DenseProjection('yz')\n            self.pool_zx = DenseProjection('zx')\n\n        self.conv_xy = nn.Conv2d(in_channels, mid_channels, kernel_size, padding=padding) \n        self.conv_yz = nn.Conv2d(in_channels, mid_channels, kernel_size, padding=padding)\n        self.conv_zx = nn.Conv2d(in_channels, mid_channels, kernel_size, padding=padding)\n\n        offset_ch = 2 * 3 * 3\n        self.offset_xy = nn.Conv2d(mid_channels, offset_ch, 3, padding=1)\n        self.offset_yz = nn.Conv2d(mid_channels, offset_ch, 3, padding=1)\n        self.offset_zx = nn.Conv2d(mid_channels, offset_ch, 3, padding=1)\n\n        self.def_xy = DeformConv2dTPV(mid_channels, mid_channels, 3, padding=1, bias=False)\n        self.def_yz = DeformConv2dTPV(mid_channels, mid_channels, 3, padding=1, bias=False)\n        self.def_zx = DeformConv2dTPV(mid_channels, mid_channels, 3, padding=1, bias=False)\n\n        self.gn_xy = nn.GroupNorm(gn_groups, mid_channels)\n        self.gn_yz = nn.GroupNorm(gn_groups, mid_channels)\n        self.gn_zx = nn.GroupNorm(gn_groups, mid_channels)\n        self.act   = nn.ReLU(inplace=True)\n\n        self.fuse_conv = nn.Conv3d(mid_channels, in_channels, 1, bias=False)\n        self.fuse_gn   = nn.GroupNorm(gn_groups, in_channels)\n        self.fuse_act  = nn.ReLU(inplace=True)\n\n    def forward(self, x):\n        if self.adaptive_pooling:\n            w  = self.weights_conv(x)\n            xy = self.pool_xy(x, w)\n            yz = self.pool_yz(x, w)\n            zx = self.pool_zx(x, w)\n        else:\n            xy = x.mean(4); yz = x.mean(2); zx = x.mean(3)\n\n        # TPV spatial decoupling\n        def proc(f, conv, off, dconv, gn):\n            f = self.act(conv(f)) # Conv_{(T)}^{(P)}\n            o = off(f)\n            return self.act(gn(dconv(f, o))) # DefConv_{(T)}^{(P)}\n\n        dxy = proc(xy, self.conv_xy, self.offset_xy, self.def_xy, self.gn_xy)\n        dyz = proc(yz, self.conv_yz, self.offset_yz, self.def_yz, self.gn_yz)\n        dzx = proc(zx, self.conv_zx, self.offset_zx, self.def_zx, self.gn_zx)\n\n        B, C, X, Y, Z = x.shape\n        dxy = dxy.unsqueeze(-1).expand(-1, -1, -1, -1, Z)\n        dyz = dyz.unsqueeze(2 ).expand(-1, -1, X, -1, -1)\n        dzx = dzx.unsqueeze(3 ).expand(-1, -1, -1, Y, -1)\n\n        out = dxy + dyz + dzx\n        out = self.fuse_act(self.fuse_gn(self.fuse_conv(out)))\n\n        return out\n```\n\n## Response to tiny weakness\n\nThank you for your careful review and constructive suggestions.  \n1. We will include the following related works [#4, #5], which investigate object instances in 2D space for SSC and further support the paradigm used in previous studies.\n2. We will revise accordingly as suggested.  \n3. We underline “Vox” and “Det” in our claim to highlight the name of our method, VoxDet.  \n4. We will adjust the font size according to the final version.\n\n[#4] Disentangling Instance and Scene Contexts for 3D Semantic Scene Completion (ICCV 25)    \n[#5] VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion (AAAI 25)"}, {"review_id": "RrT3RvLaql", "reviewer": "Reviewer_3a7T", "summary": "The paper proposes a new method for semantic scene completion. It formulates the task as object detection. Given that there are no instance-level labels in the data, the authors propose a heuristic for extracting object instances from semantic labels. It is based on the assumption that individual objects are separable in 3D so there is a change in semantic label at the object boundaries. The method achieves state-of-the-art results on two semantic scene completion benchmarks.", "questions": "Do the authors believe that the situations where their assumption is not true are not significant?\n\nOr did I oversee something in the method that makes it robust against wrong supervision due to the proposed heuristic?\n\nAnswers to those are extremely important for the method's applicability.", "limitations": "No, see the strengths and weaknesses section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "I acknowledge that the paper achieves state-of-the-art results on two semantic scene completion benchmarks. However, I believe the fundamental assumption behind the method design is wrong and can lead to dangerous consequences on other datasets.\n\nThe authors suggest heuristically extracting instance-level labels from semantic labels based on the fact that there is a semantic label change on the object boundaries. This is, however, not always correct. In L.127 they even acknowledge that sometimes \"minor ambiguities may arise in cases such as the densely overlapping foliage of adjacent trees\". I would argue this is not a minor ambiguity but rather a fundamentally flawed assumption that can be harmful in many real-world situations.\n\nFor example if there is a row of cars parked on the side of the road close to each other, they might end up as a single semantic blob when discretized into a voxel grid. The community often works with voxel sizes between 20 and 40 cm so situations like this might not be rare at all. This in turn would lead to the proposed method learning wrong geometric priors. In the context of semantic scene completion this can have dire consequences: the model would to predict a wrong geometry because of the wrong supervision used during training. Automated driving is a safety-critical application so I believe introducing wrong assumptions like this is extremely dangerous.\n\nThe fact that this assumption happens to improve the results on some datasets does not justify its use in general. First, because on other datasets those cases where it does not hold true might be statistically more significant. Second, because even a small fraction of safety-critical mistakes introduced by this assumption is enough to refrain from using it.", "was_revised": true, "final_justification": "I appreciate the response by the authors and update my rating accordingly. I would kindly ask the authors to discuss the issue in more detail in the main paper and also move the description of the safeguarding mechanism from Appendix B.3 to the main paper.", "rebuttal": "Thank you for raising this concern. We understand the reviewer's point that closely packed objects (e.g., adjacent cars) could, in principle, lead to incorrect instance separation and potentially introduce noisy supervision. We address this in three ways:\n\n1. **Impact of Incorrect Labels**: In cases where such ambiguities occur, our method will not degrade compared to existing SSC approaches; it simply does not gain the benefit of additional geometric cues. When the instance-level labels are correct (which is the dominant case), our approach yields significant improvements, as evidenced by state-of-the-art results across **four benchmarks** (not just two). \n\n2. **Empirical Evidence of Robustness**: Despite the theoretical concern, in practice, we observed that even for challenging scenarios (e.g., cars parked with less than 10 cm separation), the heuristic extraction works reliably. This robustness is reflected in consistent performance gains across diverse datasets, which contain many such dense object arrangements.\n\n3. **Mitigation via Self-Correction (Overlooked Detail)**: Importantly, we already incorporate a **self-correction mechanism** (Appendix B.3) to prevent harmful supervision signals. Specifically, we filter out any pseudo-instance annotations whose size falls outside a reasonable object-size range. As a result, outlier offsets and class labels are automatically ignored during training. This ensures that the model does not learn incorrect geometric priors from erroneous cases.\n\nThis mechanism is a unique advantage of our approach: unlike prior SSC methods that implicitly rely on potentially incorrect semantic boundaries without correction, our method actively suppresses invalid labels, thereby reducing risk and enhancing geometric awareness.\n\nIn summary, while we acknowledge that the heuristic is not perfect, we have both **empirical evidence** (robust results across multiple datasets) and **built-in safeguards** (size-based filtering) that mitigate the reviewer’s concern. Far from introducing additional risk, our approach makes SSC supervision more informative and inherently safer than existing methods. In the following sections, we provide sufficient evidence and justifications to support the safety, robustness, and generalizability of our method.\n\n## [Justification 1] Our assumption is free of flaws and better (safer) than prior works\n\nOur VoxNT trick generates pseudo offset labels based on the observation (assumption): **Compared with the 2D image space, object instances are largely free of occlusion in the 3D voxel space; thus, most instances can be discovered to enhance SSC performance, except for tightly overlapping cases** (e.g., intertwined trees). This is a more effective and safer formulation than existing SSC works, which assume **all voxels of the same class as a single class group, totally ignoring per-instance geometry**. Specifically,\n\n1. For the instances that can be individually discovered, like traffic signs, poles, and most cars (Fig. 6 of the Appendix), **our method can successfully identify their geometric information, which no previous SSC works can achieve.** According to the statistical experiments (Fig. 2 of the Appendix), our algorithm successfully discovers these instances by extracting statistically correct scale offsets. \n2. For the tightly overlapping cases, like intertwined trees (mentioned in l.127 of the main paper), **our work and all existing SSC methods cannot separate them without real instance labels**.\n\nTo better illustrate the safety of our assumption in real-world settings, we list a formal comparison below.\n\n\n| Real-world Scenario                                        | Ground-truth Connectivity        | Prior SSC Work Assumption                                          | Our Assumption                                                    | Safer Choice     |\n|-------------------------------------------------|----------------------------------|----------------------------------------------------------|-----------------------------------------------------------|------------------|\n| 1. Naturally separate (e.g., traffic signs, poles, and trunks) | Separate                         | Single class group                                       | Separate instance groups                                  | Ours             |\n| 2. Naturally merged (e.g., overlapping tree crowns) | Merged                           | Single class group                                       | Fallback to single class group                                  | Equally safe     |\n| 3. Falsely merged (e.g., two cars fused by coarse voxels without air voxels between them) | Merged (e.g., Fig.7 of main paper, sub-optimal label) | Single class group (blindly trusts wrong label)          | Separate instance groups (self-corrected supervision, Appendix B.3) | Ours             |\n\nHence, compared with previous SSC works, our method demonstrates clear advantages in naturally separate (the dominant case) and falsely merged scenarios, while performing equally well in naturally merged cases. This ensures that our approach does not degrade performance in the complex real-world scenarios, providing a safer and more robust solution.\n\n## [Justification 2] Our method generalizes well to other datasets\n\nWe further conduct experiments on two additional SSC benchmarks: Occ-ScanNet (top table) and NYU v2 (bottom table). In both cases, VoxDet substantially surpasses the current state-of-the-art (including the concurrent work DISC and MonoMRN), showing its adaptability across diverse dataset settings. VoxDet achieves the best performance on all **four major SSC benchmarks** (not just 2), covering both LiDAR and camera settings. To the best of our knowledge, our method has been  **thoroughly evaluated on far more benchmarks and strict settings** than any existing SSC work, fully justifying its general adoption.\n\n| Method           | Reference    | IoU  | mIoU   | \n|:-----------------|:----------------|:----------|:-----------|\n| MonoScene           |   CVPR 2022        |    41.60|     24.62|\n| ISO           |   ECCV 2024       |    42.16 |    28.71 |\n| CGFormer           |    NeurIPS 2024       |  43.02   |29.11     |\n| DISC           |   ICCV 2025              |  43.18   | 28.92      |\n| **VoxDet**           |   **Ours**         |    **46.47** |**30.02**  |\n\n\n| Method           | Reference    | IoU | mIoU  | \n|:-----------------|:----------------|:----------|:-----------|\n| MonoScene           |   CVPR 2022        |    42.51|    26.94|\n| ISO           |   ECCV 2024       |    47.11 |    31.25 |\n| CGFormer           |   NeurIPS 2024      |  52.72   | 30.97    |\n| MonoMRN           |   ICCV 2025        |  53.16   |   30.73    |\n| **VoxDet**           |   **Ours**         | **55.88**    | **32.15** |\n\n\n## [Justification 3]  Our method is robust to the merged cases\n\nFirst, to evaluate robustness under the scenarios where more instances are merged, we increase the voxel size from 20 cm to 40 cm by downsampling the voxel resolution (Existing SSC benchmarks use a 20 cm setting). Although this change induces substantially more voxel-level merging (e.g., adjacent vehicles become a single blob), VoxDet outperforms the state-of-the-art method by 3.66 % in IoU and 2.45 % in mIoU. **This justifies that our method is more robust for merging instances, achieving better performance than previous works.**\n\nSecond, to further evaluate the correctness of our offset labels generated by the VoxNT trick, we train with the ground-truth instance supervision, denoted as \"VoxDet with GT\". Compared to training with the real instance-level GT, using our VoxNT-generated pseudo labels to train the model achieves competitive performance, with a performance difference of less than 1%. This demonstrates that our **VoxNT can generate sufficiently accurate instance-level pseudo labels**, even in cases of severe instance merging, fully validating its correctness and robustness.\n\n\n| Method             | Reference       | IoU (40 cm)      | mIoU (40 cm)      | IoU (20 cm)      | mIoU (20 cm)      |\n|:-------------------|:----------------|:-----------------|:------------------|:-----------------|:------------------|\n| CGFormer           | NeurIPS 2024   | 43.01            | 15.72             | 45.99            | 16.89             |\n| VoxDet             | Ours           | 46.67 (+3.66)    | 18.17 (+2.45)     | 47.36 (+1.37)    | 18.73 (+1.84)     |\n| VoxDet with GT     | Ours           | **47.21 (+4.20)**| **18.49 (+2.77)** | **47.64 (+1.65)**| **18.85 (+1.96)** |\n\n## Response to the rest of the questions\n\n1. **Do the authors believe that the situations where their assumption is not true are not significant?**\n\n   In fact, our assumption is both rigorous and intuitive, as it explicitly accounts for both separate and merged instances. This makes it applicable to all real-world scenarios. Both cases are significant, and our method consistently performs better than all prior SSC approaches in empirical evaluations.\n\n2. **Or did I overlook something in the method that makes it robust against wrong supervision due to the proposed heuristic?**\n\n   We apologize for any confusion. We have already designed a self-correction strategy to prevent incorrect supervision when the labels are wrong (see Appendix B.3), which is a unique strength of our method compared to previous SSC works."}, {"review_id": "OSiSP6EHMb", "reviewer": "Reviewer_6r4q", "summary": "This work introduces a semantic scene completion algorithm based on voxel-to-instance class label offset prediction. After 2D to 3D lifting, the framework conducts two separate spatial decoupling modules, where one branch predicts voxel semantic labels while the other aims to capture instance-level information of each voxel. The learned instance-level voxels are integrated into the semantic-level voxels for scene completion. The authors have conducted exhaustive experiments across multiple benchmarks and comparisons with various baselines.", "questions": "See weaknesses.", "limitations": "See weaknesses.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n- The idea is simple yet effective.\n- The experiments are quite strong. The authors have conducted extensive ablation studies and analysis of different designs of different modules.\n- The paper is well-written, with proper content for high-level motivations, low-level technical details, and clear figures.\n\n**Weaknesses**\n- The algorithm relies on additional instance-level supervision, which is not always available in the training datasets. Is it possible to leverage existing instance segmentation algorithms (SAM) to create such labels? How large a negative effect would it have on the system?\n- The authors are expected to discuss some existing instance detection algorithms [1,2] in the related works, as they do complementary research (integrating 3D knowledge for 2D tasks).\n- Though this algorithm is designed for outdoor scene completion. I am wondering if it is also applicable to indoor scenarios [3,4]? And if it does not, what would be the key bottleneck / potential ways to address it?\n\n[1] Li, Bowen, et al. \"VoxDet: voxel learning for novel instance detection.\" Advances in Neural Information Processing Systems 36 (2023): 10604-10621.\n[2] Shen, Qianqian, et al. \"Solving instance detection from an open-world perspective.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n[3] Straub, Julian, et al. \"The replica dataset: A digital replica of indoor spaces.\" arXiv preprint arXiv:1906.05797 (2019).\n[4] Dai, Angela, et al. \"Scannet: Richly-annotated 3d reconstructions of indoor scenes.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.", "was_revised": true, "final_justification": "The detailed explanations and response have resolved my concerns, and I will maintain my score.", "rebuttal": "We sincerely appreciate the constructive comments. We want to summarize and highlight three key clarifications. \n1. Our method **requires no additional manually annotated instance-level labels** and can be further boosted with 2D foundation models such as SAM.  \n2. We will broaden our related-work section to include a discussion of instance-detection research. \n3. Our approach offers a general and powerful SSC paradigm that achieves **state-of-the-art results on almost all existing SSC benchmarks (4 datasets)**, across (1) indoor and outdoor scenes and (2) camera and LiDAR-based settings. \n\nWe would be grateful if these clarifications prompt a reconsideration of our score, and we remain fully available for any further questions during the author-reviewer discussion period.\n\n## [W1] Clarify using extra instance labels and using SAM to generate labels\n\n**No additional instance-level labels needed.** We are sorry for the confusion. We'd like to kindly clarify that our VoxNT trick (Fig. 4 in the main paper) automatically generates instance-level pseudo-offset labels from the standard semantic occupancy annotations used in previous SSC works. This process requires **no extra manual instance-level labeling** and thus preserves strict fairness in data usage.\n\n**Use SAM to create pseudo labels.**  We appreciate your insightful suggestion! To validate the potential of external instance-level labels, we conducted an additional experiment using Grounded SAM [#1] to produce 2D instance pseudo labels. Specifically, for each image, we first apply Grounded SAM to generate pseudo instance masks corresponding to the defined classes in the SSC datasets. We then follow previous SSC work [#2] to introduce an auxiliary MaskDINO [#3] head after the ResNet-50 backbone as an additional instance prediction head, supervised by the instance-level loss functions of MaskDINO. It is worth noting that this MaskDINO head is used only during training to enhance instance-aware representation of the backbone, which **does not incur extra parameters or computational overhead at inference time**. \n\nIn the table below, we present a comprehensive comparison with existing state-of-the-art methods. After incorporating the labels from Grounded SAM, our method achieves consistent performance gains in both IoU and mIoU, surpassing previous works on all metrics. These results indicate that our approach effectively leverages the external knowledge and labels provided by 2D foundational models, verifying its effectiveness. Therefore, **using SAM to generate labels not only has no negative impact but also further boosts performance**.\n\n\n| Method           | Reference    | IoU ↑ | mIoU  ↑ | Parameters ↓ | Inference Time ↓ |\n|:-----------------|:----------------|:----------|:-----------|:-----------|:-----------|\n| CGFormer           |   NeurIPS 2024        |    45.99|      16.89 | 122 | 205|\n| SGFormer           |   CVPR 2025       |     45.01 |     16.88 |126 | -|\n| ScanSSC           |   CVPR 2025       |     45.95 |      17.12 |145 | 261|\n| DISC           |   ICCV 2025        |     45.93 |      17.05 |-|-|\n| **VoxDet**           |   **Ours**         |    **47.36** | **18.73** | **53** | **159** |\n| **VoxDet w. Grounded SAM**    |    **Ours**  |     **47.92** |      **19.02** | **53** | **159** |\n\n[#1] Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks (ArXiv 2024)  \n[#2] Symphonize 3d Semantic Scene Completion with Contextual Instance Queries (CVPR 2024)  \n[#3] Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation (CVPR 2023)\n\n## [W2] Discuss some instance detection works\n\nThank you very much for your suggestion! This makes our manuscript more comprehensive, especially by deepening the review of instance-centric learning, which is the key focus of our work. Li et al. [1] propose a novel 3D-inspired paradigm for geometrically reliable detection in 2D views, enhancing geometric perception through voxel-driven instance reconstruction. Recently, IDOW [2] innovatively adapts foundational knowledge into a 3D-aware instance detection pipeline, significantly improving generalization and robustness in open-world scenarios. Unlike these approaches, which leverage 3D information to augment 2D instance detection, our method discovers object instances directly in dense 3D voxel space, eliminating 2D limitations such as occlusion and perspective distortion. We will revise the final version to review these related works comprehensively.\n\n## [W3] Explore the adaptability to indoor scenes\n\nThank you for your constructive suggestions! Our approach represents a generic SSC paradigm adaptable to both indoor and outdoor scenes. To further demonstrate the generalizability of our generic SSC paradigm, we further compare VoxDet on two widely-used indoor datasets: Occ-ScanNet (upper table) and NYU v2 (lower table). On both datasets, VoxDet decisively outperforms all prior SSC methods, underscoring its **ability to adapt to highly diverse voxel settings**. Note that our method even surpasses the co-occurrence works such as DISC and MonoMRN with noticeable gains.\n\nWith these new results, VoxDet now sets the state of the art on all four major SSC benchmarks. To the best of our knowledge, **this is the first method to be exhaustively validated across the entire SSC benchmarks**: most prior work reports on at most two of these datasets. This comprehensive superiority highlights our effectiveness and affirms its applicability as a **truly general-purpose solution** for semantic scene completion.\n\n\n| Method           | Reference    | IoU   | mIoU  | \n|:-----------------|:----------------|:----------|:-----------|\n| MonoScene           |   CVPR 2022        |    41.60|     24.62|\n| ISO           |   ECCV 2024       |    42.16 |    28.71 |\n| CGFormer           |   NeurIPS 2024       |  43.02   |29.11     |\n| DISC           |   ICCV 2025              |  43.18   | 28.92      |\n| **VoxDet**           |   **Ours**         |    **46.47** |**30.02**  |\n\n\n| Method           | Reference    | IoU  | mIoU   | \n|:-----------------|:----------------|:----------|:-----------|\n| MonoScene           |   CVPR 2022        |    42.51|    26.94|\n| ISO           |   ECCV 2024       |    47.11 |    31.25 |\n| CGFormer           |   NeurIPS 2024      |  52.72   | 30.97    |\n| MonoMRN           |   ICCV 2025        |  53.16   |   30.73    |\n| **VoxDet**           |   **Ours**         | **55.88**    | **32.15** |"}, {"review_id": "3SWxLrMuy4", "reviewer": "Reviewer_8beq", "summary": "This paper aims to tackle semantic scene completion (SSC), which reconstructs 3D semantic occupancy from camera input. Starting from an interesting observation that the object instance can be regressed in the voxel space, the authors first propose a simple yet effective VoxNT trick to freely convert the voxel-level class labels to the instance-level offsets. Leveraging these free offset labels, they further propose VoxDet, an instance-centric framework that reformulates SSC as a dense object detection task with two subtasks: semantic prediction and offset regression. VoxDet first employs a spatially decoupled voxel encoder to learn task-specific volumetric representations via distinct spatial deformations. Then it introduces a task-decoupled dense predictor to regress instance borders and aggregate instance semantics. Extensive experiments on both camera and LiDAR benchmarks show that VoxDet achieves new state-of-the-art results jointly, which also gives the highest 63.0 IoU on the SemanticKITTI hidden test set (online leaderboard). Overall, this work is both exciting and inspiring for the community.", "questions": "Overall, this is an interesting paper. All questions are listed above, I would appreciate if those issues can be addressed.", "limitations": "Yes.", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths\n\n- The paper is clearly organized and well-motivated with good technical flow.\n- Section 3 offers practical, interesting observations that clearly motivate the core method.\n- The proposed VoxNT trick cleverly extracts instance-level scale information from voxel-level class labels without extra supervision and training.\n- In VoxDet, the reformulation of SSC as dense object detection is both novel and effective, providing new insights and perceptive. The detailed algorithm designs are also tailor-designed and technically sound.\n- The approach is computationally efficient and delivers state-of-the-art results on both camera and LiDAR benchmarks, with the highest IoU on the online leaderboard. The multi‐run experiment in appendix demonstrates good robustness.\n- A comprehensive experimental suite, such as feature visualizations and quantitative studies, supports the method design and yields additional insights.\n\n\nAfter a thorough review of the main paper and appendix, I found no major weaknesses and have only the following minor question:\n\n- In the appendix, the method achieves remarkable performance on LiDAR benchmark, which is suggested to reported in the main paper. Accordingly, given the strong LiDAR-based results, I recommend reframing the paper’s focus from camera-based SSC to the more general SCC task across sensing modalities.\n- Can the proposed method be further improved by using both camera and LiDAR inputs together?", "was_revised": true, "final_justification": "Thank the authors for their response, which addressed my questions. After carefully reading the other reviews and the rebuttal, I find that the authors have not only provided strong clarifications but also highlighted additional advantages of their method, including (a) strong performance and generalization ability on other datasets, (b) superior problem formulation and novelty compared to previous works, (c) robustness to different voxel resolutions without ethical concerns, and (d) easy-to-reproduce implementation details, with the core source code available.\n\nOverall, I believe this work is insightful and high-quality, which can make significant contributions to the occupancy community.", "rebuttal": "We sincerely appreciate the careful review of the details in both the manuscript and the Appendix. The comments precisely captured the key aspects we hoped to convey, including the technical insight, methodological novelty, and experimental validation of our work, which encourages us significantly.\n\n## [Q1] Move LiDAR-based results from the Appendix to the main paper\n\nWe fully agree with this constructive suggestion and will make the change in the final version. Bringing the LiDAR results into the main paper will better highlight two core contributions: (1) VoxDet is a truly general method that works across diverse datasets and **can be deployed with either camera or LiDAR input**, which is a unique technical highlight not offered by prior SSC works. (2) VoxDet achieves **state-of-the-art performance on both camera and LiDAR settings**, ranking 1st on the public leaderboard, justifying the strong potential for general usage.\n\n## [Q2] Can the method be further improved by fusing both LiDAR and camera modalities?\n\nThank you for your insightful suggestion! Yes, our method **can be effortlessly deployed using camera and LiDAR input together**. We further evaluate our approach on SemanticKITTI by fusing two modalities: camera-based 3D volumes (after 2D-to-3D lifting) and LiDAR-encoded 3D volumes via an element-wise addition. As shown below, this multimodal fusion boosts mIoU from 26.0% to 26.9%. These results demonstrate that incorporating the camera modality, with its rich RGB cues, significantly enhances semantic understanding of VoxDet, further highlighting the adaptability and effectiveness of our method.\n\n| Method           | Reference    | IoU  | mIoU  | \n|:-----------------|:----------------|:----------|:-----------|\n| VPNet           |   NeurIPS 2024      |  60.4   | 25.0    |\n| VoxDet (LiDAR)           |  Ours          |  63.0   |    26.0    |\n| VoxDet (LiDAR-Camera-Fusion)           |   Ours         | **63.2**    | **26.9** |"}]

Author rebuttal:

Meta-review: This paper formulates semantic scene completion (SSC) as a dense object detection task, incorporating offset regression and semantic prediction to address 3D semantic scene completion.  The framework conducts two separate spatial decoupling modules, where one branch predicts voxel semantic labels while the other aims to capture instance-level information of each voxel. The learned instance-level voxels are integrated into the semantic-level voxels for scene completion. The work is based on the assumption that individual objects are separable in 3D so there is a change in semantic label at the object boundaries.  Experimental comparisons demonstrate that the proposed method achieves superior performance compared to other recent SSC methods.

The main strengths of this work are as follows: (1)  The idea is simple yet effective. The main insight of this work is the reformulation of SSC. The authors utilize two modules (e.g., spatially-decoupled voxel encoder and task-decoupled dense predictor) to achieve effective semantic scene completion, leading to improved SSC performance. (2) The experiments are quite strong. The work achieves state-of-the-art results on two semantic scene completion benchmarks.  (3) The paper is well-written, with proper content for high-level motivations, low-level technical details, and clear figures.

The main weakness of the paper is as follows:  (1) Novelty issue. The proposed method appears to be similar to previous segmentation-based methods. (2) The work is based on the assumption that there is a semantic label change on the object boundaries. This assumption does not always hold. (3) Missing some technical details. (4) Missing some details about the training process and the dataset.

This work provides a new perspective on the semantic scene completion (SSC) and demonstrates that using the object detection framework can outperform state-of-the-art SSC methods. All reviewers agree that this work is interesting and the experiments demonstrate that the method achieves superior performance compared to other recent SSC methods.

At the first round, the reviewers provide 5,5,4,3 scores. After the rebuttal, the rates become 6555 due to the main concerns about the assumption issues, and the detailed implementations are justified. I agree with the reviewers' comments and discussions. This work regards the SSC as a dense prediction or detection task. Inspired by previous dense prediction frameworks, the new SSC framework could achieve better performance than SOTA methods.  However, the final version should involve the discussions in the rebuttal to justify the assumption issues and the omission of some implementation details.
Decision: Accept (spotlight)