α-OCC: Uncertainty-Aware Camera-based 3D Semantic Occupancy Prediction
Authors: Sanbao Su
Nuo Chen
Chenchen Lin
Felix Juefei-Xu
Chen Feng
Fei Miao
Venue: Accepted by TMLR ()

Abstract: Comprehending 3D scenes is paramount for tasks such as planning and mapping for autonomous vehicles and robotics. Camera-based 3D Semantic Occupancy Prediction (OCC) aims to infer scene geometry and semantics from limited observations. While it has gained popularity due to affordability and rich visual cues, existing methods often neglect the inherent uncertainty in models. To address this, we propose an uncertainty-aware OCC method (α-OCC). We first introduce Depth-UP, an uncertainty propagation framework that improves geometry completion by up to 11.58% and semantic segmentation by up to 12.95% across various OCC models. For uncertainty quantification (UQ), we propose the hierarchical conformal prediction (HCP) method, effectively handling the high-level class imbalance in OCC datasets. On the geometry level, the novel KL-based score function significantly improves the occupied recall (45%) of safety-critical classes with minimal performance overhead (3.4% reduction). On UQ, our HCP achieves smaller prediction set sizes while maintaining the defined coverage guarantee. Compared with baselines, it reduces up to 90% set size, with 18% further reduction when integrated with Depth-UP. Our contributions advance OCC accuracy and robustness, marking a noteworthy step forward in autonomous perception systems. Our code is public on https://coperception.github.io/alpha-OCC/.

Published in Transactions on Machine Learning Research (04/2026)
a-OCC: Uncertainty-Aware Camera-based 3D Semantic Oc- cupancy Prediction
Sanbao Su
edu
School of Computing University of Connecticul
Nuo Chen
nc3144 @myu.
Tandon School of Engineering New York University
Chenchen Lin
chencken. Wicoonn edu
School of Computing University of Connecticut
Felix Juefei-Xu New York University
juefei.au@nyu edu
Chen Feng
efeng edu
Tandon School of Engineering
New York University
Fei Miao
wwm.iiaaauoonnccommmmmmm edu
School of Computing University of Connecticul
Reviewed on OpenReview: https: openreview. net/ forum? id= bUb25gBL
Abstract
Comprehending 3D scenes is paramount for tasks such as planning and mapping for au- tonomous vehicles and robotics. Camera-based 3D Semantic Occupancy Prediction (OCC) aims to infer scene geometry and semantics from limited observations. While it has gained popularity due to affordability and rich visual cues, existing methods often neglect the inherent uncertainty models. To address this, we propose an uncortainty-awaree OCC method (a-OCC). We first introduce Depth-UP, an uncertainty propagation framework that improves geometry completion by up to 11.58% and semantic segmentation by up to 12.95% across various OCC models. For uncertainty quantification (UQ), we propose the hierarchical conformal prediction (HCP) method, effectively handling the high-level class imbalance in OCC datasets. On the geometry level, the novel KL-bbasd score function significantly im- proves the occupied recall (45%) of safety-critical classes with minimal performance overhead (3.4% reduction). On UQ, our HCP achieves smaller prediction set sizes while maintaining the defined coverage guarantee. Compared with baselines, it reduces up 90% size, with 18% further reduction when integrated with Depth-UP. Our contributions advance OCC accuracy and robustness, marking noteworthy step forward in autonomous perception systems. Our code is public on https:/
wtt.gggttt.oaaa.aii... Otttuubioo/appaa-OCC///
Introduction
Achieving comprehensive understanding of 3D scenes is crucial for downstream tasks such planning autonomous vehicles (AVs) and robotics (Wang & Huang (2021)) 3D Semantic Occupancy Prediction (OCC)

---

Published in Transactions on Machine Learning Research (04/2026)
emerges solution that jointly infers the geometry completion and semantic segmentation (Song et al.
[2017] : Hu et al. (2023 ). OCC approaches typically are categorized based on the sensor input: LiDAR-
OCC and camera- baaeer--bseed OCC. While LiDAR offers precise depth information (Roldao et al. (2020): Cheng
ct al. (021]), ) they are costly and less portable. Conversely, camera- camera-based OCCs, with their affordability
and ability to caplure rich visual cues, have gained significant attention (Cao & De Charette (0022) ian
ct al. (2024))- ) For them, depth estimation is essential for the accurate 3D reconstruction of scenes. However, existing methods frequently overlook real-world depth estimation errors (Poggi et al. (2020)). ). Moreover, effectively leveraging propagated depth uncertainty and rigorously quantifying OCC uncertainty, particularly in class-imbalanced datasets, remains challenging and unexplored problem. Throughout this paper, OCC refers to camera- camera-based OCC unless stated otherwise, which is the focus of our work.
20
Rited Curve
Camera View
Basic OCC
Our HCP
Our a OCCCC HCC Depth-UP
Exp Points
16
10
5 Percent Depth Uncertainty (%)
+ eepry (12.1%
Noydlr (8.1%))
matercedion (8.60%)
(4.55%)
vegetatonn (2.22%)
Freek (8.64%)
Red 84%
(a)
(b)
Figure (a): As the percentage of depth uncertainty increases, the accuracy (mIoUt) of OCC decreases significantly. (b): High class imbalance on OCC. The percentage next to each class is its percentage in the SemanticKITTI dataset. Since the safety-critical class "bicyclist" only occupied 0.01%, the trained OCC model fails to detect the bicyelist in front, leading to a crash. However, after quantifying the uncertainty and post-processing using our HCP, the crash is avoided, for our HCP improves the occupied recall of rare classes. When applying our Depth-UP and HCP together, safety is further enhanced as the bicyclist is more accurately identified. In contrast, using only HCP often assigns the highest probability to the car (blue) for many bicyclist voxels. Due visualization limits, each occupied voxel shows the most probable nonempty class in the predicted set from HCP.
We explain the importance of considering depth uncertainty propagation and OCC uncertainty quan- tification in Fig. 1 In Fig. Πa). perturbing the ground-truth depth values by factors of (1 8) (V3 E {0%, 2%, 4%, 6%, 8%, 10%, 20%)), simulates real-world depth estimation crrors. Uncertainties of depth estimation significantly reduce the performance of OCCs. To address this, propose flexible uncertainty propagation framework (Depth-UP) from depth models to improve the performance of a variety of OCC models.
The OCC datasets exhibit a high class imbalance, with emply voxels comprising a significant proportion (92.91% for SemanticKITTI [Belley et al.) (2019)), as illustrated in the dotted box of Fig, []). Bicyclist and person voxels, crucial for safety, only occupy 0.01% and 0.007% of the datasct. Consequently, networks trained such imbalanced data, coupled with the maximum posterior classification, often overlook rare classes, leading to reduced accuracy and recall (Tian et al. (2020)). This poses a significant risk in safety-critcall applications like AVs, where detecting rare-class occupied voxels is essential for preventing collisions (Chan let al. (001))) ). As in Fig. bb), a standard OCC model fails to detect the bicyclist due to the extreme rarity of the bicyclist class in the dataset. To address this, we propose a hierarchical conformal prediction (HCP) method that enhances occupied recall for rare classes gcometry completion while generating prediction sets with class coverage guarantees for semantic segmentatiin. By quantifying uncertainty (prediction set) through HCP, the OCC model successfully identifies rare bicyclist voxels, reducing the risk of collisions. Furthermore, our complete a-OCC framework integrates both HCP and Depth-UP, further refines bicyclist. detection, significantly improving safety.
Extensive experiments on three OCC models (VoxFormer Li et al. 2023b), OccFormer Zhang et al. 20223 and CGFormer Yu et al. (2024)) and two datasets (SemanticKITT Behley et al. (2019) and KITTI 3360 et al. (2023a)) show that our Depth-UP UP improves geometry completion by 11.58% and semantic segmentation by 12.95%. Our HCP enhances person-class geometry prediction by 45% with only 3.4% ToU overhead,
2

---

Published in Transactions on Machine Learning Research (04/2026)
improving prediction rare safety-critical classes, such as persons, thereby reducing potential risks for AVs. Compared with baselines, HCP reduces set size by 90% and coverage gap by 64%, an additional 18% set size reduction when combined with Depth-UP. These results show the significant accuracy and uncertainty quantification improvements of our a-OCC.
Our contributions are unified under the proposed a-OCC, an uncertainty-aware camera-based 3D semantic occupancy prediction framework. This approach recognizes the OCC problem from fresh uncertainty perspective and consists of two major components: 1) To the best of our knowledge, we are the first to propose the uncertainty propagation framework Depth-UP to improve OCC performance. Our approach leverages uncertainty quantified through direct modeling to improve both geometry complction and semantic segmentation, resulting in substantial performance gains across common OCC models. 2) To solve the high-level class imbalance challenge on OCC, resulting in biased prediction and low recall for rare classes, we propose the HCP. On geometry completion, a novel KT.-based score function is proposed to improve the occupied recall of safety-critical classes with little performance overhead. For UQ, we achieve a smaller prediction set size under the defined class coverage guarantee. Overall, our a-OCC shows that uncertainty is integral and vital part of OCC tasks. Integrating DP (propagating depth uncertainty to OCC) and HCP (quantifying OCC uncertainty) enhances both accuracy and uncertainty of OCC models.
2 Related Work
Uncertainty Quantification and Propagation. Uncertainly quantification (UQ) holds paramount. importance in ensuring the safety and reliability of autonomous systems such as robots (Jasour & WWillaas (2019)) and AVs (Meyer & Thakurdesai (2020): ; Su et al. (0025)). Morcover, UQ for perception can significantly enhance the planning and control processes of them [u et al. (2014); He et al. (0233
Different types of UQ methods have been proposed. Monte-Carlo dropout (Miller et al. (2018)) and deep ensemble (Lakshminarayanna) et al. (2017) methods require multiple runs of inference, which makes them infeasible for real-time applications. In contrast, direct modeling (Feng et al. (2021) estimates uncertainty in one pass, used to estimate the depth uncertainty in our work.
While uncertainty has been explored in 3D tasks, our focus differs. Eldesokey et al.
2020
improved depth
completion with uncertainly by normalized convolutional neural networks. Cao et al. based OCC uncertainty with deep ensembles, increasing computational complexity. Although uncertainty propagation (UP) from depth to 3D object detection has improved accuracy (Lu ct al.
[2024)
managed LiDAR-
(2021)
Wang et al.
(2023)); no prior work has addressed UP from depth to OCCs. To bridge this gap, we propose Depth-UP, a direct modeling-basee based UP for OCCs.
Conformal prediction (CP) provides statistically guaranteed uncertainty sets for model predictions (An- gelopoulos & Bates (2021); Su et al. (2024) ). However, its application to class-inbalanced tasks is limited. Rare and safety-critical classes (e.g., person) remain challenging for OCC models. To address this, we propose a hierarchical conformal prediction method for uncertainty quantification in highly class-imbalanced OCCs. Related works on class imbalance are introduced in Appendix A.1 And related works on OCC are introduced in A Appendix A.2
3 Method
We propose a-OCC, an uncertainty-aware OCC method integrating Depth-UP for uncertainty propagation and HCP for uncertainty quantification. Figure 2 presents the whole methodology overview and the structure of our Depth-UP. Figure 3 details our HCP. The major novelties are: (1) Depth-UP quantifies the uncertainty of depth estimation by direct modeling (DM) and then propagates it through probabilistic goometry projection (for geometry completion) and depth feature extraction (for semantic segmentation).
(2) HCP calibrates OCC probability outputs. First, it predicts the voxels' occupied state by the quantile the novel KL-based score function as Eq. [ which can improve the occupied recall of rare safety-critical classes. Then it generates prediction sets for predicted occupied voxels, achieving a better coverage guarantee and smaller prediction set. sizes.
Preliminary: OCC predicts dense somantic scene within a defined volume front of the vehicle solely from RGB images (Cao & De Charette (2022)). Specifically, with an input image X E x an OCC model first extracts 2D image features F, using backbone networks like ResNet (He et al. (016 and estimates
3

---

Published in Transactions on Machine Learning Research (04/2026)
Previous work
Ours
Depth Uncertainty Propagation (Depth-UP)
Mean Head s
Depth Meen
Depth Back- bone
Std Head
Depth
Standard Deviation
Mp
Pp
View Transformation
View Transformation
Phaw
||0
Semantic Output Fa
intersccioon
Lightwoigtt Backbone
Prediction
Prediction Head
Head
HCP
Geometry Output M,
Update to
d
Updaated vcoa
Probabilatic Vowel Grid Map Ni
Figure Overview of our a-OCC method. The non-black colors highlight the novelties and important techniques in our method. C denotes the concalenation of the depth feature Fn and image feature F,. In the Depth- UP part, calculate the uncertainty of depth estimation through direct modeling. For depth model retraining, we only train the additional standard deviation head while keeping the rest of the model frozen. Then we propagate it through depth feature extraction (for semantic segmentation) and building a probabilistic voxel grid map Mp by probabilistic geometry projection (for geometry completion). Each element of Mp the occupied probability of the corresponding voxel, computed by considering the depth distribution of all rays across the voxel.
the depth value D E HR for each pixel (e.g. monocular depth estimation Bhat et al. (022)) or stereo depth estimation Shamsafar et al. (202))). Then, a probability voxel grid Ý {0, 10,1 XD is generated based on F, and D, assigning each voxel to the class with the highest probability. Each voxel within the grid is categorized is denoted as Y, H and W is the height and width of the input image, U, V and D represent the height, width, and length of the voxel grid, M denotes the total nunber of relevant classes (including the empty class), respectively.
either empty or occupied by a specific semantic class. The ground truth voxel grid
3.1 Uncertainty Propagation Framework (Depth-UP)
In OCC methods, depth models facilitate the projection from 2D to 3D space, primarily focusing geometric aspects. Nonetheless, these approaches often overlook the inherent depth uncertainty. Recognizing the potential to enhance OCC performance by utilizing this uncertainty, we introduce Depth-UP framework, which quantifies the uncertainty of depth models through a direct modeling method and propogate this uncertainty into both geometry completion and semantic segmentation improve the final performance of various OCC models.
Direct Modeling (DM). Depth UP includes DM technique (Su et al. (2023 Feng et al. 2021 to estimate the standard deviation associated with the estimated depth value of each pixel, with minimal computational overhcad. An additional regression head, with comparable structure as the original regression head for D, is tailored to predict the standard deviation Σ. Subsequently, this head is retrained based on the original depth model, with all parameters the original depth model (including the original regression head) being frozen. Assume the estimated depth value is represented as a univariate Gaussian distribution, and the ground truth depth follows a Dirac delta function (Arfken al. (2011)). For the retraining process, define the regression loss function the KT-divergence between the estimated distribution
4

---

Published in Transactions on Machine Learning Research (04/2026)
and the ground truth distribution, where D E I "R is the ground truth depth matrix for the image: (D, HW Σx
E001
(daw dou--daw"" + log Jōhol.
10m1
Propagation on Geometry Complction (PGC). Depth information used to generate the 3D voxels on geometry in OCC. There are two key challenges: lens distortion during geometric transformations and occupied probability estimation for each voxel. Lens distortion is a deviation from the ideal image formation by lens, resulting in distorted image (Zhang (2000)). Existing OCC models, such as VoxFormer (Li ct al. (2023b)), solve the lens distortion by projecting depth into 3D point cloud, and then generating the binary voxel grid map M, E 10 1)0XVD, where each voxel is marked as 1 if occupied by at least one point. However, they ignore the uncertainty of depth. Here we propagate the depth uncertainty into the geometry of OCC to solve the above two challenges.
Our Depth-UP generates probabilistic voxel grid map M, E [0, 1UUJVVD that considers lens distortion and depth uncertainty, with Í. from the depth model. For pixel (h, w) with estimated depth mean daw, we project it into point (I, 2) in 3D space: = (h-cx)xz, y (wr (w=e
drwr; where (Cu, Go) is the
camera center and fa and fe are the horizontal and vertical focal length.
When estimated depth follows a univariate Gaussian distribution, a point's exact location along its camera ray uncertain. Instead, we estimate the probability of one voxel (u, v.d) being occupied by points. Duc to the density of visual information, a single voxel may correspond to multiple pixels, which means a voxel can be passed by multiple rays. We denote this set of rays as Vaed; and single ray within this set as Phwe corresponding to pixel (h, w). When a ray Phu passes through voxel, it has two crosspoints: 2, where the ray enters the voxel, and &e where the ray exits the voxel. By cumulating the probability of the ray inside the voxel using the probability density function, we obtain the probability of voxel (u, v, d) being occupied by points:
[ N(zldnws &ha ddz
M,(u, d, min 1,
Σ
(1)
The original binary voxel grid map is replaced by the probabilistic voxel grid map Mp E [0, 1J to propagate the depth uncertainty into the geometry completion of OCC.
Propagation on Semantic Segmentation (PSS). The extraction of 2D features F, from the input image has been cornerstone for OCC to encapsulate semantic information. However, utilizing depth uncertainty on the semantic features is ignored. Through an additional lightweight backbone, such as ResNet-18 backbone (He ct al. 22166), we extract depth features F D from the concatenated depth mcan and standard deviation [D,E}. These newly acquired depth features are then seamlessly integraled with the original 2D image features, constituting a novel set of input features (F as shown in Figure 2 This integration strategy capitalizes on the extensive insights gained from prior depth prodictions, enhancing the OCC performance with enhanced semantic understanding.
3.2 Hierarchical Conformal Prediction (HCP)
3.2.1 Preliminary
Standard Conformal Prediction. For classification, conformal prediction (CP, Angelopoulos & Bates (2021); Ding et al. (2024)) is a statistical method to post-process any models by producing the set of predictions with theoretically guaranteed marginal coverage of the correct class. With M classes, considering the calibration data (X1, YI)..., (XN, YN) with N data points that are never seen during training, the standard CP (SCP) includes the following steps: (1) Define the score function sX. y) E (Smaller scores indicate better agreement between X and y). The score function is a vital component of CP. A typical score function of classifier f is s(X, y) = 11-ffo f(X)y, where f(X), represents the yth softmax output of f(X). (2) Compute as the [N-11(1--]] quantile of the calibration scores, where a E [0, 1] is a user-chosen error rate. (3) Use this quantile to form the prediction set C(X,est) C fI...., M) for one new example X,est (from the same distribution of the calibration data): C(Xtest) = (y s(X y) S q). The SCP provides a coverage guarantee that IP(Ytest E C(Xtest)) 2 1 a which has been proved in Angelopoulos & Bates (021)
N
5

---

Published in Transactions on Machine Learning Research (04/2026)
Hierarchical Conformal Prediction (HCP)
OCC Prediction Head
Prediction
Softmax
cls prob
Previous work
Argmax
vegetation
Ours
2
class
Prediction Set
SaiX 4)
Occupied
cls prob
S1X y
(vegetation, person)
KL-based
Score
2
class
1234
class
Function
S 4)
Empty
(empty)
1. empty
2. vegetation
3. car
4. person
Figure 3: Overview of our HCP method. We predict voxels' occupied state by the quantile on the KL-basod Eq. which can improve occupied recall of rare classes, and then only generate prediction sets for these predicted occupied voxels. The occupied quantile oy and semantic quantile 9 are computed during the calibration step of HCP.
Class-Conditional Conformal Prediction. The SCP achioves the marginal guarantee but may neglect the coverage of some classes, especially on class-inbalanced dalasets (Angelopoulos Bates) (2021)). Class- Conditional Conformal Prediction (CCCP) targets
coverage under the user-chosen class error rate a", Vy E {I,..., M); IP(Y test ((Xtest) Y teat y) a"a Every class y has at least 1-a" probability of being included in the prediction set when the label is y. Hence, the prediction sets are with coverage guarantee to all classes, even the rare ones.
3.2.2 Our Hierarchical Conformal Prediction
Current CP does not consider the hicrarchical structure of classification, such the geometry completion and semantic segmentation in OCCs. And it cannot achieve good coverage for very rare and safely-critical classes. Here we propose a novel HCP to address these challenges, as shown in Figure 3 The detailed algorithm is shown in Appendix A.3
Geometric Level. On the geometric level, is important and safety-critical to guarantee the occupied recall of some sensitive classes, such as the person and bicyclist for AVs. Hence, we define the occupied coverage for the specific safety-critical class y as: P(o T test = y) 2 ag, where o T means the occupancy state is true. The probability of the voxels with label y are predicted occupied is guaranteed to be no smaller than 1 - ay. The empty class is y 1 and occupied classes are y {2,..., M.....,. To achieve the above guarantee under the high dlas--iballaned dataset, propose novel score function based on the KL divergence. Here we define the ground-truth distribution for occupancy as O = (s, 1, 1]M, where s is the minimum value for the empty class to avoid the divide-by-zero problem. With the output softmax probability f(X) (Pi. P2, ... PM) from the model f, we define the KL-basod score function for y E ),;
Sn XX v) = Dia = pı log
Σ log(ps)
where D, is the considered rare class set. The quantile o is computed the [(N++1)(1=o3) No quantile of the score Sh1( X, y) on Yv, where Tv is the subset of the calibration dataset with Y = and N, = |r]] Then we predict the voxel X as occupied if By E Sur 3) E og.
Semantic Level. On the semantic level, we need achieve the same dlass-balance coverage under the geometric level coverage guarantee. For all voxels predicted occupied in the previous step, generate the

---

Published in Transactions on Machine Learning Research (04/2026)
prediction set ((eest) C {2, M to satisfy the guarantee:
PP(Ytest test E (((Xtest) test -y a
The score function here is s(X. y) f()) We compute the quantile q² for class y as the [N ((++)10000)) quantile of the score on TV. where T% is the subset of the calibration dataset that has label y and are predicted as occupied on the geometric level of our HCP. Nyo = |T The prediction set is generated as:
Ng
C(Xtest) = {y: SM 8) S d A X y) - ).
(4)
Proposition 1. For a desired error rate a we select ay and a as - a = a²) (1- a.), then the prediction set generated as Eq. satisfies PeYtest E C(Xtest) Yestt = =y) 2 a².
The proof is in Appendix A.4
4
Experiments
OCC Model. We assess the effectiveness of our approach through comprehensive experiments on three different OCC models:
VoxFormer (Li et. al. (2023b)); OccFormer (Zhang et al. (202)) and CGFormer (Yul et al. (2024)). A detailed introduction to these three models is in Appendix A.2 Dataset. We use
SemanticKITTI (Behley et al. (2019), with 20 classes) and KITTI360 (Li et al. (2023a), with 19 classes).
(more details in Appendix A.55
Detailed experiment settings are in Appendix
A.6 Experiments the
Occ3D-nuSeenos (Caosar et al. (0200) dataset are in Appendix A.9
4.1 Uncertainty Propagation Performance
Table 1:
Performance evaluation of our Depth-UP three OCC models with two datasets. Values in parentheses indicate the improvement of our DP compared with the baseline.
Dataset
OCC
Method
IoU
Precision
Recall
mloU 1
VoxFormer
Base
44.02
62.32
59.99
12.35 13:36 (+1.01)
Our
45.85 (+1.83)
63. (+0.78)
62.64 (+2.65)
SemanticKITTI
Base
37.48
48.71
61.92
12.83
OccFormer
Our
41.64 (+4.16)
53.99 (+5.28)
64.54 (42.662
14.56 (+1.73)
CGFormer
Base
41.92
60.91
63.
15.82
Our
48.64 (+3.772)
67.35 (+6.44)
(+0.441)
17.02 (+1.20)
Base
38.76
57.67
54.18
11.91
VoxFormer
KITTI360
Our
43.25 (4-4.49)
65.81 (+7.29)
55.78 (+1.60)
13.55 (+11664
OGFormer
Base
48.21
69.98
60.78
19.11
Our
48.98 (+0.777)
68.29 (-1.69)
63.40 (42.62)
19.52 (+0.44)
Metric. For OCC evaluation, we use intersection over union (IoU) assess geometric completion, independent. of semantics, as it is critical for obstacle avoidance in AVs. Semantic performance measured by mean loU (mloU) across all classes. Given the strong negative correlation between IoU and mIoU (Li ct al. (2023b)),
effective model must perform well on both metrics.
The experimental results of our Depth-UP on VoxFormer, OccFormer and CGFormer are presented in Table I As OccFormer is not available on KITTI360, we evaluate with our Depth-UP only on SemanticKITTI These results demonstrate that Depth-UP effectively leverages quantified uncertainty from the depth model to enhance OCC model performance, achicving up to 4.49 (11.58% improvement in IoU and up to 1.73 (12.95%) improvement in mloU, while also significantly improving both precision and recall in the geometry completion aspect of OCC, Even minor improvements in IoU and mloU indicate meaningful progress in OCC performance (Zhang et al. (2023: Huang et al. (2023) ). A detailed breakdown of mIoU results for each class is provided in Appendix A.7
Figure g presents visualizations of the VoxFormer with and without our Depth-UP on SemanticKITTL As highlighted by the orange dashed boxes, Depth-UP notably enhances the performance of OCC models predicting rare classes, such as persons and bicyclists. Specifically, in the third row, Depth- UP successfully detects person crossing the road at the corner, preventing potential crash, while the baseline model fails
7

---

Published in Transactions on Machine Learning Research (04/2026)
to recognize this individual. By improving the detection of such critical instances, our Depth-UP significantly reduces the risk of injury to redestrians and enhances the safety of autonomous vehicles. Additional visualization results can be found in Appendix A.7
Camera View
Ground Truth
Baseline
Ours
road
sidewalk
parking
other-grad
building
vegetation
trunk
terrain.
traf-sign
person
bicyclist
motorcyclist
fence
track
bicycle
motoreycle
Figure Qualitative results of the base VoxFormer model and that with our Depth-UFF
4.2 Uncertainty Quantification Performance
We evaluate our HCP on the geometric level and the final uncertainty quantification. Since we do not have the labeled test part SemanticKITTI, we randomly split the original validation part of SemanticKITTI into the calibration dataset (take up 30%) and the test dataset (take up 70%). For KITTI360, we use the validation part as the calibration dataset and the test part as the test datasct.
Geometric Level. For the geometric level, the target of methods to achieve the best trade-off between IoU and the occupied recall of rare classes. To show the effectiveness of our novel KL-based score function the geometric level, we compare it with two common score functions in [Agellppollss & Bates 2021 :: class
score (1 - f(X)y) and occupied score (1 Ege f(X)y). Figure 5(a) shows the ToU results across different. occupied recalls of the rare class person for different datasets. Figure 5(b) shows the loU results across different occupied recalls of the rare class bicyclist for different basic OCC models. Here "Our Depth-UP! means the basic OCC model with our Depth-UP method. We can see that our KL-based score function always achieves the best geometry performance for the same occupied recall, compared with two baselines. Our HCP significantly outperforms baselines because not ouly considers the occupied probability across all nonempty classes but also leverages the entire probability distribution.
To achicve the optimal balance between IoU and occupied recall, can adjust the desired occupied recall. For instance, in the top right subligure of Figure 5(a), the OCC without HCP shows IoU of 45.85 and an occupied recall for the person class of 20.69. By setting the occupied recall to 21.75, the IoU improves to 45.94. Increasing the occupied recall beyond 30 (45.0% improvement) results in a decrease in IoU to 44.38 (3.4% reduction). This demonstrates that our HCP method can substantially boost the occupied recall of rare classes with a minor reduction ToU.
Uncertainty Quantification. To measure the quantified uncertainty of different CP methods, usually use the average class coverage gap (CovGap) and average set size (AvgSize) of the prediction sets metrics (Ding) et al. 2024 ). For a given class y E (1(11) with the defined error rate a", the empirical class-conditional coverage of class is Cy [T] Eierv I(Y, E C(X))) The CovGap is defined as 1 - Σ Evey((1) Icp - -
This measures how far the dass-conditionall coverage is from the desired coverage 1 a", The AvgSize is
The incorrect ground truth the third occurs because SemanticKITTI LIDAR temporal fusion for annotations, which results ghosting effects dynamic objects.
8

---

Published in Transactions on Machine Learning Research (04/2026)
Baseline
Ours
Baseline
Ours
60
60
60
50
50
Ouars Occupied
Semantick
50
50
40
40
40
Ocoupled
VoxFormer
30
30
20
20
20
10:
10
10:
10
20
J0
40
30
00
70
20
30
30
00
10
10
20
30
40
00
10
20
30
40
50
60
60
60
60
50
50
KITT1360
OccFormer
40-
40
30
30
30
30
20
20
20
10
10
20 40
20
60 70
0 20
60
10
20
40
60
Occupled Recall of Person
Occupled Recall of Bicyclist
(a)
(b)
Figure 5: Compare our KL-based score function with the class and occupied scores. Evaluate OCC's geometry performance across different. occupied recalls of the rare class. The red dotted line shows the ToU of the OCC without CP. (a): Results on the basic VoxFormer across different datasets for the person class. (b): Results on ScmanticKITTI across different basic OCCs for the bicyclist class.
defined as 7 Ei= C(X;)l, where is the number of samples the test datasel and C(X:) does not contain the empty class. A good UQ method should achieve both small CovGap and AvgSize.
Table compares our HCP method with SCP and CCCP on the SemanticKITTI dataset, introduced in Subsection 3.2.1 For each class, the desired error rate is set by scaling the OCC model's original error rate by the scale À = 0.86, raising the coverage requirement. The desired error rate of "Ours** is the same as that on the base OCC model. The corresponding results on KITTI360 dataset can be found in Appendix A.8
Our results demonstrate that HCP consistently achieves robust empirical class-conditional coverage and produces smaller prediction sets. In contrast, the performance of SCP and CCCP varies across different OCC models. Specifically, for our Depth-UP based on CGFormer, HCP reduces the set size by 90% and the coverage gap by 54%, compared to SCP. For our Depth-UP based VoxFormer, HCP reduces the set size by 79% and the coverage gap by 64%, compared to CCCP. As noted in Subsection 3.2.1 SCP consistently fails to provide conditional coverage, although sometimes it provides a very small set size. Both SCP and CCCP lend to generale nonemply C(X) for most voxels, potentially obstructing AVs. In contrast, HCP only generates nonempty C(X) for these selected occupied voxels, thereby minimizing prediction set sizes while maintaining reliable class-conditional coverage. Morcover, under the same error rate sotting, our HCP with the Depth-UP model (referred as "Ours**) achieves the best UQ performance, reducing the set size by up to 18% under similar coverage gap compared to applying HCP alone. This demonstrates that a-OCC framework, which integrates Depth-UP and HCP, achieves superior performance in both accuracy and UQ
Table For the SemanticKITTI, compare our HCP (referred to as "Ours") with SCP and CCCP on CovGap and AvgSize. "Ours** are results of our HCP + Depth-UI UP under the same defined error rate for the base model.
Basic OCC|
VoxPorterr
OccFormer
CGFormer
Method
Base
Our Depth-UP
Base
Our Depth-UP
Base
Our Depth-UP
CP
[SCP
(annsO0sssSSPPCCOO Ours /SCP|CCCP Ours|Ours= SCP|CCCP
|SCP|CCCP| Ours
CovGap
0.22
0.04
[0.26]
[0.66
0.03
0..04
0.04
00.31
0.04
0.03
[0.22
0.03
0.
AvgSize
1.53
1.71
1.13
0.93
0.97
6.43
1.36
0.10
3.42
0.94
0.80
0.10
2.96
1.24
3.63
3.00
1.67
1.33
5.19
3.31
1.72
9

---

Published in Transactions on Machine Learning Research (04/2026)
4.3 Ablation Study
Table 3: Ablation study on our Depth-UP UP framework with VoxFormer and SemanticKIT'l
PGC
PSS
IoU f
Precision f
Recall
mIoU
FPS t
44.02
62.32
59.99
12.35
8.85
44.91
60.30
12.58
|
44.40
62.69
60.35
12.77
|
8.76
45.85
63.
62.64
13.36
7.08
Uncertainty Propagation. We conducted an ablation study to assess the contributions of each technique proposed in our Depth-UR, UP, as detailed in Table The best results are shown in bold. The results indicate that Propagation on Geometry Completion (PGC) significantly enhances ToU, precision, and recall, which are key metrics for geometry. This improvement demonstrates how probabilistic geometry projection effectively utilizes uncertainly during the geometry stage of OCC. Additionally, Propagation on Semanttic Segmentation (PSS) markedly improves mloU, a crucial metric for semantic accuracy, demonstrating that uncertainty effectively leveraged in semantic segmentation through depth feature extraction. Notably, the combined application of both techniques yields performance improvements that surpass the sum of their individual contributions. This shows that uncertainty propagation in OCC completion and segmentation boosts performance, Additional results on the influence of depth mean can be found in Appendix A.7
Baseline
0.25
0.25
Ours
0.25
SCP
Semantici/TTI
0.20
0.20
CovGaD
0.20
Ours
0.15
0.15
0.15
0.10
0.10
0.05
0.05
1
CovGap
0.8
1.0
1.2
1.4
1.6
0.36
0.92
0.7
6-
0.5
AvgSize
1.4
KITT360
0.3
0.2
0.1
0.1
0.96
0.89
0.92
0.95
0.98
0.86
0.89
0.92
0.95
0.98
AvgSize
Scale
(a)
Figure 6: Compare our HCP with SCP and CCCP on CovGap and AvgSize with VoxFormer. Each point denotes a desired class error rate. Lower CovGap and AvgSize indicate better performance, (a): CovGap vs. AvgSize different settings across different datasets. (b): CovGap vs. scale and AvgSize scale the SemanticKITTI where the scale represents the desired class error rate.
Uncertainty Quantification. We compare our HCP with SCP and CCCP under different desired class- specific error rate a settings with the basic VoxFormer, as shown Figure G We consider five error rate scale settings with {0.86, 0.89, 0.92, 0.95, 0.98). The points of our HCP always located in the left bottom corner of subfigures in Figure 6 a) which means HCP achieves the best performance on set size and coverage gap under all error rate settings. In Figure 6 G(b), HCP always achieves low CovGap indicating it can always satisfy the coverage guarantec even under high requirements. SCP consistently has the largest CovGap due to its disregard for conditional coverage. While CCCP addresses this issue, it often results in a high AvgSize because it generates prediction sets for empty voxels. In contrast, our HCP method not only accounts for conditional coverage but also excludes empty voxels, generating prediction sets exclusively for predicted occupied voxels. For all CP approaches, as the desired error rate becomes smaller, the set size tends to be larger. CPs increase the set size to satisfy the coverage guarantee. The results
10

---

Published in Transactions on Machine Learning Research (04/2026)
other OCC models are shown in Appendix A.8 where our HCP is applied to one LiDAR-based OCC to show its scalability.
Limitation. Our Depth-UP introduces a 20% reduction in FPS; however, this does not substantially impact the overall efficiency of OCC models. Notably, we did not apply any code-level optimization to improve runtime, indicating the added computational overhead is acceptable.
5 Conclusion
This paper introduces novel approach to enhancing camera-based 3D Semantic Occupancy Prediction by incorporating uncertainty inherent in models. Our proposed framework, a-OCC, integrates the Depth-UP from depth models to improve OCC performance in both geometry completion and semantic segmentation. A novel HCP method is designed to quantify OCC uncertainty effectively under high-level class imbalance. Our extensive experiments demonstrate the effectiveness of a-OCC. The Depth-UP siguificantly improves prediction accuracy, achieving up to 11.58% increase in IoU and up to 12.95% increase in mloU. The HCP significantly improves performance by ensuring robust class-conditional coverage while keeping compact prediction sets. Compared to baselines, it achieves up 90% reduction in set size and decreases the coverage gap by up to 64%. Additionally, integrating HCP with Depth-UP further reduces the set size by 18%. These results highlight the significant improvements in both accuracy and uncertainty quantification offered by our approach, especially for rare safety-critical classes, such as persons and bicyclists, thereby reducing potential risks for AVs. In the future, we will extend HCP to other highly imbalanced classification tasks.
Acknowledgments
Sanbao Su, Chenchen Lin and Fei Miao are supported by the National Science Foundation under Grants CNS-2047354 (CAREER). Chen Feng is supported by the National Science Foundation under Grants 2514030 and 2238968 (CAREER).
This material is based upon work supported under the AI Rescarch Institutes program by National Science Foundation and the Institute of Education Sciences, U.S. Department of Education through Award # 2229873 National AT Institute for Exceptional Education. Any opinions, findings and conclusions recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, the Institute of Education Sciences, or the U.S. Department of Education.
References
Anastasios N Angelopoulos and Stephen Bates. A gentle introduction to conformal prediction and distribution- free uncertainty quantification. arXiv preprint arXi:22107.07511, 2021.
George B Arfken, Hans J Weber, and Frank E Harris. Mathematical methods for physicists: comprehensive guide. Academic press, 2011.
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEFE transactions on pattern analysis and machine intelligence, (12):2481-249)) 2017.
Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Semantickitti: dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEFE/ /CFF international conference on computer vision, pp. 9297-9307, 2019.
Shariq Farooq Bhat, Tbraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4009 4018, 2021.
Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106:249-2500 259,, 2018
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Amush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving.
11

---

Published in Transactions on Machine Learning Research (04/2026)
In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 11621-11631, 2020.
Anh- Quan Cao and Raoul De Charette. Monoscene: Monocular 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitàon, pp- 3991-4001, 2022.
Anh-Quan Cao, Augela Dai, and Raoul de Charette. Pasco: Urban 3d panoplic scene completion with uncertainty awareness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14554 14564, 2024.
Robin Chan, Matthias Rottmann, Fabian Hüger, Peter Schlicht, and Hanno Gottschalk. Application of decision rules for handling class imbalance in semantic segmentation. arXiv preprint wrXiv:1911.0899, 2019.
Bike Chen, Chen Gong, and Jian Yang. Importance-aware semantic segmentation for aulonomous vehicles. IEEE Transactions on Intelligent Transportation Systems, 20(1):137-144 148, 2018.
Ran Cheng, Christopher Agia, Yuan Ren, Xinhai Li, and Liu Bingbing. S3cnet: A sparse semantic scene completion network for lidar point clouds. In Conference on Robot Learning, pp. 2148 2161. PMLR, 2021.
Tiffany Ding, Anastasios Angelopoulos, Stephen Bates, Michael Jordan, and Ryan J Tibshirani. Class- conditional conformal prediction with many classes. Advances in Neural Information Processing Systems, 36, 2024.
David Eigen and Rob Fergus.
Predicting depth, surface normals and semantic labels with common multi- scale convolutional architecture. In Proceedings of the IEEE international conference computer vision, pp. 2650-2655 2015.
Abdelrahman Eldesokey, Michael Felsberg, Karl Holmquist, and Michael Persson. Uncertainty-awant cnns for depth completion: Uncertainty from beginning to end. In Proceedings of the TREE/CVF Conference Computer Vision and Pattern Recognition, pp. 12014 12023, 2020.
Di Feng, Ali Harakeh, Steven L Waslander, and Klaus Dietmayer. A review and comparative study on probabilistic object detection in autonomous driving. IEEE Transactions on Intelligent. Transportation Systems,
2021.
Andreas Geiger, Philip Lenz, and Raquel Urlasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pp. 3354 3361. LEEE, 2012.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pp. 1321 1330. PMLR, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Sihong He, Songyang Han, Sanbao Su, Shuo Han, Shaofeng Zou, and Fei Miao. Robust multi-agent reinforcement learning with state uncertainty. arXic preprint 1rXi:22307.62212 2023.
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Plamning-oriented autonomous driving. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17853-17888 2023.
Yuanhui Huang, Wenzhao Zheng, Yumpeng Zhang, Jie Zhou, and Jiwen Lu. Tripperspective view for vision- based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9223-9232. 2023.
Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. Selfoce: Sclf-supervisod vision-basod 3d occupancy prediction. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19946- June 2024.
12

---

Published in Transactions on Machine Learning Research (04/2026)
Ashkan M Jasour and Brian C Williams. Risk contours map for risk bounded motion planning under perception uncertainties. In Robolies: Science and Systems, pp. 22-26, 2019.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainly estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Yiming Li, Sihang Li, Xinhao Liu, Moonjun Gong, Kenan Li, Nuo Chen, Zijun Wang, Zhiheng Li, Tao Jiang, Fisher Yu, et al. Sscbench: A large-scale 3d semantic scene completion benchmark for autonomous driving. arXiv preprint SXt::2306009001 2023a.
Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and Anima Anandkumar. Voxformer: Sparse voxel transformer for camera-based 3d semantic scene completion. In Proceedings of the (EEE/CVF conference on computer vision and pattern recognition, pp. 1087-9098, 2023b.
Yinhao Li, Han Bao, Zheng Ge, Jinrong Yang, Jianjian Sun, and Zeming Li. Bevstereo: Enhancing depth estimation in multi-view 3d object detection with temporal stereo. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 1486-1494, 2023c.
Zhiqi Li, Wenhai Wang, Hongyang Li, Eaze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird's serd representation from multi-camera images via spatiotemporal transformers. In European conference on computer vision, pp. 1 18. Springer, 2022.
Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3): -310, 2022.
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
Yan Lu, Xinzhu Ma, Lei Yang, Tianzhu Zhang, Yating Liu, Qi Chu, Junjic Yan, and Wanli Ouyang. Geometry uncertainty projection network for monocular 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3111-3121 2021.
Laurent Lucas, Céline Loscos, and Yannick Remion. Camera calibration: geometric and colorimetric correction.
3D Video: From Capture to Diffusion, pp. 91- 111-12, 2013.
Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Paltern Recognition (CVPR), pp. 19936 19945, June 2024.
Fadel M Megahed, Ying-Ju Chen, Aly Megahed, Yuya Ong, Naomi Altman, and Martin Krzywinski. The class imbalance problem. Nat Methods, 18(11):1270- 7, 2021.
Gregory P Meyer and Niranjan Thakurdesai. Learning an uncertainly-aware object delector for autonomous driving. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10521-10527. IEEE, 2020.
Dimity Miller, Lachlan Nicholson, Feras Dayoub, and Niko Sünderhauf. Dropout sampling for robust object detection in open-set conditions. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3243 3249. IEFF, 2018.
Jingyi Pan, Zipeng Wang, and Lin Wang. Co-oce: Coupling explicit feature fusion with volume rendering regularization for modti-modal 3d semantic occupancy prediction. IEEE Robotics and Automotion Letters, 9(6):5687-5644, 2024a.
Mingjie Pan, Jiaming Liu, Renrui Zhang, Peixiang Huang, Xiaoqi Li, Hongwei Xic, Bing Wang, Li Liu, and Shanghang Zhang. Renderoce: Vision-centric 3d occupancy prediction with 2d rendering supervision. In 2024 IEFE International Conference on Robotics and Aulomation (ICRA), pp. 12404-1241 TEFE, 2024b.
13

---

Published in Transactions on Machine Learning Research (04/2026)
Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In Computer Vision-F.CCO 2020: 16th European Conference, Glasgow, UK, August 28, 2020, Proceedings, Part XIV 16, pp. 194 210. Springer, 2020.
Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mattoccia. On the uncertainty of self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3227 3237, 2020.
Fiana Raiber and Oren Kurland. Kullback-leibler divergence revisited. In Proceedings of the ACM SIGIR internalional conference on theory of information relrieval, pp. 117- 1774124 2017.
Luis Roldao, Raoul de Charette, and Anne Derroust-Blondct. Lmsenet: Lightweight multiscale 3d semantic completion. In 2020 International Conference 3D Vision (8DV), pp. 111-119. IEEE, 2020.
Faranak Shamsafar, Samuel Woerz, Rafia Rahim, and Andreas Zell. Mobilesterconet: Towards lightweight deep networks for stereo matching. In Proceedings of the ieee/cof winler conference on applications of computer vision, pp. 1417-2420 2022.
Shuran Song, Fisher Yu, Andy Zeng, Angel X Chang, Manolis Savva, and Thomas Funkhouser. Semantic scene completion from a single depth image. In Proceedings of the IEFE conference on computer uision and pottern recognition, pp. 1746 1754, 2017.
Sanbao Su, Yiming Li, Sihong He, Songyang Han, Chen Feng, Caiwen Ding, and Fei Miao. Uncertainty quantification of collaborative detection for self-driving. In 2023 IEEE International Conference on Robolics and Automation (ICRA), pp. 5588- 5594. IEEE, 2023.
Sanbao Su, Songyang Han, Yiming Li, Zhili Zhang, Chen Feng, Caiwen Ding, and Fei Miao. Collaborative multi-object tracking with conformal uncertainty propagation. IEEF Robotics and Automation Letters, 2024.
Sanbao Su, Xin Li, Thang Doan, Sima Behpour, Wenbin He, Liang Gou, Fei Miao, and Liu Ren. Metaat: Active testing for label-efficintt evaluation of dense rocognition tasks. In European Conference on Computer Vision, pp. 325-342 Springer, 2025.
Mingkui Tan, Zhuangwei Zhuang, Sitao Chen, Rong Li, Kui Jia, Qicheng Wang, and Yuanqing Li. Epmf: Efficient perception-aware multi-sensor fusion for 3d semantic segmentation. IEEE Transactions on Pallern Analysis and Machine Intelligence, 46(12) (1255-8273, 2024.
Junjiao Tian, Yen-Cheng Liu, Nathaniel Glaser, Yen-Chang Hsu, and Zsolt Kira. Posterior re-calibration for imbalanced datasets. Advances in neural information processing systems, 33:8101-8113, 2020.
Xiaoyu Tian, Tao Jiang, Longlei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Newral Information Processing Systems, 36, 2024.
Jason Van Hulse, Taghi M Khoshgoftaar, and Amri Napolitano. Experimental perspectives on learning from imbalancod data. In Proceedings of the 24th international conference on Machine learning, pp. 935-942, 2007.
Antonin Vobecky, Oriane Siméoni, David Hurych, Spyridon Gidaris, Andrei Bursuc, Patrick Pérez, and Josef Sivic. Pop-3d: Ooca-vocabulary 3d occupancy prediction from images. Advances in Neural Information Processing Systems, 36, 2024.
Lele Wang and Yingping Huang. A survey of 3d point cloud and deep learning-based approaches for scene understanding in autonomous driving. IEEE Intelligent Transportation Systems Magazine, 14(6):135 154,
2021.
Yuqi Wang, Yuntao Chen, and Zhaoxiang Zhang. Frustumformer: Adaptive instanco-aware resampling for multi-view 3d detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 5096-5105, 5105, 2023.
14

---

Published in Transactions on Machine Learning Research (04/2026)
Yuqi Wang, Yuntao Chen, Xingyu Liao, Lue Fan, and Zhaoxiang Zhang. Panoocc: Unified occupancy representation for camera-based 3d panoptic segmentation. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pp. 17158 17168, 2024.
Songlin Wei, Haoran Geng, Jiayi Chen, Congyue Deng, Cui Wenbo, Chengyang Zhao, Xiaomeng Fang, Loonidas Guibas, and He Wang. D3roma: Disparity diffusion-bascd depth scnsing for material-agnostic robolic manipulation. In ECCV 2024 Workshop on Wild 3D: 3D Modeling, Reconstruction, and Generalion in the Wild, 2024.
Wenda Xu, Jia Pan, Junqing Wei, and John M Dolan. Motion planning under uncertainty for on-road autonomous driving. In 2014 IEEE International Conference Robotics and Automation (TCRA), pp- 2507 2512. IEEE, 2014.
Xu Yan, Jiantao Gao, Chaoda Zheng, Chao Zheng, Ruimao Zhang, Shuguang Cui, and Zhen Li. 2dpass: 2d priors assisted semantic segmentation on lidar point clouds. In European conference on compuler vision, pp. Springer, 2022.
Zhu Yu, Runmin Zhang, Jiacheng Ying, Junchen Yu, Xiaohai Hu, Lun Luo, Si- Yuan Cao, and Hui- Liang Shen. Context and geometry aware voxel transformer for semantic scene completion. ar preprint or 1756:2405513665, 2024.
Haiming Zhang, Xu Yan, Dongfeng Bai, Jiantao Gao, Pan Wang, Bingbing Liu, Shuguang Cui, and Zhen
Li. Radocc: Learning cross- roos-modallity occupancy knowledge through rendering assisted distillation. Proceedings of the AAAT Conference on Arlificial Intelligence, volume 38, pp. 7060 7068, 2024.
Yunpeng Zhang, Zheng Zhu, and Dalong Du. Oceformer: Dual-path transformer for Assionbasse 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9433-9443, 2023.
Zhengyou Zhang. A flexible new technique for camera calibration. IEEE Thansactions pattern analysis and machine intelligence, 22(11):1330-1338 2000.
15

---

Published in Transactions on Machine Learning Research (04/2026)
A Appendix
A.1 More Related Work
Class Imbalance. In real-world applications like robotics and autonomous vehicles (AVs), datasets often face the challenge of class imbalance |Chen et al. (2018)). Rare classes, typically encompassing high safety-critical entities such as persons, are significantly outnumbered by lower safety-criticll classes like trees and buildings. Various strategies have been proposed to tackle class imbalance. Data-level methods involve random under- sampling of majority classes and over-sampling of minority classes during training (Van Hulse et. al. (2007)
However, they struggle to address the pronounced class imbalance encountered in OCC (Megahed et al. (021
as shown Scction [ Algorithm-lovel methods employ cost-sensitive losses to adjust the training process for different lasks, such as depth estimation (Eigen & Fergus (2015) and 2D segmentation (Badrinaraynnan el al.
(017 ). While algorithm-level methods have been widely implemented in current OCC models (Voxformer Li et al. 2023b) utilizes Focal Loss Lin et al. (2017) the loss function), they still fall short in accurately predicting minority classes. In contrast, classifier-level methods postprocess output class probabilities during the testing phase through posterior calibration (Buda et. al. (2018); Tian et. al. (002))) ). In this paper, we propose a hierarchical conformal prediction method falling within this category, aimed at enhancing the recall of rare safety-critical classes in the OCC task.
A.2 Introduction Semantic Occupancy Prediction Models
Semantic Occupancy Prediction. The concept of 3D Semantic Occupancy Prediction (OCC), which is also known 3D semantic scene completion, was first introduced by SSCNet (Song et al. (2017)) integrating both geometric and semantic reasoning. Since its inception, numerous studies have emerged, categorized into
two streams: LiDAR-based OCC (Roldao el. al. (2020); Cheng el. al.] (2021) and camera-based OCC (Cao &
De Charette (002)] Li et al.
2023bb Zhang et al. 2023 Huang et al. (202)) Vobecky et al. 2024
Recently, casera-based OCC has gained increasing attention due to its advantages in visual recognition and cost-effectiveness (Ma et al. (2024)) ). Depth predictions from depth models are instrumental in projecting 2D information into 3D space for it. Existing methodologics can be classified into two paradigms based on their utilization of depth information:: 3D-to-2D querying and 2D-to-3D lifting. The former Li et al. (2023) 2022 generates query proposals using depth estimation and leverages them to extract rich visual features from the 3D scene. The latter Tian et al. (2024); Zhang et al. (2023): ;; Yu et al. (2024), meanwhile, projects multi-view 2D image features into depth-aware frustums, as proposed by LSS Philion & Fidler (220)).
However, these methods overlook depth estimation uncertainty. Despite leveraging latent depth distribution, 2D-to-3D lifting technique sacrifices precise information and neglects lens distortion issues during geometry completion (Lucas et al. (2013)). To address this, we propose an uncertainty propagation framework from depth models to enhance OCC performance.
During the experiments, we used three OCC models: VoxFormer Li et al. 2023b)), OccFormer Zhang et al. (023 and CGFormer (Yu et al. (2024)). VoxFormer is the 3D-to-2D querying approach. OccFormer and CGFormer are the 2D-to-3D lifting approaches. So we have considered both paradigms that utilize depth information on OCC models in our experiments.
Recent advancements have explored leveraging diverse data sources and training strategies to enhance occupancy accuracy. 2DPASS (Yan et al. (202)) demonstrates the utility of using 2D priors to assist 3D semantic sogmentation on LiDAR point clouds. EPMF (Tan et al. (2024)) focuses on efficient multi-scnsor fusion for robust perception. And Recent works like Co-Oce (Pan el al. (2024a)) and RadOce (Zhang) et al. 2024b) introduce volume rendering and knowledge distillation to couple explicit feature fusion with global scene consistency in OCC. While these methods improve base performance, they remain primarily deterministic. Our OCC framework complementary to these architectures, providing a layer of uncertainty awareness that enhances robustness for OCC.
16

---

Published in Transactions on Machine Learning Research (04/2026)
A.3 Algorithm of HCP
Algo. I
shows the detailed algorithm of our HCP.
A.4 Proof of Proposition 1
Proposition 1. For a desired at value, we select ag and a as 1 - -0 = (1 11-aa a1 (1-))1o0)))) o.), then the prediction
set generated as Eq. 4 satisfies that P( E C(X(est) Yeest v) 2 1 a².
Proof.
P(Ytest test E C(Xtest) test = y) Σ P(Y E (((X)test) ((tsst) Y = y, P(o =
=P(Ytost tost E C(Xtest) test = y, 0 = P) = T Y test = 1)
Peytts test E ((Xtest)] Y test = 0 F) F test = y)
P( test E C(Xtest) ( Yeest test = P) T Y lest
:) 2 (1 (1-
+ P(Y test E C(Xtest) Yestt= =y) 2 - a² when 1 - a² = =(1 =1--o)) a.)
A.5 Introduction on Datasets
During the experiments, we used two datasets: SemanticKITTI (Behley et al. (2019) and KTTT1360 [Li et al. (2023a). SemanticKITTI provides dense semantic annotations for each LiDAR sweep composed of 22 outdoor driving scenarios based on the KITTI Odometry Benchmark (Geiger et al. 2012 Regarding the sparse input to an OCC model, it can be either single voxelized LiDAR sweep grids are labeled with 20 classes (19 semantics and 1 empty), with the size of 0.2m x 0.2m 0.2m. We only used the train and validation parts of SenTanicKKTTI as the annolations of the test part are not available. SSCBench-KITTTI336 provides dense semantic annotations for each image based on KITTI360 (Liao et al. (2022)), which also called KITTI360 for simplification. The voxel grids are labeled with 19 classes (18 semantics and 1 empty), with the size of 0.2m x 0.2m 0.2m. Both SemanticKITTI and KITTI360 are interested in volume of 51.2m ahead of the car, 25.6m to left and right side, and 6.4m in height.
RGB image. The voxel
A.6 Experimental Setting
We used two different servers to conduct experiments on the SemanticKTTTI and KITTT360 datasets. For the SemanticKITTI dataset, we employed a system equipped with four NVIDIA Quadro RIX 8000 GPUs, each providing 48GB of VRAM. The system was configured with 128GB of system RAM. The training process required approximately 30 minutes per epoch, culminating in a total training duration of around 16 hours for 30 epochs. The software environment included the Linux operating system (version 18.04), Python 3.8.19, CUDA 11.1, PyTorch 1.9,1+cull1, and CuDNN 8.0.5.
For the KITTI360 dataset, we used a different system equipped with eight NVIDIA GeForce RTX 4090 GPUs, each providing 24GB of VRAM, with 720GB of system RAM. The training process required approximatcly 15 minutes per epoch, culminating in a total training duration of around 8 hours for 30 epochs. The software enviromment comprised the Linux operating system (version 18.04), Python 3.8.16, CUDA 11.1, PyTorch 1.911 and CuDNN 8.0.5. These settings ensure the reproducibility of our experiments on similar hardware configurations.
In our training, we used the AdamW optimizer with a learning rate of 2e-4 and a weight decay of 0.01. The learning rate schedule followed a Cosine Anncaling policy with a lincar warmup for the first 500 itcrations, starting a warmup ratio of 3. The minimum learning rate ratio was set 1c-3. We applied gradient clipping with maximum norm of 35 to stabilize the training.
17

---

Published in Transactions on Machine Learning Research (04/2026)
Algorithm 1: Our HCP
Data: number of classes is M, calibration dataset Deats (X, Y) with N samples, test dataset Dtest(X)
with samples, the considered rare class set Dr, the occupied error rate a vy E Vr ,desired class-specifical error rate a Vy E MM1,,, the OCC model
Result: Prediction set C(X,), VX, E Diest
/* Calibration Step: Geometric Level
S 0 Vy E J/; O {e, (e,,,,,,, 13M;
for (X,, Y;) E Deals do
Sal (X, y) = (aa(f(X))
-0=YE EJE Eq. 2 add SA((X,)) v) into SV;
end
Ø = Quantilo +11 a 5*) where N, = |S Vy E Vi:
N, / Calibration Step: Semantic Level
S 0, Ipy = and fny Vy E {1);; for (X,, Y.) Deals and Y, E [1) do
10
if By E Vr, su1 saa (X, S o then
add 1 - x(X,)x, into 8 and fpy, = fpx. + 1;
11
12
else
13
fny. = fny, + 1;; end
14
15 end
16 for y E y([1} do
a.!² 11 tpy if y $ yr
17
tpy+
a²!-
T-og 9 Quantile(
Sy) where Ny |
Nyo
19 end
20
/* Test Step
21 for X; E Drest do
22
if By E Vr, sui (X.) 9) S 9% then
C(X,) ty f(X,), ) y S ]
clsc
25
C(X,) = 0 which means il is emply class.
26
end
27 end
The user-defined target error rate a for each class y is decided according to the prediction error rate of the original model. For each class, It is set by multiplying the original prediction error rate of OCC models with the scale À < 1, which raises the coverage requirement. For example, for the person class, if the original model has 90% prediction error rate and we set the scale À = 0.9, the user-defined target error rate o person of person is docidod as 90% x 0.9 = 81%.
A.7 More Results on Depth-UP
Table presents comparative analysis of our Depth-UP models against various OCC models, providing detailed mloU results for different classes. Our Depth-UP demonstrates superior performance in geometry completion and semantic sogmentation, outperforming all other OCC models and even surpassing LiDAR- based OCC models on the SemanticKITTI dataset. The VoxFormer with our Depth-UP achieves the best loU on SemanticKITTI and the OccFormer with our Depth-UR achieves the best mloU on SemanticKITTL This propagating the inherent uncortainty of the depth model into both geometry
improvement attributed completion and semantic segmentation of OCCs. Notably, on the KITTI360 dataset, our Depth UP achieves the highest mloU for bicycle, motorcycle, and person classes, which are crucial for safety.
18

---

Published in Transactions on Machine Learning Research (04/2026)
For the person and bicyclist categories the SemanticKITTI dataset, our Depth-UP decreases the mloU. This issue primarily stems from annotation defects, particularly for dynamic objects such as persons and bicyelists. The SemanticKITTI dataset generates annotations using LiDAR temporal fusion, which introduces ghosting effects for moving objects. This problem has been documented in Figure of the SSCBench (Li jet al. (2023a)). While cars are also alfected, most are stationary, so the impact is minimal. However, nearly all persons and bicyclists in the SemanticKITTI validation set are moving, leading to erroncous annotations. SSCBench has acknowledgee this issue, and thus KITTI360 proposed in SSCBench does not suffer from ghosting problems. Our Depth-UP shows mloU improvements in both person and bicyclist categories on KTTT1360, aligning with our expectations. This may also explain why our Depth-UP enhances VoxFormer significantly more on KITTI360 compared to SemanticKITTI, showing a 1.64 mloU improvement versus a 1.01 mloU improvement.
Visualization. Figure provides additional visualizations of the OCC model's performance with and without our Depth-UP on the SemanticKITTI dataset. These visualizations demonstrate that our Depth- UP enhances the model's ability to predict rare classes, such as persons and bicyelists, which are highlighted with orange dashed boxes. Notably, in the fourth row, our Depth-UL UP successfully predicts the presence of a person far from the camera, whereas the baseline model fails to do so. This indicates that Depth-UP improves object prediction in distant regions. By enhancing the detection of such critical objects, our Depth- significantly reduces the risk of accidents, thereby improving the safety of autonomous vehicles.
Ablation Study. We further conducted the ablation study on the influence of depth information on the semantic feature part of our Depth-UD framework, as shown in Table | When Propagation on Geometry Completion (PGC) is not applied, using only the depth mean in the Propagation on Semantic Segmentation (PSS, second row) improves IoU by 0.85 and mloU by 0.16. When PGC is applied, compared to using only PGC (fourth row), incorporating only the depth mean in PSS (fifth row) increases IoU by 0.45 and mIoU by 0.08. These results indicate that the depth mean enhances geometric performance (loU) and slightly improves semantic performance (mloU). In contrast, incorporating depth uncortainty into PSS further improves mIoU by 0.26 when PGC is not applied and by 0.7 when PGC is applied. This demonsttrtts that leveraging depth uncertainty in PSS significantly enhances semantic performance. The depth mean primarily accounts for the improvement in IoU, suggesting a partial decoupling of the effects of uncertainty. One additional observation is that when PGC is not applied, PSS with only the depth mean achieves higher IoU than the complete PSS, due to an imbalance in precision and recall. However, since the complete Depth-UP achieves the best. performance in both IoU and mloU across all settings, incorporating depth uncertainty into semantic features remains essential for optimal performance.
The depth mean estimation D in our OCC model remains identical to that in the corresponding base OCC model, as we only retrained the additional regression head for Σ in the depth model. The improvement achieved by our OCC model does not result from nonexistent enhancements to the depth mcan estimation; rather, it arises from effectively incorporating the uncertainty of depth estimation into the OCC models.
Discussion.
Table 4: Ablation study on the depth information for our Depth-UP framework with VoxFormer and SemanticKITTI.
PGC
PSS
IoU f
Precision
Recall
mIoU
FPS
62.32
59.99
12.35
mean
44.87
59.8
64.25
| 12.51
8.77
mean & std
44.40
62.69
60.35
| 12.77
8.76
11.91
63.76
60.30
12.58
7.14
45.96
62.76
62.06
12.66
7.08
mean & std
45.85
63.10
62.64
13.36
7.08
19

---

Published in Transactions on Machine Learning Research (04/2026)
Table 5: Separate results on SemanticKITTI and KITTI360. We evaluate our Depth-UL UP models on two datasets. The default evaluation range is 51.2x51.2x6. 2.. Due to the label differences between the two subsets, missing labels are replaced with K-". "Depth-UP"* means the VoxFormeer with our Depth-UP method. "Depth-UPf* means the OccFormer with our DPpth-UP method. "Depth-UP! means the CGFormer with our UP method. The top three performances on each dataset are marked by red, green, and blue respectively.
motorcyolles
Dataret
Mothod
Input
bieyyele
motoreyelt
other-vah.
rowd
packing
sidewalx
other-graa
building
vegetation
pole
traf-sigo
blogollet
10U
LMSCNet
1 44
14.77
30.58
0.00
0.00
2.11
0.08
0.00
57.41
12.86
31.80
0.00
34.69
9.32
11.38
38.86
19.52
0.43
0.00
23.52 0.00
1. 40.93
10.27
MonoScene
C 36.80
11.30
28.29
0.69
9:29
2.68
65.89
14.76
26.60
1.68
19.55
0.60
17.95
29.84
3.91
2.43
1.07
0.00
VoxFormer
44.02
3.77
04.76
26.35
7.64
24.30
20
4:15
0.00
Semantiak
TPVFommer
OocPormer
C 36.50
1.19
68.85
26.88
19:68
2.80
0.00
CGFormer
C 44.92
15.82
34.07
2.87
4.26
6.80
6.03
1.51
68.48
21:27
$3.06
0.00
21.77
9.18
25.87
40.18
10.78
7.02
8:18
8.62
0.00
Dpp
[ones]
C 45.85
13.36
26.01
0.12
3.67
12:01
4.23
2.24
00.73
14.35
26.20
0.10
20.58
7.70
26.24
30.26
8.03
5.81
1:38
7:03
0.00
Depth-UD? (onrs)
C 41.97
14.56
26.53
1.12
1.54
10.84
9:37
2.655
65:38
21.58
29.79
3.97
18.85
7.09
24.68
$4.09
7.86
5.82
1.61
7:40
0.00
Depth-UP (ours)
C 48.04
17.02
94.84
2.43
0.40
6.26
10.07
2:20
00.47
20.47
99.12
0.26
27
27.80
10.11
20.82
42.72
12:12
7.71
2.80
8.52
0.00
LMBCNet
I. 47.33
13:88
26.80
88 et.
L 5.5.5a
16.95
$1.95
0.17
10.29
0.58
0.07
65.7
17.33
41:24
1..2
44.41
6.77
45.72
28.87
0.78
0.75
MoacSecaee
C 87.87
12.31
19.34
0.48
0.68
8.02
2.08
0.80
48.35
11:36
26.13
8.22
92.89
8.68
26.16
16.75
6.92
5.67
Vive Pormerr
C 38.76
11:91
17:84
1.16
0.89
4.55
2.00
1.63
07.01
9.67
27.31
3.89
31.18
4.97
25.98
14.09
6:51
6.92
OGPormer
C 48.21
19:11
29.82
7.77
4.95
12.77
8.39
6.74 65.52
18.52
40.71
4.71
42.84
8.34
38.82
24.30
16.51
19.29
Dopth-UPp [oucs]
C 43:25
13.56
22.32
1.90
1.68
0.43
2.27
8.18
63.50
11.86
81.68
8.20
34.49
6.11
32.01
18:78
11.40
18.65
Dep [ours) [ 48.98 18.53
29.63 4.10 4.07 16,80 7.88 6.05
63.12
18.12
40.70
5.80
43.74
9.02
39.47
24.53
17.99 20.48
Table 6: For the KITTI360 and VoxFormer, compare HCP (referred to as "Ours') with SCP and CCCP CovGap and AvgSize. "Ours*' are results of our HCP + Depth-UP under the same defined error rate for the base model.
Method
Base
Our Depth-UP
CP
CCCP)OOs Ours
SCP [CCCP]Ours
CovGap
100.4
0.26
00.10
0.08
0.62
0.25
0.100
AvgSize
66.30
1.03
0.56
0.49
13.24
1.51
1.12
A.8 More Results on HCP
Geometric Level. Similar to Figure we evaluate the geometry performance of our HCP module by comparing the proposed KL-based score function against standard class score and occupied score baselines for the bicycle and motorcyele classes. As shown in Figure g we asscss the geometry performance (IoU) of the VaxFormer baseline across varying levels of occupied recall for these rare classes. The results demonstrate that our KT--based score function consistently maintains superior geometry performance across the entire range of occupied recalls for all safety-critical rare classes. By effectively capturing the nuances of the probability distribution through our hierarchical approach, the HCP ensures that these rare entities are recovered with significantly higher precision than traditional scores. This consistent performance across diverse classes and datascts underscores the robustness and generalizability of our uncotaiitty-awary framework in addressing the extreme class imbalance inherent in 3D occupancy tasks.
More Discussion. Our HCP significantly outperforms baselines because it not only considers the occupied probability across all nonempty classes but also leverages the entire probability distribution. Compared with the class score, which only considers individual class probabilities, our score function accounts for all nonempty classes. Predicting rare classes is challenging for models, but they tend to identify these as occupied, assigning lower probabilitics to the empty class and higher probabilities to all nonempty classes. Therefore, it's crucial to consider the probability of all nonempty classes. Although the occupied score addresses this by summing probabilities of all nonempty classes, it loses sensitivity to the distribution. When facing difficult.
20

---

Published in Transactions on Machine Learning Research (04/2026)
Camera View
Ground Truth
Baseline
Ours
road
silewalk
parking
other-grad
building
vegetation
trunk
terrain.
traf.-ign
car
bicyclist
motorcyclist
fence
truck
bleyele
motorcycle
pole
Figure 7: Qualitative results of the baseline OCC model and that with our Depth-UP method.
samples (such rare classes), models tend to produce output probabilities that more evenly distributed across the possible classes (Guo et al. (2017)). The Kullback-Leiillr (KT.) divergence measures how one probability distribution diverges from reference distribution, considering the entire shape of the probability distribution (Raiber Kurland (2017)) This sensitivity to distribution shape enables our KL-based score function to identify rare classes more effectively.
Uncertainty Quantification. Similar Table 2 we compare our HCP method with SCP and CCCP on the KITTI-360 dataset in Table 0 For each class, the desired error rate is obtained by scaling the original error rate of the OCC model using factor of A 0.86, thereby increasing the coverage requirement. The desired error rate for *Ours*" remains identical to that of the base OCC model. Our results demonstrate that HCP consistently achieves robust. empirical class-conditional coverage while producing significantly smaller prediction sets. In contrast, the performance of SCP and CCCP varies across different OCC models. Spocifically, for our Depth-UP model based on VoxFormer, HCP reduces the prediction size by 92% and the coverage gap by 84% compared to SCP. For the base VoxFormer model, HCP reduces the set size by 62% and the coverage gap by 46% compared to CCCP.
21

---

Published in Transactions on Machine Learning Research (04/2026)
Baseline
Ours
Baseline
Ours
50
50
40
40
40
SemanticK
20
30
30
20
Class Score
Ours
Occupled Score
20
30
20
Cass Score
Ours
Occupied Score
10-
10
10
10
20
30
40
50
60
70
20
30
40
50
60
70
30
40
50
60
70
80
90
30
40
60
50
50-
40
KIT11660
30
30
20
20
20
10
10
10
0
20
30
40
50
60
20
30
40
50
70
60
Occupied Recall of Bicycle
Occupied Recall of Motorcycle
(a)
Figure 8: Compare our KL-based score function with the class and occupied scores. Evaluate OCC's geometry performance across different occupied recalls of the rare class on the basic VoxFormer across different datasots. The red dotted line shows the IoU of the OCC without CP. (a): Results for the bicycle class. (b): Results the motoreycle class.
Basic OccFormer
Our OccFormer
Basic CGFormer
CGFormer
LMSCNet
0.35
0.16
0.30
0.40
0.14
CovGap
0.17
0.10
0.00
0.10
0.10
SDP
Ous
0.05
OCOP
0.
05
100
155
2.0
25
3.0
35
0.0
0.5
1.0
15
2.0
2.5
3/2
1.5
2.0
25
3,0
3,5
0.02
155
2.0
25
3.0
35
40 45 5.0
05 3.0
1.5
2.0
25 30 35 6.0
AugSize
Figure 9: The results of CovGap vs, AvgSize for our HCP, SCP and CCCP on SemanticKITTIL The considered OCC models are the basic OccFormer, the OccFormer with our Depth-UP, the basic CGFormer, the CGFormer with Depth-UP, and the LiDAR-based OCC model LMSCNet..
Ablation Study. We compare our HCP with SCP and CCCP under different desired class-specific error rate settings on more OCC models: the basic OccFormer, the OceFormer with our Depth-UP, the basic CGFormer, the CGFormer with our Depth- -U, and the LisAR-based OCC model LMSCNet (Roldao et al. (2220) show the scalability of our HCP. The dataset used here is SemanticKITTI For each class, the desired error rate is set by multiplying the original error rate of OCC models with the scale X, À E {0.86, 0.89, 0.92, 0.95, 0.98), ) which raises the coverage requirement. Figure 9auure]sshr shows the CovGap vs, AvgSize results. We can see that our HCP always outperforms the two baselines for the points of our HCP are located in the left bottom corner, compared with points of SCP and CCCP. Figure 10 shows the dotailod results CovGap vs. scale and AvgSize vs. scale. For most cases, as the desired error rate becomes smaller, the set size tends to be larger order to satisfy the coverage guarantee. The results on the LiDAR-based OCC model LMSCNet (Roldao) et al. (2020)) show that HCP is effective in LiDAR-based OCCs, even though they not the primary focus of our work.
A.9 Experimental Results on Occ3D-nuScenes Dataset
To demonstrate the scalability of our a-OCC approach, we applied to the Occ3D-muSeenes dataset (Cacsar et al. (0200) and the OCC model BEVStereo [Li et al. (023c
22

---

Published in Transactions on Machine Learning Research (04/2026)
Basic OccFormer
Our OccFormer
Basic CGFormr
Our CGFormer
LMSCNet
3.5
0.25
0.50
0.20
0.25
3.70
0.20
0.50
3.25
CovGap
0.15
0.30
0.15
0.15
SCP
Our
0.35
2:13
CCCP
0.10
2:30
0.05
0.05
2.05
0.05
3.05
0.86
0.89
0.92
0.05
0.58
0.86
0.83
0.52
0.05
0.58
0.80
0.88
0.82
0.90
0.90
0.59
0.95
0.86
0.80
0.92
0.95
0.96
4.0
3.0
4.5
3.3
2.3
3.0
4.0
3.0
7.0
3.5
1.5
1.0
2.0
2.0
1.5
2.0
1.0
0.0
0.0
1.5
Scale
Figure 10: The results of CovGap vs. scale and AvgSize vs. scale for our HCP, SCP and CCCP on SemanticKITTI. The considered OCC models are the basic OccFormer, the OccFormer with our Dep the basic CGFormer, the CGFormer with our Depth-UP, and the LiDAR-based OCC model LMSCNet. The scale represents the desired class error rate.
The OeuSDnnnseenes dataset consists of 1,000 outdoor driving scenes captured using six surround-view cameras. The sparse imput to the OCC model comprises six RGB images from these cameras. The dataset features voxel grids labeled with 17 classes (16 semantic classes and I empty class) at a resolution 0.4m x 0.4m x 0.4m. We utilized only the training and validation sets of NuScenes, as the test set annotations are unavailable. The 3D volume of interest. covers a range of 40m ahead and behind the vehicle, 40m to the left. and right sides, Im below, and 5.4m above the vehicle. BEVStereo {Li et al. (2023c)) serves used OCC baseline for the Occ3D-nuScenes dataset in many works, such RenderOcc (Pan et al. (202b)) and PanoOce (Wang et al. [2024)). Due to time and computational constraints, both the base BEVStereo model and BEVStereo with our Depth-UP were trained for 12 epochs and 20 batch sizes on the server with 4 Tesla V100 GPUs while the original work trained on 32 batch sizes. The input image size here is 416 x 704 while the original work used the 512 x 1408 input image size.
commonly
Table 7 presents the mIoU across all classes and the IoU for each individual class for both the base BEVSterco model and BEVStereo enhanced with our Depth-UP on the Occ3D-nuScenes dataset. Depth-UP demonstrates notable improvements over the base OCC model, achieving a 1.61 (8.77%) increase in mloU. Furthermore, our DP significantly enhances performance for small, safety-critical classes, including a 9.43 loU improvement for the motorcycle class and a 1.09 IoU improvement for the person class. These improvements are altributed to the effective integration of uncertainty information from the depth model into the OCC model.
HCP method with SCP and CCCP on the Oce3D-nuScenes dataset using the BEVSterco model, similar to Table 2 The results demonstrate that HCP consistently achieves robust em empirical class- conditional coverage while generating smaller prediction sets. Compared to SCP, it reduces the set size by up to 87% and the coverage gap by up to 97%. Similarly, compared to CCCP, it achieves reductions of up to 10% set size and 6% coverage gap. These findings are consistent with experimental results on the SemanticKITTI and KITTI360 datasets, further validating the scalability of our approach.
Table
A.10 Ablation Study on Model Parameter
Due to space limitations, did not include the changes in model parameters for our Depth-UP approach Here, we providc discussion. The original model size is 59.98MB. Adding a head for depth uncertainty estimation increases the size to 60.09MB, representing a 0.18% increase. The Propagation on Geometry Completion (PGC) does not introduce additional parameters beyond this head. In contrast, Propagation on Semantic Segmentation (PSS) incrcases the parameter size 71.53MB, a 19.26% increase, primarily due to the additional ResNet backbone required for depth feature extraction. The whole Depth-UP UP approach, which includes all the above parts, increases the parameter size to 71.53MB. To address this, we
in Table B
23

---

Published in Transactions on Machine Learning Research (04/2026)
Table 7: Separate results of our UP on the Occ3D-nuScenes dataset. CCessrr al. (2020 and the
BEVSterco (Li et al. (2023c)) model.
motorcycle
traffic-cone
drive--surt
other-flat
sidewalk
vegetation
manmado
Method
person
terrain
barries
bicycle
const.
trailer
truck
others
mlou
bus
Base
18.35
0.00
6.57
0.00
50.85
24.54 28.89
21.56
16.14 14 16.09 29.54 0.00 33.58
36.62
10.60
11.23
25.81 0.01
Our
19.96
9.43
7.66
1.71 54.01 26.84 29.76
26.19 19.14 14.44 31.74 0.01 32.55 33.43 14.25
10.00
25.02 3.10
Table 8: Compare our HCP (referred to "Ours') with the standard conformal prediction (SCP) and class-conditional conformal prediction (CCCP) on CovGap and AvgSize for the Occ3D-nuSeenes dataset and the BEVStereo model.
Method
Base
Our Depth-UP
CP
SCP
CCCP
Ours
SCP
CCCP
Ours
CovGap AvgSize
0.1931
0.0070
0.0069
0.2058
0.0075
0.0070
1
0.2481
0.0341
0.0316
0,2585
0.0376
0,0336
plan to explore using a simpler backbone for depth feature extraction in future work to reduce the added parameters.
A.11
Uncertainty vs. Distance
0.40
3.0
0.35
2.5
0.30
pepth Std
2.0
OCC ECE
0.25
0.20
1.5
0.15
1.0
0.10
0.5
0.05
0.0
0.00
10
20
30
40
50
0
10
20
30
40
50
Distance (m)
Distance (m)
(a)
(b)
Figure 11: (a) The estimated depth standard deviation from the direct modeling method, representing uncertainty, varies with distance. (b) The Expected Calibration Error (ECE) uncertainty of the VoxFormer model output, representing OCC ECE, varies with distance.
Figure I1 illustrates the relationship between uncertainty and distance in the occupancy prediction model. Figure I a), we show the correlation between the estimated standard deviation (uncertainty) of depth and the distance from the camera to the object. For clarity, we divided the distance into 256 bins, cach 0.2 meters in length, and calculated the average estimated standard deviation for each bin. The results reveal that the depth uncertainty is highest. when the object is very close to the camera. This phenomenon arises
24

---

Published in Transactions on Machine Learning Research (04/2026)
because, in stereo vision systems, objects at close range result in minimal disparity between the two images, making depth estimation inherently challenging (Wei et al. (2024))- The uncertainly reaches its lowest point at approximately 15 meters, beyond which it increases with distance. This trend aligns with the inverse relationship between depth and disparity, as well as the reduced pixel resolution available for objects further away from the camera (Wei al. (0244) ). These observations confirm that the depth uncertainty estimation in our model is consistent with thcorctical expectations.
Figure 11 b) presents the relationship between the Expected Calibration Error (ECE) metric of the VoxFormer model and the distance to the voxels. ECE is standard metric for assessing the calibration of uncertainty estimates in probabilistic models (Feng et al. (2021)). In this case, we applied the voxcl-based ECE compulation method described in Cao et al. (0024). The results show that the OCC uncertainly is minimized at approximately 15 meters, consistent with the depth uncertainty trend observed in Figure a). When voxels are very close to the camera, the OCC ECE relatively high, likely due to depth estimation errors. Similarly, when voxels are far from the camera, the OCC ECE increases, attributed the limited pixel resolution for distant objects.
Notably, the similarity in the shapes of the curves in Figure 11 and I1 b) highlights the significant influence of depth uncertainty on OCC performance, as discussed in Section | of utilizing the depth uncertainty in improving final OCC performance.
These findings reinforce the importance
A.12 Table of Abbreviations
Abbreviation
Full Term
OCC
3D Semantic Occupancy Prediction
HCP
Hierarchical Conformal Prediction
Depth-UP
Uncertainty Propagation framework from depth models
UQ
Uncertainty Quantification
PGC
Propagation on Geometry Completion
PSS
Propagation on Semantic Segmentation
DM
Direct Modeling
mloU
mean Intersection over Union
CovGap
Average Class Coverage Gap
AvgSize
Average Set Size
SCP
Standard Conformal Prediction
CCCP
Class-oonditional Conformal Prediction
Table 9: Table of Abbreviations and Definitions
25


Reviews:
[{"review_id": "PkAe0Qcoou", "reviewer": "Reviewer_ivqq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focused on the uncertainty about depth and semantic in 3D occupancy. Specifically, this paper introduced an uncertainty-aware\ncamera-based 3D semantic occupancy prediction framework, which contains the uncertainty propagation (DepthUP) from depth models to improve OCC performance and a hierarchical conformal prediction (HCP) module to quantify the uncertainty of OCC. Finally, the proposed method was validated on the KITTI and nuScenes dataset.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper conducted a series of related experiments regarding proposed modules (e..g, Depth-UP and HCP) with VoxFormer and SemanticKITTI. Moreover, the proposed method achieved obvious performance improvements compared to the baseline VoxFormer on the. semantic-KITTI and KITTI-360 dataset.", "audience_yn": "Yes", "audience_explanation": "This paper introduced a view of uncertainty regarding the geometric and semantic in the 3D occupancy, which is appealing for some individuals in TMLR's audience.", "requested_changes": "1. This paper should conduct more experiments to make this paper more convincing. \n\n(1) Authors should provide more experiments of comparison about the proposed DepthUP because there are a lot of different methods about introducing depth information to image features in existing works. For example, BEVDepth [1] and OPEN [2] introduced depth information from two different views to enhance the geometric capacity of models. \n\n(2) Authors should conducted more experiments based on more advanced baselines to illustrate its effectiveness on the KITTI or nuScenes datasets. \n\n(3) Authors need to provide more analysis regarding the running time and parameter size brought by proposed modules, which is important for real-time application in autonomous driving.\n\n2.  This paper should provide more discussion regarding the uncertainty of geometric and semantic. What is the relationship between uncertainty and geometry and semantic completion. There should be more deeper analysis to improve the quality of this paper.\n\n3. This paper can add more visualizations regarding uncertainty to illustrate the problem more clearly.\n\n[1] Bevdepth: Acquisition of reliable depth for multi-view 3d object detection\n\n[2] Open: Object-wise position embedding for multi-view 3d object detection", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JsXIQw8Mh3", "reviewer": "Reviewer_soEr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an uncertainty-aware method (α-OCC) for camera-based 3D semantic occupancy prediction, which consists of Depth-UP (Depth Uncertainty Propagation framework) and HCP (Hierarchical Conformal Prediction). Depth-UP enhances OCC performance by explicitly modeling the uncertainty of depth estimation and propagating it into both the geometry completion and semantic segmentation modules. HCP addresses the class imbalance by introducing a KL-based score function and a hierarchical calibration mechanism, which improves the occupied recall for rare classes and generates statistically guaranteed prediction sets. The experiments on SemanticKITTI and KITTI360 show the effectiveness of the proposed method. However, I have some concerns about this paper. My detailed comments are as follows.\n\n**Positive points\n1. The authors propose a Depth Uncertainty Propagation (Depth-UP) framework that propagates depth estimation uncertainty into both geometry completion and semantic segmentation through probabilistic voxel occupancy modeling and depth-aware feature fusion, improving occupancy prediction performance.\n2. The authors introduce a Hierarchical Conformal Prediction (HCP) method that addresses class imbalance by employing a KL-based score function for rare-class recall enhancement and generating calibrated prediction sets with guaranteed coverage while minimizing set size.\n3. The experiments on SemanticKITTI and KITTI360 show that the proposed method achieves competitive performance in camera-based 3D semantic occupancy prediction.", "claims_and_evidence_yn": "Yes", "claims_explanation": "**Positive points\n1. The authors propose a Depth Uncertainty Propagation (Depth-UP) framework that propagates depth estimation uncertainty into both geometry completion and semantic segmentation through probabilistic voxel occupancy modeling and depth-aware feature fusion, improving occupancy prediction performance.\n2. The authors introduce a Hierarchical Conformal Prediction (HCP) method that addresses class imbalance by employing a KL-based score function for rare-class recall enhancement and generating calibrated prediction sets with guaranteed coverage while minimizing set size.\n3. The experiments on SemanticKITTI and KITTI360 show that the proposed method achieves competitive performance in camera-based 3D semantic occupancy prediction.", "audience_yn": "Yes", "audience_explanation": "This paper proposes an uncertainty-aware method (α-OCC) for camera-based 3D semantic occupancy prediction, which consists of Depth-UP (Depth Uncertainty Propagation framework) and HCP (Hierarchical Conformal Prediction). This topic is quite important to Autonomous Driving.", "requested_changes": "**Negative points\n1.\tThe innovation of the depth uncertainty propagation mechanism is limited. The core operation of computing the probability integral of a ray through voxels under a known depth distribution is a standard technique in geometric occupancy modeling (e.g., LSS-based methods). The authors have not sufficiently distinguished their method from prior approaches.\n2.\tIt would be beneficial to evaluate the performance of Depth-UP under adverse weather conditions (e.g., rain, fog, and low-light scenarios), where depth estimation errors are significantly exacerbated. Such an evaluation would better demonstrate the robustness and advantages of Depth-UP.\n3.\tThe necessity of the hierarchical prediction design is questionable, as it decouples the inherently joint reasoning of geometry and semantics in occupancy prediction tasks, potentially compromising the model's capacity for unified voxel-wise prediction.\n4. The specific value of ε in the KL-based score function is not clearly defined.\n5. The performance on other rare classes (e.g., bicycle, motorcycle) has not been reported, which limits the comprehensive evaluation of the HCP’s generalizability across all safety-critical rare classes. \n6. In Table1, the results of OccFormer and CGFormer on KITTI360 are missing.\n7. In Table 7, it would be better to include more baselines, such as BEVFormer [A] and COTR [B].\n8. Some related work can be considered and discussed, such as [C,D,E,F].\n\n**Minor issues\n1. In Table 3, the maximum value in the precision column is 63.76 instead of 63.10.\n2. In Table 5, according to the original LMSCNet paper, it achieves 54.22% IoU and 16.78% mIoU on the SemanticKITTI validation set.\n\n**Reference\n[A] BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers. ECCV 2022.\n[B] COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction. CVPR 2024.\n[C] 2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds. ECCV 2022.\n[D] EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation. TPAMI 2024.\n[E] Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction. RAL 2024.\n[F] RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation. AAAI 2024.", "broader_impact_concerns": "NA", "additional_comments": "NA", "extra_scores": {}, "extra_text": {}}, {"review_id": "5JxtLPUlQ5", "reviewer": "Reviewer_y4jm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "α-OCC is a camera-based 3D semantic occupancy prediction framework that incorporates uncertainty to improve reliability and safety in autonomous driving. It introduces two components: 1. Depth-UP, which estimates depth uncertainty and propagates it into the OCC pipeline, improving both 3D geometry reconstruction and semantic segmentation. 2. HCP, which produces calibrated uncertainty-aware prediction sets and boosts recall for rare but safety-critical classes (like pedestrians and bicyclists).", "claims_and_evidence_yn": "No", "claims_explanation": "I will review the paper again after the following questions are clarified:\n1. In the experiment section, which one shows the rare-class improvements as you've mentioned in the introduction?\n2. Explain how do you generate Figure 1a to compute the percent of depth uncertainty, and which dataset do you use to generate this figure. How do you compute the depth uncertainty here?\n3. For direct modeling $\\mathcal{L}_\\text{KL}$, explain how did you get the loss expression (under the distributional assumption).\n4. For PSS, explain how do you use Res18 to extract deep features from depth mean and std.\n5. In HCP, \n    - in Geometric Level, why do you use the formulation of $P(o = T \\mid Y_\\text{test} = y)$ here, but not using the standard conformal prediction expression, where you measure the probability of $Y_\\text{test}$ in a prediction set?\n    - The bullet point above seems to lead to your score design in Eq. 2 if my understanding is correct. I wonder why $\\epsilon$ seems to be set as a small value (I infer this when you say we use $\\epsilon$ to avoid the divide-by-zero problem), and is this because of some assumptions in the problem setting for the empty class? Or is this tied to some practical use case?\n    - A follow-up question is how exactly you use Sec 4.2 class score and occupied score for Geometric Level uncertainty quantification? Are we using the same coverage mentioned in 3.2.2?\n    - Why do we need Eq.3 also conditioned on $o = T$ in Semantic Level UQ?\n5. In the experiment to evaluate uncertainty quantification, why do we want to measure CovGap?\n6. In Table 2, why does including Depth-UP make uncertainty quantification better (Ours* > Ours)?", "audience_yn": "Yes", "audience_explanation": "Yes, UQ is an important research topic in ML research and this paper can be applied to high-impact real-world scenarios like autonomous driving.", "requested_changes": "For major concerns, see questions above. Here are some minor writing style suggestions:\n1. Only define abbreviations once (probably somewhere at the beginning, or put a table in the appendix to explain all word abbreviations) to improve the flow of the paper. For example, now I think there are at least 10 (HCP)s defined all over the paper.\n2. For your contribution, I'd recommend you rewrite it as 1 main major contribution $\\alpha$-OCC with 2 components, Depth-UP and HCP. This will match the structure of the paper and your main figure Fig. 2 better.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)