# Demystifying IoU score prediction in 3D LIDAR-based object detectors

Simon Berthoumieux, Emanuel Aldea

► To cite this version:

Simon Berthoumieux, Emanuel Aldea. Demystifying IoU score prediction in 3D LIDAR-based object detectors. 2025 IEEE International Conference on Advanced Visual and Signal-Based Systems (AVSS 2025), Aug 2025, Tainan, Taiwan. 10.1109/AVSS65446.2025.11149884. http-05522540

HAL Id: hal-05522540

https://hal.science/hal-05522540v1

Submitted on 22 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Demystifying IoU score prediction in 3D LIDAR-based object detectors

Simon Berthoumieux, Emanuel Aldea

SATIE - CNRS UMR 8029, Paris-Saclay University, France

{simon.berthoumieux, emmanuel.aldea}@universite-paris-saclay.fr

## Abstract

In this paper, we aim to identify how various architectural choices in two-stage one-to-many unimodal LIDAR 3D detectors influence the calibration of their IoU score ($S_{IoU}$) prediction branch. We select a relevant family of freely available networks, and extensively test their output scores w.r.t. $S_{IoU}$ regression, and score-based ordering of predicted bounding box (BBox). We observe that the interpretation of predicted localization score ($\hat{S}_{IoU}$) as a confidence score is poorly justified, that geometric features seem to be of importance when it comes to accurately predicting $S_{IoU}$, and that accurate ranking over the whole score range seems to be favored by the use of a more balanced, in terms of target metric values, distribution of BBoxes in training the score.

## 1. Introduction

Not unlike its image-based counterpart, 3D object detection has been linked to several potential downward tasks, such as scene understanding [10] or tracking [14], and has become a prominent task in certain fields associated with public safety and smart cities, such as autonomous driving [21], or augmented reality [7].

In this paper, we study the calibration of 3D object detectors, to determine how well their predicted Intersection over Union (IoU)-based scores [8] map to the real localization quality of their predicted 3D Bounding Boxes (BBoxes). To this aim, we identify and extensively adapt several usual calibration metrics, to allow for the study of 3D object detectors. We then study some relevant 3D detectors using those metrics, and identify characteristics that contribute to the calibration of their IoU score branch.

### 1.1. Theoretical base

Our interest in the adequate characterization of the quality of BBoxes is threefold. (1) In object detection, one-to-

many detectors generate a great number of proposal BBoxes for each ground-truth BBox. Those proposals are then refined using a *post-processing* step (usually *Non-Maximum Suppression (NMS)*) that relies on per-BBox quality scores. [5] first showed that localization-based scores worked better to that end than classification scores in the 2D case, leading to their relative popularity in 2D and 3D detection. Thus, a study of the calibration of localization scores could help refine better boxes from initial proposals. (2) One usual metric for detector quality in 3D detection datasets [2] is per-class Average Precision (AP). AP is computed as the area under the precision-recall (PR) curve, which is obtained by accounting for detections in order of decreasing score. To maximize that metric, networks must be able to detect all objects (ie. have a recall of 1) and to correctly rank all valid proposals before all non-valid ones, and thus must produce a well calibrated score. (3) Accurate localization information on predicted BBoxes is generally useful to any downstream application using those predicted BBoxes.

We also note that, while some works exist on the calibration of IoU scores for 2D detection [5, 13, 6], such works are severely lacking for 3D detection, beyond a few articles briefly analyzing the calibration of their own proposed network [9, 22, 23]. Given the fact that IoU score prediction is more challenging for 3D detection than for 2D detection, due to the additional degrees of freedom for BBoxes (from 4 in 2D to 7 in 3D, including rotation), and the increased sparsity of data used in convolution, an extension beyond the scope of those works seems useful to us.

### 1.2. Methodology

To ensure reproducibility, our study will be performed on the three classes of the publicly available validation split of the KITTI dataset [2], only on models with an official public release for code and weights (except for Voxel R-CNN [1], whose weights we train ourselves)<fnref n="1" />. We find ourselves with a selection of six 3D detectors producing IoU score ($S_{IoU}$) predictions ($\hat{S}_{IoU}$) as indicators of BBoxes quality. These models are all LIDAR-based, one-to-many, and two-staged.

<footnote>This study was supported by the OLICOW project, funded by the French National Research Agency (ANR) grant ANR-22-CE22-0005.</footnote>

<footnote n="1">¹All models used here are based on https://github.com/openmmlab/OpenPCDet, which facilitates the comparison of their output.</footnote>
We define those terms and justify our choices in Section 2.1, the models themselves are presented in Table 1.

## 1.3. Objectives and contributions

In this paper, we adapt existing metrics [3, 11, 13] to the evaluation of approaches for $S_{IoU}$ prediction w.r.t. two questions: (1) How well does the predicted score $\hat{S}_{IoU}$ estimate the real score $S_{IoU}$? (2) How well does the network rank the predicted BBoxes by order of quality?
We also studied the validity of interpreting $\hat{S}_{IoU}$ as a classical confidence score, but we will limit our explanations on the matter in this paper.

To summarize, in this article: (1) We select and extensively adapt some existing calibration metrics to evaluate the quality of predicted scores provided by some 3D object detectors; (2) We use the adapted metrics to study the impact of certain architectural approaches using a representative subset of networks, and we propose some guidelines for future design.

# 2. RELATED WORKS

## 2.1. LIDAR-based 3D Object detection

3D detectors aim to predict the localization, shape, and orientation of objects-of-interest in an observed 3D space. LIDAR point clouds and camera images are usual inputs in outdoor 3D detection [2] (with RGB-D being sometimes considered for indoor detection).

In this paper, we limit ourselves to unimodal LIDAR-based detections as: (1) The local and global geometric information stored within point clouds is sufficient to get high-quality detections; (2) LIDAR methods perform well regardless of scene luminosity [12]; (3) A unimodal approach avoids potentially costly fusion mechanisms.

We further limit ourselves to one-to-many detectors, that predict many BBoxes for each Ground-Truth (GT) BBox, before removing many redundant BBox proposals, to keep one prediction per initial cluster of predicted BBoxes. Our choice is partly motivated by their relative popularity, due to their ability to receive far more foreground information during training than their one-to-one counterpart, and partly by the fact that their post-processing step, which is usually done through non-maximum suppression (NMS), heavily relies on a correct estimation of the quality of BBoxes.

We find that most one-to-many LIDAR-based 3D detectors in the literature do not possess an official public release for code or weights. Among all the models we identify, we choose to focus our study on two-stage detectors, which create BBoxes proposals in a first stage of the network, before refining them in a second stage. The general architecture for the kind of network we will use is given in Fig.1. Our justification for this choice of focus is twofold: (1) We need to have a relatively coherent set of models, in order to be

able to infer some general properties on the calibration of $S_{IoU}$ prediction. (2) Two-stage detectors are relatively popular, because they offer an interesting compromise, in terms of precision and HW performance, between one-stage networks [20] that do not refine their initial BBoxes proposals, and multi-stage networks [19] that continue to iteratively refine proposals past the first stage.

## 2.2. Calibration

In addition to targeting a high task performance, it is of paramount importance to design models which are rigorously calibrated, i.e., the models output scores and estimations which are reliable indicators - in the statistical sense - of the likelihood of the prediction. We will introduce below one well-grounded approach, that we further adapted for our specific IoU score calibration. A second approach, based on ECE and reliability diagrams, is discussed in 5.1.

Now, let us consider a sparsification [11] procedure which, to the best of our knowledge, has not been used for IoU score analysis. It consists in evaluating across a dataset how accurately a given estimation relates to the error committed by the model. Sparsification is extremely useful as it highlights how well the model score orders the samples relatively to the actual error being performed. In consequence, we argue that this indicator is critical in one-to-many detection frameworks, for which associations are intensively relying on ordering operations. In more practical terms, the samples are processed and then sorted in decreasing order of the score. Iteratively, a predefined quantile of samples is removed and the ground-truth error is computed on the remaining samples until all the samples are removed from the sorted sample set. The resulting curve, denoted as the sparsification curve, is to be compared with the ideal curve (i.e., the oracle), which is obtained when the score used to sort the sample set is the actual ground-truth IoU value. Quantitatively, the area between the model curve and the oracle highlights the relative ordering error that the model introduces; and we denote this value as the sparsification error.

<figure><img src="image_2.png" /><figcaption>Figure 1. General architecture of LIDAR-based one-to-many two-stage detectors.</figcaption></figure>
# 3. IoU AND IoU SCORES

In this section, we recall the definition of the IoU, and give a proper definition of the IoU score $S_{IoU}$ that the 3D detectors being considered in this paper will aim to provide for each predicted BBox.

## 3.1. IoU : Intersection over Union

Intersection over Union is a very common way to characterize the quality of a predicted BBox in object detection. It is heavily inspired by the Jaccard index used in statistics, and, for two BBoxes *Pred* and *GT*, it can be defined as:

$$IoU(\text{Pred}, \text{GT}) = \frac{|\text{Pred} \cap \text{GT}|}{|\text{Pred} \cup \text{GT}|} \quad (1)$$

where $| \cdot |$ is the volume (or area in the 2D case) of the intersection/union of BBoxes.

The validity of an outputted BBox is conditional to the IoU between it and its closest unmatched GT BBox surpassing a certain threshold.

## 3.2. IoU scores

As we previously established, a per-BBox score is required to filter them through NMS, and to compute average precision. Classification scores are still somewhat used to that aim, but, since [5], localization-based scores have known a huge increase in popularity. This can be explained by the fact that well-localized BBoxes tend to be classified correctly, but well-classified BBoxes bear limited information on localization quality. This was originally observed in 2D, but seems to generalize well to 3D. Moreover, in two-stage detectors, classification scores are usually computed well before final proposals.

IoU scores $S_{IoU}$ are one such type of localization-based score, they measure the localization quality of BBoxes the following way:

$$S_{IoU} = \min \left( 1, \max \left( 0, \frac{IoU - \theta_L}{\theta_H - \theta_L} \right) \right) \quad (2)$$

The detectors we study in this paper produce a prediction of this score ($\hat{S}_{IoU}$), using parameters $\theta_L = 0.25$ and $\theta_H = 0.75$. The associated prediction branches will be trained through Binary Cross Entropy (BCE).

We note that options other than $S_{IoU}$ exist, such as class-dependent thresholds ($\theta_L, \theta_H$), straightforward IoU prediction, combined $\hat{S}_{IoU}$/classification scores [16, 23].

# 4. MODELS

The models used in this paper are presented in Table 1. We will also include a few additional comments.

Our experimentations are done on the 3 classes of the valuation split of the KITTI [2] dataset. We note that we

<table><caption>Table 1. Specificities of the different models used in this paper.</caption><thead><tr><th>Models :</th><th>Backbone :</th><th>RPN head :</th><th>Roi-pooling :</th><th>Transformers :</th></tr></thead><tbody><tr><td>PV-RCNN[17]</td><td>Second-like</td><td>Anchor-based</td><td>From points</td><td>×</td></tr><tr><td>Part-A<sup>2</sup>-free[18]</td><td>UNet-based</td><td>Seg.-based</td><td>From points w/ part & seg. pred.</td><td>×</td></tr><tr><td>Part-A<sup>2</sup>-anchor[18]</td><td>UNet-based</td><td>Anchor-based</td><td>From points w/ part & seg. pred.</td><td>×</td></tr><tr><td>Voxel R-CNN[1]</td><td>Second-like</td><td>Anchor-based</td><td>From voxels</td><td>×</td></tr><tr><td>CT3D[15]</td><td>Second-like</td><td>Anchor-based</td><td>N/A</td><td>Encoder-decoder</td></tr><tr><td>PDV[4]</td><td>Second-like</td><td>Anchor-based</td><td>From points, density-aware</td><td>Self-attention</td></tr></tbody></table>

removed the NMS step done during post-processing, as the networks are trained using the unfiltered output or the RoI-head, and as it would be strange to exclude low-scoring but potentially highly matching BBoxes from our study of the link between $\hat{S}_{IoU}$ and $S_{IoU}$. We do not remove the NMS step after the RPNs, as their detections are filtered during training, and thus the RoI-heads are not trained to process unfiltered proposals. We also, once again, note that all the networks in Table 1 use BCE to train $S_{IoU}$ prediction.

We note that PDV uses self-attention on its density-aware RoI-pooled BB features, while CT3D does not reuse any BB features and uses its transformer encoder-decoder directly on initial points around its RoIs. We also note that PDV is the only network that computes $\hat{S}_{IoU}$ from RoI-head refined BBoxes and not alongside them.

# 5. RESULTS & PROPOSED METRICS

We collect post-processing-free scored BBox predictions for all the studied networks, and evaluate them on two criteria, namely (1) **quality**: ability to predict target localization metric, and (2) **ranking**: ability to correctly order detections based on quality. **Calibration** (ie. usefulness of localization score as confidence score) is also briefly discussed.

## 5.1. Calibration

A discussion on calibration is motivated by the fact that some works refer to the IoU score prediction, $\hat{S}_{IoU}$, as a confidence score.

First of all, $\hat{S}_{IoU}$ is not a confidence score, as, despite the fact that the validity of a detection depends on the successful check of an IoU threshold, the training objective $S_{IoU}$ is not the probability of a detection being valid, and so $\hat{S}_{IoU}$ is not a predicted probability. Secondly, we evaluated the detections using some usual metrics for confidence (ie. ECE and reliability diagrams), and found that, if $\hat{S}_{IoU}$ was a confidence score, the studied networks would be highly over-confident, while mostly outputting low confidence predictions (as seen in Fig.2). This last fact might seem worrying, but in a real use case, most low $\hat{S}_{IoU}$ detections would be removed through NMS and score thresholding, which we
<figure><img src="image_3.png" /><figcaption>Figure 2. Confidence histograms for studied networks</figcaption></figure>

<table><caption>Table 2. Performances of the different models used in this paper, with: mAP40 (computed on NMS post-processed output) is a standard metric of model quality on KITTI, other metrics explained in the two "Proposed metrics" sections.</caption><thead><tr><th rowspan="2">Models:</th><th colspan="3">mAP40 (w/NMS):</th><th rowspan="2">MAE:</th><th rowspan="2">ABC<br>(Fig.4):</th><th rowspan="2">Score of max<br>(Fig.5&6):</th><th rowspan="2">AUC<br>(Fig.5):</th><th rowspan="2">AUC<br>(Fig.6):</th></tr><tr><th>car</th><th>cycl.</th><th>ped.</th></tr></thead><tbody><tr><td>PV-RCNN</td><td>86.32</td><td>75.17</td><td>55.69</td><td>9.90e-02</td><td>1.38e-01</td><td>0.66</td><td>3.48e-02</td><td>1.01e-01</td></tr><tr><td>Part-A<sup>2</sup>-free</td><td>83.37</td><td>79.28</td><td>66.28</td><td>1.48e-01</td><td>1.50e-01</td><td>0.60</td><td>3.59e-02</td><td>5.81e-02</td></tr><tr><td>Part-A<sup>2</sup>-anchor</td><td>85.69</td><td>75.80</td><td>60.40</td><td>1.16e-01</td><td>1.78e-01</td><td>0.66</td><td>3.88e-02</td><td>1.03e-01</td></tr><tr><td>Voxel R-CNN</td><td>86.86</td><td>81.59</td><td>59.18</td><td>1.28e-01</td><td>1.42e-01</td><td>0.64</td><td>3.83e-02</td><td>1.02e-01</td></tr><tr><td>CT3D</td><td>86.73</td><td>76.27</td><td>55.91</td><td>1.20e-01</td><td>2.23e-01</td><td>0.66</td><td>2.62e-02</td><td>9.07e-02</td></tr><tr><td>PDV</td><td>86.75</td><td>79.70</td><td>57.95</td><td>1.16e-01</td><td>1.52e-01</td><td>0.62</td><td>3.77e-02</td><td>1.08e-01</td></tr></tbody></table>

removed in our paper. We fact that $\hat{S}_{IoU}$ is not a predicted probability leads us to exclude a lot of classical statistical tools in calibration from our analysis.

## 5.2. Estimator quality

We now seek to determine how well the various studied models estimate the IoU score $S_{IoU}$ defined in Eq.(2).

**Proposed metrics:** One usual way to measure the quality of an estimator is to focus on the statistical relationship between its predictions and the ground truth (eg. $n^{th}$ moments of the error, Pearson/Spearman correlation coefficients). Unfortunately, these metrics broadly rely on the assumption that the noise of the estimation is Gaussian or centered, which will not be the case in our work, as most non-post-processed BBoxes ($> 80\%$ for anchor-based models, $> 65\%$ for Part-A$^2$-free), will be associated to $S_{IoU} = 0$, with these detections having $\hat{S}_{IoU}$ across the entire score range. We thus rely on usual calibration approaches and the work done in [22], and keep our detections split in ten bins, as shown in Fig.2.

We first qualitatively study the detection noise using boxplots, as in Fig. 3. Then, in order to have a quantitative measurement of the estimation error over all detections and the whole score range, for each model, we compute (1) the Mean Absolute Error (MAE), with $MAE = \frac{\sum_{Pred} |S_{IoU} - \hat{S}_{IoU}|}{\#Pred}$ and (2) the Area Between Curves (ABC), which is the area between the curve defined by the mean points per bins of the model and the target curve in Fig.4, which we approximate as the unweighted average of the error on mean points. The MAE and the ABC for all models are given in Table 2.

**Results:** Fig. 3 is computed for Voxel R-CNN, but similar

profiles are obtained for all other models; we thus only show this one for the sake of readability.

In this figure, we see that the detections associated with $S_{IoU} = 0$, which mostly correspond to detections that do not match any ground truth ($IoU = 0$), are heavily present in all bins ($> 75\%$ in $[0.0, 0.3)$, $> 50\%$ in $[0.3, 0.6)$, $> 25\%$ in $[0.6, 0.8)$, and they drive the mean point outside the boxplot in $[0.9, 1.0)$). This highlights a real problem with the ability of these networks to correctly score non-GT-intersecting BBoxes, as we would wish for them to be scored in $[0 - 0.1)$, in order to guarantee that they will be properly filtered through NMS and score thresholding. By conjointly analyzing the relative repartition in the first few bins in Fig.2, and the MAE in 2, we observe that PV-RCNN seems to outperform the other networks w.r.t this problem, as it assigns a larger proportion of detections to $[0 - 0.1)$, a smaller proportion of detections in $[0.1 - 0.3)$, and has a significantly lower MAE than other models. It is interesting to note that Voxel R-CNN, which closely resembles PV-RCNN but does not use geometric information from points in its RoI-pooling, performs significantly worse than other models (Part-A$^2$-free not included, as it produces far less detections with $S_{IoU} = 0$). We thus hypothesize that geometric information from points is important to correctly score detections associated with $S_{IoU} = 0$, but, since that feature is also shared by other networks performing worse than PV-RCNN, there are likely also other elements at play.

Our analysis on the MAE mostly informs us on the behavior of the lowest bins, and is heavily affected by the large noise of the estimator (as seen by the range between $1^{st}$ and $3^{rd}$ quartiles covering almost the entire $S_{IoU}$ range for bins across $[0.5 - 0.8)$). We complement it with an analysis of the ABC between the curves of mean points and the ideal curve in Fig.4. Based on this, two networks (PV-RCCN and Voxel R-CNN) seem to slightly outperform the rest when it comes to accurate prediction equilibrated over the whole $\hat{S}_{IoU}$ range. These networks extract features from all levels of the BB to refine their predictions, while PDV only uses features from some levels, Part-A$^2$ only features from its output, and CT3D no BB features at all. We thus hypothesize that backbone features are important when it comes to accurately regress $S_{IoU}$ over the whole score range.

## 5.3. Prediction ranking

We now seek to see how well the studied networks rank detections. This is partly linked with the preceding problem, since a network with high-quality prediction of $S_{IoU}$ would perform well on this task, but it goes beyond it, as it would be possible to produce a set of scores that wildly misestimate $S_{IoU}$, while still accurately ranking a set of given detections by order of quality.

**Proposed metrics:** We measure the quality of the rankings resulting from $\hat{S}_{IoU}$ by following sparsification procedures
<figure><img src="image_4.png" /><figcaption>Figure 3. Evolution of $S_{IoU}$ depending on $\hat{S}_{IoU}$ for Voxel R-CNN [1]. Detections are split in bins based on $\hat{S}_{IoU}$ (x-axis) same as Fig.2. $S_{IoU}$ (y-axis) is computed for every prediction. For each bin, the green point correspond to the mean, and the red bar and blue boxplot to the median and first/fourth quartile of $S_{IoU}$.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4. Evolution of the mean $S_{IoU}$ per bin against the mean $\hat{S}_{IoU}$ per bin. In black is the target curve ($S_{IoU} = \hat{S}_{IoU}$). ABC of a model is the area between the target curve and the curve of means.</figcaption></figure>

dures, as presented in Section 2.2. For that, we compute the oracle curve by sorting detections by ascending *IoU*, and the sparsification curve by order of ascending $\hat{S}_{IoU}$. This means that for each point of the curves, we compute the mean Jaccard distance between the remaining predictions and their closest GT ($\frac{\sum_{Pred} 1 - IoU}{\#Pred}$), with detections being eliminated by order of ascending *IoU*/$\hat{S}_{IoU}$. Since all networks output different BBoxes, and thus have different oracle curves, we only plot the difference between the sparsification and oracle curves for all networks (in Fig. 5 and Fig.6), and we measure the area under the curve (AUC) obtained through this difference.

Since, contrary to usual applications of sparsification plots [11], the number of predictions will generally not match the number of GT, a direct analysis of Fig. 5 is not recommended, as one possible way to reduce one model's AUC is to produce a higher number of non-GT-matching detections ($IoU = 0$), and to give them a low $\hat{S}_{IoU}$, squeezing the error peak to the right. To account for this problem, and the fact that low $\hat{S}_{IoU}$ detections are usually removed or accounted last during AP computation, we refactor the abscissa of curves in Fig. 5, to instead plot curves based on the score associated with ordered detections used to compute the sparsification curve (Fig. 6). That way, we can evaluate the correct ordering of detection over the whole $\hat{S}_{IoU}$ range,

<figure><img src="image_6.png" /><figcaption>Figure 5. Difference between oracle curve and sparsification curve, plotted against the fraction of removed detection. AUC is the area under curve.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6. Difference between oracle curve and sparsification curve, plotted against the score associated with the index last removed detection in the set of detection with ordered score used to compute the sparsification curve. AUC is the area under curve.</figcaption></figure>

and not mostly on its lower parts.

The AUC for the curves of distance between oracle and sparsification curves in Fig. 5 and Fig. 6, and the $\hat{S}_{IoU}$ associated with the last removed detection to reach the max of these curves, are in Table 2.

**Results**: In Table 2, we can see that, for all networks, the predicted score associated with the maximal difference between the oracle and the sparsification curve always lies in [0.59, 0.71]. This is relatively coherent with what we observed in Fig. 3, where we saw that, for $\hat{S}_{IoU}$ in bins between 0.5 and 0.8, the range between the 1st and 3rd quartile of $S_{IoU}$ tended to approach [0, 1]. We can also see that, outside of this $\hat{S}_{IoU}$ range, networks perform relatively well when it comes to ordering relatively low and high scoring detections. This can be explained by the fact that the vast majority of BBoxes used in training the $\hat{S}_{IoU}$ computation are associated with $S_{IoU} = 0$ or $S_{IoU} = 1$.

Also of note on Fig. 6 is the behaviour of Part-A²-free, the only model with a non-anchor based RPN among the ones we test. It generates far less non-intersecting detections than other networks (65% instead of slightly more than 80%), and far more high scoring detections (cf. Fig. 2). We can see that it has a far lower AUC over the whole score range than detectors using anchor-based RPN. This might be due to too much importance being given to learning accurate score for non-intersecting detection when training the anchor-based detectors, due to the higher number of these detection being
present in the output of the networks. Thus, accurate ranking over the whole score range might be favored by using less low quality BBoxes in training score prediction. This may be achieved by modifying the loss used to train score, or by generating less non-intersecting detections (for example, by using an anchor-free detector).

# 6. CONCLUSION

In this paper, we adapt existing approaches in calibration and study the calibration of 3D object detectors regarding mainly 2 objectives: estimating $S_{IoU}$, and ranking detections.

We investigate and underline why the predicted score $\hat{S}_{IoU}$ cannot be treated as a confidence, and that local geometry (contained in point cloud shape) aware approaches with features from backbone seem to help predict $S_{IoU}$ more accurately. For ranking, we highlight that the network using anchor-free RPN gets a better ordering over the entire score range, which we attribute to a more balanced repartition of the $S_{IoU}$ of BBoxes used to train them. We thus hypothesize that better ordering could be obtained by accounting for the imbalance in the repartition of $\hat{S}_{IoU}$ and $S_{IoU}$ in the training loss, or by simply providing less detections non-intersecting with the GT by using well designed anchor-free detectors.

We also note some limitations to our work, that open possibilities for further efforts. Mainly, the facts that : (1) the family of studied detectors is limited, and could be extended to cover other input modalities (e.g. unimodal image, multimodal image-LIDAR), other architectural approaches (eg. one-to-one, one-stage or multi-stages) and other scoring modalities (eg. IoU, classification scores, and combined $\hat{S}_{IoU}$-classification scores) ; (2) we only use Kitti, and could extend our evaluation to more datasets (eg. Waymo, NuScenes) to study more diverse environmental conditions, sensor setups, and class balances ; (3) ablation and other experiments could be used to support our observations and obtain more prescriptive results ; (4) our analysis could be more fine-grained, through evaluation on a per-class level or by comparing some metrics w/ or w/o NMS.

# References

[1] J. Deng, S. Shi, P. Li, W. Zhou, Y. Zhang, and H. Li. Voxel r-cnn: Towards high performance voxel-based 3d object detection. In AAAI, volume 35, pages 1201–1209, 2021.

[2] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The kitti dataset. IJRR, 32(11):1231–1237, 2013.

[3] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger. On calibration of modern neural networks. In ICML, pages 1321–1330. PMLR, 2017.

[4] J. S. K. Hu, T. Kuai, and S. L. Waslander. Point density-aware voxels for lidar 3d object detection. In CVPR, pages 8469–8478, June 2022.

[5] B. Jiang, R. Luo, J. Mao, T. Xiao, and Y. Jiang. Acquisition of localization confidence for accurate object detection. In ECCV, September 2018.

[6] K. Kim and H. S. Lee. Probabilistic anchor assignment with iou prediction for object detection. In ECCV, pages 355–371, Cham, 2020. Springer International Publishing.

[7] W. Lee, N. Park, and W. Woo. Depth-assisted real-time 3d object detection for augmented reality. In ICAT, volume 11, pages 126–132, 2011.

[8] B. Li, W. Ouyang, L. Sheng, X. Zeng, and X. Wang. Gs3d: An efficient 3d object detection framework for autonomous driving. In CVPR, June 2019.

[9] J. Li, S. Luo, Z. Zhu, H. Dai, A. S. Krylov, Y. Ding, and L. Shao. 3d iou-net: Iou guided 3d object detector for point clouds. arXiv preprint arXiv:2004.04962, 2020.

[10] D. Lin, S. Fidler, and R. Urtasun. Holistic scene understanding for 3d object detection with rgbd cameras. In ICCV, December 2013.

[11] O. Mac Aodha, A. Humayun, M. Pollefeys, and G. J. Brotow. Learning a confidence measure for optical flow. TPAMI, 35(5):1107–1120, 2012.

[12] J. Mao, S. Shi, X. Wang, and H. Li. 3d object detection for autonomous driving: A comprehensive survey. IJCV, 131(8):1909–1963, 2023.

[13] M. Schubert, K. Kahl, and M. Rottmann. Metadetect: Uncertainty quantification and prediction quality estimates for object detection. In IJCNN, pages 1–10, 2021.

[14] J. Shackleton, B. VanVoorst, and J. Hesch. Tracking people with a 360-degree lidar. In AVSS, pages 420–426, 2010.

[15] H. Sheng, S. Cai, Y. Liu, B. Deng, J. Huang, X.-S. Hua, and M.-J. Zhao. Improving 3d object detection with channel-wise transformer. In ICCV, pages 2743–2752, October 2021.

[16] G. Shi, R. Li, and C. Ma. Pillarnet: Real-time and high-performance pillar-based 3d object detection. In ECCV, pages 35–52, Cham, 2022. Springer Nature Switzerland.

[17] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li. Pv-rcnn: Point-voxel feature set abstraction for 3d object detection. In CVPR, June 2020.

[18] S. Shi, Z. Wang, J. Shi, X. Wang, and H. Li. From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network. TPAMI, 43(8):2647–2664, 2021.

[19] H. Wu, J. Deng, C. Wen, X. Li, C. Wang, and J. Li. Casa: A cascade attention network for 3-d object detection from lidar point clouds. IEEE Trans. on Geoscience and Remote Sensing, 60:1–11, 2022.

[20] Y. Yan, Y. Mao, and B. Li. Second: Sparsely embedded convolutional detection. Sensors, 18(10):3337, 2018.

[21] E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda. A survey of autonomous driving: Common practices and emerging technologies. IEEE Access, 8:58443–58469, 2020.

[22] Y. Zhang, Q. Zhang, Z. Zhu, J. Hou, and Y. Yuan. Glenet: Boosting 3d object detectors with generative label uncertainty estimation. IJCV, 131(12):3332–3352, 2023.

[23] W. Zheng, W. Tang, S. Chen, L. Jiang, and C.-W. Fu. Cia-ssd: Confident iou-aware single-stage object detector from point cloud. In AAAI, volume 35, pages 3555–3562, 2021.