LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection
Authors: Sifan Zhou
Liang Li
Xinyu Zhang
Bo Zhang
Shipeng Bai
Miao Sun
Ziyu Zhao
Xiaobo Lu
Xiangxiang Chu
Venue: ICLR 2024 poster (2024)
Keywords: Quantization
3D Object Detection
Autonomous Driving
Primary area: applications to robotics, autonomy, planning

Abstract: Due to highly constrained computing power and memory, deploying 3D lidar-based detectors on edge devices equipped in autonomous vehicles and robots poses a crucial challenge. Being a convenient and straightforward model compression approach, Post-Training Quantization (PTQ) has been widely adopted in 2D vision tasks. However, applying it directly to 3D lidar-based tasks inevitably leads to performance degradation. As a remedy, we propose an effective PTQ method called LiDAR-PTQ, which is particularly curated for 3D lidar detection (both SPConv-based and SPConv-free). Our LiDAR-PTQ features three main components, (1) a sparsity-based calibration method to determine the initialization of quantization parameters, (2) an adaptive rounding-to-nearest operation to minimize the layerwise reconstruction error, (3) a Task-guided Global Positive Loss (TGPL) to reduce the disparity between the final predictions before and after quantization. Extensive experiments demonstrate that our LiDAR-PTQ can achieve state-of-the-art quantization performance when applied to CenterPoint (both Pillar-based and Voxel-based). To our knowledge, for the very first time in lidar-based 3D detection tasks, the PTQ INT8 model's accuracy is almost the same as the FP32 model while enjoying 3X inference speedup. Moreover, our LiDAR-PTQ is cost-effective being 6X faster than the quantization-aware training method. The code will be released.

Published as a conference paper at ICLR 2024
LIDAR-PTQ: POST-TRAINING QUANTIZATION FOR POINT CLOUD 3D OBJECT DETECTION
Sifan Zhou Liang Li Xinyu Zhang?, Bo Zhang", Shappeg Bai"", Miao Sun Ziyu Zhao', Xiaobo Lu Xiangxiang Chu
Southeast University "Meituan Inc ZZejjaang University "Nanyang Technological University fannjaa janjjaygmaill com, xblu2013e126 com, chuxlaangxlanggessttua...coo
ABSTRACT
Due to highly constrained computing power and memory, deploying 3D lidar-based detectors edge devices equipped in autonomous vehicles and robots poses crucial challenge. Being a convenient and straightforward model compression approach, Trainira Quantization (PTQ) has been widely adopted in 2D vision tasks. However, applying it directly to 3D lidar-based tasks inevitably leads to performance degradation. As remedy, propose effective PTQ method called LiDAR-PTQ, which is particularly curated for 3D lidar detection (both SPConv-based and SPConv-free). Our LiDAR-PTQ features three main compo- nents, (1) sparsity- based calibration method to determine the initialization of quantization parameters, (2) Task-guided Global Positive Loss (TGPL) reduce the disparity between the final predictions before and after quantization, (3) an adaptive rounding-too secliinggtt-n-eeee to-nearest operation to minimize the layerwise reconstruction error. Extensive experiments demonstrate that our LiDAR-PTO can achieve state- of-the-art quantization performance when applied to CenterPoint (both Pillar-based and Voxel-based). To our knowledge, for the very first time in lidar-based 3D detection tasks, the PTQ INT8 model's accuracy is almost the same as the FP32 model while enjoying 3 inference speedup. Moreover, effective being 30x faster than the quantization aware training method, Code will be released at https: /ttp///ggtttttttttte/teete htw.th...ccooo..... wtt../////spppayyaaaaaaa...
LiDAR- P-PQPQQQQR is cost-
INTRODUCTION
LiDAR-basee 3D detection has wide range of applications in self-driving and robotics. It is important to detect the objects in the surrounding environment fastly and accurately, which places high demand for both performance and latency. Currently, mainstream grid-based 3D detectors convert the irregular point cloud into arranged grids (voxels/pillarss, and achieve top-ranking perfor- mance (Jiageng Mao. 2023) while facing crucial challenge when deploying 3D lidar-based models on resource-limite edge devices. Therefore, it is important to improve the efficiency of grid-based based 3D perception methods (e.g., reduce memory and computation cost).
Quantization is an efficient model compression approach for high-efficiency computation by reducing the number of bits for activation and weight representation. Compared quantization-awwe aware training (QAT) methods, which require access to all labeled training data and substantial computation re- sources, Post-training quantization (PTQ) more suitable for fast and effective industrial applications. This is because PTQ only needs small number of unlabeled samples as calibration set. Besides, PTQ does not necessitate retraining the network with all available labeled data, resulting in a shorter quantization process. Although several advanced PTQ methods (Nagel et al., 2020; Li al., 2021; Wei et al., 2022; Yao et al., 2022) have been proposed for (GB-b detection tasks, applying directly 3D lidar-base tasks inevitably leads to performance degradation due to the differences between images and point clouds.
As shown in Fig 1, the inherent sparsity and irregular distribution of LiDAR point clouds present new challenges for the quantization of 3D Lidar-based detectors. (1) The sparsity of point cloud.
Work done as intern at Meituan. Corresponding author. PPooecct leader.

---

Published as a conference paper at ICLR 2024
Redundant empty pillar
(90% pixels BEV features)
Non-empty pillar
Foreground points
(only 10% non-emply grid projected from 2D/3D Pollar/Voxel
(4m x 2m vehicle only occupics 40 20 pixels the final 1504 x 1504 feature map)
Figure The sparsity of point cloud on 3D LiDAAR-based object detection. Orange area means empty area, blue point means the point cloud (non-empty area) in scenario, green box means the 3D Bboxes, and red point means foreground points.
Different from dense RGB images, non-zero pixels only occupy a very limited part of the whole scenario (about 10% in Waymo dataset (Sun et al., 2020)). For example, the huge number of zero pixel lead to significant differences activation distribution compared to dense RGB-based tasks. (2) Larger arithmetic range. Compared with the 8-bit (0-255) RGB images, the point coordinates after voxelization are located a 1504 X 1504 x 40 (voxel size = 0.1m) 3D space in Waymo dataset, which makes it more susceptible to the effects of quantization (such as clipping error). (3) Imbalance between foreground instances and large redundant background area. For example, based on CenterPoint- Voxel (Yin al., 2021), vehiele with 4m 2m occupies only 40 20 pixels in the input 1504 x 1504 BEV feature map. Such small foreground instances and large perception ranges in 3D detection require the quantized model have less information loss to maintain detection performance. Therefore, these challenges hinder the direct application of quantization methods developed for 2D vision tasks to 3D point cloud tasks.
To tackle the above challenge, we propose an effective PTQ method called LiDAR-PTQ. which specifically curated for 3D LiDAR- sparsity-based calibration method to determine the initialization of quantization parameters on parameter space. Secondly, we propose Task-guided Global Positive Loss (TGPL) to find the quantization parameter on model space that is suitable for final output performance. Thirdly, we utilize an adaptive rounding value to mitigate the performance gap between the quantized and the full precision model. The proposed LiDAR-PTQ framework is general and effective quantization method for both SPConv-based and SPConv-free 3D detection models. Extensive experiments on various datasets evaluate that our LiDAR-PTQ can achieve state-of-the-art quantization performance (Fig 2) when applied to CenterPoint (both Pillar- based and Voxel-based). To our knowledge, for the very first time in LiDAR- tasks, the PTQ INTS model's accuracy is almost the same as the FP32 model while enjoying 3x inference speedup. Moreover, our LiDAR-PTO,
object detection tasks. Firstly, we introduce
INT8
QDROP OCLEZUS
PD ED-Quuat.cCvaan
82
3002
LIDAR PTQ coan
ConterPuint- Vonel
-4.54
60
-0.20
FP32
CanterPiintPillee
4.05
54
3D detection
Inference Speed (FPS)
Figure 2: Performance comparison
cost-effective being x0x faster than QAT method. We will release code to the community.
Here, we summarize our main contributions as follows:
Unveiling the root cause of performance collapse in the quantization of the 3D LiDAR-bases detection model. Furthermore, we propose the sparsity- -bsed calibration method initialize the quantization parameter.
TGPL: A Task-guided Global Positive Loss (TGPL) function to minimize the output disparity on model space which helps improve the quantized performance.
LiDAR-PTQ: a general and effective quantization method for both SPConv- basedd and SPConv-free 3D detection models. Extensive experiments demonstrate LiDAR-PTO can achieve state-of-the-an the-art quantization performance on CenterPoint (both Pillar-based and -oxel-
2

---

Published as a conference paper at ICLR 2024
To our knowledge, for the very first time in LiDAR-based 3D detection tasks, the PTQ INT8 model's 's accuracy is almost the same as the FP32 model while enjoying 3x inference speedup. Moreover, LiDAR-PTQ is cost-effective being 30x faster than QAT method.
2 PRELIMINARIES
LiDAR-based 3D object detection, Given point set with N points in the 3D space, which is defined as P = {p, = 1 yi, 24, ra]" RNx4), where 2,, yr, 24 denote the coordinate values of each point along the axes X, Y. Z, respectively, and r; is the laser reflection intensity. Given set of object in the 3D scene B {bj- = [ Vy, 2g, h2, Wy, 4, ";," E IR [RMx*)), where M is the total number of objects. b, is the i-th object in the scene, 2; Mj. 2j the object's center, h2. wg. L is the object's 's size, 82 is the object's 's heading angle and Cj is the object's 's class. The task of LiDAR-bas based 3D object detection
detect the 3D boxes B from the point cloud P accurately.
Quantization for tensor. The quantization operation is defined as the mapping of a floating ppint (FP) value (weights activations) an integer value Tint according to the following equation:
Tint = clamp( T 2, Qmin: Jmnz.
where [.] is the rounding-to-neart operator, which results in the rounding error Ar. The function clampp() clips the values that lie outside of the integer range [fmin Imaz] incurring clipping error Ac. Tint represents the quantized integer value. 2 is zero-point. s denotes the quantization scale factor, which reflects the proportional relationship between FP values and integers. Imii Imanl is the quantization range determined by the bit-width b. Here, we adopt uniform signed symmetric quantization, as it is the most widely used in TensorRT (Migacz, 2017) and brings significant accelcration effect. Therefore, Jmin = -2h and qmaz = 2 1. Nonuniform quantization (Jeon al., 2022) challenging to deploy on hardware, so we disregard in this work. Generally, weights can be quantized without any need for calibration data. Therefore, the quantization of weights is commonly solved using grid search or analytical approximations with closed-form solution (Banner et al., 2019; Nagel et al., 2021) to minimize the mean squared error (MSE) in PTQ. However, activation quantization is input-dependent, so often requires few batches of calibration data for estimation of the dynamic ranges converge. To approximate the real-valued input x, we perform the de-quantization step:
(Iint 2)
where à is the de-quantized FP value with an error that is introduced during the quantization process.
increase the quantization scale factor 8 to expand the quantization range. However, increasing s leads to increased rounding error Ar because Ar lies in the range [-3. 2.. Therefore, the key problem is how to choose the quantization range (Emin: mmax) to achieve the right trade-off between clipping and rounding error. Specifically, when we set fixed bit-width b, the quantization scale factor s is determined by the quantization range:
Quantization range. If want reduce clipping enror Ac,
s (Imar Iminn (2
There are two common methods for quantization range setting. i): Max-min calibration. We can define the quantization range as:
Emax = maz([z) Tmin = -Imaz
(4)
to cover the whole dynamic range of the floating-pon -ponn value This leads to clipping error. However, this approach is sensitive to outliers as strong outliers may cause excessive rounding errors.
ii): Entropy calibration. TensorRT (Migacz, 2017) minimize the information loss between and f based on the KL divergence to determine the quantization range:
arg min (xI
Eman Foaa
where DEL denotes the Kullback-Leii divergence function. The entropy calibration will saturate the activations above certain threshold to remove outliers. More details refer to the appendix.
Quantization for network. For a float model with layer, we primarily focus on the quantization of convolutional layers linear layers, which mainly involves the handling of weights and activations.
3

---

Published as a conference paper at ICLR 2024
For given layer L,, we initially execute quantization operations its weight and input tensor, as illustrated in Eq 14 and 2, yielding W; and 1,. Consequently, the quantized output of this layer can be expressed as follows.
A S(BN(), @ W.)
where denotes the convolution operator, BN(() is the aatch-Normalization procedure, and f(-) the activation function. Quantization works generally take into account the convolution, Batch Normalization (BN), and activation layers.
METHODOLOGY
Here, first conduct PTQ ablation study the CenterPoint-Pilla (Yin al., 2021) model using two different calibrators (Entropy and Max-min) on Waymo val set. As shown in Table 1, when using INTS quantization, the performance drop is severely compromised for both the calibration method, especially for the entropy calibrator with significant accuracy drop of -38.67 mAPH/L.. However, directly employ- ing the Max-min calibrator yielded better results, yet not unsatisfactory. It is entirely contrary to our experience in 2D model quantization, where entropy calibration effectively mitigates the impact of out-
Table 1: Ablation study.
LEVEL 2 MAPH Vehicle
Method
Bits(W/A)
Mean
Pedestrian
Full Prec.
32/32
60.32
65.42
55.23
Entropy Max-Min
8/8
21.65 (-38.67)
29.41 (-36.02)
11.89 (-43.34)
8/8
52.91 (-7.41)
55.37 (-10.05)
50.45 (-4.78)
liers, thereby achieving superior results (Nagel et al., 2021). Similar observations are also discussed in Stäcker et al. (2021). This anomaly prompts us propose a general and effective PTQ method for 3D
detectors. In Waymo dataset, the official evaluation tools evaluated the methods two difficulty levels: LEVEL_1 for boxes with more than five LiDAR points, and LEVEL for boxes with at least one LiDAR point. Here we report the metrics in Mean Average Precision with Heading/ LEVEL 2 (mAPH/1.2), which a widely adopted metric by the community.
3.1 LIDAR-PTQ FRAMEWORK
In this paper, we propose post-training quantization framework for point cloud models, termed LiDAR-PTQ. Our LiDAR-PTQ could enable the quantized model to achieve almost the same performance the FP mode, and there is extra huge computation cost and access labeled training data. This framework primarily comprises three components.
i) Sparsity- based calibration: We employ Max-min calibrator equipped with lightweight grid search to appropriately initialize the quantization parameters for both weights and activations.
ii) Task-guided Global Positive Loss (TGPL): This component utilizes specially designed foreground-aware global supervision to further optimize the quantization parameters of activation.
iii) Adattive tounding-to-nearest: This module aims to mitigate the weight rounding error Ar by minimizing the layer-wise reconstruction error.
In summary, method first initializes the quantization parameters for weights and activations through a search in the parameter space, and then further refine them through a process of supervised optimization in the model space. Consequently, accuracy that almost matches their float counterparts for certain lidar detectors.
method is capable of achieving a quantized
We formulate the our LiDAR-PTO algorithm for a full precision 3D detector in Algorithm Next, we will provide detailed explanations for these three parts.
3.2 SPARSITY-BASED CALIBRATION
Here, in order to delve into the underlying reasons for the huge performance gap (31.29 mAPH/L2 in Tab 1) between Max-mil min and entropy calibrator. We statistically analyze the numerical distribution of feature maps of both RGB-R based models and LiDAR- object detection models, and visualize the main diversity as shown in Fig 3. The main reasons affecting the quantization performance can be summarized in points:
4

---

Published as a conference paper at ICLR 2024
Algorithm LiDAR-PPO quantization
Input: Pretrained FP model with N layers; Calibration dataset D iteration Output: quantization parameters of both activation and weight in network, i.c., weight scale Sw weight zero-point Zw. activation scale Sa, activation zero-point 2u and adaptive rounding value for weight 0.
1: Optimize only weight quantization parameters sw and zu minimize Eq 16 in every layer using the grid search algorithm;
2: input D FP network get the FP final output Of
3: for L,, = Li 1,2, ...N. do
4: Optimize only activation quantization parameters sa and za to minimize Eq 16 in layer L;
using the grid search algorithm:
5: Collect input data I the FP layer L;;
6: Input I to quantized layer I. and FP layer I, to get quantized output A, and output A;
7: Input A, to the following FP network to get output Opar of partial- quantizzd network:
8:
for all 1,2 ... -iiteration do
9:
Check quantized output A, and FP output and calculate Liocal using Eq 11;
Check partial-quantized network output Oper and FP final output Sz calculate LTGPI. using Eq 9;
Optimize quantization parameters Sws 2 Sas and B 0 of layer L3 minimize Leotal using Eq 12;
end for
13: Quantize layer L, with the learnable quantization parameters sw: our Sa. and 2n, A:
14: end for
Min-max range
Entropy range
Minomax range
Entropy range
(0) RGB image data distribution
(b) LIDAR point cloud data distribution
Figure 3: The diagram of data distribution for RGB-base and LiDAR-basce object detection. Orange and green denote the data distribution of the entire feature map and foreground feature.
i) Huge sparsity lead to inappropriate quantization range. As shown Fig and 3, the sparsity of point cloud makes the whole BEV feature map exist a large number of zero pixels. Therefore, the entropy calibrator will statistic the feature value including zero pixels (zs 90%) to minimize the information loss, which leads to the values outside the quantization range being clipped. However, these truncated values contain rich geometric representations that could used for final object detection.
ii) Point cloud features more sensitive to quantization range. Point cloud explicitly gauges spatial distances, and shapes of objects by collecting laser measurement signals from environments. During voxelization process, the raw point cloud coordinates. i.e., x. y, in the ego-vehicle coordinate system are encoded as part of voxel features that preserve essential geometric information. Specif- ically, the arithmetic range of the input point cloud coordinates increases with detection distance. Therefore, the arithmetic range in the voxel feature is strongly correlated with detection distance. In other words, the arithmetic range of point cloud is relevant to the geometrics.
Furthermore, we also conduct an ablation study with different range distances on waymo val set. As shown Tab 2, we find that the decline in accuracy exacerbated as the distance increases. For entropy calibrator, the quantized performance on long-range metrics (50m- inf) is terribly damaged (5.90 mAPH/L2, up to 84.5% drop).
Table 2: Ablation study in different range
Method
Biss
Vehicle LEVEL 2 mAPH
0130
32/32
Entropy Max min
8/8
60.03 (-28.74) (33..4))
22. 46 (-42777) (48
5,90 (-32.19) (845%)
S/8
87.14 (-14))))))8))) (-1.8%)
$7.49 (-7.74 (1.9%%
22.98 (-15.11) (-39.7%
while accuracy on short-range
metrics (0-30m) remains well (60.03 MAPH/L22, 32.4% drop). This is because the entropy calibrator
5

---

Published as a conference paper at ICLR 2024
provides an inappropriate quantization range, resulting in significant clipping error. Therefore, large number of values with geometrics info are truncated and consequently a substantial degradation in model accuracy. On the contrary, for the Max-min calibrator, which covers the whole dynamic range of the FP activation, the values with geometric information are preserved effectively. Therefore, its performance in different range metrics performs well, especially on short-range metries (0-30m), which only drops 1.63 mAPH/L2 (1.8%) than FP model.
Drawing upon the above findings, we conclude that the commonly used calibration method for RGB images is sub-optimal, while Max-min is more suitable for 3D point clouds. Therefore, we adopt Max-min calibrator for both weights and activations to mitigate the impact of high sparsity. Besides, to get more tner-grained scale factor and avoid the influence of outliers on rounding error Ar, lightweight grid search (Banner et al., 2019; Choukroun et al., 2019) is incorporated to further optimize the quantization parameters.
Specifically, for a weight or activation tensor X, firstly obtain the Tmaz and Emii according to Eq.4, and calculate the initial quantization parameter 80 following Eq. 15. Then linearly divide the interval [assp, Bsp] into T' candidate bins, denoted as {st) 3 and T are designed to control the search range and granularity. Finally, search {st)/=1 to find the optimal Sopt that minimizes the quantization error,
arg min || - (s)))l
|| -1 is the Frobenius norm (MSE Loss). Refer to appendix for more details about grid search.
3.3 YASK-GUIDER GLOBAL POSITIVE Loss
The aforementioned calibration initialization approach can effectively improve the quantization accuracy of lidar detectors, but there is still significant gap compared with the float model.
Both empirical and theoretical evidence (Li et al., 2021; Wei et al., 2022; Liu et al., 2023a) suggest that solely minimizing the quantization error in parameter space dose not guarantee equivalent minimization in final task loss within model space. Therefore, it becomes imperative to devise global supervisory signal specifically tailored for 3D LiDAR-basce detection tasks. This supervision would enable further fine-tuning of the quantization parameters p to achieve higher quantized precision. It is crucial to emphasize that this fine-tuning process does not involve labeled training data. Only need to minimize the distance between float output Of and the quantized model's output Ó, as depicted in Eq 8
arg min -Ō) -0)
(8)
P
In this paper, we propose Task-guided Global Positive Loss (TGPL) function to constrain the output disparity between the quantized and FP models. Our TGPL function features two characteristics that contribute improving the performance of the quantized method:
i) Optimal quantization parameter on model space. The TGPL. function compares the final output difference between the FP and the quantized models rather than the difference in each layer's output.
ii) Task-guided. As mentioned in Secl and Fig 1, there exists extreme imbalance between small informative foreground instances and large redundant background areas in Lidar-based detection tasks. For sparse 3D scenes, it is sub-optimal to imitate all feature pixels on dense 2D images. TGPL function is designed to leverage cues in the FP model's classification response to guide the quantized model to focus on the important area (i.e. positive sample location) that is relevant to final tasks.
In detail, we filter all prediction boxes from the FP model by threshold ?, then we select the top K boxes. Then we perform NMS (Neubeck & Van Gool, 2006) to get the final prediction as positive boxes (pusedo-labels). Specifically, inspired by the Gaussian label assignment CenterPoint (Yin et al., 2021), we define positive positions in soft way with center-peak Gaussian distribution. Finally, for the classification branch, we use the focal loss (Lin et al., 2017) as the heatmap loss Lats- For the 3D box regression, we make use of the L1 loss freg to supervise their localization offsets, size and orientation. The overall TGPL loss consists of two parts as follows:
LTGP, Lets + aLregy

---

Published as a conference paper at ICLR 2024
3.4 ADAPTIVE ROUNDING- BEEIING-TO-NEEE TO-NEAREST
Through grid search initialization and TGPL function constrain, the performance of quantized model has been greatly improved, but there is still gap in achieving comparable accuracy with FP model. Recently, some methods (Wei et al., 2022; Liu et al., 2023a) optimize a variable, called rounding values, to determine whether weight values will be rounded up or down during the quantization process. In this way, the Eq 14 in weight quantization can be formulated as follows:
Tint = clamp(l 0 + 2, 9min {maz).
(10)
where 0 is the optimization variable for each weight value decide rounding results up or down
(Nagel et al., 2020), i.e., g ranges from 0 to I. Inspired by Adagound(Nagel et al., 2020), we add
local reconstruction item to help learn the rounding value 8. The local reconstruction item as follows:
Local = I(W, 1 - W, @ L)
(11)
where || is the Frobenius norm and W; are the soft-quantized weights are calculated by Eq 10 and Eq 2. This operation allows us to adapt the rounding value minimize information loss according to the calibration data, ensuring that the quantization process preserves important details. By adjusting the rounding value, we can achieve better performance of LiD.R-PTQ. Finally, the overall loss of
our LiDAR-PTQ consists of two parts follows:
Liotat = X1 Lioeat + XehTGPt.
(12)
4 EXPERIMENTS
Dataset. To evaluate the effectiveness of our proposed Lidar-PTO, we conduct main experiments on large-scale autonomous driving datasets, Waymo Open Dataset (WOD) (Sun et al., 2020).
Implementation Details. In WOD dataset, we randomly sample 256 frames point cloud data from the training set as the calibration data. The calibration set proportions 0.16% (256/158,08) for WOD. We set the first and the last layer of the network to keep full precision. The learning rate for the activation quantization scaling factor is 5e-5, and for weight quantization rounding is Se-3. In TGPL loss, we set as 0.1, and K as 500. More details in supplements.
4.1 PERFORMANCE COMPARISON WAYMO DATASET
Table 3: Performance comparison on Waymo val set. t: reimplementation by us.
Mean (L2)
Vehicle (L.2)
Pedestrian (L22
Cyclist (L2)
Models
Methods
(WA)
mAp
mAPH
mAP
mAPH
mAP
mAPH
mAP
mAPH
Full Prec.
32/32
65.78
60.32
65.92
65.42
65.65
55.23
BRECQ
8/8
61.73
56.27
61.87
61.36
61.59
51.18
CP-
QDROP
8/8
63.60
58.14 14
63.74
63.23
63.46
$3.04
Pillar
PD- QUANT
8/8
64.59
64.87
64.21
64.32
53.91
QAT
8/8
65.56
60.08
65.69
65.17
65.4 65.44
$4,99
LIDAR-PTQ
8/8
65.60
60.12
65.64
65.14
65.55
55.11
Full Prec.
32/32
67.67
65.25
66.29
65.79
68.04
62.35
68.69
67.61
BRECQ
8/8
63.15
60.71
62.53
62.03
63.22
$7.49
63.71
62.60
CP-
QDROP
8/8
64.70
62.23
63.97
63.38
64.90
59.17
65.24
64.13
Voxel
PD-QUANT
8/8
66.45
64.00
65.11
64.56
66.91
61.18
67.32
66.25
QAT
8/8
67.63
65.20
66.28
65.76
67.98
62.28
68.62
67.55
LiDAR-PTQ
8/8
67.60
65.18
66.27
65.78
67.95
62.24
68.60
67.52
Due to there are no PTQ methods specially designed for 3D LiDAR-based detection tasks, we reimplement several advanced PTQ methods in 2D RGB-based vision tasks, which are BRECQ (Li et al., 2021), QDROP (Wei et al., 2022) and PD-Quant (Liu et al., 2023a). Specifically, we select well-known CenterPoint (Yin et al., 2021) our full precision model and report the quantized performance on WOD (Sun et al., 2020) dataset. Because it includes SPConv- based and SPConv-frre free models, which could effectively verify the generalization of our LIDRR-PTO As shown in Tab 3, LiDAR-PTQ achieves tfate-of-the-as art performance and outperforms BRECQ and QDrop by large margin of 3.87 and 2.00 CenterPoint- model and 4.45 and 2.90 CenterPoint- Voxel model.
7

---

Published as a conference paper at ICLR 2024
For PD-Quant, state-of-the-art PTQ method specially designed for CGB-based vision tasks, but it has suboptimal performance on LiDAR-basee tasks. Specifically. to solve the over- wer-fitting problem on the calibration set, PD-Quant adjusts activation according to FP model's BN layer. However, for the point cloud which is more sensitive to the arithmetic range, this design is ineffective and time-consuming, and will lead to accuracy loss. Notably, our LiDAR-PTQ achieves on-par superior accuracy than the QAT model and almost without performance drop than the float model.
4.2
THE EFFECTIVENESS OF LIDAR-PTQ FOR FULLY SPARSE DETECTOR
Table 4: The performance of FSD on Waymo val set.
Mean (L2)
Vehicle (L.)
Pedestrian (L.2
Cyclist (L2)
Models
Methods
(/A)
mAP
mAPH
mAP
mAPH
mAP
mAPH
mAP
mAPH
Full Prec.
32/32
73.01
70.94
70.34
69.98
73.95
69.16
74.75
73.69
Entropy
8/8
10.54
9.44
0.06
0.06
21.88
18.86
9.41
FSD
Max-min
8/8
71.24
68.42
68.18
72.09
67.60
73.22
72.34
LiDAR-PTQ
8/8
72.84
70.73
69.95
69.62
73.85
68.95
74.71
73.63
Recently, there are emerging of some fully sparse 3D detectors, like FSD (Fan et al., 2022),
FSD++ (Fan et al., 2023) and VoxelNext (Chen et al., 2023), Here, we take FSD as an example to validate the effectiveness of our LiDAR-PTQ on fully sparse detectors. As shown in Tab 4, adopting entropy calibration still leads significant accuracy drop of -61.50. We discover that quantized FSD readily delivers the desired performance while employing vanilla max- min calibration. Nonetheless, using LiDAR-PTQ can further achieve comparable accuracy to float counterpart. The experiments demonstrate that LiDAR-PTO is also applicable to fully sparse detectors.
4.3
ABLATION STUDY
Table Ablation study of different components of LiDAR-PTQ on Waymo val set.
Mean (L.2)
Vehicle (I.2)
Pedestrian (I.2)
Models
Methods
Bits(W/A)
mAP
mAPH
mAP
mAPH
mAP
mAPH
Full Prec.
32/32
65.78
60.32
65.92
65.42
65.65
55.23
Max-min
8/8
$7.33
52.91
55.64
55.37
$9.02
50.45
CP-
+GRID S
8/8
63.66
58.39
63.37
62.87
63.96
53.91
Pillar
+TGPL
8/8
64.81
59.40
65.12
64.53
64.50
54.27
+Round
8/8
65.60
60.12
65.64
65.14
65.55
55.11
Here, we conduct ablation study of different components our LiDAR-PTO based on CenterPoint- Pillar model to verify their effects. As shown in Tab 5, based on the selected Max-min calibrator, we could obtain 5.48 mAPH/L22 performance gain by using lightweight grid search method. However, grid search only minimizes reconstruction error in parameter space, which is not equivalent to minimize the final performance loss. Therefore, by introducing the proposed TGPL function to fine-tune quantization parameters in model space. the performance of quantized model coud be 59.40 mAPH/L2. Finally, by introducing an adaptive rounding value, a freedom degree (Eq 10) added to mitigate the final performance gap and achieve almost the same performance the FP model (60.12 vs 60.32). Notably, the performance of FP model is the upper limit of the quantized model because focus on post-training quantization without labeled training data.
4.4 INFERENCE ACCELERATION
Here, we compared the speed of CenterPoint before and after quantization on an NVIDIA Jeston edge GPU platform that is widely used in real-word AGX Orin. This is a quantized model enjoying 3x inference speedup, which demonstrates self-driving cars. The speed of that our LiDAR-PTQ can effectively improve the efficiency of 3D detection model on edge devices.
Method
Bits(W/A)
Latency
FPS
CP-Vaxel
(NT8)
CP-
32/32
26.01
38.4
Real thee tn
Pillar
8/8
8.82
113.4
CP-Pilur
(FP3))
AUN
CP-
32/32
80.64
12.4
1134
Voxel
8/8
32.26
31.0
20
E20
8

---

Published as a conference paper at ICLR 2024
4.5 COMPUTATION EFFICIENCY
LiDAR-PTO requires additional computation and fine-tuning process compared to other traditional PTQ methods, resulting in increased time costs. While the quantization time is a limitation of LiDAR- PTQ. compared with other advanced PTQ methods, LiDAR-PTQ's additional time cost is acceptable.
Furthermore, QAT method, the quantiza- tion time of LiDAR-PTQ is very short. For example, CenterPoint-Pille
Model
QAT
BRECQ
QDROP
PD-QUANT
LIDAR-PTQ
will take 94 GPU/hour to achieve the same performance as the FP model on WOD dataset, while LiDAR-PTE PAQ takes only 3 GPU/hour, which is 30 x faster than the QAT method. It also proves that our LiDAR-PTQ is cost-effective.
CP-Pillar
CP-Voxel
93.82
80.51
1.73
1.93
1882
1.64
6.13
6.72
2.12
2.75
5 RELATED WORKS
Post-training Quantization (PTQ). As mentioned in (Krishnamoorthii 2018), existing quantization
methods can be divided into two categories: (1) Quantization-Awart Training (QAT) and (2) Post- Training Quantization (PTQ). QAT methods (Wei et al., 2018; Li al., 2019; Esser et al., 2019; Zhuang et al., 2020; Chen et al., 2021) require access to all labeled training data, which may not be feasible due to data privacy and security concerns. Compared to Quantization-awee aware Training (QAT) methods, Post-training quantization (PTQ) methods are simpler to use and allow for quantization with limited unlabeled data. Currently, there are many methods (Wu et al., 2020; Nahshan et al., 2021; Yuan et al., 2022; Liu et al., 2023b; Chu et al., 2024) designed for 2D vision tasks. AdaRound (Nagel et al., 2020) formulates the rounding task as a layer-wise quadratic unconstrained binary optimization problem and achieves better performance. Based on AdaRound, BRECQ (Li et al., 2021) proposes utilizing block reconstruction to further enhance the accuracy of post-traine post-training quantization (PTQ). After that, QDrop (Wei et al., 2022) randomly drops the quantization of activations during PTQ and achieves new state-of-the-an accuracy. PD-Quant (Liu et al., 2023a) considers the global difference of models before and after quantization and adjusts the distribution of activation by BN layer statistics to alleviate the overfitting problem. However, these methods are specially designed for RGB images, and they are not readily transferable to LiDAR point cloud with substantial modal differences.
Quantization for 3D Object Detection. With the wide application of 3D object detection, in autonomous driving and robotics, some quantization methods are designed to improve inference speed for onboard deployment applications. With the advance of quantization techniques based on RGB image, QD-BEV (Zhang et al., 2023) achieves smaller size and faster speed than baseline BevFormer (I.i et al., 2022) using QAT and distillation on multi-camera 3D detection tasks. For LiDAR-based 3D detection, especially for fully convolutional methods, like PointPillars (Lang et al., 2019), FCOS-LIDAR (Tian al., 2022), FastPillars (Zhou et al., 2023), etc., effective quantization solutions could significantly speedup their latency to meet the practical requirements. (Stäcker et al., 2021) find that directly using INT8 quantization for 2D CNN will bring significant performance drop on PointPillars (Lang et al., 2019), where the reduction is even more severe for the entropy calibrator. Besides, BiPointNet (Qin et al., 2021) is a binarization quantization method, which focuses on classification and segmentation tasks based on point cloud captured from small CAD simulation. To the best of our knowledge, there is no quantization solution designed for large-scale outdoor LiDAR- -DDRebasedd3 3D object detection methods in driving
CONCLUSION AND FUTURE WORK
In this paper, we analyze the root cause of the performance degradation of point cloud data during the quantization process. Then we propose an effective PTQ method called LiDAR-PTQ, which is particularly designed for 3D LiDAR-basee based object detection tasks. Our LiDAR-PTO features three main components: (1) a sparsity- based calibration method to determine the initialization of quantiza- tion parameters (2)= a Task Task-guided Global Positive Loss (TGPL) to reduce the disparity on the final task. (3), an adaptive
mondinn-tt--eeares operation to minimize the layer-wise reconstruction error. Extensive experiments demonstrate that our LiDAR-PTQ can achieve state-of-the-art performance on CenterPoint (both dillar-based and based). To our knowledge, for the very first time in lidar-based 3D detection tasks, the PTQ INT8 model's 's accuracy is almost the same the FP32 model while enjoying 3x inference speedup. Moreover, our LEDAR-PPQQ is cost-effective being 30x faster than the quantization-aware training method. Given its effectiveness and efficiency, we hope that our LiDAR-PTO PTQ can serve as a valuable quantization tool for current mainstream grid-based 3D detectors and push the development of practical deployment of 3D detection models on edge devices. Besides, we believe that the low-bit quantization of 3D detectors will bring further efficiency improvement. This remains an open problem for future research.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
We thank anonymous reviewers for their kind help of this work. This work was supported by National Key R&D Program of China (No. 2022ZD01 18700), National Natural Science Foundation of China (No.62271143), and the Big Data Computing Center of Southeast University.
REFERENCES
Ron Banner, Yury Nahshan, and Daniel Soudry. Post training 4-bit quantization of convolutional networks for rapid-deployment. Advances Neural Information Processing Systems, 32, 2019.
Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Jurgen Gall. Semantickiti:: A dataset for semantic scene understanding of lidar sequences. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9297-9307, 2019. 13
Holger Caesar, Varun Bankiti, Alex Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. Nuscenes: A multimodal dataset for autonomous driving. pp. 11621-11631, 2020. 13, 14
Peng Chen, Jing Liu, Bohan Zhuang, Mingkui Tan, and Chunhua Shen. Aqd: Towards accurate quantized object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 104-113, 2021. 9
Yukang Chen, Jianhui Liu, Xiangyu Zhang, Xiaojuan Qi, and Jiaya Jia. Voxelnext: Fully sparse voxelnet for 3d object detection and tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 8
Yoni Choukroun, Eli Kravchik, Fan Yang, and Pavel Kisilev. Low-bit quantization of neural networks for efficient inference. In 2019 JEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3009-3018. IEEE, 2019. 6
Xiangxiang Chu, Liang Li, and Bo Zhang. Make repvgg greater again: A quantization-awan approach. AAAI, 2024. 9
Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmen- dra Modha. Learned step size quantization. arXiv preprint arXiv:1902.08111 08000.,0813,.... 2019.
Lue Fan, Feng Wang, Naiyan Wang, and Zhaoxiang Zhang. Fully sparse 3d object detection. Advances in Neural Information Processing Systems, 35:351-363, 2022.
Lue Fan, Yuxue Yang, Feng Wang, Naiyan Wang, and Zhaoxiang Zhang. Super sparse 3d object detection. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2023. 8
Yongkweon Jeon, Chungman Lee, Eulrang Cho, and Yeonju Ro. Mr. biq: Post-training non-uniform quantization based on minimizing the reconstruction error. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12329-12338, 2022. 3
Xiaogang Wang Hongsheng Li Jiageng Mao, Shaoshuai Shi. 3d object detection for autonomous driving: A comprehensive survey. IJCV, 2023. 1
Raghuraman Krishnamoorthi. Quantizing deep convolutional networks for efficient inference: A
whitepaper. arXiv preprint arXiv:1806.0833 08342, 2018. 9
Alex Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpil- lars: Fast encoders for object detection from point clouds. In CVPR, pp. 12697-12705, 2019. 9, 13
Rundong Li, Yan Wang, Feng Liang, Hongwei Qin, Junjie Yan, and Rui Fan. Fully quantized network for object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 28 2810-2819, 2019. 9
10

---

Published as a conference paper at ICLR 2024
Yuhang Li, Ruihao Gong. Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu. Breeq: Pushing the limit of post- -os--training quantization by block reconstruction. 2021. 1, 6, 7,
Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chonghao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: Learning bird's bud's-eee-vieww representation from multi-camera images via spatiotemporal transformers. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Purt IX, pp. 1-18. Springer, 2022.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017. 6
Jiawei Liu, Lin Niu, Zhihang Yuan, Dawei Yang, Xinggang Wang, and Wenyu Liu. Pd-quant: Post-training quantization based on prediction difference metric. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24427-24433 2023a. 6,
Yijiang Liu, Huanrui Yang. Zhen Dong, Kurt Keutzer, Li Du, and Shanghang Zhang. Noisyquant: Noisy bias-enhanced post-training activation quantization for vision transformers. 2023b. 9
Szymon Migacz. 8-bit inference with tensorrt. In GPU technology conference, volume 2, pp. 5, 2017. 3
Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pp. 7197-7206. PMLR, 2020. 7,
Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tij- men Blankevoort. A white paper on neural network quantization. arXiv preprint orXiv:2106.08295 2021. 3, 4
Yury Nahshan, Brian Chmiel, Chaim Baskin, Evgenii Zheltonozhskii, Ron Banner, Alex Bronstein, and Avi Mendelson. Loss aware post-training quantization. volume 110, pp. 3245-3262. Springer, 2021. 9
Alexander Neubeck and Luc Van Gool. Efficient non-maximum suppression. In /8th International Conference on Pattern Recognition (ICPR (IC) volume 3. pp. 850-855. IEEE, 2006. 6
Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Liu, and Hao Su. Bipointnet: Binary neural network for point clouds. In ICLR, 2021.
Lukas Stäcker, Juncong Fei, Philipp Heidenreich, Frank Bonarens, Jason Rambach, Didier Stricker, and Christoph Stiller. Deployment of deep neural networks for object detection edge ai devices with runtime optimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop, pp. 1015-1022, 2021. 4.9
Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. pp. 1446-2454 2020. 2, 7, 14
Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han. Searching efficient 3d architectures with sparse point-voxed convolution. In European conference on computer vision, pp. 685-702. Springer, 2020. 13
Zhi Tian, Xiangxiang Chu, Xiaoming Wang, Xiaolin Wei, and Chunhua Shen. Fully convolutional one-stage 3d object detection on lidar range images. Advances in Neural Information Processing Systems, 35:34899-3491 2022. 9
Xiuying Wei, Ruihao Gong, Yuhang Li, Xianglong Liu, and Fengwei Yu. Qdrop: randomly dropping quantization for extremely low-bit post-training quantization. 2022. 1, 6, 9,
Yi Wei, Xinyu Pan, Hongwei Qin, Wanli Ouyang, and Junjie Yan. Quantization mimic: Towards very tiny enn for object detection. In Proceedings of the European conference on computer vision (ECCV), pp. 267-283, 2018. 9
11

---

Published as a conference paper at ICLR 2024
Di Wu, Qi Tang, Yongle Zhao, Ming Zhang, Ying Fu, and Debing Zhang. Easyquant: Post-training quantization via scale optimization. arXiv preprint arXiv:2006. 2020. 9
Hongyi Yao, Pu Li, Jian Cao, Xiangcheng Liu, Chenying Xie, and Bingzhang Wang. Rapq: Rescuing accuracy for power-of-two low-bit post-training quantization. arXiv preprint arXiv:220411522. 2022.
Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. pp. 11784-11799 2021. 2, 6, 7, 13, 14
Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, and Guangyu Sun. Ptq4vit: Post-training quantization for vision transformers with twin uniform quantization. pp. 191-207, 2022.
Yifan Zhang, Zhen Dong, Huanrui Yang, Ming Lu, Cheng-Ching Tseng, Yandong Guo, Kurt Keutzer, Li Du, and Shanghang Zhang. Qd-bev: Quantization-awe aware view-guided distillation for multi-view 3d object detection. 2023.
Sifan Zhou, Zhi Tian, Xiangxiang Chu, Xinyu Zhang, Bo Zhang, Xiaobo Lu, Chengjian Feng, Zequn Jie, Patrick Yin Chiang, and Lin Ma. Fastpillars: A deployment-friendly pillar-based 3d detector. arXiv preprint 070222330..02367, 2023. 9
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. 13
pp. 4490 4499, 2018.
Benjin Zhu, Zhengkai Jiang, Xiangxin Zhou, Zeming Li, and Gang Yu. Class-balanced grouping and sampling for point cloud 3d object detection. arXiv preprint arXiv: 1908. 0949099 2019. 14
Bohan Zhuang, Lingqiao Liu, Mingkui Tan, Chunhua Shen, and lan Reid. Training quantized neural networks with full-precision auxiliary module. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1488-1497, 2020. 9
12

---

Published as a conference paper at ICLR 2024
APPENDIX A: LIDAR-PTO FOR DIFFERENT DETECTORS
CenterPoint (Yin et al., 2021) integrates two milestone works in LiDAR-based BEV de- tection, VoxelNet (Zhou & Tuzel, 2018) and PointPillars (Lang et al., 2019) as CP- Pillar and CoxVexel. In particular, CP-Pillar and CP-Voxel have different network design.
The CP-Pillar model is fully dense convolutional network, while the CP- Voxel model includes SP-
Method
representation
backbone
neek head
Conv and dense convolution. Our results on Cen- terPoint (-pillar and -voxel) demonstrate that: i) Lidar-PTQ applicable to pillar-based and voxel- based detectors. ii) Lidar-PTQ is applicable to SPConv and dense convolution operations.
CP-Pillar
Pillar
dense
dense
dense
CR-Voxel
Voxel
sparse
dense
dense
FSD
Point+
sparse
sperse
sparse
Table 6: Performance comparison nuScene vol set. We show the NDS, and mAP for each class. Abbreviations: construction vehicle (CV), pedestrian (Ped), motorcycle (Motor), bicycle (BC) and traffic cone (TC).
Models
Methods
(is(W)))
NDS
mAP
Car
Truck
Bus
Trailer
CV
Ped
Motoe
BC
TC
Barrier
Full Prec.
32/32
60.3
50.0
83.8
50.6
61.8
31.2
9.2
79.4
44.1
20.2
57.7
61.3
BRECQ
56.9
75.9
54.3
21.6
15.7
53.3
CP:
QDROP
8/8
57.8
45.9
78.8
44.2
$7.0
23.8
5.2
78.4
40.1
17.6
56.7
56.8
Pillar
PD-QUANT
8/8
59.6
48.3
81.8
59.4
28.2
7.8
78.4
41.6
19.8
57.6
61.0
LIDAR-PTQ
8/8
60.2
49.8
83.7
50.8
61.8
30.6
9.0
79.0
43.6
20.4
57.8
61.0
Full Prec.
32/32
64.8
56.6
84.6
54.5
66.7
36.4
16.9
83.1
56.1
39.6
64.0
64.3
BRECQ
76.5
60.5
28.9
12.5
34.8
CP: Vexel
QDROP
8/8
63.2
54.0
82.1
48.5
64.9
32.9
15.1
81.1
55,1
36.9
60.9
63.77
PD-QUANT
66.6
36.4
63.2
LIDAR-PTO
8/8
64.7
56.5
84,6
54.2
66.7
36.4
16.6
83,3
56.0
39.4
63.6
64.4
APPENDIX B: PERFORMANCE COMPARISON ON NUSCENES DATASET
To further evaluate the effectiveness of LiDAR-PTQ, we also conducted experiments on nuScenes (Caesar et al., 2020) dataset. Our performance evaluation involves two metrics, average precision (mAP) and nuScenes detection score (NDS). NDS is a weighted average mAP and other attributes metrics, including translation, scale, orientation, velocity, and other box attributes. As shown in Tab 6, LiDAR-PTQ achieves state-of-the-art performance and outperforms BRECQ and QDrop by a large margin of 6.2 mAP and 3.9 mAP on CenterPoint-Pillaa model and 5.3 mAp and 2.5 mAP CenterPoint Voxel model. Consistent with the accuracy on the Waymo dataset, our LiDAR-PTO also achieves almost the same performance as the full precision model on nuScenes dataset.
APPENDIX C: LIDAR-PTO FOR POINT CLOUD SEGMENTATION
Table 7: The PTQ performance of SPVNAS on SemanticKITTI val set.
other- vehicle
other- other-groum -roundd
motorcyele
motoreyclis
vegetation
traffic sign
bicycle
person
bicyclist
parking
sidewalk
building
terrain
mloU
truck
road
fence
trunk
pole
Method
Full Prec.
65.0 96.3 49.0 77.6 74.4 51.8 75.2 88.2.. 5.7 93.4 44.6 81.0 3.5 89.3 56.5 87.8
68.4
75.1
67.1
49.6
Entropy
46.9
92.9 34.7 72,1
20.4 37.2 48.5 80.9 5.1 47.8
16.9 28.7
0.2 79.9
47.5
82.9
57.0
44.0
55.9
38.8
Max-min
62.4
94.5
46.2
75.3
73.0
50.2
73.6
86.4
5.7
92.3
41.5
78.9
2.1
87.3
53.4
85.1
65.3
71.8
63.0
48.6
LIDAR-PQ 64.9 96.3 48.7 78.0 74.3 52.2 74.5 87.9 5.9 93.3 44.0 80.9
3.5 89.4 56.4 87.6 68.3
74.5
67.2 49.5
Additionally, we conducted experiments SemanticKITTI (Behley et al., 2019) dataset for point cloud segmentation to further evaluate the generalization of LiDAR-PTQ. Specifically, we utilize SPVNAS (Tang et al., 2020) our baseline, which is representative work in point cloud segmenta- tion task. As shown in Tab 7, adopting entropy calibration leads to a significant accuracy drop of 18.09 mIOU. As for vanilla max-min calibration, there is still performance drop 2.64 mlOU
13

---

Published as a conference paper at ICLR 2024
for quantized SPVNAS. However, LiDAR -TQ can further achieve comparable accuracy to its float counterpart. This demonstrates the effectiveness of LiDAR-PTE on point cloud segmentation tasks well.
APPENDIX D: EXPERIEMNTS DETAILS
Dataset. NuScenes dataset (Caesar et al., 2020) uses a LiDAR with 32 lines to collect data, containing 1000 scenes with 700, 150, and 150 scenes for training, validation, and testing, respectively. The metrics of the 3D detection task are mean Average Precision (mAP) and the nuScenes detection score (NDS). Waymo Open Dataset (Sun et al., 2020) uses a LiDAR with 64 beams to collect data, containing 1 1.50 sequences in total, 798 for training, 202 for validation, and 150 for testing. The metrics the 3D detection task are mAP and mAPH (mAP weighted by heading). In Waymo, LEVELI and LEVEL2 are two difficulty levels corresponding to boxes with more than five LiDAR points and boxes with at least one LiDAR point. The detection range in nuScenes and WOD is 50 meters (cover of 100m 100m) and 75 meters (cover area of 150m 150m).
Implementation Details. All the FP models in our paper use CenterPoint Yin et al., 2021) official open-source codes based Det3D (Zhu al., 2019) framework. In WOD dataset, we randomly sample 256 frames point cloud data from the training set as the calibration data. The calibration set proportions 0.16% % (256/158,88) for WOD, nuScenes dataset, the calibration set proportions We set the first and the last layer of the network to keep full precision. We
are 0.91 % (256/28,130) execute block reconstruction for the backbone and layer reconstruction for the neck and the head with batch size of 4, respectively. Note that we do not consider using Int8 quantization for the PFN in CenterPoint-Pinn Pillar, since the input is 3D coordinates, with approximate range =10 m and accuracy 0.01 m. so that Int8 quantization in FPN would result in significant loss of information. The learning rate for the activation quantization scaling factor is 5e-5, and for weight quantization rounding, the learning rate is 5e-3. In TGPL loss, we set as 0. and K as 500. We execute all experiments single Nvidia Tesla V100 GPU. For the speed test, the inference time of all comparison methods is measured on an NVIDIA Jeston AGX Orin, a resource-constrained edge GPU platform widely used in real-world autonomous driving.
APPENDIX E: ENTROPY CALIBRATION METHOD
Given the original and quantized data distribution p(t) and q(i) as follows:
(KL(p(i), (()) = Σ p(i) log p(i) - -(() log ((1)
(13)
The entropy calibration method in Algorithm3
Algorithm 2 Entropy calibration method
Input: FP32 histogram H with N bins, and bit-width b.
Output: threshold with (ee))Dx
Require: len(p) = len(g)
I: for in range(2 .N do
2: ref dist sois_p(()-
1]
3: outliers ccuurstccuuttt summmunnn (in 1), bin[N 1])
4: ref dist (i)sf 1]+ outliess count
5: p(i) rref dist (rm) dist p())
6: quantize candidate_dist_(() from Oin bin bin (22() (bin (bin())....,,inn bi.. 1]) memmhotho'lineatt
bin(i 1] into 2 levels
7: paaiidaate_dss_qqiiiieeeeeee
8: q(i) =wwnddddateddisteeccoo.. swm cummcandidaa (candidate.dist_q(()))
9: divergenceji = Dx (p)),, (()) using Eq 13
10: end for
11: m = argmin DD = [divergence 2 1], divergence[N - -1])
12: threshold (m + 0.5) (widthsa))
13: return threshold
14

---

Published as a conference paper at ICLR 2024
APPENDIX F: GIRD SEARCH
For a weight or activation tensor X, we can get their initial quantization scale factor using the following equation:
④ (clamp(l 2
2, 9roin: 9maw) - 2)
(14)
(Xmar Immin) (2 -1)
(15)
arg min ||X (s
(16)
|| 112 is the Frobenius norm (MSE Loss). Refer to appendix for more details about grid search. Then linearly divide the interval [ass, Ssu] into T candidate bins, denoted {sc)(=1- a, 8 and T are designed to control the search range and granularity. Finally, search {st to find the optimal Sopt that minimizes the quantization error, The entropy calibration method in Algorithm3
Algorithm 3 Grid search
Input: the input of full precision tensor X, bit-width b and T bins.
Output: scale factor Sopt with min X (s
using maax maz(1=]) get max value of tensor X
2: set range = Emax, Chest = 100
set Umin = mmin and Umar Imaz
4: for in range(1, do
5: threshold = range/TT
6: Zmin -threshold, Zmar = threshold
7: get scale 8; with Umin and waa using Eq 15
8: input the quantized value & and FP value using Eq 14 get score
9: update Umin and Umaz when < Cbest and update Cbest = c
10: end for
11: get Umin and Uman with the minimal score c
12: get final scale Sept with Umin and Umax using Eq 15
13: return scale Sopt
15


Reviews:
[{"review_id": "kGG0bSazcg", "reviewer": "Reviewer_szHU", "summary": "This paper describes an approach for model compression for applications using 3D Lidar data, namely 3D object detection.  The approach is based on post-training quantization.  The paper shows that direct application of post-training quantization methods used with 2D images leads to a decrease in performance. The method proposed in the paper is based on a sparsity-based calibration followed by the application of a task -guided global positive loss and finally by the application of an adaptive rounding-to-nearest operation. Experimental results show that the proposed approach achieves state-of-the-art performance, namely that the accuracy of the model with PTQ INT8 is at the level of the accuracy of the model using FP32. The inference speed is increased 3 times.", "questions": "Two main questions:\n--Would  similar results be obtained with other object detectors based on Lidar data?\n--Would similar results be obtained with other applications using Lidar data, e.g., segmentation?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper shows that the simple and direct application of approaches commonly used for model compression in applications with 2D images leads to a decrease in performance (when applied to object detection with 3D lidar data) and proposes a new method for model compression (for 3D Lidar data and for object detection). This new model achieves similar accuracy to a full precision model and achieves an increase in inference speed (3x). The authors also analyze and discuss the reasons that may be the cause of such behavior. Such analysis is qualitative---no experimental validation. The proposed approach is clear and contributes a solution to a well-defined problem.", "weaknesses": "The main weakness of the paper results from the experimental results (and validation) having been obtained for a single application: CenterPoint (in both versions, Pillar and Voxel).  This is too specific, and it is not clear that similar results could be obtained with other applications using 3D Lidar data, e.g., segmentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Gl8jbTgUk", "reviewer": "Reviewer_Dy1B", "summary": "This paper proposes a Post-Training Quantization (PTQ) compression approach, LiDAR-PTQ, for the real applications of 3D LiDAR point cloud detection that are deployed on edge GPU systems. The key ideas include density-aware calibration, task-guided global positive loss and layerwise-reconstruction-based adaptive rounding.  The proposed method is evaluated on the Waymo dataset to validate the low performance-drop with INT8 compared with FP32.", "questions": "- Comparisons of inference acceleration performance with other baselines are not available in Tab.3.  Accuracy together with speed performance can better demonstrate the advantages of the proposed method.\n- How is γ set for TGPL?\n- It is wrotten that “there is no extra access to labeled training data”. Dose this work like the knowledge distillaatio  from the full FP32 model  for an INT8 quantizated model?\n- For results on Waymo, can authors give some explaination about why  select LEVEL2?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is  easy to understand.  The analysis of why 2D-PTQ can’t be directly applied to 3D is reasonable and the corresponding solution is practical. \n-  The proposed LiDAR-PTQ achieves competitive  detection performance as INT8 quantization for bothe CP-Pillar and Voxel.", "weaknesses": "- The method is only evalutated on the PointCenter model, how about the effect for other types of detectors (Transform-based, etc.).\n- It is said that the method is evaluated on various datasets (Waymo and NuScenes). However, results on  NuScenes seem not available.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fl9ZFh9MQS", "reviewer": "Reviewer_3wrT", "summary": "Running neural nets for object detection on an edge device requires quantization of the network so that it can fit on the limited memory of the device and perform inference at an adequate (real-time) speed.\nTraditionally, post-training quantization (PTQ) for vision tasks utilizes a small set of unlabeled samples as a calibration set. These samples are used to quantize the weights and activations of the network using max-min and information theory to minimize KL-divergence between the quantized and non-quantized tensor. The information-theory based method which performs well for 2D object detection does quite poorly on 3D LIDAR object detection because of sparsity in the point cloud. This paper describes a LIDAR-PTQ method that quantizes existing LIDAR object detectors like the Centre-point-pillar. It uses a sparsity based calibrator, a Task-Guided Loss (which calibrates the quantization parameters using performance on the final object detection task) and an adaptive rounding-to-nearest loss which determines whether the weight in a particular neuron should be rounded up or down.\n\nExperiments demonstrate similar performance to the floating point model on the NuScenes and Waymo datasets, but with a 3x inference speedup on a Jetson AGX Orin edge device.", "questions": "-What is LEVEL_2 mAPH in Table 1? Please clarify this closer to the table\n-Page 5: Huge ‘sparisity’ lead to inappropriate quantization range. \n‘Sparisity’ should be sparsity\n-Page 7: ‘… are the soft-quantized weights follows Eq 11’. \nI think you mean equation 10?\n-Page 7: ‘NuSecens’ should be NuScenes", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is on the whole well written and does a good job of introducing the reader to the subject of neural network quantization. It describes an algorithm for the important task of adapting large neural nets for operation on the edge, while preserving model accuracy. Since there is relatively little work on LIDAR network quantization, the paper is timely and important.", "weaknesses": "There is not enough detail about the optimization steps: particularly which parameters are optimized when and what percentage of training data is required for this post-training quantization. The task-guided loss minimizes the distance between the output of the quantized network vs the non-quantized network and so presumably requires the labelled training data. This seems to deviate from the requirements of PTQ, which only seems to need a small number of unlabeled samples as the calibration set.\nEven though the combined system with max-min quantization + grid search + TGPL + Adaptive Rounding together approach the mAP of the FP model, the contribution in narrowing the gap between max-min and the final model seems to be mainly by the grid search. The TGPL and Adaptive Rounding contribute relatively small gains to the mAP. This could be a quirk of the dataset / optimization parameters / training regime and I would ideally like to see more experiments to be convinced of the efficacy of TGPL and Adaptive Rounding, even though from a conceptual point of view, they make sense.\nI would like the code to be released so that independent validation of the authors’ experiments can be performed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MKC1v6qMun", "reviewer": "Reviewer_fLuR", "summary": "This paper proposes a post-training quantization method for LiDAR-based 3D object detection. It uses min-max quantization and proposes task-guided global positive loss.\nThe experiments are conducted in the Waymo Open Dataset with CenterPoint detector, achieving competitive performance among other quantization methods.", "questions": "No more additional questions. Looking forward to the authors' response to my concerns in the weaknesses box.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- This paper achieves competitive results compared with other quantization methods.\n- The writing is clear and easy to follow.\n- Although the proposed method is quite simple, it is fast and pratical.", "weaknesses": "### This paper lacks novelty and originality. \n- Although the authors propose the so-called sparsity-based calibration and conduct the spatial sparsity analysis, the implementation simply adopts the max-min calibration without any unique designs.\n- The proposed TGPL is also a trivial technique, which basically selects some high-confidence prediction for supervision and this is widely-adopted in many fields such as distillation. \n### Some claims are inaccurate or wrong.\n\"Point cloud coordinates have explicitly encoded the shapes and geometrics of objects, so the larger arithmetic range of input point\ncloud preserves essential geometric information.\"  By this sentence, the authors indicate the large range of point cloud coordinates makes the detectors sensitive to quantization range. However, in fact, LiDAR-based object detectors do not take the absolute point cloud coordinates as input. Voxel encoders usually calculate the relative coordinates within each voxel as input.\n### The application is a little bit narrow.\nThere are emerging methods of \"fully sparse detectors[1][2][3]\", which show impressive efficacy and efficiency and do not lead to zero feature maps. Thus the proposed method (sparsity-based calibration) has no advantages in these methods.\n\n[1] Fully Sparse 3D Object Detection, NeurIPS 2022. \\\n[2] Super Sparse 3D Object Detection, TPAMI 2023. \\\n[3] VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking, CVPR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)