SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 3: Fine-tuning performance on Waymo benchmark (LEVEL_2 metric). Note that we only use a single checkpoint parameter to initialize all downstream baselines including SECOND, CenterPoint, PV-RCNN++. Semi denotes the semi-supervised method training on unlabeled ONCE split.
Method	Paradigm	Data amount	L2 AP / APH
Overall	Vehicle	Pedestrian	Cyclist
From scratch (SECOND)	-	3%	52.00 / 37.70	58.11 / 57.44	51.34 / 27.38	46.57 / 28.28
From scratch (SECOND)	-	20%	60.62 / 56.86	64.26 / 63.73	59.72 / 50.38	57.87 / 56.48
ProposalContrast (SECOND) [31]	SS-PT	20%	60.91 / 57.16	64.50 / 63.90	60.33 / 51.00	57.90 / 56.60
BEV-MAE (SECOND) [13]	SS-PT	20%	61.03 / 57.30	64.42 / 63.87	59.97 / 50.65	58.69 / 57.39
MeanTeacher (SECOND) [21]	Semi	20%	60.93 / 57.31	64.22 / 63.73	59.54 / 50.80	58.66 / 57.41
Ours (SECOND)	AD-PT	3%	55.41 / 51.78	60.53 / 59.93	54.91 / 45.78	50.79 / 49.65
Ours (SECOND)	AD-PT	20%	61.26 / 57.69	64.54 / 64.00	60.25 / 51.21	59.00 / 57.86
From scratch (CenterPoint)	-	3%	59.00 / 56.29	57.12 / 56.57	58.66 / 52.44	61.24 / 59.89
From scratch (CenterPoint)	-	20%	66.47 / 64.01	64.91 / 64.42	66.03 / 60.34	68.49 / 67.28
GCC-3D (CenterPoint) [12]	SS-PT	20%	65.29 / 62.79	63.97 / 63.47	64.23 / 58.47	67.68 / 66.44
ProposalContrast (CenterPoint) [31]	SS-PT	20%	66.67 / 64.20	65.22 / 64.80	66.40 / 60.49	68.48 / 67.38
BEV-MAE (CenterPoint) [13]	SS-PT	20%	66.92 / 64.45	64.78 / 64.29	66.25 / 60.53	69.73 / 68.52
MeanTeacher (CenterPoint) [21]	Semi	20%	66.66 / 64.23	64.94 / 64.43	66.35 / 60.61	68.69 / 67.65
Ours (CenterPoint)	AD-PT	3%	61.21 / 58.46	60.35 / 59.79	60.57 / 54.02	62.73 / 61.57
Ours (CenterPoint)	AD-PT	20%	67.17 / 64.65	65.33 / 64.83	67.16 / 61.20	69.39 / 68.25
From scratch (PV-RCNN++)	-	3%	63.81 / 61.10	64.42 / 63.93	64.33 / 57.79	62.69 / 61.59
From scratch (PV-RCNN++)	-	20%	69.97 / 67.58	69.18 / 68.75	70.88 / 65.21	69.84 / 68.77
ProposalContrast (PV-RCNN++) [31]	SS-PT	20%	70.30 / 67.78	69.45 / 69.00	71.42 / 65.68	70.04 / 69.05
BEV-MAE (PV-RCNN++) [13]	SS-PT	20%	70.54 / 68.11	69.53 / 69.07	71.50 / 65.69	70.60 / 69.56
MeanTeacher (PV-RCNN++) [21]	Semi	20%	70.62 / 68.14	69.21 / 68.81	71.96 / 66.42	70.17 / 69.21
Ours (PV-RCNN++)	AD-PT	3%	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00
Ours (PV-RCNN++)	AD-PT	20%	71.55 / 69.23	70.62 / 70.19	72.36 / 66.82	71.69 / 70.70
$$
\mathcal {L} _ {\text {c o n s i s t}} = \frac {1}{B K} \sum_ {i = 1} ^ {B} \sum_ {j = 1} ^ {K} \left(\hat {f} _ {j} ^ {\Gamma_ {1}} - \hat {f} _ {j} ^ {\Gamma_ {2}}\right) ^ {2}, \tag {4}
$$
where  $B$  is the batch size and  $K$  is the number of the corresponding activation features.
Overall Function. The overall loss function can be formulated as:
$$
\mathcal {L} _ {\text {t o t a l}} = \mathcal {L} _ {\text {c l s}} + \mathcal {L} _ {\text {r e g}} + \mathcal {L} _ {\text {c o n s i s t}}, \tag {5}
$$
where  $\mathcal{L}_{cls}$  and  $\mathcal{L}_{reg}$  represent classification loss and regression loss of the dense head, respectively, and  $\mathcal{L}_{\text{consist}}$  is the activation consistent loss as shown in Eq. 4.
Pre-training Dataset. ONCE [14] is a large-scale dataset collected in many scenes and weather conditions. ONCE contains 581 sequences composed of 20 labeled ( $\sim$ 19k frames) and 561 unlabeled sequences ( $\sim$ 1M frames). The labeled set divides into a train set with 6 sequences ( $\sim$ 5K samples), a validation set with 4 sequences ( $\sim$ 3k frames), and a test set with 10 sequences ( $\sim$ 8k frames). We merge Car, Bus, Truck into a unified category (i.e., Vehicle) when performing the pre-training. Our main results are based on ONCE small split and use a larger split to verify the pre-training scalability.
Description of Downstream Datasets. 1) Waymo Open Dataset [20] contains  $\sim 150\mathrm{k}$  frames. 2) nuScenes Dataset [1] provides point cloud data from a 32-beam LiDAR consisting of 28130 training samples and 6019 validation samples. 3) KITTI Dataset [5] includes 7481 training samples and is divided into a train set with 3712 samples and a validation set with 3769 samples.
Description of Selected Baselines. In this paper, we compare our method with several baseline methods including both Self-Supervised Pre-Training (SS-PT) and semi-supervised learning methods. SS-PT methods: as mentioned in Sec. 2.2, We mainly compare with contrastive learning-based (i.e., GCC-3D [12], ProposalContrast [31]) and MAE-based SS-PT methods (i.e., Voxel-MAE [6], BEV-MAE [13]). All these methods are pre-trained and fine-tuned on the same dataset. Semi-supervised learning methods: to verify the effectiveness of our method under the same experimental setting, we also compare with commonly-used semi-supervised technique (i.e., MeanTeacher [21]).
Table 4: Fine-tuning performance on nuScenes benchmark. C.P. denotes that CenterPoint is employed as the baseline detector and D.A. represents the Data Amount. We fine-tune on  $5\%$  and  $100\%$  data for 20 epochs. Compared with other works [12, 13], our pre-training process is performed on a unified dataset rather than nuScenes.
Method	Setting	D.A.	mAP	NDS	Car	Truck	CV.	Bus	Trailer	Barrier	Motor.	Bicycle	Ped.	TC.
From scratch (SECOND)	-	5%	29.24	39.74	67.69	33.02	7.15	45.91	17.67	25.23	11.92	0.00	53.00	30.74
From scratch (SECOND)	-	100%	50.59	62.29	-	-	-	-	-	-	-	-	-	-
Ours (SECOND)	AD-PT	5%	37.69	47.95	74.89	41.82	12.05	54.77	28.91	34.41	23.63	3.19	63.61	39.54
Ours (SECOND)	AD-PT	100%	52.23	63.04	83.12	52.86	15.24	68.58	37.54	59.48	46.01	20.44	78.96	60.05
From scratch (C.P.)	-	5%	42.68	50.41	77.82	43.61	10.65	44.01	18.71	52.95	36.26	16.76	67.62	54.52
From scratch (C.P.)	-	100%	56.2	64.5	84.8	53.9	16.8	67.0	35.9	64.8	55.8	36.4	83.1	63.4
GCC-3D (C.P.) [12]	SS-PT	100%	57.3	65.0	85.0	54.7	17.6	67.2	35.7	65.0	56.2	36.0	82.9	63.7
BEV-MAE (C.P.) [13]	SS-PT	100%	57.2	65.1	84.9	54.9	16.5	67.2	35.9	65.2	56.0	36.2	83.2	63.5
Ours (C.P.)	AD-PT	5%	44.99	52.99	78.90	43.82	11.13	55.16	21.22	55.10	39.03	17.76	72.28	55.43
Ours (C.P.)	AD-PT	100%	57.17	65.48	84.86	54.37	16.09	67.34	36.06	64.31	58.50	40.58	83.53	66.05
Table 5: Fine-tuning performance  $\left( {\mathrm{{AP}}}_{3\mathrm{D}}\right)$  on KITTI benchmark.
Method	Setting	Data amount	mAP (Mod.)	Car	Pedestrian	Cyclist
Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
From scratch (SECOND)	-	20%	61.70	89.78	78.83	76.21	52.08	47.23	43.37	76.35	59.06	55.24
From scratch (SECOND)	-	100%	66.70	89.63	80.78	78.21	58.05	52.61	48.24	84.25	66.71	62.50
Ours (SECOND)	AD-PT	20%	65.95	90.23	80.70	78.29	55.63	49.67	45.12	83.78	67.50	63.40
Ours (SECOND)	AD-PT	100%	67.58	90.36	81.39	78.41	58.30	53.58	48.72	86.04	67.78	63.95
From scratch (PV-RCNN)	-	20%	66.71	91.81	82.52	80.11	58.78	53.33	47.61	86.74	64.28	59.53
ProposalContrast (PV-RCNN) [31]	SS-PT	20%	68.13	91.96	82.65	80.15	62.58	55.05	50.06	88.58	66.68	62.32
From scratch (PV-RCNN)	-	100%	70.57	-	84.50	-	-	57.06	-	-	70.14	-
GCC-3D (PV-RCNN) [12]	SS-PT	100%	71.26	-	-	-	-	-	-	-	-	-
STRL (PV-RCNN) [7]	SS-PT	100%	71.46	-	84.70	-	-	57.80	-	-	71.88	-
PointContrast (PV-RCNN) [25]	SS-PT	100%	71.55	91.40	84.18	82.25	65.73	57.74	52.46	91.47	72.72	67.95
ProposalContrast (PV-RCNN) [31]	SS-PT	100%	72.92	92.45	84.72	82.47	68.43	60.36	55.01	92.77	73.69	69.51
Ours (PV-RCNN)	AD-PT	20%	69.43	92.18	82.75	82.12	65.50	57.59	51.84	84.15	67.96	64.73
Ours (PV-RCNN)	AD-PT	100%	73.01	91.96	84.75	82.53	68.87	60.79	55.42	91.81	73.49	69.21
Implementation Details. We fine-tune the model on several different detectors including SEC-OND [27], CenterPoint [32] and PV-RCNN++ [19]. Note that the transformer-based detectors are not used, since we consider directly applying the pre-trained backbone parameters to the most commonly used downstream baselines. For different views generation, we consider 3 types of data augmentation methods, including random rotation  $\left(\left[-180^{\circ}, 180^{\circ}\right]\right)$ , random scaling  $\left([0.7, 1.2]\right)$ , and random flipping along X-axis and Y-axis. We use Adam optimizer with one-cycle learning rate schedule and the maximum learning rate is set to 0.003. We pre-train for 30 epochs on ONCE small split and large split using 8 NVIDIA Tesla A100 GPUs. The number of selected RoI features is set to 256 and the cross-view matching threshold  $\tau$  is set to 0.3. Our code is based on 3DTrans [22].
Evaluation Metric. We use dataset-specific evaluation metrics to evaluate fine-tuning performance on each downstream dataset. For Waymo, average precision (AP) and average precision with heading (APH) are utilized for three classes (i.e., Vehicle, Pedestrian, Cyclist). Following [13, 31], we mainly focus on the more difficult LEVEL_2 metric. For nuScenes, we report mean average precision (mAP) and NuScenes Detection Score (NDS). For KITTI, we use mean average precision (mAP) with 40 recall to evaluate the detection performance and report  $\mathrm{AP}_{\mathrm{3D}}$  results.
Table 6: Ablation study on data preparation.
Method	Enhancement	Waymo L2 AP/APH	nuScenes
Overall	Vehicle	Pedestrian	Cyclist	mAP	NDS
Baseline	None	67.12 / 64.55	67.45 / 66.97	67.74 / 61.15	66.19 / 65.24	36.26	45.04
Baseline+re-scaling	Object-size	67.39 / 64.68	67.52 / 67.03	67.82 / 61.24	66.83 / 65.79	39.72	49.93
Baseline+re-sampling	LiDAR-beam	67.37 / 64.70	67.70 / 67.21	68.21 / 61.71	66.15 / 65.18	41.35	51.03
Baseline+re-scaling+re-sampling	Both	67.77 / 65.09	68.01 / 67.61	68.32 / 61.69	66.99 / 65.98	43.11	52.41
Results on Waymo. Results on Waymo validation set are shown in Tab. 3. We first compare the proposed method with previous SS-PT methods. It can be seen that all three detectors achieve the best results using AD-PT initialization, surpassing previous SS-PT methods even using a smaller pre-training dataset (i.e.,  $\sim 100\mathrm{k}$  frames on ONCE small split). For example, the improvement achieved by PV-RCNN++ is  $1.58\% / 1.65\%$  in terms of L2 AP/APH. Note that the compared SS-PT
Table 7: The impact of pseudo-labeling methods on downstream datasets.
Pseudo-labeling Method	ONCE	Waymo L2 AP/APH	nuScenes
Overall	Overall	Vehicle	Pedestrian	Cyclist	mAP	NDS
SECOND (Low Performance)	57.10	65.96 / 63.29	65.95 / 65.46	66.87 / 60.36	65.07 / 64.06	41.49	50.82
CenterPoint (Middle Performance)	60.84	66.79 / 64.10	67.09 / 66.60	67.79 / 61.16	65.51 / 64.55	41.91	51.64
Ours (High Performance)	69.90	67.77 / 65.09	68.01 / 67.61	68.32 / 61.69	66.99 / 65.98	43.11	52.41
Figure 6: Different budgets
Table 8: The performance scalability using KITTI and ONCE.
Pre-training dataset	Waymo L2 AP/APH
Overall	Vehicle	Pedestrian	Cyclist
KITTI (~4k)	64.28 / 63.16	64.73 / 64.19	64.43 / 57.30	63.69 / 62.60
ONCE (~4k)	64.28 / 61.36	66.11 / 65.64	66.26 / 59.51	65.39 / 64.35
ONCE (~10k)	66.94 / 64.24	67.41 / 66.91	67.97 / 61.39	65.45 / 64.43
ONCE (~100k)	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00
ONCE (~500k)	69.04 / 66.52	68.69 / 68.23	69.81 / 63.74	68.61 / 67.60
methods are pre-trained on Waymo  $100\%$  unlabeled train set ( $\sim 150\mathrm{k}$  frames), which has a smaller domain gap with fine-tuning data. Further, to verify the effectiveness of fine-tuning with a small number of samples, we conduct experiments of fine-tuning on  $3\%$  Waymo train set ( $\sim 5\mathrm{K}$  frames). We can observe that, with the help of pre-trained prior knowledge, fine-tuning with a small amount of data can achieve much better performance than training from scratch (e.g.,  $3.41\% / 14.08\%$  in L2 using SECOND as baseline). In addition, to ensure the fairness of the experiments, we compare our method with semi-supervised learning methods which are identical to our pre-training setup.
Results on nuScenes. Due to the huge domain discrepancies, few works can transfer the knowledge obtained by pre-training on other datasets to nuScenes dataset. Thanks to constructing a unified large-scale dataset and considering taxonomic differences in the pre-training stage, our methods can also significantly improve the performance on nuScenes. As shown in Tab. 4, AD-PT improves the performance of training from scratch by  $0.93\%$  and  $0.98\%$  in mAP and NDS.
Results on KITTI. As shown in Tab. 5, we further fine-tune on a relatively small dataset (i.e., KITTI). Note that previous methods often use models pre-trained on Waymo as an initialization since the domain gap is quite small. It can be observed that using AD-PT initialization can further improve the accuracy when fine-tuning on both  $20\%$  and  $100\%$  KITTI training data under different detectors. For instance, the  $\mathrm{AP}_{3\mathrm{D}}$  in moderate level can improve  $2.72\%$  and  $1.75\%$  when fine-tuning on  $20\%$  and  $100\%$  KITTI data using PV-RCNN++ as the baseline detector.
4.3 Insight Analyses
In this part, we further discuss the 3D pre-training. Note that in ablation studies, we fine-tune on  $3\%$  Waymo data and  $5\%$  nuScenes data under the PV-RCNN++ and CenterPoint baseline setting.
4.3.1 Insight Analyses on the Data Preparation
Discussion on Diversity-based Pre-training Processor. From Tab. 6, we observe that both beam re-sampling and object re-scaling during the pre-training stage can improve the performance for other downstream datasets. For example, the overall performance can be improved by  $0.65\% / 0.54\%$  on Waymo and  $7.37\%$  on nuScenes. Note that when pre-training without our data processor, the fine-tuning performance on nuScenes will drop sharply compared with training from scratch, due to the large domain discrepancy. Since our constructed unified dataset can cover diversified data distributions, the backbone can learn more general representations.
Pseudo-labeling Performance. Tab. 7 indicates the impact of pseudo-labeling operation on the downstream perception performance. We use three types of pseudo-labeling methods to observe the low, middle, and high performance on ONCE. We find that the performance of fine-tuning is positively correlated with the accuracy of pseudo-labeling on ONCE. This is mainly due to that pseudo labels with relatively high accuracy can guide the backbone to activate more precise foreground features and meanwhile suppress background regions.
Scalability. To verify the scaling ability of the AD-PT paradigm, we conduct experiments in Tab 8 to show the Waymo performance initialized by different pre-trained checkpoints, which are obtained using pre-training datasets with different scales. Please refer to Appendix for more results.
4.3.2 Insight Analyses on the Unified Representations Learning
Discussion on Unknown-aware Instance Learning Head. It can be seen from Tab. 9 that, Unknown-aware Instance Learning (UIL) head and Consistency Loss (CL) can further boost the perception accuracy on multiple datasets. It can be observed that the gains are larger on the Pedestrian and Cyclist categories. The reason is that the UIL head can better capture downstream-sensitive instances, which are hard to be detected during the pseudo-labeling pre-training process.
Table 9: Ablation study on the designed UIL and CL.
Method	Waymo L2 AP/APH	nuScenes
Overall	Vehicle	Pedestrian	Cyclist	mAP	NDS
Baseline	67.77 / 65.09	68.01 / 67.61	68.32 / 61.69	66.99 / 65.98	43.11	52.41
Baseline+UIL	67.97 / 65.35	67.99 / 67.58	68.62 / 62.12	67.32 / 66.35	43.92	52.65
Baseline+UIL+CL	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00	44.99	52.99
Training-efficient Method. In the main results, we verify that our methods can bring performance gains under different amounts of fine-tuning data (e.g.,  $1\%$ ,  $5\%$ ,  $10\%$  budgets). As shown in Fig. 6, results demonstrate that our method can consistently improve performance under different budgets.
4.3.3 Insight Analysis on Different Types of Baseline Detectors
To verify the generalization of our proposed method, we further conduct experiments on different types of 3D object detection backbones (i.e., pillar-based and point-based). As shown in Tab. 10 and Tab. 11, the performance of multiple types of detectors can improve when initialized by AD-PT pre-trained checkpoints which further shows that AD-PT is a general pre-training pipeline that can be used on various types of 3D detectors.
Table 10: Ablation study on the pillar-based backbone. We conduct experiments on Waymo dataset.
Method	Data amount	Waymo L2 AP/APH
Overall	Vehicle	Pedestrian	Cyclist
From scratch (PointPillar)	20%	48.56 / 39.30	54.28 / 53.51	47.11 / 25.50	44.29 / 38.89
AD-PT (PointPillar)	20%	52.01 / 43.99	58.51 / 57.85	50.22 / 32.52	47.31 / 41.59
From scratch (PointPillar)	100%	57.85 / 50.69	62.18 / 61.64	58.18 / 40.64	53.18 / 49.80
AD-PT (PointPillar)	100%	59.71 / 53.49	64.10 / 63.54	59.00 / 43.13	56.04 / 53.80
Table 11: Ablation study on the point-based backbone. We conduct experiments on KITTI dataset.
Method	Data amount	mAP	Car	Pedestrian	Cyclist
(Mod.)	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
From scratch (PointRCNN)	20%	64.12	-	75.30	-	-	52.52	-	-	69.55
AD-PT (PointRCNN)	20%	67.67	88.75	77.20	75.18	64.58	54.16	48.24	84.25	71.86
From scratch (PointRCNN)	100%	68.40	-	78.70	-	-	54.41	-	-	72.11
AD-PT (PointRCNN)	100%	70.47	90.90	80.25	78.05	64.80	57.13	50.37	92.45	74.04
In this work, we have proposed the AD-PT paradigm, aiming to pre-train on a unified dataset and transfer the pre-trained checkpoint to multiple downstream datasets. We comprehensively verify the generalization ability of the built unified dataset and the proposed method by testing the pre-trained model on different downstream datasets including Waymo, nuScenes, and KITTI, and different 3D detectors including PV-RCNN, PV-RCNN++, CenterPoint, and SECOND.
6 Limitation
Although the AD-PT pre-trained backbone can improve the performance on multiple downstream datasets, it needs to be verified in more actual road scenarios. Meanwhile, training a backbone with more generalization capabilities through data from different sensors is also a future direction.
This work is supported by National Natural Science Foundation of China (No. 62071127), National Key Research and Development Program of China (No. 2022ZD0160100), Shanghai Natural Science Foundation (No. 23ZR1402900) and Shanghai Artificial Intelligence Laboratory, in part by Science and Technology Commission of Shanghai Municipality under Grant 22DZ1100102, in part by National Key R&D Program of China under Grant No. 2022ZD0160104.
[32] Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center-based 3d object detection and tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11784-11793, 2021. 1, 3, 5, 8
[33] Jiakang Yuan, Bo Zhang, Xiangchao Yan, Tao Chen, Botian Shi, Yikang Li, and Yu Qiao. Bi3d: Bi-domain active learning for cross-domain 3d object detection. arXiv preprint arXiv:2303.05886, 2023. 2, 5
[34] Bo Zhang, Jiakang Yuan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Uni3d: A unified baseline for multi-dataset 3d object detection. arXiv preprint arXiv:2303.06880, 2023. 2, 5
[35] Jiyang Zheng, Weihao Li, Jie Hong, Lars Petersson, and Nick Barnes. Towards open-set object detection and discovery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3961-3970, 2022. 6
In this supplementary material, we provide more details and experimental results not included in our main text.
Outlines:
- Sec. A: More details about large-scale pre-training dataset preparation.
- Sec. A.1: Preliminary experiments on class-aware pseudo label generator.
- Sec. A.2: Analysis on pseudo label threshold on different classes.
- Sec. A.3: Visualization results of pseudo labels.
- Sec. A.4: Details of object re-scaling.
- Sec. A.5: Taxonomy difference between different datasets.
- Sec. B: Detailed dataset description and evaluation metrics.
- Sec. B.1: Dataset description.
- Sec. B.2: Evaluation metrics.
- Sec. C: More implementation details.
Sec. D: More experimental results.
- Sec. D.1: Ablation studies on unknown-aware instance learning head.
- Sec. D.2: More results of pre-training scalability.
- Sec. D.3: Results of fine-tuning on ONCE.
- Sec. D.4: Visualization results.
A More Details about Large-scale Pre-training Dataset Preparation.
In this section, we give some preliminary experimental results and analysis on large-scale pre-training dataset preparation.
A.1 Preliminary Experiments on Class-aware Pseudo Label Generator
As mentioned in Sec. 3.2.1 in our submission, we explore how to improve the performance on ONCE. We first analyze the results in the ONCE benchmark and find that CenterPoint reaches the SOTA performance on pedestrian and cyclist while PV-RCNN achieves the best performance on vehicle. To use a stronger baseline to further improve the performance, we conduct experiments using PV-RCNN++ as the baseline detector. As shown in Tab. 12, PV-RCNN++ with center head can not obtain a satisfactory performance on ONCE while PV-RCNN++ with anchor head can achieve better accuracy on vehicle and pedestrian.
Further, to obtain more accurate pseudo labels, we use a semi-supervised learning method to further improve the performance as shown in Tab. 13. Finally, we individually train pedestrian using CenterPoint and other classes using PV-RCNN++.
Table 12: Effects of using different heads on PV-RCNN++. We report mAP using the ONCE evaluation metric.
Detector	Head Choice	Vehicle	Pedestrian	Cyclist
PV-RCNN++	Center Head	71.61	45.27	61.15
PV-RCNN++	Anchor Head	81.72	43.86	66.17
A.2 Analysis on Pseudo Label Threshold on Different Classes
Fig. 7 shows the precision under different IoU thresholds. The precision can be calculated by Precision = TP/(FP + TP), where FP and TP denote false positive and true positive, respectively. We can observe that when IoU thresholds are more than 0.8, 0.7, 0.7 for vehicle, pedestrian and cyclist, the number of TP instances is significantly more than that of FP instances.
Table 13: Effects of using MeanTeacher. We report mAP using the ONCE evaluation metric.
Detector	MeanTeacher	Vehicle	Pedestrian	Cyclist
CenterPoint	X	-	46.22	-
CenterPoint	✓	-	56.01	-
PV-RCNN++	X	81.72	-	66.17
PV-RCNN++	✓	82.50	-	71.19
Figure 7: The Precision under different IoU thresholds.
The visualization of the pseudo label results under different thresholds in Fig. 8, we can see that some FP pseudo labels will be annotated when setting low thresholds, while some TP instances can not be annotated when the thresholds are relatively high. To more intuitively see the impact of the threshold on pseudo labeling, we use the model to annotate the samples of the ONCE validation set for comparison with ground-truths.
A.3 Visualization Results of Pseudo Labels
Fig. 9 shows the visualization results of our final pseudo label results.
A.4 Details of Object Re-scaling
In detail, given a bounding box  $b = (c_x, c_y, c_z, l, w, h, \theta_h)$  and point clouds  $(p_i^x, p_i^y, p_i^z)$  within it, where  $(c_x, c_y, c_z), (l, w, h)$  and  $\theta_h$  denote the center, size and heading angle of the bounding box. We first transform points into the local coordinate with the following formula:
$$
\left(p _ {i} ^ {l}, p _ {i} ^ {w}, p _ {i} ^ {h}\right) = \left(p _ {i} ^ {x} - c _ {x}, p _ {i} ^ {y} - c _ {y}, p _ {i} ^ {z} - c _ {z}\right) \cdot R,
$$
$$
R = \left[ \begin{array}{c c c} \cos \theta_ {h} & - \sin \theta_ {h} & 0 \\ \sin \theta_ {h} & \cos \theta_ {h} & 0 \\ 0 & 0 & 1 \end{array} \right], \tag {6}
$$
where  $\cdot$  is matrix multiplication. Then, to derive the scaled object, the point coordinates inside the box and the bounding box size are scaled to be  $\alpha(p_i^l, p_i^w, p_i^h)$  and  $\alpha(l, w, h)$ , where  $\alpha$  is the scaling factor. Finally, the points inside the scaled box are transformed back to the ego-car coordinate system and shifted to the center  $(c_x, c_y, c_z)$  as
$$
\tilde {p _ {i}} = \alpha \left(p _ {i} ^ {l}, p _ {i} ^ {w}, p _ {i} ^ {h}\right) \cdot R ^ {T} + \left(c _ {x}, c _ {y}, c _ {z}\right). \tag {7}
$$
A.5 Taxonomy difference between different datasets
As shown in Tab. 14, there exists a huge taxonomy difference between some fine-tuning datasets and the pre-training dataset. As a result, some foreground instances may be regarded as background if only using pseudo label as supervision.
(d)
(g)
(e)
(h)
(f)
(i)