SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 8: Visualization results under different pseudo label thresholds. (a-c): annotations with low thresholds (i.e., 0.6, 0.5, 0.5 for vehicle, pedestrian and cyclist, respectively). (d-f): the thresholds used in our methods. (g-i): high thresholds (i.e., 0.9, 0.8, 0.8 for vehicle, pedestrian and cyclist, respectively). The green and red bounding boxes represent ground-truths and detector predictions, respectively.
(a)
(d)
Figure 9: Pseudo-labeled annotation results on unlabeled set.
(b)
(e)
(c)
(f)
B Detailed Dataset Description and Evaluation Metrics
B.1 Dataset Description
ONCE dataset. ONCE dataset [14] is a large-scale dataset that is built to encourage the exploration of self-supervised and semi-supervised learning in the autonomous driving scenario. ONCE is collected by a 40-beam LiDAR in multiple cities in China and contains diverse weather conditions
Table 14: Taxonomy difference between different datasets.
Dataset	classes
ONCE (Pre-train)	Car, Truck, Bus, Pedestrian, Cyclist
Waymo (Fine-tune)	Vehicle, Pedestrian, Cyclist
nuScenes (Fine-tune)	Car, Truck, Construction vehicle, Bus, Trailer, Barrier, Motorcycle, Bicycle, Pedestrian, Traffic cone
KITTI (Fine-tune)	Car, Pedestrian, Cyclist
(e.g., sunny, cloudy, rainy), traffic conditions, time periods (e.g., morning, noon, afternoon, night) and areas (e.g., downtown, suburbs, highway, tunnel, bridge).
Waymo Open Dataset. Waymo Open Dataset [20] is a widely-used large-scale autonomous driving dataset that is composed of 1000 sequences and divided into a train set with 798 sequences ( $\sim$ 150k samples) and a validation set with 202 sequences ( $\sim$ 40k samples). The Waymo dataset is gathered in the USA by a 64-beam LiDAR and 4 200-beam short-range LiDAR with annotations in full  $360^{\circ}$ . We use the 1.0 version of Waymo Open Dataset.
nuScenes Dataset. NuScenes dataset [1] provides point cloud data from 32-beam LiDAR collected from Singapore and Boston, USA. It consists of 28130 training samples and 6019 validation samples. The data is obtained during different times in the day, different weather conditions and a diverse set of locations (e.g., urban, residential, nature and industrial).
KITTI Dataset. KITTI dataset [5] is a common-used autonomous driving dataset that contains 7481 training samples and is divided into a train set with 3712 samples and a validation set with 3769 samples. The point cloud data is collected by a 64-beam LiDAR in Germany. KITTI dataset only provides the annotations for the objects within the field of view of the front RGB camera.
B.2 Evaluation Metrics
ONCE evaluation metric. Following ONCE official evaluation metric, we merge the car, bus and truck class into a super-class (i.e., vehicle).  $\mathrm{AP}_{3D}^{Ori}$  is used to evaluate the performance of the ONCE dataset, which can be obtained by the following formula:
$$
A P _ {3 D} ^ {O r i} = 1 0 0 \int_ {0} ^ {1} \max  \left\{p \left(r ^ {\prime} \mid r ^ {\prime} \geq r\right) \right\} d r, \tag {8}
$$
where  $r$  is recall rates from 0.02 to 1.00 at step 0.02 and  $p(r)$  denotes the precision-recall curve. Mean average precision (mAP) is the average of the scores of the three categories. The Intersection over Union (IoU) thresholds are set to 0.7, 0.3 and 0.5 for vehicle, pedestrian and cyclist, respectively.
Waymo evaluation metric. Two difficulty levels (i.e., LEVEL 1 and LEVEL 2) are utilized to evaluate the detection accuracy of Waymo dataset and we mainly focus on more difficult L2 performance. Among each difficulty level, we report AP and APH which can be formulated as:
$$
A P = 1 0 0 \int_ {0} ^ {1} \max  \left\{p \left(r ^ {\prime}\right) \mid r ^ {\prime} \geq r \right\} d r, A P = 1 0 0 \int_ {0} ^ {1} \max  \left\{h \left(r ^ {\prime}\right) \mid r ^ {\prime} \geq r \right\} d r, \tag {9}
$$
where the different between  $h(r)$  and  $p(r)$  is  $h(r)$  is weighted by the accuracy of heading accuracy.
nuScenes evaluation metric. Following the official NuScenes Evaluation Metric, we report mAP and nuScenes detection score (NDS). AP is defined as matches by thresholding the 2D center distance on the ground plane and the mAP can be calculated by:
$$
m A P = \frac {1}{\mathbb {C}} \frac {1}{\mathbb {D}} \sum_ {c \in \mathbb {C}} \sum_ {d \in \mathbb {D}} A P, \tag {10}
$$
where  $\mathbb{C}$  is the set of classes and  $\mathbb{D}$  is the set of thresholds (i.e.,  $\{0.5,1,2,4\}$ ). We mainly focus on 10 classes. NDS is the weighted of mAP and five true positive metrics, including Average Translation
Error (ATE), Average Scale Error (ASE), Average Orientation Error (AOE), Average Velocity Error (AVE) and Average Attribute Error (AAE). The NDS can be formulated as:
$$
m T P = \frac {1}{\mathbb {C}} \sum_ {c \in \mathbb {C}} T P _ {c}, \quad N D S = \frac {1}{1 0} [ 5 m A P + \sum_ {m T P \in \mathbb {T P}} (1 - \min (1, m T P)) ], \tag {11}
$$
where  $\mathbb{TP}$  is the set of true positive metrics.
KITTI evaluation metric. We report mAP with 40 recall positions to evaluate the detection performance and the 3D IoU thresholds is set to 0.7 for cars and 0.5 for pedestrians and cyclists.
C More Implementation Details
As shown in Tab. 15, we list some details about pre-training and fine-tuning datasets. Note that the voxel size of nuScenes is set to [0.1, 0.1, 0.2] following [13]. It can be seen that different datasets may have different dimensions of input features (e.g., ONCE use 4 dimension features as input while Waymo and nuScenes use 5 dimension features) causing the input dimension of the first layer network to be different. We simply do not load the parameters of the first layer when this happens while fine-tuning. In the pre-training phase, we merge the pseudo-labeled data and a small amount of labeled data (i.e., ONCE train set) as the pre-training dataset. In the fine-tuning phase, we fine-tune 30 epochs for Waymo, 20 epochs for nuScenes and 80 epochs for KITTI.
Table 15: Some implementation details about pre-training and fine-tuning datasets.
Dataset	Point cloud range	voxel size	input features
ONCE (Pre-train)	[-75.2, -75.2, -5.0, 75.2, 75.2, 3.0]	[0.1, 0.1, 0.2]	[x, y, z, intensity]
Waymo (Fine-tune)	[-75.2, -75.2, -2.0, 75.2, 75.2, 4.0]	[0.1, 0.1, 0.15]	[x, y, z, intensity, elongation]
nuScenes (Fine-tune)	[-51.2, -51.2, -5.0, 51.2, 51.2, 3.0]	[0.1, 0.1, 0.2]	[x, y, z, intensity, timestamp]
KITTI (Fine-tune)	[0.0, -40.0, -3.0, 70.4, 40.0, 1.0]	[0.05, 0.05, 0.1]	[x, y, z, intensity]
D.1 Ablation Studies on Unknown-aware Instance Learning Head
In this part, we conduct experiments to ablate the hyper-parameters in unknown-aware instance learning head (i.e., the number  $M$  of selected features and the distance threshold  $\tau$ ).
Tab. 16 shows the results using different numbers of selected features in unknown-aware instance learning head when pre-training. When  $M$  is small, some foreground instances with relatively low scores are ignored, while when  $M$  is large, the matched background regions are increased. Considering these factors, we choose  $M$  to be 256.
Tab. 17 shows the performance under different distance thresholds in Eq. 4 in the main submission. The number of matched features is relatively small when using a lower  $\tau$ , thus can not fully exploit the unknown foreground instances. When using a larger threshold, some mismatches may occur. Finally, we set  $\tau$  to 0.3 as mentioned in our main submission.
Table 16: Ablation studies of the number  $M$  of selected features.
M	Waymo L2 AP / APH
Overall	Vehicle	Pedestrian	Cyclist
128	67.71 / 64.98	67.91 / 67.45	68.54 / 61.87	66.67 / 65.63
256	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00
512	67.93 / 65.24	68.04 / 67.36	68.63 / 62.12	67.14 / 66.23
D.2 More Results of Pre-training Scalability
In this section, we show more results to verify the pre-training scalability. We pre-train the model on the small, middle and large splits of the ONCE dataset and then fine-tune the model on  $3\%$  Waymo
Table 17: Ablation studies of the distance threshold  $\tau$ .
τ	Waymo L2 AP/APH
Overall	Vehicle	Pedestrian	Cyclist
0.1	67.90 / 65.22	68.01 / 67.54	68.52 / 62.01	67.17 / 66.12
0.3	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00
0.5	67.82 / 65.15	67.73 / 67.26	68.26 / 61.73	67.49 / 66.46
and  $20\%$  KITTI train data. As shown in Tab. 18, as the scale of the pre-training dataset and the diversity of scenarios increases, the performance of fine-tuning on the downstream dataset will also improve.
Table 18: The pre-training scalability. We use ONCE to pre-train and Waymo and KITTI to fine-tune.
Pre-training dataset	Waymo L2 AP/APH	KITTI Moderate mAP
Overall	Vehicle	Pedestrian	Cyclist	Overall	Car	Pedestrian	Cyclist
ONCE (~100k)	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00	69.43	82.75	57.59	67.96
ONCE (~500k)	69.04 / 66.52	68.69 / 68.23	69.81 / 63.74	68.61 / 67.60	71.36	83.17	58.14	72.78
ONCE (~1M)	69.63 / 67.08	69.03 / 68.57	70.54 / 64.34	69.33 / 68.33	72.37	83.47	59.84	73.81
D.3 Results of fine-tuning on ONCE.
(d)
(g)
Figure 10: Visualization of fine-tuning results. We visualize the results of three downstream datasets. (a-c): results of Waymo. (d-f): results of nuScenes. (g-i): results of KITTI.
The green and red bounding boxes represent ground-truths and detector predictions, respectively.
In our main submission, we report the fine-tuning performance on multiple datasets which are different from the pre-training dataset. Here, we show some fine-tuning performance on ONCE. As shown in Tab. 19, the performance can be largely improved when the baseline detectors are initialized by AD-PT. For example, when using SECOND as the baseline detector, the overall performance can be improved from  $56.47\%$  to  $64.10\%$ $(+7.63\%)$ . We use the ONCE train set to fine-tune the model.
Table 19: The fine-tuning performance on ONCE validation set.
Init.	SECOND	CenterPoint
Overall	0-30m	30-50m	>50m	Overall	0-30m	30-50m	>50m
Random Initialization	56.47	65.94	51.05	36.44	64.94	74.52	59.47	44.28
AD-PT Initialization	64.10	74.34	57.69	41.23	67.73	76.48	61.85	46.29
D.4 Visualization Results.
Fig. 10 shows the visualization results of three downstream datasets (i.e., Waymo, nuScenes, KITTI).