SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- Regarding the point cloud, we adopt a random rotation around the  $z$ -axis and flip the  $x$ -axis or  $y$ -axis with a  $50\%$  probability. Besides, we randomly drop the cuboid where the length of each side covers no more than  $10\%$  range of point coordinates on the corresponding axis, and the cuboid center is located on a randomly chosen point in the point cloud. We also ensure that the dropped cuboid retains at least 1024 pairs of points and pixels; otherwise, we select another new cuboid instead. For the temporal frame, we apply the same data augmentation to it.
- Regarding the multi-view image, we apply a horizontal flip with a  $50\%$  probability and a cropped resize which reshapes the image to  $416 \times 224$ . Before resizing, the random crop fills at least  $30\%$  of the available image space with a random aspect ratio between  $14:9$  and  $17:9$ . If this random cropping does not preserve at least 1024 or  $75\%$  of the pixel-point pairs, a different crop is chosen.
For the downstream fine-tuning tasks, we apply a random rotation around the  $z$ -axis and flip the  $x$ -axis or  $y$ -axis with a  $50\%$  probability for all the points in the point cloud. If the results are mentioned with LaserMix [55] augmentation, we augment the point clouds via the LaserMix before the above data augmentations. We report both results in the main paper so that we can fairly compare the point cloud segmentation performance with previous works.
A.3.4 Model Configuration
For the model pretraining, the 3D backbone  $F_{\theta_p}$  is a Minkowski U-Net [19] with  $3 \times 3 \times 3$  kernels; while the 2D image encoder  $G_{\theta_i}$  is a ResNet-50 [33] initialized with 2D self-supervised pretrained model of MoCoV2 [12]. These configurations are kept the same as SLidR [85]. The channel dimension of  $G_{\theta_i}$  head and  $F_{\theta_p}$  head is set to 64. For the linear probing task, we adopt a linear classification head, as mentioned in previous sections. For the downstream fine-tuning tasks, the same 3D backbone  $F_{\theta_p}$ , i.e., the Minkowski U-Net [19] with  $3 \times 3 \times 3$  kernels, is used.
Figure E: The convergence rate comparison between SLidR [85] and the proposed Seal framework.
B Additional Quantitative Result
B.1 Self-Supervised Learning
We report the complete results (i.e., the class-wise IoU scores) for the linear probing and downstream fine-tuning tasks shown in the main paper. We report the official results from previous works whenever possible. We also report our reproduced results for random initialization, PPKT [65], and SLidR [85]. Specifically, the complete results on the nuScenes [26], SemanticKITTI [3], Waymo Open [88], and Synth4D [82] datasets are shown in Table D, Table E, Table F, and Table G, respectively. We observe that Seal constantly outperforms prior methods for most semantic classes on all datasets.
We also compare the convergence rate of Seal with SLidR [85] in Fig. E. As can be seen, our framework is able to converge faster with the use of semantic superpixels, where these higher qualitative contrastive samples gradually form a much more coherent optimization landscape.
B.2 Robustness Probing
We report the complete results of the robustness evaluation experiments in the main paper, including the per-corruption CE scores and the per-corruption RR scores. Specifically, the complete results on the nuScenes-C [53] dataset are shown in Table H and Table I. We observe that Seal is superior to previous methods in terms of both CE and RR metrics.
C Additional Qualitative Result
C.1 Cosine Similarity
We provide more examples for the cosine similarity study in Fig. F, Fig. G, and Fig. H.
C.2 Downstream Tasks
We provide more qualitative results for downstream fine-tuning tasks in Fig. I, Fig. J, and Fig. K.
Table D: The per-class IoU scores of different pretraining methods pretrained on nuScenes [26] and linear probed or fine-tuned on different proportions (1%, 5%, 10%, 25%, and Full) of the nuScenes [26] data. Symbol  $\mathbb{I}$  denotes our reproduced results and the remaining are reported scores. All IoU scores are given in percentage (\%). The best mIoU score is highlighted in bold.
Method	mIoU	barrier	bicycle	bus	car	con. veh.	motor	ped	traf. cone	trailer	truck	driv. surf.	other flat	sidewalk	terrain	mammade	veg
Linear Probing
Random ¶	8.4	0.5	0.0	0.0	3.9	0.0	0.0	0.0	6.4	0.0	3.9	60.4	0.0	0.1	16.2	30.6	12.2
PointContrast	21.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
DepthContrast	22.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PPKT	35.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
SLidR ¶	39.2	44.2	0.0	30.8	60.2	15.1	22.4	47.2	27.7	16.3	34.3	80.6	21.8	35.2	48.1	71.0	71.9
ST-SLidR	40.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal (Ours)	45.0	54.7	5.9	3.6	61.7	18.9	28.8	48.1	31.0	22.1	39.5	83.8	35.4	46.7	56.9	74.7	74.7
Fine-tuning (1%)
Random	30.3	0.0	0.0	8.1	65.0	0.1	6.6	21.0	9.0	9.3	25.8	89.5	14.8	41.7	48.7	72.4	73.3
PointContrast	32.5	0.0	1.0	5.6	67.4	0.0	3.3	31.6	5.6	12.1	30.8	91.7	21.9	48.4	50.8	75.0	74.6
DepthContrast	31.7	0.0	0.6	6.5	64.7	0.2	5.1	29.0	9.5	12.1	29.9	90.3	17.8	44.4	49.5	73.5	74.0
PPKT	37.8	0.0	2.2	20.7	75.4	1.2	13.2	45.6	8.5	17.5	38.4	92.5	19.2	52.3	56.8	80.1	80.9
SLidR	38.8	0.0	1.8	15.4	73.1	1.9	19.9	47.2	17.1	14.5	34.5	92.0	27.1	53.6	61.0	79.8	82.3
ST-SLidR	40.8	0.0	2.7	16.0	74.5	3.2	25.4	50.9	20.0	17.7	40.2	92.0	30.7	54.2	61.1	80.5	82.9
Seal (Ours)	45.8	0.0	9.4	32.6	77.5	10.4	28.0	53.0	25.0	30.9	49.7	94.0	33.7	60.1	59.6	83.9	83.4
Fine-tuning (5%)
Random	47.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PointContrast	44.5	50.1	2.9	57.3	70.3	1.1	6.1	39.1	18.3	17.3	44.8	92.3	38.6	54.9	61.1	80.3	77.9
PPKT ¶	52.7	56.1	8.2	65.3	79.0	9.1	15.5	54.3	34.5	26.7	58.6	93.2	44.1	63.1	64.8	85.1	83.6
SLidR	52.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
ST-SLidR	54.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal (Ours)	55.6	61.0	7.4	70.4	82.4	11.9	30.4	59.2	34.0	33.6	61.1	94.7	46.2	63.4	63.9	85.7	84.9
Fine-tuning (10%)
Random	56.2	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PointContrast	53.2	56.8	5.2	66.3	74.5	5.7	36.1	49.5	38.2	29.2	54.4	94.0	47.7	61.4	67.6	82.8	81.1
PPKT ¶	60.3	64.0	12.0	67.8	77.6	16.0	56.8	63.3	49.8	28.3	56.3	94.1	62.7	66.4	68.7	85.9	85.4
SLidR	59.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
ST-SLidR	60.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal (Ours)	63.0	64.9	15.1	78.9	83.5	22.5	61.0	63.0	51.5	37.4	65.2	95.2	59.4	67.7	69.6	86.2	86.4
Fine-tuning (25%)
Random	65.5	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PointContrast	63.0	64.9	15.1	78.9	83.5	22.5	61.0	63.0	51.5	37.4	65.2	95.2	59.4	67.7	69.6	86.2	86.4
PPKT ¶	67.1	63.7	12.1	87.4	85.2	42.0	61.2	69.6	54.7	50.1	74.9	95.8	64.6	69.9	70.4	87.4	85.1
SLidR	66.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
ST-SLidR	67.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal (Ours)	68.4	67.4	15.5	90.5	85.1	40.0	62.3	68.1	58.3	54.5	76.0	95.8	65.6	69.8	71.0	87.9	86.7
Fine-tuning (Full)
Random	74.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PointContrast	74.9	77.4	34.4	90.4	86.5	51.1	78.4	77.0	64.9	67.8	77.7	96.6	72.2	74.3	73.2	89.6	86.7
PPKT ¶	74.5	75.4	37.4	91.7	86.1	45.3	77.3	75.6	65.1	65.0	79.9	96.6	72.1	74.6	73.4	89.2	86.6
SLidR	74.8	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
ST-SLidR	75.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal (Ours)	75.6	76.7	26.0	93.4	86.1	55.8	82.7	77.3	66.0	67.9	83.9	96.5	73.6	74.3	73.2	89.3	87.1
Table E: The per-class IoU scores of different pretraining methods pretrained on nuScenes [26] and fine-tuned on  $1\%$  of the SemanticKITTI [3] data. Symbol  $\mathbb{Q}$  denotes our reproduced results and the remaining are reported scores. All IoU scores are given in percentage (\%). The best mIoU score is highlighted in bold.
Method	mIoU	car	bicycle	motorcycle	truck	other-vehicle	person	bicyclist	motorcyclist	road	parking	sidewalk	other-ground	building	fence	vegetation	trunk	trunk	trunk
Random	39.5	91.2	0.0	9.4	8.0	10.7	21.2	0.0	0.0	89.4	21.4	73.0	1.1	85.3	41.1	84.9	50.1	71.4	55.4	37.6
PPKT	43.9	91.3	1.9	11.2	23.1	12.1	27.4	37.3	0.0	91.3	27.0	74.6	0.3	86.5	38.2	85.3	58.2	71.6	57.7	40.1
SLidR	44.6	92.2	3.0	17.0	22.4	14.3	36.0	22.1	0.0	91.3	30.0	74.7	0.2	87.7	41.2	85.0	58.5	70.4	58.3	42.4
ST-SLidR	44.7	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal	46.6	92.3	14.9	18.7	16.1	23.7	43.0	34.4	0.0	91.3	27.2	75.3	0.7	85.7	38.8	85.1	61.9	71.3	57.7	47.7
Table F: The per-class IoU scores of different pretraining methods pretrained on nuScenes [26] and fine-tuned on  $1\%$  of the Waymo Open [88] data. Symbol  $\mathbb{Q}$  denotes our reproduced results and the remaining are reported scores. All IoU scores are given in percentage  $(\%)$ . The best mIoU score is highlighted in bold.
Method	mIoU	car	truck	bus	other vehicle	motorcyclist	bicyclist	pedestrian	sign	traffic light	pole	construction	bicycle	motorcycle	building	vegetation	tree trunk	curb	road	lane marker	other ground	walkable	sidewalk
Random	39.4	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
PPKT¶	47.6	92.3	48.8	34.8	6.9	0.0	22.0	73.0	58.3	17.6	55.8	20.4	25.5	7.8	91.5	87.1	53.8	55.1	89.1	38.6	34.7	71.4	62.6
SLiDRe	47.1	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
ST-SLiDRe	44.9	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
Seal	49.3	92.5	52.6	33.5	3.7	0.0	31.9	73.1	61.0	23.5	57.0	31.4	20.1	12.4	91.4	87.1	53.2	57.2	89.5	38.8	34.9	72.1	68.7
Table G: The per-class IoU scores of different pretraining methods pretrained on nuScenes [26] and fine-tuned on  $1\%$  of the Synth4D [82] data. Symbol  $\mathbb{Q}$  denotes our reproduced results and the remaining are reported scores. All IoU scores are given in percentage (\%). The best mIoU score is highlighted in bold.
Method	mIoU	building	fences	other	pedestrain	pole	roadlines	road	sidewalk	vegetation	vehicle	wall	traffic sign	sky	ground	bridge	rail track	guardrail	traffic light	static	dynamic	water	terrain
Random	20.2	33.2	13.8	0.0	16.3	13.9	0.0	88.6	51.0	48.9	95.7	27.6	0.0	0.0	0.0	0.0	0.1	19.4	0.0	0.2	0.0	0.0	36.2
PPKT¶	61.1	84.4	67.1	62.9	77.4	75.0	2.2	92.0	76.3	92.5	99.2	73.4	67.3	0.0	62.2	39.6	83.5	66.8	0.0	63.6	55.3	37.4	66.0
SLidR¶	63.1	83.7	66.3	64.9	77.4	76.4	6.5	92.8	78.7	92.5	99.0	72.5	64.2	0.0	74.3	48.9	85.3	67.1	0.0	67.1	60.0	47.1	63.6
Seal	64.5	84.8	70.5	64.8	80.3	76.3	9.3	92.9	79.8	92.7	98.9	73.0	60.7	0.0	75.2	55.3	84.6	67.0	0.0	68.2	60.7	53.2	70.9
Table H: The Corruption Error (CE) scores of different pretraining methods pretrained on nuScenes [26] and probed under the eight out-of-distribution corruptions in the nuScenes-C dataset from the Robo3D benchmark [53]. All CE scores are given in percentage (\%). The best CE score for each corruption type is highlighted in bold.
Initial	Backbone	mCE	Fog	Wet Ground	Snow	Motion Blur	Beam Missing	Crosstalk	Incomplete Echo	Cross-Sensor
LP	PPKT	MinkUNet18	183.44	149.59	247.53	120.50	266.03	213.54	109.62	199.03	161.65
SLidR	MinkUNet18	179.38	140.47	237.29	112.93	276.44	210.65	107.86	189.27	160.16
Seal (Ours)	MinkUNet18	166.18	135.18	219.36	117.47	234.01	189.70	108.54	172.16	153.03
Full	Random	PolarNet	115.09	90.10	115.33	58.98	208.19	121.07	80.67	128.17	118.23
Random	FIDNet	122.42	75.93	122.58	68.78	192.03	164.84	57.95	141.66	155.56
Random	CENet	112.79	71.16	115.48	64.31	156.67	159.03	53.27	129.08	153.35
Random	WaffleIron	106.73	94.76	99.92	84.51	152.35	110.65	91.09	106.41	114.15
Random	Cylinder3D	105.56	83.22	111.08	69.74	165.28	113.95	74.42	110.67	116.15
Random	SPVCNN18	106.65	88.42	105.56	98.78	156.48	110.11	86.04	104.26	103.55
Random	SPVCNN34	97.45	95.21	99.50	97.32	95.34	98.73	97.92	96.88	98.74
Random	MinkUNet18	112.20	79.90	112.50	74.64	181.47	120.76	93.22	111.58	123.53
PPKT	MinkUNet18	105.64	77.63	104.22	68.60	160.95	114.81	86.71	108.96	123.20
SLidR	MinkUNet18	106.08	74.61	106.13	73.75	165.09	118.02	79.08	107.38	124.57
Seal (Ours)	MinkUNet18	92.63	58.97	98.47	56.63	127.25	108.20	57.97	110.95	122.63
Table I: The Resilience Rate (RR) scores of different pretraining methods pretrained on nuScenes [26] and probed under the eight out-of-distribution corruptions in the nuScenes-C dataset from the Robo3D benchmark [53]. All RR scores are given in percentage (\%). The best RR score for each corruption type is highlighted in bold.
Initial	Backbone	mRR	Fog	Wet Ground	Snow	Motion Blur	Beam Missing	Crosstalk	Incomplete Echo	Cross-Sensor
LP	PPKT	MinkUNet18	78.15	85.38	98.66	78.33	81.36	91.42	54.37	78.02	57.69
SLidR	MinkUNet18	77.18	89.90	98.17	84.12	68.14	86.93	53.63	81.29	55.26
Seal (Ours)	MinkUNet18	75.38	83.05	95.15	66.59	83.94	89.70	45.18	83.94	55.48
Full	Random	PolarNet	76.34	81.59	97.95	90.82	62.49	86.75	57.12	75.16	58.86
Random	FIDNet	73.33	90.78	95.29	82.61	68.51	67.44	80.48	68.31	33.20
Random	CENet	76.04	91.44	95.35	84.12	79.57	68.19	83.09	72.75	33.82
Random	WaffleIron	72.78	73.71	97.19	65.19	78.16	85.70	43.54	80.86	57.85
Random	Cylinder3D	78.08	83.52	96.57	79.41	76.18	87.23	61.68	81.55	58.51
Random	SPVCNN18	74.70	79.31	97.39	55.22	78.44	87.85	49.50	83.72	66.14
Random	SPVCNN34	75.10	72.95	96.70	54.79	97.47	90.04	36.71	84.84	67.35
Random	MinkUNet18	72.57	84.33	94.63	74.31	69.26	83.06	42.27	79.88	52.79
PPKT	MinkUNet18	76.06	85.90	97.71	79.28	76.72	85.72	48.77	81.31	53.10
SLidR	MinkUNet18	75.99	87.46	96.68	74.89	74.97	84.06	56.08	81.78	52.01
Seal (Ours)	MinkUNet18	83.08	96.11	98.29	87.59	87.49	87.25	75.98	79.19	52.71
Figure F: The cosine similarity between the query point (denoted as the red dot) and the feature learned with SLIC [1] and different VFMs [50, 122, 111, 123]. The color goes from violet to yellow denoting low and high similarity scores, respectively. Best viewed in color.
Figure G: The cosine similarity between the query point (denoted as the red dot) and the feature learned with SLIC [1] and different VFMs [50, 122, 111, 123]. The color goes from violet to yellow denoting low and high similarity scores, respectively. Best viewed in color.
Figure H: The cosine similarity between the query point (denoted as the red dot) and the feature learned with SLIC [1] and different VFMs [50, 122, 111, 123]. The color goes from violet to yellow denoting low and high similarity scores, respectively. Best viewed in color.
Figure I: The qualitative results of different point cloud pretraining approaches pretrained on the raw data of nuScenes [26] and fine-tuned with  $1\%$  labeled data. To highlight the differences, the correct / incorrect predictions are painted in gray / red, respectively. Best viewed in color.
Figure J: The qualitative results of different point cloud pretraining approaches pretrained on the raw data of nuScenes [26] and fine-tuned with  $1\%$  labeled data. To highlight the differences, the correct / incorrect predictions are painted in gray / red, respectively. Best viewed in color.
Figure K: The qualitative results of different point cloud pretraining approaches pretrained on the raw data of nuScenes [26] and fine-tuned with  $1\%$  labeled data. To highlight the differences, the correct / incorrect predictions are painted in gray / red, respectively. Best viewed in color.
D Public Resources Used
We acknowledge the use of the following public resources, during the course of this work:
- nuScenes5 CC BY-NC-SA 4.0
- nuScenes-devkit<sup>6</sup> Apache License 2.0
- SemanticKITTI7 CC BY-NC-SA 4.0
- SemanticKITTI-API<sup>8</sup> MIT License
- Waymo Open Dataset  ${}^{9}$  . - Waymo Dataset License
- ScribbleKITTI<sup>10</sup> Unknown
- RELLIS-3D11 CC BY-NC-SA 3.0
- SemanticPOSS<sup>12</sup> Unknown
- SemanticSTF13 CC BY-NC-SA 4.0
- SynLiDAR<sup>14</sup> MIT License
-  $\mathrm{Synth4D^{15}}$  GNU General Public License 3.0
DAPS-3D16 MIT License
- nuScenes-C17 CC BY-NC-SA 4.0
- MinkowskiEngine18 MIT License
-  $\mathbf{SLidR}^{19}$  Apache License 2.0
- spvnas20 MIT License
- Cylinder3D21 Apache License 2.0
- LaserMix22 CC BY-NC-SA 4.0
- mean-teacher $^{23}$  Attribution-NonCommercial 4.0 International
- PyTorch-Lightning24 Apache License 2.0
- mmdetection3d25 Apache License 2.0
$^{5}$ https://www.nuscenes.org/nuscenes.
$^{6}$ https://github.com/nutonomy/nuscenes-devkit.
<sup>7</sup>http://semantic-kitti.org.
<sup>8</sup>https://github.com/PRBonn/semantic-kitti-api.
9https://waymo.com/open.
10 https://github.com/ouenal/scribblekitti.
11http://www.unmannedlab.org/research/RELLIS-3D.
12http://wwwposs.pku.edu.cn/semanticposs.html.
$^{13}$ https://github.com/xiaoaoran/SemanticSTF.
$^{14}$ https://github.com/xiaoaoran/SynLiDAR.
<sup>15</sup>https://github.com/saltoricristiano/gipso-sfouda.
16https://github.com/subake/DAPS3D.
17https://github.com/ldkong1205/Robo3D.
18https://github.com/NVIDIA/MinkowskiEngine.
19https://github.com/valeoai/SLidR.
20https://github.com/mit-han-lab/spvnas.
21https://github.com/xinge008/Cylinder3D.
$^{22}$ https://github.com/ldkong1205/LaserMix.
$^{23}$ https://github.com/CuriousAI/mean-teacher.
$^{24}$ https://github.com/Lightning-AI/lightning.
25https://github.com/open-mmlab/mmdetection3d.