Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentation
Authors: Binhui Xie
Shuang Li
qingju guo
Chi Harold Liu
Xinjing Cheng
Venue: NeurIPS 2023 poster (2023)
Keywords: Active Learning
LiDAR Semantic Segmentation
Domain Adaptation
TL;DR: This paper presents a simple, general, and efficient active learning baseline for label-efficient LiDAR semantic segmentation from a voxel-centric perspective.

Abstract: Active learning, a label-efficient paradigm, empowers models to interactively query an oracle for labeling new data. In the realm of LiDAR semantic segmentation, the challenges stem from the sheer volume of point clouds, rendering annotation labor-intensive and cost-prohibitive. This paper presents Annotator, a general and efficient active learning baseline, in which a voxel-centric online selection strategy is tailored to efficiently probe and annotate the salient and exemplar voxel girds within each LiDAR scan, even under distribution shift. Concretely, we first execute an in-depth analysis of several common selection strategies such as Random, Entropy, Margin, and then develop voxel confusion degree (VCD) to exploit the local topology relations and structures of point clouds. Annotator excels in diverse settings, with a particular focus on active learning (AL), active source-free domain adaptation (ASFDA), and active domain adaptation (ADA). It consistently delivers exceptional performance across LiDAR semantic segmentation benchmarks, spanning both simulation-to-real and real-to-real scenarios. Surprisingly, Annotator exhibits remarkable efficiency, requiring significantly fewer annotations, e.g., just labeling five voxels per scan in the SynLiDAR → SemanticKITTI task. This results in impressive performance, achieving 87.8% fully-supervised performance under AL, 88.5% under ASFDA, and 94.4% under ADA. We envision that Annotator will offer a simple, general, and efficient solution for label-efficient 3D applications.

Annotator: A Generic Active Learning Baseline for LiDAR Semantic Segmentation
Binhui Xie Beijing Institute of Technology binhuixie@bit.cc edu. cn
Shuang Li Beijing Institute of Technology shuangli edu. on
Qingju Guo
Beijing Institute of Technology qingjuguo@bit. edu. cn
Chi Harold Liu
Xinjing Cheng
Beijing Institute of Technology chiliu@bit edu. cn
Tsinghua University & Inceptio Technology cnorbot@gmail.com com
Abstract
Active learning, label-efficient paradigm, empowers models to interactively query an oracle for labeling new data. the realm LiDAR semantic segmentation, the challenges stem from the sheer volume of point clouds, rendering annotation labor-intensive and cost-prohibitvee This paper presents Annotator, general and efficient active learning baseline, in which a voxel-centric online selection strategy is tailored to efficiently probe and annotate the salient and exemplar voxel girds within each LiDAR scan, even under distribution shift. Concretely, we first exe- cute an in- in-eepth analysis of several common selection strategies such as Random, Entropy, Margin, and then develop voxel confusion degree (VCD) to exploit the local topology relations and structures of point clouds. Annotator excels in diverse settings, with particular focus on active learning (AI.), active source-free domain adaptation (ASFDA), and active domain adaptation (ADA). It consistently delivers exceptional performance across LiDAR semantic segmentation benchmarks, span- ning both simulation-to-res and real-to-real scenarios. Surprisingly, Annotator exhibits remarkable efficiency, requiring significantly fewer annotations, e.g., just labeling five voxels per scan in the SynLiDAR + SemanticKITTI task. This results in impressive performance, achieving 87.8% fully-supervised performance under AL, 88.5% under ASFDA, and 94.4% under ADA. We envision that Annotator will offer simple, general, and efficient solution for label-efficient 3D applications]
Introduction
3D perception and understanding have become indispensable for machines effectively interact with the real world, LiDAR (Light Detection And Ranging) [50 52] is a widely-used methodology for capturing precise geometric information about the environment, spurring significant advancements in areas like autonomous vehicles and robotics [] 25 However, semantic segmentation of LiDAR presents an enormous challenge. The high-speed collection of millions of point clouds per second by on- an-board sensors sharply contrasts with the laborious and cost- prohibitohvee nature of annotating them. Consider, for instance, the vast number of outdoor scenes an autopilot can encounter, which practically limitless. Yet, acquiring annotations for these large-scale point clouds entails intensive human labor. This underscores the urgency of establishing label-efficient learning mechanism capable of boosting performance in the low-data regime [19 76 of models to new domains
90
97 or facilitating the adaptation
15
85
Corresponding author. Project
io/ /n/.oatorrotreeb
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Extensive solutions encompass semi-supervisee 8 11
31 83,, weakly-supervised 22 3d 76 99
or self-supervised [5] 44 63 7
TOT
learning. Semi- and weakly -sueev-- learning methods
aim to alleviate the annotation bur-
den by harnessing partially labeled or weakly labeled data. In contrast, self- supervised ones learn representations from point clouds via pretext tasks and then transfer to downstream tasks for weight initialization. Although these works offer scalability and practicality for weal-world utility, they also con- front new challenges, such as varia- tions in LiDAR configurations, sensor biases, and environmental conditions. That is, the majority of prior works has endeavored to in-distribution sce- narios, with limited consideration for label-efficient paradigms in out-of- distribution scenarios, especially for sparse outdoor point clouds. Recent efforts turn large-scale auxiliary
Fully--upervised
80.
LaserMix"
Annotator (ADA)
Scribbia
CPS
Mean Teacher
Annotator (ASFDA)
Annotator (AL.)
10x fewer
moou m
SQN'
labels
CoSMix
32.
PolarMix
1000x fower labols
CRST
23.5
SynLiDAR-only
voxels (-0.1%)
10%
100%
The proportion SemanticKITTI (MinkNet/ "Cylinderr))
Figure 1: Performance vs. annotated proportion on Se-
manticKITTI val [3] of existing label-efficient LiDAR seg-
mentation paradigms including domain adaptation (+) [55
X
(A) 22 76 and semi-supervised 72 learning. As a reference, fully supervised 29
10:5 weakly-
datasets [56. [4]] and delve into do- main adaptation (DA) algorithms []] 28 [1 92 to significantly reduce the annotation workload under a domain shift. Nevertheless, the performance of these methods still lags behind the fully-supervised ap- proaches. In Figure [ we provide an intuitive comparison of results across various paradigms. becomes evident that there is ample room for improvement in the performance of these methods.
) is reported as well. Annotator attains excellent balance between performance and annotation cost.
counterpart (*)
To surmount these obstacles and promote performance the domain of interest, active learning (AL)
is being an optimal paradigm 27 35 60 82). Given the limited annotation budget, a common scenario that only an unlabeled target domain of large amounts of point clouds is available with the goal to interactively select a minimal subset of data to be annotated to maximally improve the segmentation performance. In reality, this setting faces a significant hurdle known as cold start problem: the lack of prior information to guide the initial selection of annotated data. A recent work has explored the impact of seeding strategies on the performance of AL methods [58]. Differently, we put forward a new path to access auxiliary model via pre-training on the open-access auxiliary (source) dataset. This auxiliary model serves as a warm-up stage, allowing for smart target data selection for initial annotation. We formulate this new setting active source-free domain adaptation, termed ASFDA. Take a further step, drawing inspiration from recent trends in 2D images 38
88 we delve into the third setting, active domain adaptation (ADA) for semantic segmentation of 3D point clouds. In this setting, a labeled auxiliary dataset is available, and the objective is to select target instances for annotating and learn model with higher segmentation performance the target test set.
Overall, in this work, we benchmark three distinct active learning settings for LiDAR semantic segmentation and deliver a simple and general baseline, Annotator, as illustrated in Figure 2 Bor- rowing the idea of modeling and computational techniques in geometry processing, we introduce voxel-centric selection strategy dedicated to point clouds. Specifically, an input LiDAR scene is first voxelized into voxel grids, with a large voxel size to expand the local areas during the selection process. After obtaining final network predictions, importance estimation is carried out for each voxel grid using several common strategies such as Random, the softmax entropy (Entropy), and the margin between highest softmax scores (Margin). But considering only uncertainty for selection would be suboptimal 22 4 86 Therefore, we introduce the concept of voxel confusion degree ( (VCD), which takes into account nearby predictions, capturing diversity and redundancy within voxel grid. VCD enables the exploitation of local topology relations and point cloud structures. As a result, VCD can represent both uncertainty and diversity of voxel grid in the LiDAR scene. In each active round, we query the top one voxel grid within each scan for annotation until the budget is exhausted. Despite the simplicity of our Annotator, achieves performance on par with the fully-supervised counterpart requiring 1000x fewer annotations and significantly outperforms all prevailing acquisition strategies.
2

---

A generalist
Unlabeled pool
Voxelization
Active function te.g. Margin, VCD)
Candidate voxel pool
Online selection (top one voxel)
Picked
voxel pool
Labeled pool
Raw point cloud
Current network
Iterate rounds
Annotate
(e.g. human)
Query
(i) Annotator for AL
(ii) Annotator for ASFDA
(iil) Annotator
1
Pre-train with auxiliary (source) data
generalist Annotator
Unlabeled pool
L
Le
Labeled source (available)
Unlabeled target
selection
Labeled source (unavailable)
Warm-up network
CE loss
Labeled pool
Labeled pool
Anmontatio
Labeled pool
Die
Unlabeled target
Lige
Dee
A active target supervised (CE) loss
Unlabeled target
Access to an auxiliary model
Access to auxiliary data
Figure 2: An illustration of Annotator. Annotator is a new active learning baseline with broad applicability, capable interactively querying tiny subset of the most informative new (target) data points based on available inputs without task-specific designs. This includes (i) only unlabeled new (target) data being available (active learning, AL); (ii) access an auxiliary (source) pre-trained model (active source-free domain adaptation, ASFDA); and (iii) availability of labeled source data and unlabeled target data (active domain adaptation, ADA). Remarkably, Annotator attains excellent results not only in in-domain settings but also manifests adaptive transfer to out-of-domain settings.
The contribution of this paper can be summarized three aspects. First, we present a voxel-centric active learning baseline that significantly reduces the labeling cost and effectively facilitates learning with limited budget, achieving near performance to that of fully-supervised methods with 1000 fewer annotations. Second, we introduce label acquisition strategy, the voxel confusion degree (VCD), which is more robust and diverse select point clouds under domain shift. Third, Annotator is generally applicable for various network architectures (voxel-, range- and bev-views), settings (in-distribution and out-of-distribution). and scenarios (simulation-to- and real-to-real) with consistent gains. We hope this work could lay a solid foundation for label-efficient 3D applications.
A Generic Baseline
2.1 Preliminaries and overview
Problem setup. In the context LiDAR semantic segmentation, LiDAR scan is made of a set of point clouds and let X E I Nx4 ,Y K respectively denote N points and the corresponding labels. K is a predefined semantic class vocabulary K (1..... ,K} of K categorical labels. Each point in X is and an intensity value of returning laser beam. Our baseline works in the following settings: active learning (AL), active source-free domain adaptation (ASFDA), and active domain adaptation (ADA). First, we are given an unlabeled target domain D {Xx UX*), where X denotes unlabeled target point clouds and X denotes the selected points to be annotated and is initialized empty set, i.c., X 0. Next, for ASFDA and ADA, labeled source domain D {X*, Y*} can be utilized only in pre-training stage and anytime respectively. Ultimately, given limited budget, our goal is to iteratively select subset of data points from D' to annotate until the budget is exhausted, all the while catching up with the performance of the fully-supervised model.
4 vector with 3D Cartesian coordinate relative to the scanner (a; b.,c))
Overview. Figure displays an overview of Annotator, which is a label-efficient baseline LiDAR semantic segmentation. It is composed of two parts: 1) generalist Annotator which contains voxelization process to get voxel grids and an active function with online selection for picking the most valuable voxel grid of each input scan in each active round; 2) the pipelines of distinct active

---

learning settings are described. For AL, we interactively select a subset of voxels from the current scan be annotated and train the network with these sparse annotated voxel grids. In the case of ASFDA, we begin by pre-training network on the source domain through standard supervised learning. This warm-up network then serves as a strong initialization to aid the initial selection. As for ADA, except for the pre-training stage, we also make use of annotated source domain to promote the selection in each round and facilitate domain alignment. In the following, we will detail why we select salient and exemplar data points from voxel-centric perspective and how to address cold start problem via an auxiliary model. After that, overall objectives for all three settings are elaborated.
2.2 A generalist Annotator
In this section, we proposed a general active learning baseline called Annotator. The core idea is to select salient and exemplar voxel grids from each LiDAR scan. It's important to note that previous researches have proposed frame-based [33 941
region-based [82). and point-based [55] selection strategies. The first two usually require an offline stage, which may be infeasible at large scales. The last one is costly due to the sparsity of outdoor point clouds. By contrast, our voxel-centric selection focuses on querying salient and exemplar areas and annotating all points within those areas. This approach is more efficient and flexible. Moreover, it can be seamlessly applied to various network architectures, including voxel-, range- and bev-views, as demonstrated in the experiment section.
To implement it, we begin with the voxelization process as introduced in 1002 Each input LiDAR scan X is transformed into 3D voxel grid set V. This process involves sampling the continuous 3D input space into discrete voxel grids, where points falling into the same grid are merged. Each voxel grid serves as a selection unit. Mathematically, for a point x, E X, the corresponding voxel grid coordinate is (a, .b. , = L(a: bj c4) JJJ, with A denoting predefined voxel size. In our experiments, we have found that using large voxel grid is more robust against noise and sparsity. Unless otherwise specified, we use A 0.05 for training and 42 = 0.25 for the selection process.
Selection strategies. For each voxel grid 1 V. we assess its importance and select the best voxel grid per LiDAR sean in each active round. Initially, we employ a Random selection strategy. Subsequently, we explore softmax entropy (Entropy) and the margin between highest softmax scores (Margin). It's essential to note that while these common selection strategies are not technical contributions, they are necessary build our baseline. Detailed calculations provided below.
Random: randomly select target voxel grid " from V to be annotated in each round.
Entropy: first calculate the softmax entropy of each point r; E 12 and then adopt the maximum value as the Entropy score of this grid, i.e., Entropy(e,) = maxg,e0, -p; log Pi. where p; is the softmax score of point x4. The voxel grid with the highest Entropy score is selected in each sean.
Margin: first calculate the margin between highest softmax score of each point 2, E Uy and then adopt the maximum value as the Margin score of this grid, i.e., Margin( (1y) max (max(p:) max2(p:)), ():)), where max2(.) is the second-largest value operator. In each scan, the voxel grid with the lowest Margin score is chosen.
The VCD strategy, Our voxel confusion degree (VCD) motivated by an important observation: the previously mentioned selection strategies become less effective when models are applied in new domains due to mis- calsbbabbdt uncertainty estimation. Therefore, the VCD is designed to estimate category diversity within voxel grid rather than uncertainty, making it more robust under domain shift. Here's how it works:
begin by obtaining pseudo label y, for each point zj. Next, we of -,,55>>/00 Jar E f/y . = k). This allows to collect statistical information about the categories present in the voxel grid. With this information, we calculate VCD assess the significance of voxel grids as follows:
divide points within by into K clusters:
K 1e [ng
10,401 |n5l
VCD(,,) =
log
where | denotes the number of points in a set. Finally, voxel grid with the highest VCD score is selected in each scan. The insight is that a higher score indicates a greater category diversity within voxel, which would be beneficial for model training once being annotated. In all experiments, Annotator is equipped with VCD by default, and the results indicate the superiority of VCD strategy.
Making a good first impression. To avoid the cold start problem mentioned before, we introduce a warm start mechanism that pre-trains an auxiliary model with an auxiliary (source) dataset, and then it is used to select voxel grids in the first round. This warm start stage is applied in ASFDA and ADA.

---

Discussion: balancing annotation cost and computation cost. Our primary focus is on reducing annotation cost while maintaining performance comparable to fully-supervised approaches. Let's consider simulation-to-reat tasks as example. The simplest setup involves active learning within the real dataset. However, this setup yields less satisfactory results due to the cold-start problem: the lack of prior information for selecting an initial annotated set. To address this, we utilize synthetic dataset to train an auxiliary model in a brief warm-up stage, enabling smarter data selection in the first round. Importantly, this warm-up process is short, conducted only once, and results in minimal costs (both annotation and computation). For a detailed analysis, please refer to Appendix B.
2.3 Optimization
The overall loss function is the standard cross-entropy loss, which is defined as:
Lee (X)
2/ logg,,
IEEX
where K' is the number of categories, Vs is the one-hot label of point 2; and p. is the predicted probability of point z; belonging to category k. Hereafter, for AL, the objective is ming Lce ))";; for ASFDA, the objective is ming, (x"" for ADA, the objective is ming, (x, (X²). Here, 0 and 0g denote training from scratch and training from the source pre-trained model, respectively.
3 Experiments
In this section, we conduct extensive experiments on several public benchmarks under three active learning scenarios: (i) AL setting where all available data points are from unlabeled target domain; (ii) ASFDA setting where we can only access pre-trained model from the source domain; (iii) ADA setting where all data points from source domain can be utilized and a portion of unlabeled target data is selected to be annotated. We first introduce the dataset used this work and experimental setup and then present experimental results of baseline methods and extensive analyses of Annotator.
3.1 Experiment setup
Datasets. We build all benchmarks upon SynLiDAR SemanticKITTI SemanticPOSS 12], and nuScenes [4] constructing two simulation-to-reat and two real-to-real adaptation scenarios.
SynLiDAR is a large-scale synthetic dataset, which has 198,396 LiDAR scans with point-level segmentation annotations over 32 semantic classes. Following [84] the training data. SemanticKITTI (KITTI) 3 is popular LiDAR segmentation dataset, including 2,9130 training scans and 6,019 validation scans with 19 categories. SemanticPOSS (POSS) [2 consists 2,988 world scans with point-level annotations over 14 semantic classes. As suggested in 4p use the sequence 03 for validation and the remaining sequences for training. nuScenes contains 190 training scans and 4,071 validation scans with 16 object classes.
19,840 point clouds as
Class mapping. To ensure compatibility between source and target labels across datasets, perform class mapping. Specifically, we map SynLiDAR labels into 19 common categories for SynLiDAR + KITTI and 13 classes for SynLiDAR - POSS. Similarly, map labels into 7 classes for KITTI nuScenes and nuScenes > KITTI. We refer readers to Appendix A. for detailed class mappings.
Implementation details. We primarily adopt MinkNet 9 and SPVCNN 771 as the segmentation backbones. Note that, all experiments share the same backbones and are within the same codebase, which are implemented using PyTorch 4 on single NVIDIA Tesla A 100 GPU. We use the SGD optimizer and adopt a cosine learning rate decay schedule with initial learning rate of 0.01. And the batch size for both source and target data is 16. For additional details, please consult Appendix A.2 Finally, we evaluate the segmentation performance before and after adaptation, following the typical evaluation protocol 47 in LiDAR domain adaptive semantic segmentation
3.2 Experimental results
Quantitative results summary. We initially evaluate Annotator on four benchmarks and two backbones while adhering to fixed budget of selecting and annotating five voxel grids in each scan.
5

---

Simulation-to-Real
Real-to-Real
Method
SynLLiDAR 19, KITTI SynLiDAR 19, POSS
KITTI 4 nuScenes nuScenes 4 KITTI
Source-TTaget-Only
21.0
30.4/56. 56.7
28.4/ 82.5
34.6 / 83.3
Random
35.3/ / 36.3 45.3
27.4 / 30.9 43.4
06.0/6..5711
70.9/ / 69.7 / 74.7
Entropy 78
39.8 / 49.6 50.1
42.8 / 45.5 49.9
59.7/ 100.3/73. 73.1
70.7 / 69.1 / 74.0
Margin
26
46.9 / 44.3 49.0
41.6 / 44.1 46.9
60.2/ 222/59.//71
73.1 / 70.3 76.7
Annotator
53.7 / 54.1 / 57.7
44.9 / 48.2 / 52.0
70.4/72.4/75 75.9
76.8 75.3 / 81.8
Table 1: Quantitative summary of all baselines' performance based on MinkNet 9 over various LiDAR semantic segmentation benchmarks using only voxel grids. Source-/larget-Onf correspond to the model trained on the annotated source/target dataset which are considered as lower/upper bound. Note that results are reported following the order of AL/ ASFDA ADA in each cell.
Simulation-to PililattiontoRRell -e-Real
Real-to-Real
Method
SynLiDAR 19, KITTI SynLiDAR 18 POSS
KITTI 4 nuSeenes nuScenes 4 KITTI
Source-Target-Only
24.2/ 63.7
37.0/51.9
21.3/81.3 81.3
47.1 / 85.0
Random
40.9/41.7 / 51.0
35.5 37.8 42.3
65.0 66.9 64.3
70.4 / 68.1 1 / 75.8
Entropy 78
52.7 / 52.1 / 52.8
35.2 / 40.5 / 46.8
61.3 / 66.0 66.3
69.5 / 67 72.6
Margin 26
47.1 / 49.9 50.7
42.9 44.8 47.1
57.8 60.3 63.2
72.3 73.0 75.3
Annotator
52.8 54.6 55.6
44.9 47.5 50.9
71.4 / 72.1 72.3
79.5 / 80.5 / 78.4
Table 2: Quantitative summary of all baselines' performance based on SPVCNN
over various
LiDAR semantic segmentation benchmarks using only voxel grids.
pors.
road
sidow.
trank
Model
mioUI
Source-Only
59.4
6.2
27.3
0.6
5.8
37.9
9.3
8.8
31.0
0.1
24.5
22.6
62.7
27.7
43.4
22.8
22.0
ADDA [3]
52.5
4.5
11.9
0.3
3.9
27.9
52.8
27.4
0.0
61.0
17.0
57.4
34.5
42.9
23.2
23.0
AdvEat
58.3
5.1
14.3
0.3
1.8
14.3
44.5
0.5
50.4
4.3
34.8
0.0
43.3
19.7
67.5
34.5
52.0
35.0
6.1
25.8
2
CRST 1014
62.0
5.0
12.4
1.3
9.2
16.7
44.2
0.4
$1.0
2.5
28.4
0.0
57.1
18.7
69.8
35.0
48.7
32.5
6.9
26.5
ST-PCT
708
7.3
15.1
1.9
8.4
12.6
44.0
0.6
56.4
4.5
31.8
0.0
66.7
23.7
73.3
34.6
48.4
39.4
11.7
28.9
CoSMix
13
75.1
6.8
29.4
27.1
11.1
22.1
25.0
24.7
79.3
14.9
46.7
0.1
53.4
13.0
67.7
31.4
32.1
37.9
13.4
32.2
PalarMax
54
76.3
8.4
17.8
3.9
6.01
26.6
40.8
15.9
70.3
0.0
44.4
0.0
68.4
14.7
6916
38.1
37.1
40.6
10.6
31.0
Randons
90.6
1100
01.0
4.5
11.1
0.0
0.0)
0.0
84.5
19.1
68.3
0.01
$4.4
45.5
85.8
53.9
73.3
47.8
2.11
35.3
Eatropy FX
94.2
1.0
19.8
23.4
24.7
6.4
0.0
0.2
79.0
19.5
62.4
2.4
85.1
50.4
M6.9
56.5
74.2
52.9
13.6
34.8
Margin 2
92.0
0.0
35.0
45.3
34.0
40.7
61.0
0.0
80.5
19.8
67.0
0.1
90.6
47.3
83.4
55.5
67.33
51.6
30.0
46.9
Annotator
40.3
56.3
46.8
76.9
64.0
23.4
69.2
2.0
$1.9
85.8
62.6
61.6
53.7
ASYDA
Random
90.3
0.0
0.0
16.5
0.0
64.4
20.3
68.9
0.1
$4.7
45.9
85.8
55.0
72.8
53.7
36.3
Eanapyy
59.9
61.4
Margin
55.6
79.2
Asnotator
47.8
Random
30.0
25.0
0.1
Eauopy
80.3
31.4
81.2
62.9
58.1
0.1
M0.4
52.7
53.2
57.5
29.6
Margin
39.3
30.2
51.0
76.1
69.7
51.2
55.5
57.4
Aunotatov
95.2
22.0
39.7
49.4
82.1
84.1
71.4
1.7
65.4
58.8
60.1
73.2
60.3
Terget-Only
95.7
204
63.9
70.3
45.5
78.5
93.5
49.6
81.0
0.2
91.1
63.8
87.3
68.5
72.3
64.4
49.1
19 Table 3: Per-class results on task of SynLiDAR Domain adaptation (DA) results reported from
-> KITTI (MinkNet 9 using only 5 voxel budgets.
55 3
The results in Table and Table 2 paint a clear picture overall: all baseline methods achieve significant improvements over the Source-Only model, especially for Annotator with VCD strategy, underscoring the success of the proposed voxel-centric online selection strategy. In particular, Annotator achieves the best results across all simulation-to- to-l and real-to-real tasks. For SynLiDAR - KITTI task, Annotator achieves 87.8% / 88.5% / 94.4% supervised performance under AL/ ASFDA ADA settings respectively. For SynLiDAR + POSS task, they are 79.0% / 85.0% / 99.7% respectively. On the task of KITTI- + nuScenes, they are 85.3 3% 87.8% 92.0% respectively. And on the task of nuScenes - KITTI, they are 92.2% / 90.3% / 98.2%. respectively. It is also clear that domain shift between simulation and real-world is more significant than those between real-worke datasets. Therefore, simulation-tor ro-l tasks show poorer performance. Further, we compare Annotator with additional AL algorithms and extend to indoor semantic segmentation in Appendix B.2 and
Per-class performance. To sufficiently realize the capacity of our Annotator, we also provide the class-wise loU scores on two simulation-to-real tasks (Table and Table H for different algorithms and comparison results with state-of-the-art DA methods 555 X Other results of the remainder

---

Model
car
hike
pers.
rider
grou.
buil.
fence 30.3
plants 63.8
trunk 22.0
pole
traf.
garb.
cone.
mlot
Source-Only
44.7
33.5
38.3
77.0
54.2
11.2
4.7
30.4
CRST 003
22.0
6.8
23.5
31.8
60.3
58.2
9.1
63.2
18.9
41.6
1.9
13.5
1.0
27.1
ST-PCT R4
27.8
6.6
28.9
34.8
63.9
64.1
12.1
63.7
18.6
41.0
4.9
16.6
1.6
29.6
CoSMix
10.6
55.8
51,4
78.7
66.2
71.3
23.5
22.5
28.9
20.4
PolarMix
33
25.0
10.7
32.6
39.1
79.0
44.8
23.8
64.2
11.9
29.6
5.8
15.3
13.3
30.4
Random
24.0
47.8
28.9
0.1
79.3
66.7
27.7
76.4
0.1
5.5
0.2
0.0
0.0
27.4
Entropy
45.2
56.3
39.3
Margin 26
30.1
44.2
55.3
46.8
79.2
63.7
44.0
74.3
34.1
23.4
34.5
9.7
1.2
41.6
Annotator
52.0
66.4
73.4
31.1
34.5
ASEDA
Random
46.4
37.9
0.4
69.8
77.7
17.4
5.5
30.9
Entropy
73
32.1
46.5
65,7
58.0
74.4
62.9
45.5
69.6
41.5
34.5
33.7
13.2
14.3
45.5
Magin 2
47.6
59.5
44.5
79.7
66.8
51.7
73.5
28.6
30.1
35.2
0.1
44.1
Annolator
56.2
54.2
63.6
58.7
80.9
64.6
58.1
73.4
37.8
26.3
34.0
6.3
11.9
48.2
Random
65,0
10.9
59.3
54.3
58.6
70.0
54.2
63.9
39.6
39.8
20.8
27.8
0.0
43.4
ADA
Entropy
78
53.3
29.1
62.9
52.7
80.3
71.9
48.2
72.3
38.9
30.0
27.6
44.2
37.8
49.9
Margin
61.3
25.2
60.6
56.2
54.2
46.6
66.7
38.1
30.8
40.8
20.4
46.9
Amnotator
67.4
18.0
64.0
52.0
78.5
61.5
1.5
68.6
48.5
32.7
37.9
50.8
43.8
52.0
Target-Only
73.7
60.4
68.6
62.2
81.7
79.2
60.8
78.9
36.5
31.2
44.1
12.9
46.6
56.7
Table 4: Per-class results task of SynLiDAR
13
POSS (MinkNet 9
using only 5 voxel budgets.
wrged
Burshon Entmpy
Yargin
Sagg::
Fundoon Entropy
Mugn Antolator
Tang::
Randon Entrppy
Magg Antorator
Target
Randoon Eneray
Margh
Annstater
Mimber bosted vaees
Nomter isteed
Nimter offeced wooks
nimber ofiseeee wides
(a) SynLiDAR 19 KITTI (b) SynLiDAR 13 POSS
(c) KITTI In nuScenes
(d) nuScenes In KITTI
Figure 3: Active learning results on various benchmarks varying active budget.
Tarpet
Bandam
Vangi
Targe:
Tandam
Harg
Target
Random
Marghn
Brgat
Bundom
Minner
Numster lacett woots
(a) SynLiDAR 19
KITTI (b) SynLiDAR 13 POSS
(c) KITTI I nuScenes
(d) nuScenes 4 KITTI
Figure 4: Active source-free domain adaptation results on various benchmarks varying active budget.
Sourre-s- South---
Annorater
Number tazeted woots
lcbater
(a) SynLiDAR - KITTI (b) SynLiDAR 12, POSS
(c) KITTI 4 nuScenes
(d) nuScenes I KITTI
Figure Active domain adaptation results on various benchmarks varying active budget.
tasks and backbones are listed Appendix B.4 It is noteworthy that Annotator under any active learning settings significantly outperform DA methods with respect to some specific categories such as "traf.", "pole", "garb." and "cone" etc. These results also showcase the class-balanced selection of the proposed Annotator, which is testified in Figure d as well.
Results with varying budgets. We investigate the impact of varying budgets and compare the performance with baseline methods, as illustrated in Figure Figure and Figure 5 A consistent

---

Model
car
hike
pers.
rider
grou.
buil.
fence
plants
trunk
pole
traf.
garb.
cone.
mloU
Random
30.9
40.6
22.8
10.4
74.7
$7.2
26.6
66.6
15.6
8.3
10.6
28.4
SalsaNet
Entropy
32.8
45.2
33.1
18.6
76.8
52.6
40.2
64.7
20.1
5.5
11.2
12.7
4.3
32.1
Margin
29.8
38.2
33.6
28.0
71.1
48.0
27.7
61.3
24.5
12.5
20.6
18.4
0.0
31.8
Annotator
45.1
39.7
76.5
53.9
64.7
26.2
11.8
13.7
35.9
Target Only
39.2
51.0
52.7
40.2
79.3
66.1
50.1
71.5
28.1
18.7
28.3
8.0
16.7
42.3
Random
36.6
50.5
40.8
0.1
76.1
69.3
50.3
74.0
3.1
17.8
1.3
0.0
0.0
32.3
PolarNet
Entroyy
44.5
48.8
50.3
11.8
77.9
63.6
45.4
71.0
10.0
13.3
19.1
0.0
0.0
35.0
Margin
22.0
42.8
64.1
64.2
19.4
17.5
Annotator
44.4
51.7
55.9
39.2
76.2
64.3
51.9
70.3
22,4
18.6
28.7
6.9
21.7
42.5
Target-Only
66.3
57.2
62.3
51.8
80.8
74.9
61.3
75.5
22.8
21.8
29.4
46.1
50.4
Table 5:
Per-class results on the SemanticPOSS val (range-view: SalsaNet under active learning setting using only 10 voxel budgets.
T0 and bev-view:
PolarNet (T00)
Category frequency
0.35
SemanticPO55
AL selected
0. Details grab.
0.30
ASFDA selected
ADA selected
0. 0,0050
0 0025
0.25
0
0.20
July
Details for cone
0.15
0.10
o 000
0.05
0.00
Fill
car
bike pers. rider grou. buil. fence plants trunk pole traf. garb. cone
Figure 6: Category frequencies on SemanticPOSS train 42 of Annotator selected voxel grids under AL, ASFDA, ADA scenarios, with the model trained SynLiDAR - POSS (MinkNet (9).
observation across these experiments is that Anmotator consistently outperforms the baseline methods regardless of the budget allocation. In particular, Annotator achieves the best performance with about five voxel grids of each LiDAR scan, highlighting the effectiveness of our method in selecting informative areas for active learning. Additionally, we notice that the performance of Annotator tends to saturate when the budget exceeds four voxel grids, particularly in the nuScenes - KITTI adaptation task. This phenomenon can be attributed to the fact that the selected voxels at this point provide a sufficient foundation for training highly competent segmentation model.
3.3 Analysis
More network architectures. As general baseline, Annotator can be easily applied to other non- voxelization based backbones. Here, we conduct experiments on both SalsaNext [IO) (range-view) and PolarNet [T00] (bev-view) and per-class results are presented in Table The findings reveal that Annotator continues to yield significant gains, even when applied to range- bev-view backbones, with limited budget. However, the performance gains these cases are somewhat less pronounced compared to the voxel-view counterparts. Also, to achieve fully-supervised performance of 85%, a budget twice as large is required. We suspect that some annotations derived from voxel-centric selection may not be entirely applicable to other non-voxelization based methods.
0.05
Effeet of voxel size 22- We conduct experiments different 42 while keeping the same budget for selection process and results are listed in Table & We can observe that active rounds (# round) decreases as Ag increases since the number of voxels (# voxel) will be small when A2 large. Notably, the performance of the large voxel grid (2 0.2) more adequate and robust.
A yonel
54973
54543
43795
36091
30414
25992
22539
R Foondd
11
6.
AL
39.6
42.9
43.9
44.2
44.9
45.1
44.8
ASFDA
40.0
46.0
48.0
48.3
48.0
ADA
44.5
52.4
$2.0
Table 6: Experiments different values of A
(from 0.05 to 0.35) for selection process, con-
ducted on SynLiDAR 13, POSS (MinkNet
Category frequency. To obtain deep insight into Annotator, we also visualize a detailed plot of the class frequencies of voxel grids selected by Annotator (AL, ASFDA. ADA) and true class
8

---

:ln
truskk
modd
aideos
build
Senow
munnk
Target Only
(Ouns)
Figure 7: Visualization of segmentation results for the task SynLiDAR MinkNet 19. Each row shows results of Ground-Trun Target-Only, Source-Only, our Annota- for under AI,, ASFDA, and ADA scenarios one by one. Best viewed in color.
19 KITTI using
frequencies of the SemanticPOSS train in Figure g As expected, we clearly see that the true distribution is exactly long-tail distribution while Annotator is able to pick out more voxels that contain rare classes. Particularly, it asks labels for more annotations of "rider", "pole", "trunk", "traf.", "grab." and "cone". This, along with the apparent gains in these classes in Table confirms the diversity and balance of voxel grids selected by Annotator.
Qualitative results. Figure visualizes segmentation results for Source-/Tagget-Olly, our Annotator under AL, ASFDA, and ADA approaches on SemanticKITTI val. The illustration demonstrates the ability of Annotator enhance predictions not only in distant regions but also to effectively eliminate false predictions across a wide range of directions around the center. By employing voxel-centric selection, Annotator successes in enhancing segmentation accuracy even when faced with extremely limited annotation availability. More qualitative results are shown in Appendix B.5
3.4 Limitations
Currently, Annotator has two main limitations. First, high annotation cost and potential biases. Annotator has made substantial strides in enhancing LiDAR semantic segmentation with human involvement. Nonetheless, the annotation cost remains challenge. It's imperative to acknowledge the existence of label and sensor biases, which can be a safety concern in real-world deployments. Second, expansion beyond semantic segmentation. Annotator current focus on LiDAR semantic segmentation represents a significant limitation fully realizing its potential. In the future work, we plan to extend Annotator to other 3D tasks, such LiDAR object detection. This may involve two key changes: i) shifting to frame-level selection; ii) reformulating the VCD strategy to consider the diversity for each box annotation.
4 Related Work
LiDAR perception. Deep learning has made LiDAR perception tasks such classification 1i
49 70 98) and detection f
30
65
X easy to solve, allowing deployment in outdoor scenarios.
Differently, LiDAR semantic segmentation
23
45 46), receiving a class label for
each point, is an indispensable technology understand scene that is beyond the scope of modern
object detectors 35 There exist various techniques to segment the 3D LiDAR point clouds, e.g.,
point 32 45 79 voxel [18
39 103), range [ا 8I], bird's eye [100], and multiple view [68] 89 96
methods. As the best approaches for LiDAR perception are typically trained under full supervision,
which can be costly more than capturing data itself, several methods resort more frugal learning techniques [4], such as semi- 22 BT, weak-
[2 34] and self-supervision
f 101 zero-shot
and few-shot 63 learning and, as studied here, active learning 66 and domain adaptation 74
Active learning for LiDAR point clouds. avoid the burden of complete point cloud annotation, these methods iteratively select and request the most exemplar scans [94] regions [22
points 35

---

boxes 37 to be labeled during the network training. Most selection strategies lean on uncertainty 124 criteria. Uncertainty sampling can be measured over each point prediction scores of the model, e.g., softmax entropy 77 or the margin between the two highest scores [53 to select the most confusing of the current model. For example, Hu et al. 24 estimate the inconsistency across frames to exploit the inter-frame uncertainty embedded in LiDAR sequences. On the other side, diversity sampling has been ensured by selecting core sets [59]. Leveraging the unique geometric structure of LiDAR point clouds, Liu et al. [S5] partition the point could into collection of components then annotate a few points for each component. Recently, the need for an initially annotated fraction of the data to bootstrap an active learning method has been investigated [ 200 58 95 [04]], which is termed as cold start problem. In this work, we show that a smart selection of the first set of data with the aid of an auxiliary model can boost all baseline methods drastically.
88 or diversity 60 91
Domain adaptation for LiDAR point clouds To tackle the sensor- problem encountered in LiDAR deployment, large body of literature on domain adaptation (DA) 12
57 02 has
been developed. These methods aim to overcome the challenges posed by variations in data collection, sensor characteristics, and environmental conditions, enabling machines to perceive the real world more accurately and reliably. To name few, Kong et al. (28) explore cross-city adaptation for uni-modal LiDAR segmentation. Rochan er al. [51] propose a self-supervised adaptation technique with gated adapters. Saltori et al. [55] mitigate the domain shift by creating two new intermediate domains via sample mixing. Similarly, with the intermediate domain, Ding el al. [12] propose a data-oriented framework with a pretraining and self-training stage for 3D indoor scenes. Despite the significant progress made in DA, the label scareity of target domain severely handicaps its utility as the performance of such models often lags far behind the supervised learning counterparts. With this consideration, given an acceptable annotation budget, we explore a simple annotating strategy to assist adaptation process and significantly boost the performance of target domain.
Up to now, active learning coupled with domain adaptation has great practical significance 15 38 41
69 [6]. Nevertheless, rather little work has been done to consider the problem in 3D domains. A recent effort, UniDA3D [I3,, effectively tackles domain adaptation and active domain adaptation tasks for 3D semantic segmentation. UniDA3D employs unified multi-modal sampling strategy, selecting informative pairs of 2D-3D data from both source and target domains through domain discriminator, primarily for ADA tasks. The primary distinction is that our Annotator serves as a benchmark for active learning, active source-free domain adaptation, and active domain adaptation tasks, delivering simple and general AL algorithm for LiDAR point clouds. Annotator focuses on enabling AL in both in-distribution and out-of-distribution scenarios. In contrast, UniDA3D places greater emphasis adaptation tasks. On the other hand, Annotator minimizes human labor in a new domain, regardless of the availability of samples from an auxiliary domain. Methodically, Annotator adopts a voxel-centric representation for structured LiDAR data, which is different from the scan-based representation in UniDA3D, Furthermore, Annotator is more efficient than UniDA3D in terms of both computation and annotation cost.
22 67
Conclusion
In this work, we present Annotator, a generalist active learning baseline, to tackle LiDAR semantic segmentation under three distinct label-efficient settings: active learning (AL), active source-free domain adaptation (ASFDA), and active domain adaptation (ADA). Annotator harnesses the power of a purpose-designes voxel confusion degree selection strategy, enabling it to make optimal use of limited budgets while achieving efficient selection and effective performance. Experiments conducted on widely- simulation-to to-rlll and real-to-real LiDAR semantic segmentation benchmarks demon- strate a substantial performance improvement. Looking forward, we believe the effectiveness and simplicity of Annotator has the potential to serve as a powerful tool for label-efficient 3D applications.
Acknowledgements
This paper was supported by National Key R&D Program of China (No. 2021 YFB330110033 the National Natural Science Foundation of China (No. 62376026), and also sponsored by Beijing Nova Program (No. 20230484296).
10

---

References
[1] Idan Achituve, Haggai Maron, and Gal Chechik. Self-supervised learning for domain adaptation point clouds. In WACV, pages 123-133, 2021.
[2] Jordan Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In ICLR, 2020.
[3] Jens Behley, Martin Garbade, Andres Milioto, Jan Quenzel, Sven Behnke, Cyrill Stachniss, and Jürgen Gall. Semantickiti: A dataset for semantic scene understanding of lidar sequences, In ICCV, pages 9296-9306, 2019.
[4] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: multimodal dataset for autonomous driving. In CVPR, pages 11621-11631, 2020.
[5] Devendra Singh Chaplot, Murtaza Dalal, Saurabh Gupta, Jitendra Malik, and Ruslan Salakhutdinov. SEAL: Self-supervised embodicd active learning using exploration and 3d consistency. In NeurlPS, pages 13086-13098, 2021.
[6] Liangyu Chen, Yutong Bai, Siyu Huang, Yongyi Lu, Bihan Wen, Alan L Yuille, and Zongwei Zhou. Making your first choice: To address cold start problem in vision active learning. CoRR, abs/2210.02442,, 2022.
[7] Qi Chen, Lin Sun, Emest Cheung, and Alan L. Yuille. Every view counts: Cross-view consistency 3d object detection with hybbiil-yyiindrical-pphrriiaa voxelization. In NeurlPS, pages 21224-21235, 2020.
[8] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic segmentation with cross pseudo supervision. In CVPR, pages 261.32222,, 2021.
[9] Christopher B. Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In CVPR. pages 3075-3084, 2019.
[10] Tiago Cortinhal, George Tzelepis, and Eren Erdal Aksoy. Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds. In ISVC, pages 207-222, 2020.
[11] Shuang Deng. Qiulei Dong, Bo Liu, and Zhanyi Hu. Superpoint-guie semi-super semantic segmentation of 3d point clouds. In ICRA, pages 9214-9220, 2022.
[12] Runyu Ding, Jihan Yang. Li Jiang, and Xiaojuan Qi. Doda: Data-oriented sim-to-real domain adaptation for 3d semantic segmentation. In ECCV, pages 284-303, 2022.
[13] Ben Fei, Siyuan Huang. Jiakang Yuan, Botian Shi. Bo Zhang, Weidong Yang. Min Dou, and Yikang Li. Unida3d: Unified domain adaptive 3d semantic segmentation pipeline. CoRR, abs/2212.10390, 2022.
[14] Ben Fei, Weidong Yang, Liwen Liu, Tianyue Luo, Rui Zhang. Yixuan Li, and Ying He. Self-supervised learning for pre-training point clouds: survey. CoRR, abs/2305 04691,, 2023.
[15] Bo Fu, Zhangjie Cao, Jianmin Wang. and Mingsheng Long. Transferable query selection for active domain adaptation. In CVPR, pages 7272-7281, 2021.
[16] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun, Vision meets robotics: The KITTI dataset. Int. J. Robotics Res., 32(11):1233 123111123311171111 2013.
[17] Ankit Goyal, Hei Law, Bowei Liu. Alejandro Newell, and Jia Deng. Revisiting point cloud shape classification with simple and effective baseline. In ICML, pages 3809-3820, 2021.
[18] Benjamin Graham, Martin Engeleke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. CVPR, pages 9224 9232, 2018.
[19] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE Trans. Pattern Anal. Mach. Intell., 43(12))4338 4364, 2020.
[20] Neil Houlsby, José Miguel Hernández-Lobend and Zoubin Ghahramani. Cold-start active leamning with robust ordinal matrix factorization. In ICML, pages 766-774, 2014.
[21] Ping Hu, Stan Sclaroff, and Kate Saenko. Uncertainty-awer aware learning for zero-shot semantic segmentation. In NewrIPS, pages 11713-21722 2020.

---

[22] Qingyong Hu, Bo Yang, Guangchi Fang, Yulan Guo, Ales Leonardis, Niki Trigoni, and Andrew Markham. Syn: Weakly-supervised semantic segmentation of large-scale 3d point clouds. In ECCV, pages 600-619, 2022.
[23] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang. Niki Trigoni, and Andrew Markham. Learning semantic segmentation large-scale point clouds with random sampling. IEFE Trans. Pattern Anal. Mach. Intell., 41(1444(111)11)))1 1:8833888344 2022.
[24) Zeyu Hu, Xuyang Bai, Runze Zhang, Xin Wang, Guangyuan Sun, Hongbo Fu, and Chiew-Lan Tai. Lidal: Inter-frame uncertainty based active learning for 3d lidar semantic segmentation. In ECCV, pages 248-265, 2022.
[25] Braden Hurl, Krzysztof Czarnecki, and Steven L. Waslander. Precise synthetic image and lidar (presil) dataset for autonomous vehicle perception. IV. pages 2522-2529. 2019.
[26] Ajay J. Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. In CVPR, pages
2009.
[27] Alexander Kirillov, Fric Mintun, Nikhila Ravi, Hanzi Mao, Chloé Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross B. Girshick, Segment anything. CoRR, abs/2304.02643, 2023.
[28] Lingdong Kong, Niamul Quader, and Venice Erin Liong, Conda: Unsupervised domain adaptation for lidar segmentation via regularized domain concatenation. In ICRA, pages 9338 9345, 2023.
[29] Lingdong Kong, Jiawei Ren, Liang Pan, and Ziwei Liu. Lasermix for semi-supervised lidar semantic segmentation. In CVPR, pages 21705-21715, 2023.
[30] Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang. and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. CVPR, pages 12697-12705, 2019.
[31] Li, Hubert PH Shum. and Toby P Breckon. Less is more: Reducing task and model complexity for 3d point cloud semantic segmentation. In CVPR, pages 9361-9371, 2023.
[32] Yangyan Li, Rui Bu, Mingehao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. Pointenn: Convolution on x transformed points. In NeurlPS, pages 828 838, 2018.
[33] Kangcheng Liu. Rm3d: Robust data-efficient 3d scene parsing via traditional and learnt 3d descriptors- based semantic region merging. Int. J. Comput. Vis., 131(4):938-90 967, 2023.
[34] Kangcheng Liu, Yuzhi Zhao, Qiang Nie, Zhi Gao, and Ben M Chen. Weakly supervised 3d scene segmentation with region-Ievel boundary awareness and instance discrimination. In ECCV, pages 37-55, 2022.
[35] Minghua Liu, Yin Zhou, Charles R Qi. Boqing Gong, Hao Su, and Dragomir Anguelov. Less: Label- efficient semantic segmentation for lidar point clouds. In ECCV, pages 70-89, 2022.
[36] Zhengzhe Liu, Xiaojuan Qi. and Chi- Bii-iing Fu. One thing one click: A self solf-training approach for weakly supervised 3d semantic segmentation. In CVPR, pages 1726-1736, 2021.
[37] Yadan Luo, Zhuoxiao Chen, Zijian Wang, Xin Yu, Zi Huang, and Mahsa Baktashmotlagh. Exploring active 3d object detection from generalization perspective. In ICLR, 2023.
[38] Antoine De mathelin, François Deheeger, Mathilde MOUGEOT, and Nicolas Vayatis. Discrepancy-basee active learning for domain adaptation. In ICLR, 2022.
[39] Hsien- Yu Meng, Lin Gao, Yu-Kun Lai, and Dinesh Manocha. Vy-net: Voxel VAE net with group convolutions for point cloud segmentation. In ICCV, pages 8499-8507, 2019.
[40] Andres Milioto, Ignacio Vizzo, Jens Behley, and Cyrill Stachniss. Rangenet Fast and accurate lidar semantic segmentation. IROS, pages 4213-4220, 2019.
[41] Munan Ning, Donghuan Lu, Dong Wei, Cheng Bian, Chenglang Yuan, Shuang Yu, Kai Ma. and Yefeng Zheng. Multi-anchor active domain adaptation for semantic segmentation. In ICCV, pages 9112-9122, 2021.
[42] Yancheng Pan, Biao Gao, Jilin Mei, Sibo Geng, Chengkun Li, and Huijing Zhao. Semanticposs: A point cloud dataset with large quantity of dynamic instances. In IV, pages 687-693, 2020.
12

---

[43] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. Pytorch: An imperative style, high- performance deep learning library. NewrIPS, pages 8024-8035 8035, 2019.
[44] Viraj Prabhu, Arjum Chandrasekaran, Kate Saenko, and Judy Hoffmann Active domain adaptation via clustering incertainty-weightee embeddings. In ICCV, pages 8505-8514, 2021.
[45] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++: Deep hicrarchical feature learning on point sets in metric space. In NeurlPS, pages 5099-5108, 2017.
[46] Haibo Qiu, Baosheng Yu, and Dacheng Tao. GFNet: Geometric flow network for 3d point cloud semantic segmentation. Trans. on Mach. Learn. Res., 2022.
[47] Md. Atiqur Rahman and Yang Wang. Optimizing intersection-over-uiinn in deep neural networks for image segmentation. In ISVC, pages 234 244, 2016.
[48] Piyush Rai, Avishek Saha. Hal Daumé III, and Suresh Venkatasubramaniia. Domain adaptation meets active learning. Proceedings the NAACI HI.T 2010 Workshop on Active Learning for Natural Language Processing, pages 27-32, 2010.
[49] Jiawei Ren, Liang Pan, and Ziwei Liu. Benchmarking and analyzing point cloud classification under corruptions. In ICML, pages 18559-18575, 2022.
[50] Stephen E Reutebuch, Hans-Eri Andersen, and Robert J McGaughey, Light detection and ranging (lidar): an emerging tool for multiple resource inventory. Journal of forestry, 103(6):286-292, 2005.
[51] Mrigank Rochan, Shubhra Aich, Eduardo R Corral-Soto, Amir Nabatchian, and Bingbing Liu. Unsu- pervised domain adaptation in lidar semantic segmentation with supf-supn and gated adapters. In ICRA, pages 2649-2655, 2022.
[52] Ricardo Roriz, Jorge Cabral, and Tiago Gomes. Automotive lidar technology: A survey. IEEE Trans. Intell. Transp. Syst., 23(7):6282 6297, 2022.
[53] Dan Roth and Kevin Small. Margin-based active learning structured oulput spaces, ECML, pages 413 424, 2006.
[54] Kwonyoung Ryu, Soonmin Hwang, and Jacsik Park. Instant domain augmentation for lidar semantic segmentation. In CVPR, pages 9350-9360, 2023.
[55] Cristiano Saltori, Fabio Galasso, Giuseppe Fiameni, Nicu Sebe, Elisa Ricci, and Fabio Poiesi. Cosmix: Compositional semantic mix for domain adaptation in 3d lidar segmentation. In ECCV, pages 586-602, 2022.
[56] Cristiano Saltori, Evgeny Krivosheew, Stéphane Lathuiliére. Nicu Sebe, Fabio Galasso, Giuseppe Fiameni, Elisa Ricci, and Fabio Poiesi. GIPSO: geometrically informed propagation for ouline adaptation 3d lidar segmentation. In ECCV, pages 567-585, 2022.
[57] Cristiano Saltori, Aljosa Osep, Elisa Ricci, and Laura Leal- Taixé. Walking your lidog: A journey through multiple domains for lidar semantic segmentation. In ICCV, pages 196-206, 2023.
[58] Nermin Samet, Oriane Siméoni, Gilles Puy, Georgy Ponimatkin, Renaud Marlet, and Vincent Lepetit, You never get second chance to make a good first impression: Seeding active learning for 3d semantic segmentation. In ICCV, pages 18445-18457, 2023.
[59] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. ICLR, 2018.
[60] Feifei Shao, Yawei Luo, Ping Liu, Jie Chen, Yi Yang. Yulci Lu, and Jun Xiao. Active learning for point cloud semantic segmentation via spatial-structura diversity reasoning. In MM, pages 2575-2585 2022.
[61] Jie-Jing Shao, Lan-Zhe Guo, Xiao-Wes Yang, and Yu-Feng Li. Log: Active model adaptation for label-efficient ood generalization. In NerulPS, pages 11023-1 1034, 2022.
[62] Jie-Jing Shao, Yunlu Xu. Zhanzhan Cheng, and Yu-Feng Li. Active model adaptation under unknown shift. In KDD, pages 1558-1566, 2022.
[63] Charu Sharma and Manohar Kaul. Self-superviser few-shot learning point clouds. In NewrlPS, pages 7212-7221, 2020.
13

---

[64] Yuefan Shen, Yanchao Yang, Mi Yan, He Wang. Youyi Zheng, and Leonidas J Guibas. Domain adaptation on point clouds via geometry
implicits. In CVPR. pages 7223-7232, 2022.
[65] Shaoshuai Shi, Li Jiang, Jiajun Deng. Zhe Wang. Chaoxu Guo, Jianping Shi, Xiaogang Wang, and Hongsheng Li. PV-RCNN++: point-voxel feature abstraction with local vector representation for 3d object detection. Int. J. Comput. Vis., 131(2)55315555 2023.
[66] Xian Shi, Xun Xu, Ke Chen, Lile Cai, Chuan Sheng Foo, and Kui Jia. Label-efficient point cloud semantic segmentation: An active learning approach. CoRR, abs/2101.06991 2021.
[67] Inkyu Shin, Dong jin Kim, Jae Won Cho, Sanghyun Woo, Kwanyong Park, and In So Kweon. Labor: Labeling only if required for domain adaptive semantic segmentation. In ICCV, pages 8588-8598, 2021.
[68] Inkyu Shin, Yi-Hsuan Tsai, Bingbing Zhuang, Samuel Schulter, Buyu Liu, Sparsh Garg, In So Kweon, and Kuk-Jin Yoon. Mm-Ita; multi-modal test-time adaptation for 3d semantic segmentation. In CVPR, pages 16928-16999 2022.
[69] Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, and Manmohan Chandraker. Active adversarial domain adaptation. In WACV, pages 728-737, 2020.
[70] Jiachen Sun, Yulong Cao, Christopher B Choy, Zhiding Yu, Anima Anandkumar, Zhuoqing Morley Mao, and Chaowei Xiao. Adversarially robust 3d point cloud recognition using self-supervisions. In NeurlPS, pages 15512. 2021.
[71] Haotian Tang, Zhijian Liu, Shengyu Zhao, Yujun Lin, Ji Lin, Hanrui Wang, and Song Han, Searching efficient 3d architectures with sparse point-voxel convolution. In ECCV, pages 685-702, 2020.
[72] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NerulPS, 2017.
[73] Xiaoyu Tian, Haoxi Ran, Yue Wang, and Hang Zhao. Geomae: Masked geometric target prediction for self-supervised point cloud pre-training. In CVPR, pages 13570-13580, 2023.
[74] Larissa T Triess, Mariella Dreissig, Christoph B Rist, and Marius Zöllner. A survey on deep domain adaptation for lidar perception. In /V Workshops, pages 350-357, 2021.
[75] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell, Adversarial discriminative domain adapta- tion. In CVPR, pages 2962-2971, 2017."
[76] Ozan Unal, Dengxin Dai, and Luc Van Gool. Scribble-supervieed lidar semantic segmentation. In CVPR, pages 2697-2707, 2022.
[77] Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthicu Cord, and Patrick Pérez. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In CVPR, pages 2517-2526, 2019.
[78] Dan Wang and Yi Shang. A new active labeling method for deep learning. In IJCNN, pages 112- 119. 2014.
[79] Yuan Wang, Tianyue Shi, Peng Yun. Lci Tai, and Ming Liu. Pointseg: Real-time semantic segmentation based on 3d lidar point cloud. CoRR, abs/ 507.06288, 2018
[80] Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer. Squeezeseg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3d lidar point cloud. In ICRA, pages 1887- 1893, 2018.
[81] Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, and Kurt Keutzer. Squeezesegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a lidar point cloud. In ICRA, pages 4376-4382, 2019.
[82] Tsung- Haang-Han Voe Wu, Yuch-Cheng Liu, Yu-Kai Huang, Hsin- Ying Lee, Hlun--Ting Su, Ping- Chia Huang, and Winston H. Hsu. Redal: Region-based based and diversity-aware active learning for point cloud semantic segmentation. In JCCV, pages 15510-15519, 2021.
[83] Aoran Xiao, Jiaxing Huang, Dayan Guan, Kaiwen Cui, Shijian Lu, and Ling Shao. Polarmix: A general data augmentation technique for lidar point clouds. In NewrIPS, pages 1033--1008, 2022.
[84] Aoran Xiao, Jiaxing Huang, Dayan Guan, Fangneng Zhan, and Shijian Lu. Transfer learning from 2022.
synthetic real lidar point cloud for semantic segmentation. In AAAI, pages
14

---

[85] Binhui Xie, Shuang Li, Mingjia Li. Chi Harold Liu, Gao Huang, and Guoren Wang. Sepico: Semantic- guided pixel contrast for domain adaptive semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 9507)99004-002, 2023.
[86] Binhui Xie, Longhui Yuan. Shuang Li. Chi Harold Liu, and Xinjing Cheng. Towards fewer annotations: Active learning via region impurity and prediction uncertainty for domain adaptive semantic segmentation. CVPR, pages (158-8068,455 2022.
[87] Binhui Xie, Longhui Yuan, Shuang Li, Chi Harold Liu, Xinjing Cheng, and Guoren Wang. Active learning for domain adaptation: An energy-based approach. In AAAI, pages 8708-877 101--716,,, 16, 2022.
[88] Mixue Xie, Shuang Li, Rui Zhang, and Chi Harold Liu. Dirichlet-based uncertainty calibration for active domain adaptation. In ICLR, 2023.
[89] Jianyun Xu, Ruixiang Zhang, Jian Dou, Yushi Zhu, Jie Sun, and Shiliang Pu. Rpvnet: A deep and efficient range-point point-voxel fusion network for lidar point cloud segmentation. ICCV, pages 16004-16011 2021.
[90] Xun Xu and Gim Hee Lec. Weakly supervised semantic point cloud segmentation: Towards 10x fewer labels. In CVPR, pages 13706-13716 2020.
[91] Shanding Ye, Zhe Yin, Yongjian Fu, Hu Lin, and Zhijie Pan. A multi-granularity semisupervised active lcarning for point cloud semantic segmentation. Newral Comput. Appl., 35(21):1 2023.
[92] Li Yi, Boqing Gong, and Thomas Funkhouser. Complete & label: A domain adaptation approach semantic segmentation of lidar point clouds. In CVPR, pages 133633,,33,, 2021.
[93] Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark E. Camp- bell, and Kilian Q. Weinberger. Beeudo-iddr+++ Accurate depth for 3d object detection in autonomous driving. ICLR, 2020.
[94] Jiakang Yuan, Bo Zhang, Xiangchao Yan, Tao Chen, Botian Shi, Yikang Li, and Yu Qiao. Bi3d: Bi-domain active learning for cross doss-dommin 3d object detection. In CVPR, pages 15599- 15608, 2023.
[95] Michelle Yuan, Hsuan- Lin, and Jordan Boyd-Grabe Cold-start active learning through self- supervised language modeling. CoRR, abs/2010.09535, 2020.
[96] Qijian Zhang, Junhui Hou, Yue Qian, Yiming Zeng, Juyong Zhang, and Ying He. Flattening-net. Deep regular 2d representation for 3d point cloud analysis. IEEE Trans. Puttern Anal. Mach, Intell., 45(8):9726-9742, 2023.
[97] Renrui Zhang. Ziyu Guo, Peng Gao. Rongyao Fang. Bin Zhao, Dong Wang. Yu Qiao, and Hongsheng Li. Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training. In NeurIPS, pages 27061-27074. 2022.
[98] Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Lt. Pointelip: Point cloud understanding by CLIP. In CVPR, pages 8542-8552, 2022.
[99] Yachao Zhang, Yanyun Qu, Yuan Xie, Zonghao Li, Shanshan Zheng, and Cuihua Li. Perturhed self- distillation: Weakly supervised large-scale point cloud semantic segmentation. In ICCV, pages 15520- 15528, 2021.
[100] Yang Zhang, Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi, Boqing Gong, and Hassan Foroosh. Polarnet: An improved grid representation for online lidar point clouds semantic segmentation. In CVPR, pages 9598 9607, 2020.
[101] Zaiwei Zhang, Min Bai, and Erran Li Li. Self-supervised pretraining for large-scale point clouds. In NewrIPS, pages
2022.
[102] Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In CVPR, pages 4490-4499, 2018.
[103] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Wei Li, Yuexin Ma, Hongsheng Li, Ruigang Yang. and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar-based perception. IEEE Trans. Pattern Anal, Mach. Intell., 44(10)6887766222 2021.
[104] Yu Zhu, Jinghao Lin, Shibi He, Beidou Wang. Ziyu Guan, Haifeng Liu, and Deng Cai. Addressing the item cold-star start problem by attribute-driven active learning. IEEE Trans, Knowl. Data Eng., 32(4))631-644 2019.
[105] Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. Confidence regularized self- training. In ICCV, pages
2019.
15


Reviews:
[{"review_id": "voe6sFo6pl", "reviewer": "Reviewer_zYAC", "summary": "This paper proposes a voxel-centric online active learning baseline that efficiently reduces the labeling cost of enormous point clouds and effectively facilitates learning with a limited budget. The contribution of this paper can be summarized in three aspects:\n1. A voxel-centric online active learning baseline that efficiently reduces the labeling cost of enormous point clouds and effectively facilitates learning with a limited budget.\n2. A novel label acquisition strategy, voxel confusion degree (VCD), that requires 1000 times fewer annotations while reaching a close segmentation performance to that of the fully supervised counterpart.\n3. A generally applicable and works for different network architectures (e.g., MinkNet, SPVCNN, etc.), in distribution or out of distribution setting (i.e., AL, ASFDA, and ADA), and simulation-to-real and real-to-real scenarios with consistent performance gains.", "questions": "1. voxel based methods is one of the major and popular representations for lidar perception tasks, however there do exists other representations such as range view and point based methods, where the concept of voxel does not exist. question is, how does the proposed method apply to non-voxelization based methods?\n2. segmentation tasks require dense point-wise annotation, but detection tasks only require, to some extent, much sparser box level annotations, how does the proposed method apply to detection based tasks? one would expect that the data volume reduction persists but not as impressive as 1000x", "limitations": "no potential negative societal impact is noticed by the reviewer to the best of their knowledge", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. active learning is not entirely new in this field, but reducing the data volume requirement at the order of 1000 times is still very impressive\n1. active learning coupled with domain adaptation has great practical significance, but little work has been done to consider the problem in 3D domains. The Annotator aims to minimize human labor in a new domain, regardless of whether samples from an auxiliary domain are available or not.\n2. Methodologically, Annotator adopts a voxel-based representation for structured LiDAR data, which is different from the scan-based representation in UniDA3D. \n3. Furthermore, Annotator can perform online active sampling that is more efficient than UniDA3D in terms of computation cost.", "weaknesses": "1. The cost of developing an AI based solution include multiple sectors: 1) data collection; 2) data annotation; 3) model training; 4) model deployment and integration. One of the most costly stage, both in time and money, is annotation. The AL paradigm, instead, proposes to use computation to reduce the total cost, by using computation to reduce the amount of data needed for human annotation. In this paper, an additional pre-training phase is added, it is unclear how to evaluate the cost balance, eg, increased cost of computation vs reduced cost of human annotation. In some sense, this is not the weakness/limitation of this particular paper, but rather applies to the whole AL paradigm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the Reviewer zYAC for the detailed summary and constructive comments. We are glad that the reviewer acknowledges that the problem has great practical significance, the method is novel and generally applicable, and the experiments are very impressive. Here we answer all the questions and hope they can address the concerns.\n\n**Q1: In this paper, an additional pre-training phase is added, it is unclear how to evaluate the cost balance, eg, increased cost of computation vs reduced cost of human annotation. In some sense, this is not the weakness/limitation of this particular paper, but rather applies to the whole AL paradigm.**\n\nWe agree with the reviewer that the trade-off between computation cost and annotation cost is a key problem in active learning. In this paper, we mainly focus on reducing the cost of human annotation while reaching a close segmentation performance to that of the fully supervised counterpart. Take the task of SynLiDAR $\\to$ SemanticKITTI (KITTI) as an example, the simplest setup is performing active learning within KITTI dataset. In this case, no extra pre-training phase is included, but the performance is **less than satisfactory**. The reason might be the lack of prior information to select an initial annotated set (i.e. *cold-start problem*). To address this, we utilize some open-access datasets, especially synthetic dataset, to train an auxiliary model via a pre-training phase that serves as a warm-up stage, allowing for a smart selection of the data in the first round. Importantly the pre-training phase is trained with a few epochs and only conducted once, and the cost (both time and money) is **very low**. Here we provide a detailed analysis of the cost.\n\nphase|total epoch|running time (hours)|mIoU\n-|-|-|-\npre-train on SynLiDAR|10|2.34|22.0\nAL on KITTI|50|18.04|53.7\nASFDA on SynLiDAR $\\to$ KITTI|50|18.39|54.1\nADA on SynLiDAR $\\to$ KITTI|50|28.48|**57.7**\n\nThis demonstrates that our method can achieve a good balance between high performance and low cost (computation & annotation). In the future, we will explore more efficient ways to reduce the cost of both computation and annotation. We will add these results in the revision.\n\n**Q2: voxel based methods is one of the major and popular representations for lidar perception tasks, however there do exists other representations such as range view and point based methods, where the concept of voxel does not exist. question is, how does the proposed method apply to non-voxelization based methods?**\n\nThanks for the valuable question. The main contribution consists of a plain voxel-centric online active learning baseline and a novel label acquisition strategy. As a general baseline, the proposed method can be easily applied to other non-voxelization based methods. Here, we have conducted experiments on a range-view method SalsaNext [a] and a BEV-view method PolarNet [b]. The per-class results on task of SynLiDAR$\\to$POSS under AL setting with only 10 voxel budgets are shown in the following tables.\n\nSalsaNext:\nmehotd|pers.|rider|car|trunk|plants|traf.|pole|garb.|buil.|cone|fence|bike|grou.|mIoU\n-|-|-|-|-|-|-|-|-|-|-|-|-|-|-\nRandom|22.8|10.4|30.9|15.6|66.6|8.3|5.5|0.0|57.2|10.6|26.6|40.6|74.7|28.4\nEntropy|33.1|18.6|32.8|20.1|64.7|11.2|5.5|12.7|52.6|4.3|40.2|45.2|76.8|32.1\nMargin|33.6|28.0|29.8|24.5|61.3|20.6|12.5|18.4|48.0|0.0|27.7|38.2|71.1|31.8\n**Ours**|39.7|31.8|32.0|26.2|64.7|17.7|11.8|13.7|53.9|13.1|40.9|45.1|76.5|**35.9**\nTarget-only|52.7|40.2|39.2|28.1|71.5|28.3|18.7|8.0|66.1|16.7|50.1|51.0|79.3|42.3\n\nPolarNet:\nmehotd|pers.|rider|car|trunk|plants|traf.|pole|garb.|buil.|cone|fence|bike|grou.|mIoU\n-|-|-|-|-|-|-|-|-|-|-|-|-|-|-\nRandom|40.8|0.1|36.6|3.1|74.0|1.3|17.8|0.0|69.3|0.0|50.3|50.5|76.1|32.3\nEntropy|50.3|11.8|44.5|10.0|71.0|19.1|13.3|0.0|63.6|0.0|45.4|48.8|77.9|35.0\nMargin|42.8|24.3|22.0|19.4|64.2|17.5|20.1|4.0|54.7|0.0|33.0|35.4|64.1|30.9\n**Ours**|55.9|39.2|44.4|22.4|70.3|28.7|18.6|6.9|64.3|21.7|51.9|51.7|76.2|**42.5**\nTarget-only|62.3|51.8|66.3|22.8|75.5|29.4|21.8|4.8|74.9|46.1|61.3|57.2|80.8|50.4\n\nIt is seen that our method still brings a large improvement using either range- or BEV-view backbones with only the limited budget. However, **the performance gains are less than that of the voxel-view counterparts**. Also, to achieve 85% fully-supervised performance, the budget are twice. We think that some annotations of voxel-centric selection might be invalid in other non-voxelization representations. We will add these discussions in the revision for better understanding.\n\n[a] Cortinhal et al. Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds. In ISVC 2020.\n\n[b] Zhang et al. Polarnet: An improved grid representation for online lidar point clouds semantic segmentation. In CVPR 2020.\n\n**Q3: segmentation tasks require dense point-wise annotation, but detection tasks only require, to some extent, much sparser box level annotations, how does the proposed method apply to detection based tasks? one would expect that the data volume reduction persists but not as impressive as 1000x.**\n\nThis question is very interesting and valuable. For active learning in 3D object detection task, most strategies are offline, i.e., a few frames are first selected from the entire dataset, and then all boxes in each frame are annotated manually. Our Annotator, on the other hand, is an online strategy that allows querying and annotating within each frame. \n\nTherefore, some changes need to be made to adapt to the detection task. 1) frame-level selection should be used; 2) VCD should be reformulated as follows: in each frame, obtain pseudo-label for each box, collect statistical information on the predicted categories of all boxes, and VCD is computed based on the percentage of boxes belonging to each different category. In other words, in the original algorithm, a frame is a voxel and a box is a point of the voxel. We will explore the application to other LiDAR perception tasks in the future."}, {"review_id": "Idp6dA4EW1", "reviewer": "Reviewer_N28Y", "summary": "This paper presents Annotator, a general and efficient active learning baseline for LiDAR semantic segmentation, which can adapt to different settings and scenarios with minimal annotation cost. Annotator consists of a voxel-centric online selection strategy that exploits the local topology and structure of point clouds to query the most informative voxel grids for annotation. Annotator can also leverage an auxiliary model to address the cold start problem. The paper evaluates Annotator on two datasets (SynLiDAR and SemanticKITTI) with two network architectures (MinkNet and SPVCNN), and shows that Annotator can achieve on-par performance with the fully supervised counterpart using 1000 fewer annotations and outperform existing methods under various active learning settings. The paper states that Annotator is a simple and general solution for label-efficient 3D perception.", "questions": "- Is the term '*cell*' (L190) the same meaning as '*voxel*' (L189)? If they're interchangeable, it might be better to use the same word '*voxel*'; if not, it's a good idea to explain their differences, relations, or the definition of '*cell*'.\n", "limitations": "Authors should be rewarded that the limitations and potential negative societal impact are explicitly mentioned in the Conclusion.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "**Originality**: The paper proposes 3 ideas,  a voxel-centric online active learning baseline, a label acquisition strategy (VCD), and is generally applicable and works for different network architectures.\n\n**Quality**: The paper provides a thorough **experimental evaluation** of Annotator on several simulation-to-real and real-to-real LiDAR semantic segmentation tasks, using different network architectures and baselines. The paper also conducts **ablation studies** to analyze the impact of different components of Annotator, such as voxel size, selection strategy, and auxiliary model. The paper demonstrates that Annotator can achieve **on-par or superior performance** with the fully supervised counterpart using 1000 fewer annotations, and significantly outperform other state-of-the-art methods.\n\n**Clarity**: The paper also provides sufficient background information and related work to situate the contribution of Annotator in the context of existing literature on LiDAR perception, active learning, and domain adaptation. The Supplementary Material is well-written and contains enough details and information as needed.\n\n**Significance**: The paper addresses an important and challenging problem of label-efficient LiDAR semantic segmentation, which has many applications in autonomous driving, robotics, and 3D scene understanding. ", "weaknesses": "**Major Issues:**\n- **Insufficient novelty and contribution**: this paper perform an analysis of several common selection strategies, e.g., Random, Entropy and Margin, which are all proposed in previous research. The newly proposed VCD strategy lacks justification for its design, and contains too few details (L195-L199). The pipelines of distinct active learning settings (e.g., AL, ASFDA, and ADA) seem natural and basic.\n- **Insufficient results for experiments:** Although the authors state in the main text, \"***Experimentally***, we find that the large voxel grid is also more robust to the noise and sparsity of the point clouds\", they provide no experimental results of different $\\Delta$ in their main text and Supplementary Material.\n- **Insufficient justifications**: For example, about  **Voxel-centric** selection, some justifications are missing in this paper. Why choosing voxel-based selection rather than other methods, e.g., point-based, range-image-based, BEV, etc (L101-102)? Any advantages and limitations of **Voxel-based** methods?\n\n**Minor Issues:**\n- **Readability**: Some figures and text in figures (e.g., Figure 6 and Figure A3-A4) are too small for readability. Authors should attach more detailed images/visualization to the supplementary material.\n- **Variables undefined** although they are obvious in meaning: e.g., $X_t$ in L155, $X_s$ and $Y_s$ in L157.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer N28Y for the appreciative and constructive comments. We are encouraged that the reviewer found our three original ideas, thorough experiments, sufficient background, and well-written supplementary material. Here, we would like to respond to the issues and hope that our responses will address the concern.\n\n**Q1: Insufficient novelty and contribution**\n\nWe would like to emphasize that the main contributions lie in three aspects.\n- a label acquisition strategy (VCD) is **more robust and diverse** to select samples efficiently under a domain shift;\n- a voxel-centric online active learning can largely reduce the labelling cost of enormous point clouds. Particularly, only requiring **1,000x fewer annotations** can reach a close performance to the fully-supervised counterpart;\n- **generally applicable** for different network architectures (voxel-, range-, and BEV-view, etc), settings (in distribution and out of distribution), and scenarios (simulation-to-real and real-to-real).\n\n**Q2: this paper perform an analysis of several common selection strategies, e.g., Random, Entropy and Margin, which are all proposed in previous research.**\n\nIndeed, selection strategies such as Random, Entropy, and Margin typically serve as baseline methods to verify the effectiveness of the proposed method. In this work, we introduce a novel label acquisition strategy (VCD) and a voxel-centric online active learning. Therefore, we first testify common strategies in our voxel-centric selection (the response of Q6 gives justifications). And then we compare VCD with these strategies to show its superiority. In other words, these common selection strategies \nare not technical contributions of the paper but are necessary to form our baseline.\n\n\n**Q3: The newly proposed VCD strategy lacks justification for its design, and contains too few details (L195-L199).**\n\nOur VCD design is inspired by an observation: traditional AL setting often focuses on learning a model from scratch rather than adapting under domain shift. In practice, models are trained in an auxiliary domain and deployed in a new domain of interest. In this case, existing AL strategies are less effective since uncertainty estimates on the new domain may be miscalibrated. By contrast, our label acquisition strategy (VCD) is tailored to estimate category diversity instead of uncertainty within a voxel, which is more robust under domain shift. Specifically, we first generate pseudo label for each point within a voxel based on model prediction. Second, we count the percentage of each category within the voxel. Third, we take the entropy of the statistical information within the voxel as a score. Finally, we select voxels with high score. The **higher the score, the more predicted classes within a voxel**, and we think this would help to train the model after being annotated.\n\n**Q4: The pipelines of distinct active learning settings (e.g., AL, ASFDA, and ADA) seem natural and basic.**\n\nThis question is very interesting and valuable. To mitigate the annotation burden in model training, most of existing paradigms delves into active learning (in distribution) or domain adaptation (out of distribution), separately. However, there are **not unified setting and baseline**, which hinders the research. Indeed, the two classes of approaches are deeply intertwined, but **little work has been done to consider the combination in 3D domains**. This paper targets at delivering a simple and general online active learning baseline by a voxel-centric selection. It unifies active learning and domain adaptation for LiDAR semantic segmentation. Moreover, sufficient and convincing empirical results validate the necessity of these distinct settings.\n\n**Q5: Insufficient results for experiments: provide no experimental results of different Δ.**\n\nWe agree with the reviewer, and thus conduct experiments on different Δ while keeping the same budget for selection process on SynLiDAR->POSS. The results are as follows\nΔ|0.05|0.1|0.15|0.2|0.25|0.3|0.35\n-|-|-|-|-|-|-|-\nAL|39.6|42.9|43.9|44.2|44.9|45.1|44.8\nASFDA|40.0|46.2|46.0|48.0|48.2|48.3|48.0\n\nIt can be seen that the performance of the large voxel grid is more robust to the noise and sparsity of point clouds.\n\n**Q6: Insufficient justifications: For example, about Voxel-centric selection, some justifications are missing in this paper. Why choosing voxel-based selection rather than other methods, e.g., point-based, range-image-based, BEV, etc (L101-102)? Any advantages and limitations of Voxel-based methods?**\n\nThanks for the valuable question. At first, we would like to clarify that there exist image/frame-, region-, and point-based selection strategies in previous research. The first two require an offline stage, which may be infeasible at large scales. The last one is costly due to the sparsity of point clouds. By contrast, voxel-based selection aims to query the salient and exemplar regions and annotate all points within the region, which is more efficient and flexible. It can be easily applied to other methods, e.g., point-, voxel-, range-, BEV-view. Except for the results of voxel-view method reported in the main paper, we have added experimental results on a range-view method SalsaNext and a BEV-view method PolarNet in `Table R1 of the one-page PDF`. It is seen that our method still brings a large improvement using either range- or BEV-view method. Additionally, a drawback is that one may need to tune the voxel size for different scenarios. We find that the large voxel grid (Δ>0.2) is more robust to outdoor scenes in our experiments.\n\n**Q7: Minor issues: readability & variables definition.**\n\nThank you. We will improve the readability of the paper and add a table with all the variables and their detailed descriptions.\n\n**Q8: Is the term 'cell' (L190) the same meaning as 'voxel' (L189)?**\n\nYes, both \"cell\" and \"voxel\" denote the smallest unit of selection. We will use the same term \"voxel\" through the paper."}, {"review_id": "O6Z9Zf1YfT", "reviewer": "Reviewer_dFZX", "summary": "This work proposes a general and efficient data annotation pipeline, namely Annotator, to label LiDAR data for semantic segmentation. Specifically, the proposed method introduces a voxel-centric online selection strategy to determine which voxel should be annotated by humans. Voxel confusion degree (VCD) is thus proposed based on the classification labels predicted by a semantic segmentation network. To prove its efficiency, the authors also conduct some experiments on domain adaption and they use 1,000 times fewer annotations.", "questions": "1 whether the proposed method can be applied to indoor semantic segmentation?\n\n2. Why choose voxel based partition? \n\n3. In Tables 1, 2, 3 The oracle performance (full annotations) should be provided for comparisons. \n\n4. Can the proposed method be applied to other tasks, such as detection or instance segmentation?", "limitations": "The authors have addressed the concerns properly.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This work proposes a new criterion namely voxel confusion degree (VCD) to evaluate the certainty of a voxel. It seems using this criterion the authors only need very few times of annotations to achieve better performance.\n\nThe experiments support the claim of this paper: the proposed method uses the least annotations and achieves better performance.", "weaknesses": "1. In Fig. 2 (ii), it is ASFDA source-free domain adaptation but in (1) pre-trained with source data. This is a bit confusing.\n\n2. The VCD is well defined. How to use it to determine the next annotated voxel is not clear. Should we choose a voxel with larger VCD? Why is that? The motivation or insights are missing. \n\n3. Algorithm 1 is not very informative but more like active learning pipeline. \n\n4. The proposed method Annotator can be only applied to voxelized data. How does the size of voxels affect the final annotation rounds?\n\n5. The compared baselines are all voxel based annotation. How about labeling points in BEV? In other words, there would be many ways to annotate LiDAR data. Why voxel is the best? Or is the proposed method best suitable for voxel based partition?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer dFZX for the very valuable comments. We are glad that the reviewer acknowledges our new criterion can achieve better performance with only very few times of annotations. Here we address the concerns of our paper, and hope our response can address the concern.\n\n**Q1: In Fig. 2 (ii), it is ASFDA source-free domain adaptation but in (1) pre-trained with source data. This is a bit confusing.**\n\nWe apologize for the misunderstanding. One thing that can be confirmed is that only the source model is available in source-free domain adaptation. Actually, Fig. 2 illustrates both (1) source model pre-training process and (2) adaptation and active learning process (ii). Therefore, in (1) we assume that source data is only used to pre-train the source model, while it is not available for adapting the model in (2).\n\n**Q2: The VCD is well defined. How to use it to determine the next annotated voxel is not clear. Should we choose a voxel with larger VCD? Why is that? The motivation or insights are missing.**\n\nYes, a voxel with larger VCD will be chosen. Recall that, we first obtain pseudo label $\\hat{y_i}$ of each point $x_i$ and then divide points within a voxel $v_j$ into $K$ clusters: $v_j^{<k>}=\\{x_i\\in v_j,\\hat{y_i}=k\\}$. At the moment, we can collect statistical information about the categories in a voxel. Next, we take the entropy calculated on the percentage of each distinct class as the voxel confusion degree of each voxel, that is, VCD($v_j$)$=-\\sum_{k=1}^{K}\\frac{|v_j^{<k>}|}{v_j}\\log \\frac{|v_j^{<k>}|}{v_j}$, where $|\\cdot|$ denotes the number of points in a set. The insight is that if there are many predicted classes within a voxel, we assume that it can help to train the model after being annotated. As shown in Fig. 5, we clearly see that the true class distribution of SemanticPOSS is exactly a long-tail distribution while our method can pick out more voxels that contain tail classes. Also, per-class results in Table 3-4 of the main paper and `Table R1 of the one-page PDF` show the superiority of our method in both tail and majority classes.\n\n**Q3: Algorithm 1 is not very informative but more like active learning pipeline.**\n\nGood suggestion. Most of the lines in Algorithm 1 describe active learning pipeline, to which we add additional settings, namely ASFDA and ADA (line 2), and show how to perform voxel-centric selection and model training (lines 5-6). Perhaps we can describe our algorithm in text.\n\n**Q4: The proposed method Annotator can be only applied to voxelized data. How does the size of voxels affect the final annotation rounds?**\n\nThis question is very interesting and valuable. The voxel size $\\Delta$ is default set to 0.25 for selection process and 0.05 for training process. In the case of SemantcPOSS, for example, there are about 30,000 voxels per frame. Five rounds of selection are conducted, one voxel per round, resulting in a 1,000x reduction in annotations. When the voxel size is small, the number of voxels will be large, which will increase selection rounds. Here, we conduct experiments on different size (from 0.05 to 0.35) of voxels for selection process. The following results are conducted on SynLiDAR $\\to$ POSS based on MinkNet.\n\n$\\Delta$|0.05|0.1|0.15|0.2|0.25|0.3|0.35 \n-|-|-|-|-|-|-|-\n\\# voxel per frame|64973|54543|43795|36091|30414|25992|22539|\n\\# selection rounds|11|9|7|6|5|4|4|\nmIoU in AL|39.6|42.9|43.9|44.2|44.9|45.1|44.8\nmIoU in ASFDA|40.0|46.2|46.0|48.0|48.2|48.3|48.0\n\nIt can be seen that the performance of the large voxel grid ($\\Delta$>0.2) is more robust to the noise and sparsity of the point clouds. We will add the results to the revision.\n\n**Q5: The compared baselines are all voxel based annotation. How about labeling points in BEV? In other words, there would be many ways to annotate LiDAR data. Why voxel is the best? Or is the proposed method best suitable for voxel based partition?**\n\nThanks for the valuable question. At first, we would like to clarify that there exist image/frame-, region-, and point-based selection strategies in previous research. The first two require an offline stage, which may be infeasible at large scales. The last one is costly due to the sparsity of point clouds. By contrast, voxel-based selection aims to query the salient and exemplar regions and annotate all points within the region, which is more efficient and flexible. It can be easily applied to different methods, e.g., point-, voxel-, range-, BEV-view. Except for the results of voxel-view method reported in the main paper, we have added experimental results on a range-view method SalsaNext and a BEV-view method PolarNet in `Table R1 of the one-page PDF`. It is seen that our method still brings a large improvement using either range- or BEV-view method with a limited budget.\n\n**Q6: whether the proposed method can be applied to indoor semantic segmentation?**\n\nYes, our method can be applied to indoor semantic segmentation. Following SSDR-AL [59], we have conducted experiments on S3DIS dataset. We report results of comparing the percentage of labeled points required to achieve 90% accuracy for different methods based on Randlanet as follows.\n\nmethod|Random|Entropy|Margin|SSDR-AL [59]|Ours\n-|-|-|-|-|-\nbudget|40.9%|46.7%|43.0%|11.7%|9.9%\n\nIt is observed that our method can annotate 1.8% fewer points than SSDR-AL in achieving the 90% performance of fully-supervised counterpart.\n\n**Q7: In Tables 1, 2, 3 The oracle performance (full annotations) should be provided for comparisons.**\n\nActually, \"Target-only\" in Tables 1, 2, 3, 4 denotes the oracle performance (full annotations in the target domain). Compared with the oracle performance, we find that our Annotator with 5 voxels per frame being labeled can obtain on-par or better performance.\n\n**Q9: Can the proposed method be applied to other tasks, such as detection or instance segmentation?**\n\nThis question is very interesting and valuable. Please ref to `response to Q3 for reviewer zYAC`."}, {"review_id": "DSf5zLmH7C", "reviewer": "Reviewer_a35W", "summary": "This paper introduced a baseline for active learning called Annotator for LiDAR point cloud semantic segmentation. The paper includes an analysis of various active selection strategies, including random selection, entropy-based selection, margin-based selection, and a novel strategy called voxel confusion degree (VCD). The study investigates three different setups: active learning (AL), active source-free domain adaptation (ASFDA), and active domain adaptation (ADA). Through experiments conducted on multiple Simulation-to-Real and Real-to-Real benchmarks, the proposed method demonstrates its effectiveness.", "questions": "1. Why did the annotator achieve significantly smaller performance gains in SynLiDAR->KITTI in Table 2?\n2. A minor issue: The per-class IoU performance of PolarMix is provided in its supplementary material, which is missing in Tables 2 and 3.\n3. Please also refer to the weakness.", "limitations": "This paper discusses the limitations of their work in the conclusion. However, it is suggested to have a separate section dedicated specifically to addressing these limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe paper is well-written and easy to follow.\n2.\tConsidering the significant challenge of annotating point-wise point cloud data in LiDAR segmentation, exploring active learning approaches for this task is valuable and important.\n3.\tThe idea of annotator is simple but effective.\n4.\tThe authors conducted comprehensive benchmarks across different setups, including AL, ASFDA, and ADA. This extensive evaluation covers several novel investigations that have not been explored before.", "weaknesses": "1. This paper lacks in-depth insights and analysis. What is the fundamental intuition behind it? Why do these active selection strategies contribute to performance gains? Furthermore, what sets the proposed VCD apart and enables it to achieve superior performance improvements?\n2. The selection of voxel grids plays a crucial role in determining the final performance, particularly when it comes to random selection. However, this paper did not include error bars in relation to the random seed for multiple experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer a35W for the very constructive comments. We are glad that the reviewer acknowledge that the task is valuable and important, the idea is simple but effective, and the investigation is novel. Here we address the biggest concern raised by the reviewer, i.e., the fundamental intuition behind voxel-centric selection, and hope our response can address this concern.\n\n**Q1: This paper lacks in-depth insights and analysis. What is the fundamental intuition behind it?**\n\nThanks for the valuable question. We would clarify that the intuition of this paper is to establish a simple and general baseline for label-efficient LiDAR semantic segmentation. To mitigate the annotation burden in model training, most of existing paradigms delves into active learning (in distribution) or domain adaptation (out of distribution), separately. However, there are **not unified setting and baseline**, which hinders the research. Indeed, the two classes of approaches are deeply intertwined, but **little work has been done to consider the combination in 3D domains**. This paper targets at delivering a simple and general online active learning baseline by a voxel-centric selection. It unifies active learning and domain adaptation for LiDAR semantic segmentation. Moreover, sufficient and convincing empirical results provide insights for label-efficient 3D applications.\n\n**Q2: Why do these active selection strategies contribute to performance gains?**\n\nAt first, we would like to clarify that there exist image/frame-, region-, and point-based selection strategies in previous research. Active learning aims to identify informative instances for labeling via a selection strategy. Thus, given limited budget, the selection strategy can select the most informative instances for labeling, which can be used to train the model, leading to performance gains.\n\n\n**Q3: What sets the proposed VCD apart and enables it to achieve superior performance improvements?**\nOur VCD design is inspired by an observation: traditional AL setting often focuses on learning a model from scratch rather than adapting under domain shift. In practice, models are trained in an auxiliary domain and deployed in a new domain of interest. In this case, existing AL strategies are less effective since uncertainty estimates on the new domain may be miscalibrated. By contrast, our label acquisition strategy (VCD) is tailored to estimate category diversity instead of uncertainty within a voxel, which is more robust under domain shift. Specifically, we first generate pseudo label for each point within a voxel based on model prediction. Second, we count the percentage of each category within the voxel. Third, we take the entropy of the statistical information within the voxel as a score. Finally, we select voxels with high score. The **higher the score, the more predicted classes within a voxel**, and we think this would help to train the model after being annotated. As shown in Fig. 5, we clearly see that the true class distribution of SemanticPOSS is exactly a long-tail distribution while our method can pick out more voxels that contain tail classes. Also, per-class results in Table 3-4 of the main paper and `Table R1 of the one-page PDF` show the superiority of our method in both tail and majority classes.\n\n\n**Q4: The selection of voxel grids plays a crucial role in determining the final performance, particularly when it comes to random selection. However, this paper did not include error bars in relation to the random seed for multiple experiments.**\n\nIt is a very interesting question regarding the viability of Annotator in the real world. Actually, current results in our experimental section are the mean accuracy across three runs with different random seeds, and we include error bars on task of SynLiDAR $\\to$ KITTI using three random runs in `Figure R1 of the one-page PDF`. And we will add the error bars on all benchmarks in the revision.\n\n\n**Q5: Why did the annotator achieve significantly smaller performance gains in SynLiDAR->KITTI in Table 2?**\n\nThis question is very interesting and valuable. Please note that the performance gain (0.1% under AL/ 2.5% under ASFDA/ 2.8% under ADA in SynLiDAR $\\to$ KITTI in Table 2) is compared to the state-of-the-art baseline method. We think that the numerical improvement should attribute to several reasons. First, there includes challenging variations in data distribution and annotation quality between synthetic SynLiDAR dataset and real-world KITTI dataset. Second, for simplicity, we keep all hyperparameters unchanged when change the backbone from MinkNet to SPVCNN.\n\n\n**Q6: A minor issue: the per-class IoU performance of PolarMix is provided in its supplementary material, which is missing in Tables 2 and 3.**\n\nWe apologize for missing per-class IoU performance of PolarMix. Here we report them in the following tables and will add them in the revision. \n\nSynLiDAR $\\to$ KITTI (MinkNet)\n\nmehotd|car|bi.cle|mt.cle|truck|oth-v.|pers.|b.clst|m.clst|road|park.|sidew.|oth-g.|build.|fence| veget.|trunk|terra.|pole|traf.|mIoU\n-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-|-\nPolarMix|76.3|8.4|17.8|3.9|6.0|26.6|40.8|15.9|70.3|0.0|44.4|0.0|68.4|14.7|69.6|38.1|37.1|40.6|10.6|31.0\n\nSynLiDAR $\\to$ POSS (MinkNet)\nmehotd|pers.|rider|car|trunk|plants|traf.|pole|garb.|buil.|cone|fence|bike|grou.|mIoU\n-|-|-|-|-|-|-|-|-|-|-|-|-|-|-\nPolarMix|32.6|39.1|25.0|11.9|64.2|5.8|29.6|15.3|44.8|13.3|23.8|10.7|79.0|30.4\n\nCompared with PolarMix, a strong DA method, we observe that traditional AL is more powerful than DA as it allows querying limited data to be labeled. This paper shows that, given limited annotations, DA (i.e., ASFDA & ADA) can achieve on-par performance with fully-supervised counterparts.\n\n**Q7: Limitations: it is suggested to have a separate section dedicated specifically to addressing these limitations.**\n\nThanks for the valuable suggestion. We will add a separate section to discuss the limitations in the revision."}, {"review_id": "PiaLmuwRao", "reviewer": "Reviewer_KrDz", "summary": "This work benchmarks several point selection approaches for label-efficient LiDAR point cloud semantic segmentation. Their proposed criterion leads to better results than existing selection approaches and good generalization with few annotations. They use several settings depending on the accessibility of auxiliary data to overcome the \"cold-sart\" problem.", "questions": "Q1) How do you explain that large networks can be trained with cross entropy and only a handful of points? How many epochs are you using?\n\nQ2) Why not compare to any of the other DA approaches  [25, 35, 58, 59, 78, 84]\n\nS1) proof read the entire text looking for mistakes, vague and imprecise statements.", "limitations": "not given", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "- the proposed criterion is sensible and leads to good results\n\n- the experiments are extensive\n\n- the definition of three different setting (no auxiliary data, pretrained model, auxiliary annotated data) is helpful\n\n- active learning is a critical problem for the industrialization of ML methods and is not studied nearly enough", "weaknesses": "- the entire contribution boils down to the \"voxel confusion degree\" selection criterion. Yet the authors present the entire field of active learning as a contribution\n\n- the quality of writing is quite low, with many misused technical terms (softmax cross-entropy) and vague statements (active learning [is] an optimum paradigm).\n\n- the authors only compared their approach to 2 selection criteria [27,76], but could have used many other approaches, such as  [25, 35, 58, 59, 78, 84]. Otherwise, they need to explain why these approaches do not apply\n\n- The results seem dubious. Annotating only 5 voxels (how many points total?) leads to such strong performance in a setting with many classes. Even when using 40 million parameters- networks trained from scratch and no particular measures taken to avoid the extreme overfitting bound to occur?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the comments from the reviewer KrDz. We have answered all the questions and sincerely hope they can address the concerns.\n\n**Q1: the entire contribution boils down to the \"voxel confusion degree\" selection criterion. Yet the authors present the entire field of active learning as a contribution.**\n\n*Motivation*: Recently, there has been significant interest in achieving label-efficient LiDAR semantic segmentation. Both AL within the same data distribution and DA across different data distributions hold promise as solutions to alleviate the manual annotation burden. Nonetheless, **the absence of a standardized baseline** could hinder the advancement of research. Further, AL coupled with DA has great practical significance, which has been explored in 2D image classification. Yet, **it is still unclear how to make it work well in 3D domains**. \n\n*Purpose*: This paper aims to present a simple and general active learning baseline for LiDAR semantic segmentation via the voxel-centric selection.  With its simplicity and strong performance across various backbones (e.g., MinkNet, SPVCNN, SalasNet, PolarNet, etc.), regardless of in distribution or out of distribution setting (i.e., AL, ASFDA, and ADA), and robustness over simulation-to-real and real-to-real scenarios, we hope this baseline can facilitate future research.\n\n*Contribution*: We would like to emphasize that the main contributions lie in three aspects.\n- a label acquisition strategy (VCD) is **more robust and diverse** to select samples efficiently under a domain shift;\n- a voxel-centric online active learning can largely reduce the labelling cost of enormous point clouds. Particularly, only requiring **1,000x fewer annotations** can reach a satisfactory performance;\n- **generally applicable** for various network architectures (voxel-, range-, and BEV-view, etc), settings (in distribution and out of distribution), and scenarios (simulation-to-real and real-to-real).\n\n**Q2: the quality of writing is quite low, with many misused technical terms (softmax cross-entropy) and vague statements (active learning [is] an optimum paradigm)**\n\nThank you. We will fix minor mistakes and revise paper thoroughly.\n\n**Q3: the authors only compared their approach to 2 selection criteria [27,76], but could have used many other approaches, such as [25, 35, 58, 59, 78, 84]. Otherwise, they need to explain why these approaches do not apply.**\n\nThis question is very interesting and valuable. **First**, we would like to clarify that our goal is to build a simple and general baseline, in which we benchmark several selection strategies, i.e., Random, Entropy[76], and Maigin[27] since they are typically used to testify the proposed selection strategy. **Second**, as mentioned before, there is no standardized baseline. Thus, it might be unfair to compare with some works across different scenarios and settings. Here, we compare our method with other works [25, 35, 58, 59, 78, 84] as much as possible on the basis of the benchmarks presented in this paper. These works can be categorized into three groups: \n\n1. *active learning for 2D image classification[58, 84].* core-set[58] proposes a diversity-based sample selection strategy for image classification, where a clustering algorithm is adopted and DUC[84] estimates uncertainty from the perspective of Dirichlet distribution. We modify these two selection strategies into our baseline and compare with them on two simulation-to-real tasks based on MinkNet. The detailed results are reported in `Table R2 of the one-page PDF`. Please be aware that our method continues to deliver the most favorable outcomes in two case.\n   \n2. *active learning for 3D LiDAR semantic segmentation[25, 35, 59].* Lidal [25] proposed a frame-level selection strategy. Less [35] and SSDR-AL [59] first introduced a pre-segment stage (a heuristic algorithm in [35] and superpoint in [59]) and then conducted labeling process on outdoor and indoor scenes respectively. \n\nWe first compare Lidal and Less on outdoor KITTI dataset as follows. \n\nmehotd|budget|MinkNet|SPVCNN\n-|-|-|-\nLidal|1%|47.5|48.5\nLess|0.1%|52.8|51.1\nOurs|0.1%|53.7|52.8\n\nOur method is simpler yet achieves better performance. Then, following [59], we conduct experiment on indoor S3DIS dataset. Detailed results and analyses can be found at response of Q6 for Reviewer dFZX.\n\n3. *Squeezeseg [78]* is an end-to-end road-object segmentation method with CNNs and CRFs and synthetic data (GTA-V) is used to train the model. It is not applicable to our setting.\n\n**Finally**, we would like to emphasize that your suggestion is very helpful, and we will pay attention to it when revising the paper.\n\n**Q4: the results seem dubious. Annotating only 5 voxels (how many points total?) leads to such strong performance in a setting with many classes. Even when using 40 million parameters-networks trained from scratch and no particular measures taken to avoid the extreme overfitting bound to occur?**\n\nThanks for the valuable question. On average, the total number of annotated points (5 voxels) in a frame (64,000 points in total) is about 35. `Figure R2 of the one-page PDF` provides curves of training loss and validation loss on task of SynLiDAR->POSS under AL (train from scratch) and ASFDA (train from an auxiliary model) settings. Both models are based on MinkNet (21.7 MB). The training loss and validation loss have been steadily decreasing and the final validation loss is smaller than the training loss as well, verifying that there is no extreme overfitting. Another finding is that the validation loss of ASFDA is smaller than that of AL, which confirms the power of auxiliary model. We will add this analysis in the revision.\n\n**Q5: How many epochs are you using?**\n\nThe pre-training process uses 10 epochs and others (AL/ASFDA/ADA) use 50 epochs.\n\n**Q6: limitations: not given**\n\nActually, we have discussed the limitations in the Conclusion. We will make it more clear in a separate section."}]

Author rebuttal:
Dear reviewers and AC,

We sincerely thank all the reviewers for their positive comments and helpful feedback that have certainly helped improve the quality of this paper. We have uploaded the responses w.r.t. each reviewer together with `the one-page PDF`. 

In response to the comments, we have carefully revised and enhanced the manuscript with the following additional discussions and experiments:


1. Add discussion about the cost (computation v.s. annotation) balance in AL paradigm.
2. Provide additional experiments that apply the proposed method to both range-view and BEV-view methods.
3. Provide additional comparisons with existing methods including two AL methods for image classification, two AL methods for outdoor LiDAR semantic segmentation and one for indoor scenes.
4. Add the error bars for the results.
5. Add additional experiments to analyze the effect of voxel size.
6. Provide the analysis of training the network with only a handful of points.
7. We shed light on some potential applications of our Annotator, such as indoor semantic segmentation, LiDAR object detection task.

We hope our response sincerely address all the reviewers' concerns.

Thank you very muck for your time and consideration.

Best regards,

Submission2070 Authors.

Meta-review: This paper introduces intriguing work for active learning for 3D semantic segmentation. Reviewers favor the proposed approach and the demonstrated results (especially the results on the cross-domain experiments). The authors provided comprehensive feedback regarding specific questions, and the authors provided a supplement pdf file. In particular, there were concerns from reviewer KrDz, but AC notes that the authors adequately defend it. After the rebuttal phase, all reviewers voted for the high scores, and AC saw that there were constructive suggestions for the paper. AC recommends paper acceptance. Applying the valuable discussion and additional material in the revision is strongly advised.
Decision: Accept (poster)