Learning Transferable Features for Point Cloud Detection via 3D Contrastive Co-training
Authors: Yihan Zeng
Chunwei Wang
Yunbo Wang
Hang Xu
Chaoqiang Ye
Zhen Yang
Chao Ma
Venue: NeurIPS 2021 Poster (2021)
Keywords: 3D object detection
point cloud detection
transfer learning

Abstract: Most existing point cloud detection models require large-scale, densely annotated datasets. They typically underperform in domain adaptation settings, due to geometry shifts caused by different physical environments or LiDAR sensor configurations. Therefore, it is challenging but valuable to learn transferable features between a labeled source domain and a novel target domain, without any access to target labels. To tackle this problem, we introduce the framework of 3D Contrastive Co-training (3D-CoCo) with two technical contributions. First, 3D-CoCo is inspired by our observation that the bird-eye-view (BEV) features are more transferable than low-level geometry features. We thus propose a new co-training architecture that includes separate 3D encoders with domain-specific parameters, as well as a BEV transformation module for learning domain-invariant features. Second, 3D-CoCo extends the approach of contrastive instance alignment to point cloud detection, whose performance was largely hindered by the mismatch between the fictitious distribution of BEV features, induced by pseudo-labels, and the true distribution. The mismatch is greatly reduced by 3D-CoCo with transformed point clouds, which are carefully designed by considering specific geometry priors. We construct new domain adaptation benchmarks using three large-scale 3D datasets. Experimental results show that our proposed 3D-CoCo effectively closes the domain gap and outperforms the state-of-the-art methods by large margins. 


Learning Transferable Features for Point Cloud Detection via 3D Contrastive Co-training
Yihan Zeng'* Chunwei Wang!" Yunbo Wang'
Hang Xu² Chaogiang Ye²
Zhen Yang² Chao Ma 't
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University
Huawei Noah's Ark Lab {zengyihan, weiwei0224, yunbow, chaoma]@sjtu. edu. {xu.hang, yechaoqinng yang. Sheewweehwwwii.. com
Abstract
Most existing point cloud detection models require large-scale, densely anno- tated datasets. They typically underperform in domain adaptation settings, due to geometry shifts caused by different physical environments or LiDAR sensor configurations. Therefore, is challenging but valuable learn transferable fea- tures between a labeled source domain and a novel target domain, without any access target labels. To tackle this problem, 3D Contrastive Co-training (3D-CoCo) with two technical contributions. First, 3D-CoCo is inspired by our observation that the bird-eye-view (BEV) features are more transferable than low-level geometry features. We thus propose a new co-training architecture that includes separate 3D encoders with dopain- parameters, as well BEV transformation module for learning domain-invariant features. Second, 3D-CoCo extends the approach of contrastive instance alignment to point cloud detection, whose performance was largely hindered by the mismatch between the fictitious distribution of BEV features, induced by pseudo-labels, and the true distribution. The mismatch is greatly reduced by 3D-CoCo with trans- formed point clouds, which are carefully designed by considering specific geometry priors. We construct datasets. Experimental results show that our proposed 3D-CoCo effectively closes the domain gap and outperforms the state-of tfae-offthe--a of-the-art art methods by large margins.
introduce the framework of
domain adaptation benchmarks using three large-scale 3D
Introduction
3D point cloud detection shows remarkable significance in real-world scenarios, such autonomous driving [14, 43, 38, 44], in which the recent progress is largely driven by the emergence of high- precision LiDAR sensors and large-scale, densely annotated point cloud datasets [1, 6, 26J. Most existing 3D detection models assume that the training domain and testing domain are independently and identically distributed. In practice, however, domain shifts often inevitable due differences in physical environments or LiDAR sensor configurations, including different numbers laser beams and installation positions, etc. To address this issue, we present early study of the unsupervised domain adaptation problem for point cloud detection, which aims to effectively adapt 3D detectors from the labeled source domain to novel unlabeled target domain by learning transferable features.
Previous domain adaptation approaches for image data [27, 12, 24, 2] are not readily applicable to point clouds. As shown in Fig. different from the domain shifts of 2D scenes that usually
The first two authors have equal contributions.
i C. Ma is the corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

64-Reoit LIDAR
Figure I: Comparisons of domain shifts between 2D and 3D scenes. Left: Domain shifts in 2D scenes are mainly reflected in appearance variations, egg., weather or environment changes in autonomous driving. Right: 3D domain shifts generally represented geometry variations, which not only arise from external environments, but also come from the internal sensor configurations.
exists in the image appearance, those of 3D scenes are mainly reflected in the geometry variations of point clouds. Since 2D images from different domains have the same grid topology of uniformly distributed pixels, most of the domain adaptation methods exploit image encoders with domain- sharing parameters, which also adopted by existing 3D transfer learning models such as PointDAN [20]. However, due to the severe low-level geometry shifts between different point sets, we believe that some features are transferable while some features are not on 3D object detection, which requires a rethink of the transfcrability of different levels of the point cloud representations. To this end, we propose a novel framework named 3D Contrastive Co-training (3D-CoCo), whose architecture contains separate 3D encoders with domain-specific parameters, a Nomain-agnosste BEV transformation module, and the final detection head. The key idea of the architecture design that the BEV features with similar grid structures to images can be more transferable than low-level 3D features, so that they can be better integrated with advanced transfer learning techniques in 2D vision and thus greatly reduce the geometry shift. Another benefit of the co-training architecture with domain-specific encoders that in addition to improving domain adaptation results, also maintains the in-domain performance.
Furthermore, 3D-CoCo also featured by new end-to-end contrastive learning framework, which contains two main components, i.e., contrastive instance alignment based on bird-eye-vien (BEV) features and hard sample mining with transformed point clouds. The insight of the contrastive instance alignment is to push the feature centroids of similar sample clusters, induced by pseudo-labels, closer each other, no matter they are in the same domain or different domains. Besides, we consider the mismatch between the true distribution of BEV features and the fictitious one used for contrastive learning, which is caused by biased pseudo-labels. Specifically, we take advantage of the editability of point clouds and perform hard sample mining by applying specific transformation functions to 3D data. The hard samples, as an effective supplement to contrastive co-training. can further reduce the geometry shift across domains and prevent the adaptation model from falling into the local minima.
Notably, there exists another line of work discussing transfer learning for point cloud detection [28, 35, 3], which utilizes the self-training pipeline that retrains the model with pseudo-labels on target data. Compared to these approaches, we adopt a different problem setup, using labeled source domain data and unlabeled target domain data for co-training. Through ablation studies, we observe that the separate encoders and the contrastive co-training scheme can progressively filter domain-specific features and learn more transferable knowledge across domains.
We evaluate the effectiveness and generalizability of 3D-CoCo across three widely-used autonomous driving datasets collected by heterogeneous LiDAR sensors, including Waymo [26], nuScenes [1], and KITTI [6]. 3D-CoCo shown to significantly outperform existing approaches for point cloud detection on different unsupervised domain adaptation benchmarks.
2 Preliminaries
Problem setup. The traditional setup of point cloud detection is to learn a base 3D object detector D that classifies and localizes m objects represented by Y from a point cloud P:
D: P Y,
(1)
where P= (p), ............ .P. }m} consists of n points p: Rd scattered over the 3D coordinate space. The dimension d is set to 4, including the coordinate (a, y, 2) and an additional intensity The annotations Y {vi..... }md consists of a set of class labels and bounding box coordinates.

---

BEY Transformation Module
Foeeroum
Baskground
Ground
3D Encoders
Truth
Source Samples
Es
Linter
Lets +L
64 beard LIDAR
Source
Lintra
Head
Target Samples
rey
Er
Uinter
beam LIDAR
Update
Target
Features
Labels
An illustration of the proposed 3D-CoCo model, which contains domain-speciin 3D encoders and performs contrastive adaptation BEV features for instance-level feature alignment.
Figure 2:
which is represented by size (I, w, h), center Cy Cz), and heading angle r of each object. Typically, D is trained in supervised manner by minimizing the classification and localization emross between the ground truth Y and the prediction Ý over the training (source) dataset. In this paper, specifically study the problem unsupervised domain adaptation for point cloud detection, which aims to adapt the detector Dy parametrized by B from a labeled source domain {PPS) ) to an unlabeled target domain {P? Nr. where Ng and Nr are the numbers of training samples:
(2) Compared the typical setup of point cloud detection, we focus more on the performance of the model on the test set of the target domain, which requires additional well-designed modules for learning transferable features.
Dg PS pd Y UY
Modern point cloud detectors. Modern 3D point cloud detectors [38, 43, 14] usually consist of three modules: 3D encoder E, a bird-eye-view (BEV) transformation module U, and a detection head H. The 3D encoder quantizes the point cloud into regular grids for feature extraction. The BEV transformation module, usually in forms of 2D convolutional layers, produces fixed-size BEV feature maps M R xWxLxP where width W and length L. are based the resolution of bins, and F' denotes the channel number. The detection head takes M as inputs and produces detection results Y. To demonstrate the generality of the proposed method, we adopt two mainstream architectures, ie., VoxelNet [43] and PointPillars [14], with different point cloud processing pipelines as alternative options of the 3D encoder. VoxelNet quantizes the point cloud into small 3D voxel features and then uses 3D CNN compress them into 2D BEV space along the height of the voxel, while PointPillars quantizes the point cloud into vertical pillars fixed-size 2D grids and then performs linear transformation and max-pooe max-pooling on each pillar to obtain the BEV representation. Neither of them explicitly considers the domain shift in transfer learning setups.
3 Method
We present 3D-CoCo as feasible solution to the unsupervised domain adaptation task in point cloud detection. It has two contributions to learning transferable features from heterogeneous geometries, which respectively reside the new architecture design, as shown in Fig. 2, and the framework of contrastive instance alignment enhanced by hard sample mining, shown in Alg. and Fig. 3.
3.1 3D-CoCo Architecture
Domain-specinee 3D encoders. Transfer learning in 3D scenes may suffer from dramatic geometry shifts, such density variations and different occlusion ratios point clouds, due different physical environments and sensor configurations. Although some work has explored model pretraining on 3D pretext tasks 30], in contrast with 2D scenarios. 3D vision still lacks transferable, well- pretrained backbone. One possible reason is that it is very difficult to reduce the domain shift in geometric representations at the bottom of the 3D encoder. Intuitively, we expect the 3D detection network to progressively process domain-specine non-transferable features and learn domain-invarian semantic features. As shown Fig. 2, we present novel model architecture with domain-speciff 3D encoders, which learn different mapping functions parse and convert LiDAR points into the

---

Algorithm 1:
The learning procedure of 3D contrastive co-training (3D-CoCo) J=1,
Input:
The labeled point set from the source domain {(P,) )) the unlabeled point set Nr
from the target domain {P:" Ji=1, the maximum number of update stages K
Output: Learned network weights 0
D' + {(P,, ))
b Pretrain the base detector according to Eq. (3) Generate pseudo-labels for target domain samples
2 1 + (D', (P
3 (((P)) .Y,) + HSM (E
D Mine hard samples to augment the target set
4 Dg +
b Initialize the model with the base detector
8 Dg + (((P,) y)) ((P), Y Nr)
Warm-up according to Eq. (6)
for =I...... do
1 + -Dz (P,) (T)
> Update pseudo-labels
(1(P)T |1) + HSM( Y )
Append new hard samples to the target set
D3 < (((P)) N ((P). )
Model update according to Eq. (6)
Dg t D
10
bird-eye-view (BEV) space for different domains. It is worth noting that the co-training architecture not only benefits the adaptation performance on the target domain but also contributes to maintaining the performance on the source domain, in the sense that learning transferable features upon separate encoders can facilitate bi-directional knowledge sharing.
Vomain-agnosto BEV transformation module. The 2D transformation module is co-trained with data samples from both source and target domain. It further compresses the outputs of the domain-speciftr 3D encoders into BEV feature maps M. The BEV features supposed to be more transferable, because they have similar structures to the grid-based feature maps in 2D vision and can therefore be easily integrated into off- the-shelf transfer learning techniques. Based M, perform the contrastive alignment training scheme to encourage the learning of domain invariant features.
Detection head. The detection head classifies and localizes 3D objects from the BEV feature maps M. Given a labeled sample from the source domain, the detection head is trained to minimize
Lett S. S
where LS and th respectively indicates the classification and localization error. For an unlabeled sample from the target domain, we use regularization of the localization error City between the predictions and the pseudo-labels, which encourages a rapid adaptation to the target domain.
3.2 Hard Sample Augmented Contrastive Alignment
Since the features of point clouds are sparsely distributed, it is difficult to achieve effective matching between domains by using global distribution alignment. We thus propose exploit fine- mraigrained alignment at the instance level, which is enhanced by hard sample mining to avoid bad local minima.
Foreground and background proposals. We first construct foreground and background instance- level features for contrastive domain aligmment, respectively. According to the predicted pseudo-
y"=(w7 (vT= ................................ WTT ). we obtain foreground detection proposals on the BEV features. We then apply a keypoint-based feature extraction approach to each proposal. Concretely, we assign R x R x 1 equally distributed keypoints to the proposal box, and then learn the feature of each keypoint by performing bilinear interpolation on the feature map and get the everage-pooling value as the final representation. For background samples, we randomly select grid features from areas outside the proposal boxes on the feature map. It is worth noting that the proportion of blank areas without any points in the background region relatively large, while the proportion of non-blank areas with background points is relatively small. There is a gap between the learned features of these two kinds of background samples. To alleviate the sample imbalance, we equally select certain number of features from these two groups of background samples.

---

06
Hard Sample Mining
0,20
Pacennge
Percuea Age
0.30
6.5
Original
Density transformed
Completeness
Mined Hard Sample
0.
logh
Easy Sample
transformed
Point object N
Occlusio
(a)
Source
Fereground
Push different cateegory away
Target
Background
Pull same calegory closer
Decision Boundary
Easy Samples
Hand Samplees
Contrastive Alignment
Hard Sample Augmented Contrastive Alignment
Mined Hard Sample
Area Reduction
Neglected
Timited
(d)
Aligament
(c)
Aligament
The idea of the hard sample augmented contrastive alignment. (a-b) A simple use of contrastive alignment introduces the mismatch in point density and occlusion ratio between sample distributions of pseudo-labed and ground truths in the target domain. (c) Hard sample mining transforms point clouds by considering the specific geometry mismatches.
Figure 3:
(d) The original contrastive alignment focuses more the alignment of easy samples in 3D scenes rather than the easily neglected hard samples with severe occlusions density variations. (e) The contrastive alignment scheme of 3D-CoCo is effectively augmented by the transformed hard samples to achieve further alignment.
Contrastive instance aligmment induced by pseudo-labels. The core idea of contrastive align- ment is to drive the feature centroids of similar samples closer between domains. Specifically, choose the positive instance sample pair (I,,T) based on a similarity-prion criterion as follows:
(i) arg max ( I7)), 1sis Nº, 1,2, ..2......C... |C/,
(4)
ISJSNT
where N denotes the total number of source samples in category and |C| denotes the total number of categories. For each 7:5. finds positive sample 77. from the target domain under the same category c, where (xx, y) = | calculates the cosine similarity between features source sample and a target candidate within the same category. In addition to minimizing the intra-class distance of the same category between domains, we also constrain the inter-class distance between different categories within the same domain. The objective of contrastive alignment is formulated as
x-y
[C]
exp(1 15./T)
Cintro-class (S,T)
log exp(Ie 1T./T) + So cxp(I 1T)
c=l
N
10|
exp(1e 15/
Cinter-class (S)
Σ
log
exp(Ts 1//T) + EjEN exp(Ts 155
(5)
iENe J iN
IC1
exp(IT' 1T//T
Cinter-class (T)
Σ
log exp(IT -15 EjENT exp(1, 17)'
VENT. ENT .t1
Coda = Cintraclass (S,T) Linee-etass (S) Ciner-class (T),
where denotes temperature hyper-parameterr which is set to 0.07 in our experiments. In the total adaptation loss Luda- all pairwise relations of samples between domains and within domains are con- sidered, which enhances both the intra-clas class transferability well the inter-class discriminability. Finally, by combining the loss terms Eq. (3), Eq. (5), and the regularization term for target domain fieg we optimize the model Da by
arg min Cast + ALuuaa + reg'
(6)
5

---

where À is the hyper-paramann parameter for the domain adaptation term, set as 0.5 in experiments.
Transformed point clouds as hard samples. A straightforward use of the contrastive instance alignment tends to introduce the mismatch between the sample distributions obtained by pseudo- labels and ground truths on the target domain. First, shown in Fig. 3(a), pseudo-labels are more concentrated in the patterns with dense point clouds than those with sparse point clouds. Second, as shown in Fig. 3(b), pseudo-labels cannot completely cover the patterns of severe occlusions. Therefore, most instances induced by positive pseudo-labed can be viewed as "easy samples" with sufficient points complete geometry. However, we believe that the neglected "hard samples", which are more likely to be distributed in the marginal area shown in Fig. 3(d), are equally important to 3D transfer learning. As shown in Fig. 3(e), mining the hard samples can further promote distribution alignment and prevent the model from overfitting bad local minima. The key to creating fictitious hard samples is to consider the priors of geometry variations indicated by Fig. 3(a-b). We here propose two mechanisms to create fictitious hard samples. As shown in Fig. 3(c), the first transformation method uniformly discards the points from existing dense point clouds, which simulates the change of the number of laser beams. The second method simulates object occlusions by breaking the complete geometry of easy samples. Concretely, we calculate the viewpoint of a certain sample, randomly select a part of the viewpoint, and discard the point cloud on these angles. Compared to common augmentation strategies such as rotation and flipping that were previously applied to 3D detection [38, 32], the transformed point clouds focus on effective contrastive instance alignment by reducing the distribution mismatch of the target domain induced by pseudo-labels, rather than aiming at enriching sample diversity of the source domain.
Overall training procedure. We propose step-wise training procedure with a warm-up process as shown in Alg. 1. Specifically, we first pretrain a source detector on the labeled source domain and use it to generate pseudo-labels on the target set. We then conduct hard sample mining (HSM) and augment the target set. Next, we warm up the 3D-CoCo detection model following Eq. (6), which allows more stable convergence in the early stages of training. For the remaining epochs, we update the pseudo-labels using the ensembling and voting mechanism [35]. Throughout step-wise co-training, Dg gradually adapts to the target domain while maintaining the in in-domain performance.
4 Experiments
4.1 Experimental Setup
Datasets. We evaluate 3D-CoCo three widely used LiDAR-based datasets, including Waymo [26]. nuScenes [1], and KITTI [6]. Each dataset has specific properties in external en- viromments (i.e., traffic condition) and internal sensor configurations (i.e., number of beams), so there exist huge domain gaps between them. Specifically, Waymo is collected in the United States with multiple weather conditions throughout the day using 5 LiDAR sensors. The nuScenes dataset is collected in the United States and Singapore by a 32-beam LiDAR sensor. KITTI is collected in Ger- many in sunny daytime by a 64-beam LiDAR sensor. We construct 4 domains adaptation benchmarks between datasets including: (i) Waymo Waymo--nSeenes,, (ii) nuScenes- Waymo, (iii) Waymo-KITT and (iv) KITTI.. We use the common category of the three datasets, i.e., Car/Vehicle. Here, KITTI is only used as the target domain as it is much smaller than the other two datasets.
Compared models. As shown in Table 1, 3D-CoCo is firstly compared with the "Source Only" model, which is trained with only source domain data. We include two existing approaches for cross-domain 3D detection: SN [28] normalizes the object size of the source domain by leveraging the object-level statistics of the target domain. ST3D [35] is self-training pipeline that achieves the state-of-the-art domain adaptation results by using pseudo-labels of target data for retraining. We re-implement SN and ST3D on the same base detector as ours. Finally, 3D-CoCo is also compared with the "Oracle" model, which is trained with labeled target domain data, to roughly represent the optimal performance of an adaptation model on the target domain.
Evaluation metrics. We use the average precision (AP) as the evaluation metric for both BEV IoUs and 3D IoUs under an IoU threshold of 0.7. We also adopt the domain adaptation metric named Closed Gap from Yang et al. [35], which is defined as AP AP
AP AP 100%, the higher, the better.
Implementation details. We follow Yin et al. [38] to build the base detector with two alternative 3D encoders including VoxeINet and PointPillars. We set the voxel size to (0.1, 0.1, 0.15)m for

---

VoxelNet
PointPillars Closed Gap
Task
Method
AP
Closed Gap
AP30
Closed Gap
APpav
AP
Closed Gap
Source Only
46.7
11.1
22.8
0.5
SN [28]
35.4
130.13%
22.9
+19.41%
39.3
+26.61% %
2.0
+2.11%
- ST3D [35]
54.4
+20.53%
38.3
444.74%
60.4
+00.55%
11.1
+14.91%
3D-CoCo
77.1
+81.07
65.6
89.64%%
77.0
+87.42%
47.2
+65.68%
Oracle
84.2
84.8
71.6
Source Only
55.1
18.9
47.8
11.5
SN [28]
50.6
15.46%
36.7
+33.58%
27.4
55.14%
6.4
8.49%
W + K ST3D [35]
72.6
+60.14%
54.8
+67.74%
58.1
+27.84% 87.84%
23.2
+ +19.47%
3D-CoCo
77.0
+75.26%
61.9
+81.13%
76.1
+76,49% %
42,9
+52,25 %
Oracle
84.2
71.9
84.8
71.6
Source Only
32,4
21.1
27.8
12.1
ST3D [35]
32.8
+2.34%
21.1
+0.00%
30.6
+13.21%
15.6
+ 18.23%
W ; N
3D-CoCo
39.2
+39.77
25.4
+37.07%
33.1
+25.00 %
20.7
+-4..79 %
Oracle
49.5
32.7
49.0
31.3
Source Only
40.2
22.9
28.1
8.5
ST3D [35]
42.7
6646%
29.0
11.78%
35.1
15.63%
14.0
13.13%
N-W
3D-CoCo
56.7
+ +42.64% %
33.0
+19.50%
50.3
+49.55%
29.5
+50.114
Oracle
78.9
74.7
72.9
50.4
Table 1: Results in average precision and the corresponding closed gaps for unsupervised domain adaptation. Please see the text for the definition of the metric. N: nuScenes; K: KITTI; W: Waymo.
VoxelNet and (0.1,0.1)m ym for PointPillars. We use the Adam optimizer [13] with a learning rate of 1.5 x 10-3 We set the maximum number of training epochs to 30 for KITTI and 20 for Waymo and nuScenes, with warm-up process taking half of the total epochs. For pseudo-labels generation, we apply high-pass threshold of 0.7 to IoU to obtain foreground samples, and a low-pass threshold of 0.2 for background samples. To reduce the domain shift of object size between datasets [28], we use the random object scaling (ROS) strategy [35] with a scaling factor in the range of (0.75, 0.9] when adapting the model to KITTI. In this way, different from Statistical Normalization (SN) [28]. our approach does not require accurate prior knowledge of the target domain statistics.
0.1,)m
4.2
Main Results
As shown in Table 1, 3D-CoCo outperforms all compared models by large margins on all adaptation benchmarks. Especially on BuScenes-*KKTe and Waymo->KIT based on the VoxelNet backbone, 3D-CoCo closes the domain gap in APaD by around 81% 89%. Furthermore, for the adaptation tasks between the two large-scale datasets, i.e., Waymo and nuScenes, 3D-CoCo also achieves considerable improvement, closing the domain gap in AP3D by 37% on VoxelNet and 50% on PointPillars. Notably, despite taking 3D domain shift into account, SN and ST3D achieve relatively small improvements under the domain adaptation setups, or even have a negative effect on the base model (source only). In contrast, although starting with low-quality pseudo labels, 3D-CoCo still performs well due to the effective co-training that incorporates labeled source data and augmented hard samples. The overall results validate the transferability of 3D-CoCo on different unsupervised domain adaptation benchmarks, and its ability to generalize to different detection networks.
4.3 Ablation Studies
Architecture designs. All ablation studies are conducted on nuScenes-<KITH using VoxeINet as the network backbone. First, Table 2(I) compares the results of using different parameter-shaie sharrngg strategies in the model architecture. By replacing the domain-specific 3D encoders of 3D-CoCo with
Target
Source
Target
Source
Method
Method
APpay
AP
APpay
AP
APpay
APs
APpay
AP
Source Only
45.2
32.6
32.2
19.4
Source Only
51.4
32.2
44.0
28.0
())
(II)
IE + 10
74.0
59.1
24.5
14.3
1E + IU
73.5
53.7
39.1
25.5
2E + 2U
77.5
60.3
31.6
18.1
64.0
26.0
43,5
23.5
2E IU (Ours)
77.1
65.6
30.6
19.6
2E I0 (Ours)
77.1
55.6
44.6
30.0
Table 2: Ablations on the architecture design. 1E and 2E respectively denotes using domain-sharing
3D encoder and separate domain -spncifc encoders. 1U/2U denotes using shared/separate BEV transformation module(s). All models are trained in the proposed contrastive alignment framework with the random object scaling technique. In (I), the ROS scaling factor ranges in (5,75, 0.9); (II,, it ranges in (0.75, 1. We report both in-domain and cross-domain performance.

---

Bkgd
Sim
Proto
APBEV
AP
Rand
Unif
Pers
APsev
APaa
76.5
62.1
(e)
74.0
58.8
(b)
63.3
(1)
74.2
61.3
77.1
65.6
(g)
76.2
62.5
(d)
72.7
59.2
(h)
76.0
61.4
Table 3: Ablations on contrastive alignment
77.1
65.6
schemes. Bkgd: The balanced background sam- pling strategy; Sim: Similarity -priority criterion;
Table 4: Ablations on hard sample mining. Rand:
Dropping points randomly; Unif: Dropping points uniformly; Pers:
Proto: Prototype-level alignment opposed to the instance-leve alignment.
Dropping points of certain per- spectives simulate occlusions.
baseline tne
CT- wuw.utttuto
10
15
20
MSM
0
10
15
01
10
15
20
25
[al)
(bl)
(cl)
ST+updece
5T
0.8
0.2
0.6
seseline
0.4
0.0
10 20 30
15
10 15
20
25
(n2)
(b2)
(c2)
Figure 4: Comparisons of self-training (ST) and the proposed co-training (CT) methods. (al-a2)
Training without updating pseudo-labels. (b1-b2) Training with progressively updated pseudo-labels. (cl-c2) Training with pseudo-labels filtered by different confidence scores. AP3d indicates the detection accuracy. The ratio of TPs/FPs indicates the detection noise. HSM: Hard sample mining.
domain-sharing encoder, observe that the in-domain performance in APan drops by 6.5% and the cross-domain performance drops by 5.3%, which indicates the difficulty of learning transferable features raw point clouds due to the low-level geometry shift. We further evaluate a baseline model that contains separate BEV transformation modules, and find that the domain adaptation performance yields 5.3% drop in AP3D. It demonstrates the transfcrability of the BEV features. Besides, our model can work well with ROS, which is designed reduce the object size bias on the target domain but inevitably degrades the localization accuracy the source domain. With different values of the ROS scaling factor, as shown Table 2(II), our model consistently achieves the performance gain in both in-domain and cross-domain evaluation setups.
Contrastive learning schemes. By comparing the baseline models (a) and (b) in Table 3, observe that the balanced background sampling strategy effectively improves the adaptation results. By further including model (c) into comparison, which is the final proposed model, validate the effectiveness of using the similarity-priorie criterion. We further compare 3D-CoCo with existing prototype-level alignment methods [18, 40] that calculate normalized features of all samples for each category as category-level prototypes for alignment. From Table 3, due to the ambiguity of the prototypes, the baseline model (d) performs much worse than the proposed model (c) with instance-level alignment. Furthermore, as shown in Table 4, hard sample mining (i) significantly boosts the performance of the original algorithm of contrastive instance alignment (c) with a 6.8% mAP. By comparing the models (e-g-i), we can see that the two transformation methods of uniformly dropping and occlusion simulation progressively improve the model performance.
Comparisons with the self-training pipeline. We compare our co-training procedure, denoted as CT in Fig. 4, with self-training denoted as ST, based on the same initialized pretrained source model. In Fig. 4(al), without updating pseudo-labels, both models fluctuate the early training stage but 3D-CoCo converges faster and more stably, with higher performance than the self-train self-training model.

---

Fig. 4(a2) shows the ratio of true positive and false positive predictions, denoted as TPs/FPs. It indicates that our co-training approach derives a lower detection noise than the self-training baseline. Besides, by updating pseudo-labels in the training process, as shown in Fig. 4(b1-b2), the proposed co-training framework consistently outperforms self-training in detection accuracy, and yields an extremely low detection noise with gradually improved pseudo-labels. At last, we conduct sensitivity analyses of pseudo-labels that are filtered by different confidence scores, where lower scores bring in more noisy labels while higher ones tend to miss the positive labels. As shown in Fig. 4(c1), since self-training is fully dependent on pseudo-labels, it is more sensitive to the filtering scores, while our co-training framework is more robust to the quality of pseudo-labels.
5 Related Work
3D point cloud detection. LiDAR-bused 3D detectors aim to localize and classify 3D objects from point clouds, which can be broadly grouped into two categories: point-based and grid-based. The point-based methods [25, 37, 36] take raw points as input and apply PointNet [19] to extract point- wise features and generate proposals for each point. The grid-based methods [32, 33, 14, 34, 43, 38] propose to convert point clouds into regular grids as the model input, in which Voxelization [43, a common technique to map point clouds into regular 3D voxels. Other methods quantize point clouds into certain kinds of 2D views, such as the bird-eye view [14, 34, 33J and range view [17, 4j. Compared to point-based methods, they are more efficient, accelerating the training on large-scale datasets such as nuScenes [1] and Waymo [26]. In this work, for computational efficiency, we adopt VoxelNet [43] and PointPillars [14] based on the anchor-free free detection head [38] as the base detector.
2D unsupervised domain adaptation. A variety of solutions have been proposed in 21) vision tasks including classification [5, 10), detection [12, 2] and segmentation [45], which can be roughly divided into two groups: distribution alignment [5, 10, 12] and self-training [45, 9, 11]. For the first group, adversarial learning [15, 7, 12] are leveraged to perform alignment in the feature space. Furthermore, contrastive learning has also been employed for fine- fine-grained feature alignment [10, 31, 24. 40]. Besides, some works borrow image translation techniques to performing alignment at the pixel level [23, 16, 8]. As for the second group, the self- celf-training methods [45, 11, 2, 21] generally assign pseudo-labels to guide the re-training process on the target domain. Compared to these methods, 3D-CoCo originates from the unique properties of the 3D geometry shift. By using the domain-specific encoders, domain-agnoss BEV transformation module, and the transformed point sets, it effectively extends the original contrastive adaptation methods to 3D object detection.
3D unsupervised domain adaptation. Some recent works effectively reduce the domain shift in point cloud classification [20] and semantic segmentation [29, 42]. In this paper, we focus on the domain adaptation task of 3D object detection, which has been discussed by only few works. The statistical normalization method [28] closes the 3D domain shifts by normalizing the object size of the source domain with known statisties of the target domain, which is generally unavailable in unsupervised adaptation tasks. To tackle this issue, SF-UDA [22] uses the temporal coherency to estimate the scales in the target domain, and re-trains the detection model on the target data by transforming the scales of pseudo-labels. Current works [35, 39, 3] further explore the generation mechanisms of pseudo-label, on target data as the supervision signals for self traininni Since the adaptation process of self self-training methods can be misguided by noisy pseudo-labels, those methods exploit sophisticated strategies improve the quality of pseudo-labels. Compared to self-traing self-training. 3D-CoCo leverages a co-training framework in which the labeled source data can provide more stable supervisions to the detection model in the adaptation process.
Conclusion
In this paper, we presented 3D-CoCo for unsupervised domain adaptation of point cloud detection. 3D-CoCo contains a novel model architecture and a new contrastive learning framework. Based on the observation that BEV features are more transferable than low-level geometry features in 3D scenes, we innovatively proposed to integrate domain -spaaffcc 3D encoders with BEV transformation module. We then conducted contrastive instance alignment, which is augmented by hard sample mining, on the BEV features. The experiments across three autonomous driving datasets showed the effectiveness of 3D-CoCo. As a pilot work for the transfer learning problem of 3D point cloud detection, we follow the typical training setup of unsupervised domain adaptation, which takes more memory footprint than existing self-training methods at training time.
-amaiin-ggnottic

---

Acknowledgements. This work was supported by NSFC (61906119, U19B2035), Shanghai Mu- nicipal Science and Technology Major Project (202ISHZDZX0102), Wang was supported by Shanghai Sailing Program and CAAI-Huawei MindSpore Open Fund.
References
[1] Holger Cacsar, Varun Bankiti, Alex H. Lang. Sourabh Vora, Veniee Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In CVPR, pages 11621-11631, 2020.
[2] Qi Cai, Yingwei Pan. Chong- Wah Ngo. Xinmei Tian, Lingyu Duan, and Ting Yao. Exploring object relation mean teacher for cross-domain detection. CVPR, pages 11457-11466, 2019.
[3] Benjamin Caine, Rebecca Roelofs, Vijay Vasudevan, Jiquan Ngiam, Yuning Chai, Zhifeng Chen, and Jonathon Shlens. Pseudo-labeling for scalable 3D object detection. arXiv preprint arXiv:2103.02093 2021.
[4] Lue Fan, Xuan Xiong, Feng Wang. Naiyan Wang, and Zhaoxiang Zhang. Rangedet: In defense of range view for LiDAR-based 3D object detection. arXiv preprint arXiv:2103. 10039, 2021.
[5] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In ICMI,, pages 1180-11899 2015.
[6] Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the KITTI vision benchmark suite. In CVPR, pages 3354-3361, 2012.
[7] Cheng-Chun Hsu, Yi-Hsuan Tsai, Yen-Yu Yu Lin, and Ming-Hsuan Yang. Every pixel matters: Center-aware feature alignment for domain adaptive object detector. In ECCV, pages 733-748, 2020.
[8] Han-Kai Hsu, Chun-Han Yao, Yi-Hsuan Tsai, Wei-Chih Hung, Hung- Yu Tseng, Maneesh Singh, and Hsuan Yang. Progressive domain adaptation for object detection. In WACV, pages 749-757, 2020.
[9] Naoto Inoue. Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa. Cross-domain weakly- supervised object detection through progressive domain adaptation. In CVPR, pages 5001-5009, 2018.
[10] Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann. Contrastive adaptation network for unsupervised domain adaptation. In CVPR, pages 4893-4902, 2019.
[11] Mchran Khodabandeh, Arash Vahdat, Mani Ranjbar, and William G Macready, A robust learning approach to domain adaptive object detection. In ICCV, pages 480-490, 490, 2019.
[12] Taekyung Kim, Minki Jeong, Seunghyeon Kim, Seokeon Choi, and Changick Kim. Diversify and match: domain adaptive representation learning paradigm for object detection. In CVPR, pages 12456-12465. 2019.
[13] Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6900 2014.
[14] Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Osear Beijbom. Pointpillars: Fast encoders for object detection from point clouds. In CVPR, pages 12697-12705, 2019.
[15] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. arXiv preprint orXiv: 10667, 2017.
[16] Xinhong Ma. Tianzhu Zhang, and Changsheng Xu. Gean: Graph convolutional adversarial network for unsupervised domain adaptation. In CVPR, pages 8276, 2019.
[17] Gregory P Meyer, Ankit Laddha, Eric Kee, Carlos An efficient probabilistic 3d object detector for autonomous driving. In CVPR, pages 12677-12686, 2019.
and Carl K Wellington. Lasernet:
[18] Yingwei Pan, Ting Yao, Yehao Li, Yu Wang. Chong-Wale Ngo, and Tao Mei. Transferrable prototypical networks for unsupervised domain adaptation. In CVPR, pages 2239-2247, 2019.
[19] Charles R Qi, Hao Su, Kaichun Mo, and Lconidas J Guibas. Pointnet: Deep learning on point sets for 3D classification and segmentation. In CVPR, pages 652 660, 2017.
[20] Can Qin. Haoxuan You. Lichen Wang, C-C Jay Kuo, and Yun Fu. Pointdan: A multi-scaee 3D domain adaption network for point cloud representation. arXiv preprint arXiv:1911.02744.22 2019.
10

---

[21] Aruni RoyChowdhury, Prithvijit Chakrabarty. Ashish Singh, YouYoung Jin, Huaizu Jiang, Liangliang Cao. and Erik Learned-Miller. Automatic adaptation of object detectors domains using self-training. In CVPR. pages 780-790, 2019.
[22] Cristiano Saltori. Stéphane Lathuiliére, Nicu Sebe, Elisa Ricci, and Fabio Galasso. SF-UDA ": Source-free unsupervised domain adaptation for LiDAR-based 3D object detection. In 3DV, 2020.
[23] Swami Sankaranarayanan, Yogesh Balaji, Carlos D Castillo, and Rama Chellappa. Generate adapt: Aligning domains using generative adversarial networks. In CVPR, pages 8503-8512, 2018.
[24] Astuti Sharma, Tarun Kalluri, and Manmohan Chandraker. Instance level affinity- transfer for unsupervised domain adaptation. arXiv preprint urXiv:2104.01286 2021.
[25] Shaoshuai Shi, Xiaogang Wang. and Hongsheng Li. Pointrenn: 3D object proposal generation and detection from point cloud. In CVPR, pages 770-779, 2019.
[26] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurclien Chouard, Vijaysai Patniik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, al. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, pages 2446-2454, 2020.
[27] Vihashan VS, Poojan Oza, Vishwanath A Sindagi, Vikram Gupta, and Vishal Patel. Mega-cda: Memory guided attention for category-aware unsupervised domain adaptive object detection. arXiv preprint arXiv:2103.0422 04224. 2021.
[28] Yan Wang, Xiangyu Chen, Yurong You, Li Erran Li, Bharath Hariharan, Mark Camphell, Kilian Q Weinherger, and Wei-Lun Chao. Train in germany, test in the usa: Making 3D object detectors generalize. In CVPR, pages 11713-11723 2020.
[29] Bichen Wu, Xuanyu Zhou, Sicheng Zhao, Xiangyu Yue, and Kurt Keutzer. Squeezesegv2: Improved model structure and unsupervised domain adaptation for road-object segmentation from LiDAR point cloud. In ICRA, pages 4376-4382, 2019.
[30] Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi, Leonidas Guibas, and Or Litany. Pointcontrast: Unsupervised pre-training for 3D point cloud understanding. ECCV, pages 574-591, 2020.
[31] Minghao Xu, Hang Wang, Binghing Ni, Qi Tian, and Wenjun Zhang. Cross-domain detection via graph-induced prototype alignment. In CVPR, pages 12355-12364, 2020.
[32] Yan Yan, Yuxing Mao, and Bo Li. 18(1)):3337, 2018
Second: Sparsely embedded convolutional detection. Sensors,
[33] Bin Yang, Ming Liang, and Raquel Urtasun. HDNet: Exploiting hd maps for 3D object detection. In CoRI., pages 146-155, 2018.
[34] Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Real-time 3D object detection from point clouds. In CVPR, pages 7652-7660, 2018.
[35] Jihan Yang, Shaoshuai Shi, Zhe Wang, Hongsheng Li, and Xiaojuan Qi. ST3D: Self-training for unsuper- vised domain adaptation on 3D object detection. arXiv preprint arXiv:2103.005 2021.
[36] Zetong Yang, Yanan Sun, Shu Liu, and Jiaya Jia. 3DSSD: Point-based 3D single stage object detector. In CVPR, pages 11040-11048, 2020.
[37] Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Std: Sparse-to-dense 3D object detector for point cloud. In ICCV, pages 1951-1960, 2019.
[38] Tianwei Yin, Xingyi Zhou, and Philipp Krähenbühl. Center-based 3d object detection and tracking. arXiv preprint 11X.:20006.122555 2020.
[39] Yurong You, Carlos Andres Diaz- Yan Wang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger. Exploiting playbacks in unsupervised domain adaptation for 3D object detection. arXiv preprint arXiv:2103. 14198, 2021.
[40] Xiangyu Yue, Zangwei Zheng, Shanghang Zhang, Yang Gao, Trevor Darrell, Kurt Keutzer, and Alberto San- giovanni Vincentelli. Prototypical cross cross-domain self-supervised learning for few-shot unsupervised domain adaptation. arXiv preprint arXiv: 100.::6633.6665 2021.
[41] Zaiwei Zhang, Rohit Girdhar, Armand Joulin, and Ishan Misra. Self supervised pretraining of 3D features any point-cloud, arXiv preprint arXiv:2101.0269 2021.

---

[42] Sicheng Zhao, Yezhen Wang, Bo Li. Bichen Wu, Yang Gao, Pengfei Xu, Trevor Darrell, and Kurt Keutzer. epointda: An end-to-end simulation-to-rall domain adaptation framework for LiDAR point cloud segmentation. arXiv preprint urXiv:2009. 03456, 2020.
[43] Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In CVPR. pages 4490-4499, 2018.
[44] Benjin Zhu, Zhengkai Jiang. Xiangxin Zhou, Zeming Li, and Gang Yu. balancal grouping and sampling for point cloud object detection. arXiv preprint arXiv:1908 09492, 2019.
[45] Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang. Unsupervised domain adaptation for semantic segmentation via balance self-training. In ECCV, pages 289-305, 2018.
12


Reviews:
[{"review_id": "iY10_jtjWSg", "reviewer": "Reviewer_2DjX", "summary": "This paper contributes as an early study of unsupervised domain adaptation (UDA) on 3D point cloud detection, an important step towards zero cost of labeling. Different from 2D image data that have regular grid structures, 3D point cloud data could have huge geometric variations such as point density and occlusion due to different physical environments or LiDAR sensor configurations; consequently, the low-level features of point cloud representation may be not transferable across domains. Taking it into account, this paper uses two individual encoders for the source and target domains to capture domain-specific characteristics and a shared Bird-Eye-View (BEV) transformation module to learn domain-agnostic BEV features, which have similar grid structures as images and thus are more transferable. Specifically, this paper extends contrastive instance alignment to point cloud detection, based on BEW features of foreground and background proposals. Direct use of contrastive instance alignment can cause the mismatch in point density and occlusion ratio between pseudo-labeled and true samples distributions in the target domain. To address it, this paper proposes to augment positive easy target samples (with dense points and complete geometry) by their artificial counterparts with sparse points and occlusion. Moreover, this paper constructs a new benchmark using three large-scale datasets. Empirical results show that the proposed method is effective and achieves the new state of the art.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Although the contributions and novelty of this paper are considerable, there are still many problems.\n\n1. The full name should be mentioned when BEV is shown for the first time in Section 1, where it would be better to provide a brief description of BEV and its related references for a wider range of audiences.\n\n2. It would be better to compare the high-level 3D features with the BEV features. The high-level 3D features are extracted by the same but deeper network layers than the low-level ones. The 3D domain shift is mapped into the 2D one, causing information loss.  Would it reduce the feature discriminability? Would it affect the single-domain detection performance, e.g., cases of a general detection setup in individual domains? It is also interesting to know how the features extracted by spherical convolutions [a] behave.\n\n3. The idea of separate encoders is similar to that of [b,c,d] for 2D UDA. The authors should consider them as related works and make comparisons and discussions appropriately. \n\n4.  It is interesting to approximately quantify the distribution discrepancy across domains based on the performances of passing source samples through the target encoder and passing target samples through the source encoder. In Section 4.3, the in-domain performance is evaluated on the target domain; however, according to the previous descriptions, the cross-domain performance should be evaluated on the target domain. There exists a contradiction and a clarification is required.\n\n5. Does the \"co-training\" in this paper mean two individual encoders for the source and target domains? Or the training strategy? This paper also adopts the pseudo-labeling technique from self-training. In Lines 56-59, the difference is not clear enough. \n\n6. In Lines 109-112, \"bi-directional knowledge sharing\" is similar to multi-task learning [e]? Why does it hold? Are there any related works that support this argument? In the ablation studies, it seems that the random object scaling has a great effect on the source performance and this has nothing to do with the proposed method. \n\n7. In Algorithm 1, this paper conducts three-stage training. why not two stages? The warm-up stage introduces another set of training hyperparameters. Besides, the augmented target set is denoted by the original one? The notations should be different.\n\n8. It would be more clear to expand the losses of L_cls^S, L_loc^S, and L_reg^T.\n\n9. In Section 3.2, this paper uses a keypoint-based feature extraction approach. Comparisons with other kinds of feature extraction approaches are needed. Besides, the model sensitivity to the hyperparameter R is to be examined.\n\n10. In Eq. (5), some notations are confusing, such as the summation range in the denominator in L_intra and L_inter. A negative sign is missing in L_intra and L_inter? Otherwise, samples of the same classes are pushed away? In L_intra, samples of other classes should be used as negative ones whereas this paper does not make it clear. In L_inter, other samples in the same class are used as negative ones? Is this the typical contrastive learning method of instance discrimination from self-supervised learning? How can this method increase the inter-class distance between different categories within the same domain? Are there any references for contrastive instance alignment? The main component of the proposed algorithm is so ambiguous that the experimental results may be not so convincing. \n\n11. In Eq. (5), the domain \\mathcal{D} is similar to model D; it would be better to use different notations. In Eq. (6), \"arg\" is redundant. The model sensitivity to the temperature \\tao and the hyperparameter \\lambda is to be examined. \n\n12. In Line 159, is there a specific rule to distinguish between easy and hard samples? Please make it clear. In Line 166, how many points are discarded? In Line 169, how to calculate the viewpoint of a certain sample? How many viewpoints do you select to discard? Are there any solid principles to make decisions? Are there any related references? Please make it clear. \n\n13. In Lines 170-173, the proposed augmentation can also be done in the source domain since source samples can be viewed as easy ones. Consequently, the distribution mismatch between the source and target domains can be further reduced. It is interesting to see whether the model performance can be further improved. The proposed augmentation strategy is somewhat similar to consistency regularization [f,g] from semi-supervised learning, which enforces consistent predictions between weakly and strongly augmented versions of the same sample image. Maybe the related works should be considered and discussed.\n\n14. In Line 179, can you describe how to update the pseudo labels in detail? \n\n15. Is each domain split into training and test sets? Is the performance evaluated on the training or test set of the target domain?\n\n\n[a] H. Lei, N. Akhtar and A. Mian, \"Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2020.2983410.\n\n[b] Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc., Red Hook, NY, USA, 343–351.\n\n[c] E. Tzeng, J. Hoffman, K. Saenko and T. Darrell, \"Adversarial Discriminative Domain Adaptation,\" 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2962-2971, doi: 10.1109/CVPR.2017.316.\n\n[d] W. Chang, H. Wang, W. Peng and W. Chiu, \"All About Structure: Adapting Structural Information Across Domains for Boosting Semantic Segmentation,\" 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 1900-1909, doi: 10.1109/CVPR.2019.00200.\n\n[e] Caruana, R. Multitask Learning. Machine Learning 28, 41–75 (1997). https://doi.org/10.1023/A:1007379606734.\n\n[f] D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel. Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring. In Proc. Int. Conf. on Learn. Rep., 2020.\n\n[g] S. Roy, A. Siarohin, E. Sangineto, S. R. Bulò, N. Sebe and E. Ricci, \"Unsupervised Domain Adaptation Using Feature-Whitening and Consensus Loss,\" 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 9463-9472, doi: 10.1109/CVPR.2019.00970.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Annotating point clouds is more difficult and expensive than annotating images for object detection tasks. Good 3D UDA methods are desired to reduce the annotation efforts. On the constructed new benchmark of three autonomous driving datasets, this paper achieves a significant gain in detection performance, one step closer to the goal. As for the limitation, this paper has mentioned that the proposed method takes more memory footprint than existing self-training methods at the training time whereas does not address it. The potential negative societal impact of their work has not been mentioned. The suggestions for improvement are provided in Main Review.", "review_text": "", "extra_scores": {"ethical_concerns": "There are no ethical issues with this paper.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "dye4q5qlMwS", "reviewer": "Reviewer_RhUF", "summary": "The paper proposed an unsupervised domain adaptation method for point cloud detection. The method is built on several effective strategies, including BEV feature representation, pseudo labels, instance contrastive alignment and hard mining.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Novelty: the instance contrastive joint training is interesting and novel according to my knowledge.\n\nSignificance: It clearly improves the baseline (trained only on source domain) and outperforms the existing unsupervised domain adaptation methods.\n\nPost Rebuttal: My concerns about pseudo labels have been addressed in the rebuttal. My final opinion: this is a valuable paper but not a breakthrough in the field.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1) Details about the pseudo label generation need to be described.  How did you run Eq.(4)?\n\n2) How to process or avoid the possible errors of the pseudo labels? How do such pseudo-label errors influence the final results? It’s good to have some analysis about the relationship between the quality of the pseudo labels and the final performance of the proposed model.\n\n3) Synthetic/simulation-to-real generalization experiments are interesting. If possible, it would be nice to see some tests under this setting.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "_FO5Nv6f-R", "reviewer": "Reviewer_qi8q", "summary": "This work presents an unsupervised domain adaptation for point cloud detection in the autonomous driving scenario, which employs a domain-specific 3D encoder for domain-specific feature extraction, a shared BEV feature transformation module for domain-invariant instance-level feature alignment. It adapts the contrastive instance alignment with geometry priors to align the BEV feature distributions induced by pseudo-labels and the true distribution. The proposed 3D-CoCo works on three transferring settings, effectively closing the domain gaps and can outperform recent SOTA methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This study is useful for researchers in both fields of unsupervised domain transfer and point cloud detection in the autonomous driving scenario. The proposed 3D-CoCo tries to utilize the domain knowledge induced by this special task to produce an effective unsupervised domain adaptation, rather than simply adopts the tricks used in 2D unsupervised domain adaptation. To be specific, it is nice to see that 3D-CoCo would like to specifically handle the domain drift caused by geometric variations, and designs a hard negative mining strategy by examining the statistics of pseudo-label quality. The experiments show good results on three datasets collected by heterogeneous LiDAR sensors, and outperform recent UDA methods on point cloud detection. \n\nBut the presentation needs to be polished. At first, symbols and terms in equations (4) and (5) may require a careful definition and remove some careless misuses. And the functionalities of the introduced terms and symbols should be briefly discussed. The authors claimed the effectiveness of co-training beyond self-training, but how they are especially different, or in what aspects co-training is better should be explicitly stated.\n\nOverall, this paper is well written, clearly illustrated, and appropriately structured. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It has included the limitations. The potential negative societal impact of their work  may not be available.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5 hours"}, "extra_text": {}}, {"review_id": "703FMiWz9t", "reviewer": "Reviewer_DPJL", "summary": "This paper proposes a domain adaptation point cloud object detection method termed 3D-COCO to deal with geometry shifts caused by different physical environments or LiDAR sensors. The authors use the shared BEV module to learn the domain-invariant for both source and target domain, while the augmented contrastive alignment schemes are also introduced to discover the hard samples. The mismatch is greatly reduced by 3D-CoCo with transformed point clouds. The experimental results on three benchmarks demonstrate the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength：\n\n+ The paper is well written and easy to follow.\n+ The proposed framework seems simple yet effective.\n+ The thinking and analysis about the hard instance mining are interesting and the corresponding solution achieves a great improvement of the performance.\n\n\n\nWeakness:\n- For fair comparison to ST3D, could the authors further provide the solution to combine the proposed method with PV-RCNN  and further show the performance comparison?\n- Is the proposed method sensitive to the pseudo-label generation threshold? How to determine the high-pass and low-pass threshold in line 210?\n- In line 223, the authors claim that their model achieves good performance even starting with low-quality pseudo labels. Could you please give some analysis and visualization about pseudo label adjustment during the training phase to further illustrate why and how the proposed method could achieve good results with low-quality initialized pseudo labels?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have pointed out the limitations of their work.\n\nSince it is just a basic method to deal with 3D point cloud detection, it has nothing to do with the potential negative societal impact.", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work addresses a practically important domain adaptation problem in 3D point cloud detection. The authors establish a sound perspective to this specific problem, and propose a new and effective framework that is not simply an extension of existing UDA algorithms to point cloud detection. The authors also construct new datasets to promote future research on this topic. 

All reviewers agree on the paper’s main contributions. They also recommend further polishing of the paper writing. AC agrees with the reviewers, and recommend acceptance of the paper. 

Decision: Accept (Poster)