Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This work presents an unsupervised domain adaptation for point cloud detection in the autonomous driving scenario, which employs a domain-specific 3D encoder for domain-specific feature extraction, a shared BEV feature transformation module for domain-invariant instance-level feature alignment. It adapts the contrastive instance alignment with geometry priors to align the BEV feature distributions induced by pseudo-labels and the true distribution. The proposed 3D-CoCo works on three transferring settings, effectively closing the domain gaps and can outperform recent SOTA methods.
main_review: This study is useful for researchers in both fields of unsupervised domain transfer and point cloud detection in the autonomous driving scenario. The proposed 3D-CoCo tries to utilize the domain knowledge induced by this special task to produce an effective unsupervised domain adaptation, rather than simply adopts the tricks used in 2D unsupervised domain adaptation. To be specific, it is nice to see that 3D-CoCo would like to specifically handle the domain drift caused by geometric variations, and designs a hard negative mining strategy by examining the statistics of pseudo-label quality. The experiments show good results on three datasets collected by heterogeneous LiDAR sensors, and outperform recent UDA methods on point cloud detection. 

But the presentation needs to be polished. At first, symbols and terms in equations (4) and (5) may require a careful definition and remove some careless misuses. And the functionalities of the introduced terms and symbols should be briefly discussed. The authors claimed the effectiveness of co-training beyond self-training, but how they are especially different, or in what aspects co-training is better should be explicitly stated.

Overall, this paper is well written, clearly illustrated, and appropriately structured. 
limitations_and_societal_impact: It has included the limitations. The potential negative societal impact of their work  may not be available.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a domain adaptation point cloud object detection method termed 3D-COCO to deal with geometry shifts caused by different physical environments or LiDAR sensors. The authors use the shared BEV module to learn the domain-invariant for both source and target domain, while the augmented contrastive alignment schemes are also introduced to discover the hard samples. The mismatch is greatly reduced by 3D-CoCo with transformed point clouds. The experimental results on three benchmarks demonstrate the effectiveness of the proposed method.
main_review: Strength：

+ The paper is well written and easy to follow.
+ The proposed framework seems simple yet effective.
+ The thinking and analysis about the hard instance mining are interesting and the corresponding solution achieves a great improvement of the performance.



Weakness:
- For fair comparison to ST3D, could the authors further provide the solution to combine the proposed method with PV-RCNN  and further show the performance comparison?
- Is the proposed method sensitive to the pseudo-label generation threshold? How to determine the high-pass and low-pass threshold in line 210?
- In line 223, the authors claim that their model achieves good performance even starting with low-quality pseudo labels. Could you please give some analysis and visualization about pseudo label adjustment during the training phase to further illustrate why and how the proposed method could achieve good results with low-quality initialized pseudo labels?
limitations_and_societal_impact: The authors have pointed out the limitations of their work.

Since it is just a basic method to deal with 3D point cloud detection, it has nothing to do with the potential negative societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposed an unsupervised domain adaptation method for point cloud detection. The method is built on several effective strategies, including BEV feature representation, pseudo labels, instance contrastive alignment and hard mining.
main_review: Novelty: the instance contrastive joint training is interesting and novel according to my knowledge.

Significance: It clearly improves the baseline (trained only on source domain) and outperforms the existing unsupervised domain adaptation methods.

Post Rebuttal: My concerns about pseudo labels have been addressed in the rebuttal. My final opinion: this is a valuable paper but not a breakthrough in the field.
limitations_and_societal_impact: 1) Details about the pseudo label generation need to be described.  How did you run Eq.(4)?

2) How to process or avoid the possible errors of the pseudo labels? How do such pseudo-label errors influence the final results? It’s good to have some analysis about the relationship between the quality of the pseudo labels and the final performance of the proposed model.

3) Synthetic/simulation-to-real generalization experiments are interesting. If possible, it would be nice to see some tests under this setting.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper contributes as an early study of unsupervised domain adaptation (UDA) on 3D point cloud detection, an important step towards zero cost of labeling. Different from 2D image data that have regular grid structures, 3D point cloud data could have huge geometric variations such as point density and occlusion due to different physical environments or LiDAR sensor configurations; consequently, the low-level features of point cloud representation may be not transferable across domains. Taking it into account, this paper uses two individual encoders for the source and target domains to capture domain-specific characteristics and a shared Bird-Eye-View (BEV) transformation module to learn domain-agnostic BEV features, which have similar grid structures as images and thus are more transferable. Specifically, this paper extends contrastive instance alignment to point cloud detection, based on BEW features of foreground and background proposals. Direct use of contrastive instance alignment can cause the mismatch in point density and occlusion ratio between pseudo-labeled and true samples distributions in the target domain. To address it, this paper proposes to augment positive easy target samples (with dense points and complete geometry) by their artificial counterparts with sparse points and occlusion. Moreover, this paper constructs a new benchmark using three large-scale datasets. Empirical results show that the proposed method is effective and achieves the new state of the art.
main_review: Although the contributions and novelty of this paper are considerable, there are still many problems.

1. The full name should be mentioned when BEV is shown for the first time in Section 1, where it would be better to provide a brief description of BEV and its related references for a wider range of audiences.

2. It would be better to compare the high-level 3D features with the BEV features. The high-level 3D features are extracted by the same but deeper network layers than the low-level ones. The 3D domain shift is mapped into the 2D one, causing information loss.  Would it reduce the feature discriminability? Would it affect the single-domain detection performance, e.g., cases of a general detection setup in individual domains? It is also interesting to know how the features extracted by spherical convolutions [a] behave.

3. The idea of separate encoders is similar to that of [b,c,d] for 2D UDA. The authors should consider them as related works and make comparisons and discussions appropriately. 

4.  It is interesting to approximately quantify the distribution discrepancy across domains based on the performances of passing source samples through the target encoder and passing target samples through the source encoder. In Section 4.3, the in-domain performance is evaluated on the target domain; however, according to the previous descriptions, the cross-domain performance should be evaluated on the target domain. There exists a contradiction and a clarification is required.

5. Does the "co-training" in this paper mean two individual encoders for the source and target domains? Or the training strategy? This paper also adopts the pseudo-labeling technique from self-training. In Lines 56-59, the difference is not clear enough. 

6. In Lines 109-112, "bi-directional knowledge sharing" is similar to multi-task learning [e]? Why does it hold? Are there any related works that support this argument? In the ablation studies, it seems that the random object scaling has a great effect on the source performance and this has nothing to do with the proposed method. 

7. In Algorithm 1, this paper conducts three-stage training. why not two stages? The warm-up stage introduces another set of training hyperparameters. Besides, the augmented target set is denoted by the original one? The notations should be different.

8. It would be more clear to expand the losses of L_cls^S, L_loc^S, and L_reg^T.

9. In Section 3.2, this paper uses a keypoint-based feature extraction approach. Comparisons with other kinds of feature extraction approaches are needed. Besides, the model sensitivity to the hyperparameter R is to be examined.

10. In Eq. (5), some notations are confusing, such as the summation range in the denominator in L_intra and L_inter. A negative sign is missing in L_intra and L_inter? Otherwise, samples of the same classes are pushed away? In L_intra, samples of other classes should be used as negative ones whereas this paper does not make it clear. In L_inter, other samples in the same class are used as negative ones? Is this the typical contrastive learning method of instance discrimination from self-supervised learning? How can this method increase the inter-class distance between different categories within the same domain? Are there any references for contrastive instance alignment? The main component of the proposed algorithm is so ambiguous that the experimental results may be not so convincing. 

11. In Eq. (5), the domain \mathcal{D} is similar to model D; it would be better to use different notations. In Eq. (6), "arg" is redundant. The model sensitivity to the temperature \tao and the hyperparameter \lambda is to be examined. 

12. In Line 159, is there a specific rule to distinguish between easy and hard samples? Please make it clear. In Line 166, how many points are discarded? In Line 169, how to calculate the viewpoint of a certain sample? How many viewpoints do you select to discard? Are there any solid principles to make decisions? Are there any related references? Please make it clear. 

13. In Lines 170-173, the proposed augmentation can also be done in the source domain since source samples can be viewed as easy ones. Consequently, the distribution mismatch between the source and target domains can be further reduced. It is interesting to see whether the model performance can be further improved. The proposed augmentation strategy is somewhat similar to consistency regularization [f,g] from semi-supervised learning, which enforces consistent predictions between weakly and strongly augmented versions of the same sample image. Maybe the related works should be considered and discussed.

14. In Line 179, can you describe how to update the pseudo labels in detail? 

15. Is each domain split into training and test sets? Is the performance evaluated on the training or test set of the target domain?


[a] H. Lei, N. Akhtar and A. Mian, "Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds," in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2020.2983410.

[b] Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc., Red Hook, NY, USA, 343–351.

[c] E. Tzeng, J. Hoffman, K. Saenko and T. Darrell, "Adversarial Discriminative Domain Adaptation," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2962-2971, doi: 10.1109/CVPR.2017.316.

[d] W. Chang, H. Wang, W. Peng and W. Chiu, "All About Structure: Adapting Structural Information Across Domains for Boosting Semantic Segmentation," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 1900-1909, doi: 10.1109/CVPR.2019.00200.

[e] Caruana, R. Multitask Learning. Machine Learning 28, 41–75 (1997). https://doi.org/10.1023/A:1007379606734.

[f] D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel. Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring. In Proc. Int. Conf. on Learn. Rep., 2020.

[g] S. Roy, A. Siarohin, E. Sangineto, S. R. Bulò, N. Sebe and E. Ricci, "Unsupervised Domain Adaptation Using Feature-Whitening and Consensus Loss," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 9463-9472, doi: 10.1109/CVPR.2019.00970.
limitations_and_societal_impact: Annotating point clouds is more difficult and expensive than annotating images for object detection tasks. Good 3D UDA methods are desired to reduce the annotation efforts. On the constructed new benchmark of three autonomous driving datasets, this paper achieves a significant gain in detection performance, one step closer to the goal. As for the limitation, this paper has mentioned that the proposed method takes more memory footprint than existing self-training methods at the training time whereas does not address it. The potential negative societal impact of their work has not been mentioned. The suggestions for improvement are provided in Main Review.
review_title: null

Meta-review: This work addresses a practically important domain adaptation problem in 3D point cloud detection. The authors establish a sound perspective to this specific problem, and propose a new and effective framework that is not simply an extension of existing UDA algorithms to point cloud detection. The authors also construct new datasets to promote future research on this topic. 

All reviewers agree on the paper’s main contributions. They also recommend further polishing of the paper writing. AC agrees with the reviewers, and recommend acceptance of the paper. 

Decision: Accept (Poster)