Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a polar coordinate system-based data augmentation approach, PolarMix, for scanning LiDAR point cloud understanding. PolarMix generates augmented data by mixing two different scans with scene-swapping and instance rotate-pasting. Extensive experiments and ablation studies validate the performance gain by PolarMix for three tasks（semantic segmentation, object detection, domain adaption）on three datasets.
strengths_and_weaknesses: Strengths:
- The paper is well-written and easy to follow.
- The augmentation strategy is beneficial not only for semantic segmentation tasks but also for object detection and domain adaption. I like the idea of mixing two domains’ data scan-wisely to bridge the domain gap.
- Since PolarMix is a DA strategy, generally it is model-agnostic and applicable to any model.

Weaknesses:
- The baselines for comparison of w/  and w/o PolarMix are not the latest ones. I am wondering how is the performance gain by DA of  PolarMix under some latest backbones with higher baseline performance, the improvement will still significant?
- The contribution is a little limited, considering the relevant work. PolarMix is kind of the combination of CutMix, Copy-Paste and Mix3D from the perspective of idea. 
- At least one relevant work is not mentioned and compared for object detection task. [1]

[1]J. Fang, X. Zuo, D. Zhou, S. Jin, S. Wang and L. Zhang, LiDAR-Aug: A General Rendering-based Augmentation Framework for 3D Object Detection, CVPR2021
questions: Besides the detailed ablations studies in the work, I am also curious about the following questions:
- This work tried cutting and swapping points along azimuth directions, how about further subdividing it along inclination direction.
- Is that possible to mix more than two scans? For example,  generate a new can by mixing each quarter from 4 different scans.
- What do σ_1 and  σ_2 indicate (L. 185)?
- For the rotation angles of rotate-pasting, any reason about why choosing values as L.183-185 for three datasets? 
limitations: The authors didn’t discuss the limitations of their research and potential negative societal impacts.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a data augmentation method specifically designed for LiDAR point clouds. Two layers of data augmentation are introduced: scene level augmentation and instance level augmentation. The authors demonstrated the enhancement brought by this data augmentation method on various applications, and showed that the proposed data augmentation is superior than the state-of-the-art LiDAR data augmentation methods.

strengths_and_weaknesses: + The idea is simple, in a good way
+ The evaluation is thorough: the authors demonstrated three application, namely semantic segmentation, object detection, domain gap reduction
+ The proposed method outperforms the state-of-the-art LiDAR data augmentation methods

questions: Will the authors release code after publication?

limitations: This paper presented a neat idea and showed extensive experiments to justify its contribution. I do not see any strong limitations.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes an augmentation method for point clouds, especially captured in road environments. The proposed augmentation method is to crop, cut, and mix two 3D scans at both scene-level and instance-level. This concept can be extended to unsupervised domain adaptation (UDA) by fusing source domain point cloud (known labels) and target domain point cloud (unknown labels). Series of experiments demonstrate its superiority in comparison with other point cloud augmentation methods.
strengths_and_weaknesses: 1. Strength
- Simple and straightforward ways of augmenting point clouds. 
- Potential extension to UDA tasks.
- Consistent performance improvements in various tasks.

2. Weakness
- Lack of analysis and intuition behind such a design.
Overall, the authors present extensive experiments to demonstrate the superiority of the proposed data augmentation scheme. However, I wonder why such data augmentation is helpful for point-based recognition tasks. For instance, is PolarMix effective in making the networks more robust to point cloud density? or point cloud noise? The lack of such analysis makes me suspicious of the novelty of this work. Currently, this is the dominant worrying point of this paper.

questions: 1. Intuition

1-1. I understand that this method effectively improves the quality of several target tasks. However, what is the fundamental intuition behind the proposed data augmentation? Why do such strategies bring performance improvement? While authors demonstrate lots of experiments that consistently increase the accuracy, I cannot expect why such an augmentation scheme is effective and novel. 

1-2. What kinds of wrong estimations are fixed after applying the proposed augmentation strategy, PolarMix? For instance, is the estimation of the far-away points more accurate? Or, is this method robust to the density of 3D scans? Even though I read the additional results in the supplementary material, I cannot find any deep analysis of this issue. 

2. Experiments

2-1. What if we fuse more than 3 scans?

2-2. What if there is a misalignment after data augmentation? In Fig2 of the manuscript, PolarMix crops point clouds in a certain range of azimuth. It means that the origins of the two-point clouds are aligned. I wonder why such assumption or alignment should be required for this augmentation? Why should we crop point clouds along the azimuth axis? Do you have any reason? (This is also related to Q.1-1)

2-3. Lack of comparison against recent UDA methods in Table5 of the manuscript.
In recent days, there are papers about UDA for 3D semantic segmentation, such as xMUDA [H]. In my opinion, the limited demonstration in Table5 of the manuscript does not fully support the authors' claims about the UDA setup. Technically speaking, PolarMix can be applicable to UDA and I agree with the authors' claims. However, I cannot judge whether this augmentation scheme is much more effective for the UDA for the 3D semantic segmentation task in comparison with previous methods [H]. Accordingly, the authors overclaim their novelty in UDA for the 3D semantic segmentation due to their limited comparisons.

3. Writing
3-1. (L208) As far as my understanding, MinkUNet also consists of sparse convolutional layers as SPVCNN did. However, the sentence can mislead readers as MinkUNet processes dense voxel-based architectures in contrast to SPVCNN.

3-2. I wonder why the authors did not present their qualitative results in the main manuscripts. Not just the quantitative results, qualitative results can help readers' understanding of this work. I recommend authors to revise the paper if it is accepted. 
(I found the results in the supplementary materials. However, for completeness, I still think that the qualitative results should be listed in the main manuscripts.)

> If authors relieve my worries, I am willing to change my score.

Reference
- [H] xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation, CVPR 2020

limitations: 1. While authors only focus on the LiDAR scans that are captured in the road environments, there are other types of point cloud datasets, such as S3DIS [I] or ScanNet [J]. In my understanding, this method is not applicable to the indoor point set datasets. If so, authors should have included this problem as their limitation. 

Reference
- [I] 3d semantic parsing of large-scale indoor spaces, ICCV 2016
- [J] Scannet: Richly-annotated 3d reconstructions of indoor scenes, CVPR 2017
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: - This paper introduces an approach to augmenting cylindrical LiDAR point cloud to acquire boosted performance on 3D semantic segmentation and 3D detection. The proposed approach called PolarMix enables cut, edit, and mix point clouds along the scanning direction. The augmentation happens on the scene-level and instance-level so that the augmented data provides a variety of combinations of the augmented scenes. The proposed approach is superior to conventional global rotation and scale augmentation, CutMix [37], Copy-Paste [13], and Mix3D [21].
strengths_and_weaknesses: **Strengths**
1. The paper comprehensively summarizes the related work in the point cloud augmentation field. The paper is self-contained, so the readers readily follow the problem and the recent advances.
2. The paper is straightforward to understand. I enjoyed reading the paper. The idea of mixing the 3D scenes is already known, but mixing on the polar coordinate seems to be very effective on the cylindrical LiDAR point clouds. 
3. The approach shows compelling results on the SemanticKITTI [1], nuScenes-lidarseg [2], SemanticPOSS [22], SynLidar datasets. The proposed augmentation approach is applied with recent 3D semantic segmentation networks, such as MinkNet [8], SPVCNN [30], RandLA-Net [15], Cylinder3D [41]. For the task of 3D detection, PointPillar [18], Second [36], CenterNet [10] are applied. The gain is clear, and it outperforms other baselines.
4. The approach shows the effectiveness of the unsupervised domain adaptation as well. Since the approach can mix labeled source data and unlabeled target data, the approach can be readily applied to the various combinations of the domains.
5. The proposed augmentation approach improves data efficiency, as demonstrated in the experiment section. With the small amount of 3D scans, the PolarMix can produce a similar performance.
6. The paper explains the proposed idea in detail. The supportive figures (such as Figures 1 and 2) help to understand the approach better.

**Weakness**
1. The approach is a simple extension of the idea of the mix of 3D scenes and rotating bounding boxes (limited to the azimuth angles). The idea is not entirely new, and the target domain is limited to the cylindrical LiDAR datasets, not the general 3D scenes. However, the cylindrical LiDAR domain is one of the exciting domains for the task of intelligent mobility systems, and the custom design of 3D detection for the cylindrical LiDAR data also forms a research field. Therefore, I think it is not a critical weakness.
2. It is unclear how the baseline approaches for the semantic segmentation are selected. In addition to the CutMix [37], Copy-Paste [13], Mix3D [21], there are possible options to be applied. Similarly, approaches to augment the 3D detection task, such as GT-Aug [36, 12], CutMix, or approaches of [6, 11, 13] could be applied. The paper needs some clarification on how the baseline approaches were selected.
3. It is recommended to indicate the computational overhead when the proposed approach is applied. For instance, when training MinkNet, compared with the vanilla MinkNet training, what percentage of the total time is added to use PolarMix? Depending on the additional computation burden, the baseline approaches could be reevaluated.
questions: 1. Did the authors try to mix more than two scenes? The proposed Algorithm only shows how to mix two scans, but I wonder if the idea could be generalized.
2. It is interesting to see that the rotated instances help for the data augmentation. However, I presume that the instances in a highly cluttered or distant scene would not be that effective. Such instances would make unrealistic samples because they are partially observed in a particular viewing direction. Some analysis of the data effectiveness or performance gain versus the distribution of the distant objects would be interesting.
3. Please check the weakness section and answer the questions there too.

limitations: - The paper does not address the limitation of the proposed approach. The concern about the computational overhead is clarified and stated. If it has drawbacks in any facts, the limitation section needs to describe them.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposed a data augmentation method named PolarMix for LiDAR point cloud perception. It includes two separate operations: a scene-level swapping that first cuts point cloud sectors of two LiDAR scans w.r.t. the point azimuth values and then switches the cut sectors to form a new sample for training; and an instance-level copy-paste which selects instance points for certain classes from one scan, rotates them along the LiDAR scanning direction and pastes the rotated points to another scan. Experimental results show that PolarMix yields improvements for both LiDAR semantic segmentation and 3D object detection.
strengths_and_weaknesses: Strengths:
1. This paper introduces a new data augmentation method for LiDAR point clouds.

2. The proposed PolarMix is tested for both LiDAR semantic segmentation and 3D object detection and the results show that the proposed method can improve the baselines for both tasks.

Weaknesses:
1. This paper explains mostly the “what” and “how” but not much “why” for their specific augmentation operations.  More design insights and analysis can improve this paper a lot.

2. The experimental analysis is insufficient to support all conclusions stated in this paper.

3. The elaboration of this paper is not good enough. Some assumptions are made without clear logical relations.

questions: Regarding weakness 1, there are some concerns w.r.t. the idea proposed in this work:
1. In L56, the authors state that PolarMix can maintain high fidelity for the augmented samples. However, the augmented scan shown in Figure 1 (d) contains lots of instances like cars that are scattering around the center-vehicle with arbitrary directions, which might not happen in real-world scenarios. Also, the pasted instances seem to be overlapped with the background classes. As instances like cars, bicycles, and pedestrians tend to appear only on the road surface, how to ensure that they are pasted to a proper position with high fidelity?

2. In L109, the LiDAR points in the polar coordinate system are defined by azimuth theta, depth r, and inclination phi. Why select only the azimuth to split point cloud sectors, but not depth or inclination or even the combination of all three?

3. In L207, the authors state that the proposed method can work across different LiDAR point cloud representations. Can it be applied to range view networks? How does it ensure a proper projection after concatenating or pasting new points in the current LiDAR point cloud?

4. In suppL19, randomly swapping sectors of 45 degrees could downgrade the segmentation performance. Any possible explanation on why this problem happens?

Regarding weakness 2, there are some concerns w.r.t. the experiment part of this work:
1. The main results for LiDAR semantic segmentation, e.g., Table 1, are from relatively old segmentation methods MinkNet (55.9% mIoU) and SPVCNN (58.0% mIoU), which do not yield competitive performance over state-of-the-art ones, such as Cylinder3D (65.9% mIoU). Although the authors involved RandLA-Net and Cylinder3D in later analysis, i.e., Table 3, they are only tested under the 10% data setting. Therefore, the performance gains of PolarMix from the relatively lower-score methods are not representative enough to demonstrate superiority.

2. The results for 3D object detection, i.e., Table 4, only include comparisons with the baselines. Other augmentation techniques, such as copy-paste, are recommended to include for a more comprehensive comparison.

3. The ablation studies in Section 4.4 is not comprehensive enough to support the effectiveness of each of the components in PolarMix. Besides, the scene-level swapping seems less effective compared to the rotate-pasting.

Regarding weakness 3, there are some minor problems w.r.t. the elaboration:
1. Omega and C should be properly defined after their first appearance, i.e., Eq. (1) in L116.

2. The delta_1 and delta_2 symbols in Algorithm 1 are not defined.

3. The sigma_1 and sigma_2 symbols in L185 are not defined.

4. In L247, why the cut-and-mix strategy in PolarMix can mitigate the domain discrepancy? Since PolarMix does not include any domain alignment components like the domain discriminator, the performance gains seem more related to the augmentation effect but not the adaptation effect.

limitations: This paper does not include a specific description of potential limitations. In L396, the author state that the effectiveness of the proposed method might be influenced by the parameter setting, which should be regarded as and included in ablation analysis. More general limitations are recommended to be discussed, such as the applicability of the proposed method for different types of LiDAR point clouds and networks.
review_title: null

Meta-review: metareview: The proposed augmentation method for LIDAR Scans is to crop, cut, and mix two 3D scans at both scene-level and instance-level. The approach is not novel and a simple extension of the idea of the mix of 3D scenes and rotating bounding boxes. Another limitation is that the method cannot be applied to general 3D scenes. The reviews include A(7), WA(6), BA(5), two BR (4). After carefully checking out the rebuttals and discussions, I recommend the paper to be presented for the NeurIPS community.
confidence: Certain
Decision: Accept