Paper: Surface Representation in LiDAR Scenes (ICLR-2024)

Reviewer comment (Experiments):
This work only applies the proposed method for feature extraction to segmentation. Can it be generalized to other types of point cloud problems, for instance, low-level point cloud processing problems, e.g., denoising, upsampling, etc.?

Author response:
We believe so, and that would be a great direction for future work. As the reviewer points out, our method serves as surface representation for LiDAR point clouds, and thus can be applied to other point cloud problems, as a way to address unevenness and low point density. For example, for any problem that involves upsampling, we can adopt point sliding module (proposed in Sec 3.3) to densify LiDAR point clouds.

Reviewer comment (Experiments):
A follow-up question is, since this method works well for sparse LiDAR point clouds, can it be applied back to the simpler case which is the dense point clouds or synthetic point clouds? Will the performance also be improved?

Author response:
We can apply this technique to the simpler case, however, our goal is to address issues inherent to the processing of noisy and uneven point clouds (e.g., LiDAR point clouds). Therefore, RealSurf will not be beneficial in these settings, although performance should not degrade.

Reviewer comment (Evaluation):
The Point Sliding Module generates some jittering to the centroid for augmentation. This process has certain randomness because the coefficients are drawn from uniform or Gaussian distribution. I wonder how stable this method would be in the end, because there is a certain randomness in the process.

Author response:
Experimental results may vary slightly due to random sampling. In our experiments we empirically observed a small oscillation of $0.5$, regardless of different benchmarks.

Reviewer comment (Novelty):
The authors use downsampling with FPS to obtain the RealSurf feature. However, this is also performed in PointNet++. Thus it would be better if the authors could further clarify the way they do the downsampling differs from that is proposed in PointNet++.

Author response:
As shown in Sec. 3.3, the intent of adopting FPS is not to propose a novel downsampling strategy, but to construct regular triangles on uneven point clouds. 
That is, we utilize an off-the-shelf method to obtain relatively uniform point clouds for the surface feature computation on LiDAR point clouds.

Reviewer comment (Experiments):
How this work compare to other types of advanced feature extractors? For example, those based on transformer modules, e.g., Point transformer and Voxel transformer.

Author response:
We conducted some additional experiments on the feature extractors such as PointNeXt [1] and PointMLP [2] on SemanticKITTI val set (mIoU, \%) below:


|                 | PointNeXt [1]   | PointMLP [2]    | PointNet++ ($\times$1) |
|-----------------|-----------------|-----------------|:-----------------------|
| w/o RealSurf    | 51.5            | 52.4            | 51.3                   |
| **w/ RealSurf** | **60.9** (+9.4) | **61.2** (+8.8) | **62.3** (+11.0)       |

Reviewer comment (Theory):
How is the computational complexity of this proposal compared to PointNet++ and other competing methods?

Author response:
Similar to RepSurf [3], our RealSurf introduces minor extra computation compared to PointNet++. We also compare with other competitive methods on SemanticKITTI as follows:


|                          | mIoU | \#Params (M) | Inference (s/sample) | Memory (GB) |
|--------------------------|------|--------------|:---------------------|-------------|
| Cylinder3D [4]              | 65.9 | 55.8         | 0.13                 | 1.62        |
| LiDARMultiNet [5]           | 69.1 | 29.2         | 0.41                 | 6.35        |
| **RealSurf ($\times$2)** | 68.5 | 3.9          | 1.16                 | 2.06        |
| **RealSurf ($\times$4)** | 70.2 | 15.3         | 1.72                 | 4.14        |

Reviewer comment (Writing):
The paper would benefit from further refinement in its writing. Some sections might appear a bit complex and could be clarified for easier comprehension.

Author response:
We appreciate the constructive feedback and have updated our main text to fix them. We will thoroughly proof-read the paper for the camera-ready version.

Reviewer comment (Experiments):
Naive baseline: Occupancy Network + Marching Cubes?

Author response:
Occupancy network [1], a method developed on synthetic data and based on marching cubes algorithm [2], is not common in LiDAR point clouds due to the limited efficiency and noises on larger-scale point clouds. Thus, there is very few previous work for reference. However, since occupancy network is heavily based on volumes, we consider its alternative voxel-based baselines (e.g., MinkowskiNet [3], PolarNet [4]) in LiDAR scenes. We have updated our main text to compare our proposed method to MinkowskiNet [3] and PolarNet [4] on SemanticKITTI. Ours outperforms MinkowskiNet (mIoU: 53.2) implemented by [5] and PolarNet (mIoU: 69.4) by 17.5 and 10.7 mIoU, respectively. This comparison further demonstrates the superiority of our approach as a novel baseline.

Reviewer comment (Experiments):
How many points are left after a different number of downsampling?

Author response:
We utilize *all* 3D points of a LiDAR point cloud as our input without any preprocessing like voxelization. For a point cloud of $N$ points, after each downsampling operation, the resultant number of points decreases by a factor of 4. Specifically, for the $i$-th downsampling, the output number of points becomes $N/4^i$.

Reviewer comment (Experiments):
Point Sliding is a data augmentation method that jitters the points so as to make the proposed method more robust to noises. In the ablation study (Figure 7), the authors show 0.5 performs best with Guassian noise on SemanticKITTI. I am wondering, is this hyper parameter consistent across different benchmarks?

Author response:
We have empirically determined an optimal set of hyperparameters and use the same ones on all considered benchmarks. This indicates that our method is not overly sensitive to the choice of hyperparameters. Thus, we choose the factor of 0.5 as our default setting across different benchmarks. We will include a discussion about this topic in the camera-ready version, and provide further analysis.

Reviewer comment (Experiments):
The paper misses comparison with recent point cloud works such as [1].

Author response:
We didn't notice this very recent paper and have included it in the revision. Indeed, SphereFormer [1] achieves strong performance on some datasets. However, it leverages complicated Transformer architectures which have much more parameters (32.3M vs ours: 15.3M). Due to time limit and an intensive CVPR submission during this rebuttal period, we cannot incorporate these techniques of [1] into our architecture and conduct a fair comparison. We have included [1] in the related work section.

Reviewer comment (Novelty):
In my understanding, the main purpose of the introduced strategies seems to make the real data uniform for Repsurf and make the model more robust to real noisy data by jitter augmentation. The technical contributions are somewhat limited considering the RepSurf.

Author response:
In this paper, we aim to explore the feasibility of adopting surface representation in the context of LiDAR scenes. As mentioned in Figure1 and Section 3, a direct application of surface representation (i.e., RepSurf) to LiDAR point clouds simply fails, given how much performance degrades in this challenging setting. Though simple, our contributions enable the use of surface representation in LiDAR scenes, in a way that previous methods are incapable of. Hence, we believe that our contributions are relevant and provide useful insights to the scientific community.

Reviewer comment (Experiments):
The efforts of proposed strategies such as downsampling seem to trim the data more suitable for Repsurf. How about the effects if applying the same strategies to synthetic data?

Author response:
It will not be as effective, since synthetic point clouds are already uniformly sampled in a manner similar to what FPS aims to achieve. However, performing downsampling like FPS on LiDAR point clouds will be quite different, as it can greatly mitigate the effects of the unevenness for the process of surface feature computation.