Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper targets the problem of high computational complexity in LiDAR-based long-range 3D object detection. It proposes a fully sparse 3D object detection paradigm, i.e., from the feature extraction to the proposal generation and bounding box prediction, all are conducted based on sparse voxels or points. Specifically, built upon the commonly used sparse voxel encoder, it devises a novel sparse instance recognition (SIR) module to perform 3D detection based on local points, which is efficient in implementation and resolves the "center feature missing" problem. Experiments demonstrate the efficacy of this proposed method, especially for large vehicles/tiny objects and long-range cases.
strengths_and_weaknesses: Strengths:
- The studied problem is valuable for practical use. The motivation that dense prediction can bring high computational complexity for long-range cases is reasonable. A fully sparse 3D object detector is suitable given the sparse property of LiDAR points distribution.
- The analysis of the difference between the proposed method and voxel-based semi-dense detectors/conventional voting in VoteNet makes the contribution of this paper clear.
- Illustrations are clear and concise, making the proposed method much easier to understand.
- Experiments support the main claims in this paper. The improvement in the detection performance of small objects/huge vehicles is impressive.
- The efficiency analysis in terms of different perception ranges shows the main advantage of the proposed paradigm.

Weaknesses:
- The methodology part is hard to follow. In particular, so many new concepts are unnecessarily defined in the presentation, for example, dynamic broadcast/pooling, sparse instance recognition, group feature aggregation, integration, sparse prediction, etc. For some of them, the author would have better expressions/names to explain them for easier understanding. There is no need to package some easy operations or commonly used modules with totally new names, which can only make the implementation hard to understand.

In contrast, the illustration is clear and concise to represent the overall pipeline and main modules of the proposed framework.

- Experiments: Although the proposed method can address the center feature missing problem to some extent and improve the detection performance of huge vehicles, it still performs a little worse than state-of-the-art methods (e.g., PV-RCNN++) on the vehicle detection of Waymo. The better mAP over 3 classes is mainly contributed by better performance on small objects. Maybe need some analysis on this phenomenon.

- The analysis of the center feature missing problem is a little vague. Specifically, is there any way that we can observe the effect of addressing this problem more straightforwardly? From my perspective, the detection performance of huge vehicles can not directly support the claim strongly, maybe at least we need a more detailed analysis of the center localization accuracy or some qualitative analysis.
questions: (Minor) I notice the model converges faster than SST and wonder whether there is any potential reason/discussion about it.
limitations: The author discusses the limitations and societal impact adequately in the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper studies efficient long-range LiDAR-based 3D object detection. The authors propose a fully sparse 3D object detector (FSD) to eliminate the dense BEV feature map that scales poorly with the perception range. Similar to VoteNet, FSD first applies the sparse encoder to extract point/voxel-wise features, then groups the points into instances, and finally predicts the boxes based on the instance-wise features. The authors have evaluated their proposed FSD on Waymo and Argoverse 2. FSD achieves superior performance on both benchmarks, especially Argoverse 2 where the perception range is much larger.
strengths_and_weaknesses: **[Strengths]**

This paper is well-written and easy to follow. The authors clearly explain the missing center feature problem with a good illustration (Figure 1). Generally, the figures in this paper have pretty good quality and clarity, which helps readers understand the proposed algorithm easily.

The problem studied in this paper is very important as long-range perception is very critical in high-speed driving scenarios. The proposed FSD is technically sound and achieves good empirical results on large-scale benchmark datasets.

**[Weaknesses]**

The technical novelty of this paper is a bit limited as the proposed FSD is very similar to VoteNet. The authors have clarified the differences between FSD and VoteNet in Section 3.5. However, both of them are more of engineering improvements than technical contributions. That said, the authors apply this idea to solve an important real-world problem, which could still provide valuable insights to other researchers in the community.

Researchers have spent considerable efforts on transformer-based detection heads (DETR) besides anchor-based and center-based heads. Lately, there have been several follow-ups in 3D object detection, such as Object DGCNN, DETR3D, and TransFusion. DETR head provides a potential workaround for large dense BEV feature maps. The authors could discuss the pros and cons of DETR and SIR in their rebuttal.

Although the computational cost of FSD does not grow with the perception range, it does scale linearly with the point cloud density. This could be problematic for denser point clouds (either with temporal fusion or simply with more laser beams). In contrast, dense detectors suffer less from this since the BEV feature map will always have the same resolution regardless of the point cloud size. In all experiments, the authors use single-frame point clouds for evaluation. It would be necessary to conduct experiments (or at least measure the memory footprints and inference latency) with denser multi-frame point clouds.

As this paper focuses on efficiency, it would be essential to understand where the efficiency boost comes from. For example, the authors could provide a detailed latency breakdown of different components in FSD. Besides, different inference backends could lead to very different latency results. The dense BEV encoder is composed of regular 2D convolutions, which can be easily accelerated by existing inference libraries (such as TensorRT and TVM), while the proposed SIR module is harder to be optimized. Therefore, it is unclear whether FSD could still achieve such significant speedups with highly-optimized inference engines.

Finally, there are a few aspects that could potentially be improved in the experimental evaluation:
* As FSD is based on SST, it would be necessary to report the accuracy of SST on Argoverse 2 for an apples-to-apples comparison.
* The authors could consider conducting experiments on nuScenes, where there are many competitive LiDAR baselines.
questions: I would love to see the answer to the following questions in the rebuttal:
* What are the pros and cons of DETR and SIR in dealing with long-range perception?
* How well does FSD scale with denser point clouds (*e.g.*, from temporal fusion)?
* Where does the efficiency boost of FSD come from?
* Could FSD still achieve the same level of speedup with highly-optimized inference engines (*e.g.*, TensorRT, TVM)?
* What is the accuracy of SST on Argoverse 2?

The authors could refer to the previous section for more detailed comments.
limitations: The authors have addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a novel 3D lidar point cloud detection method. the method is based on 
a sparse instance recognition module which groups points into features, making the feature
extraction and bounding boxes regression for detection more efficient than other point-based
methods.
strengths_and_weaknesses: The motivation and goals of the work are well explained and clear. 
The experimentation section is good and the ablation study brings interesting conclusions.

The paper has some little missing information listed in the questions below.
questions: However, it would be nice to have a quick explanation for the metric used as well as some qualitative
visual results, especially for the "failing"/less good results compared to other methods.
Also perhaps for the memory and latency performances, having more of the SOTA methods for comparison
would bring more value to the discussion.
Finally, even if the paper is more focused on long and sparse lidar data compared to other methods.
It would be interesting for a small comparison on the KITTI dataset, to see if the method
still performs correctly on shorter-range point clouds.
limitations: The limitation mentioned by the authors is found in the grouping strategy employed. 
I agree that a comparison with another grouping method would be out of the scope of the paper, but then perhaps an extra comparison with the grouping method used by another method would be interesting? (pointnet++ multi-scale grouping module for example).
review_title: null

Meta-review: metareview: After the rebuttal and discussion all reviewers are positive, and recommend acceptance. The AC agrees with this recommendation.
confidence: Certain
Decision: Accept