Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors pointed out the issue of the spatial redundancy of existing point-based object detection frameworks. To handle this issue, the authors proposed the dynamic ball querying module, which is adaptively activated or blocked with a suitable combination among pre-defined group set. With this module and a resource budge loss, the model can be trained for achieving a better balance between effectiveness and efficiency. In experimental section, the authors provided various comparison to demonstrate that the proposed method can reduce effectively latency by a large margin on popular point cloud detection benchmarks.
strength_and_weaknesses: [+] First of all, the authors explained the motivation of the paper very clearly and convincingly through the analysis of the various aspects in the existing point-based object detection model. Motivated by routing process of dynamic network, the proposed ball querying module seems to be well designed to adaptively select a subset of input points as queries and extract local features in a suitable spatial factor. Lastly, the proposed model can be easily used into the existing point-based detection model framework to replace the set abstraction layer.

[-] One concern is how well routing procedure works. There seems to be a lack of detailed explanation about whether it works well if it is divided more finely than 2 groups (large and small), and what kind of points is when multiple selections are made. 

For efficiency, the authors used the nearest sampling, but it seems that the density change in terms of the distance from the LiDAR sensor does not take into account. Object nearby sensor has many points and farthest objects have few points, so nearest sampling can collect points that are completely unrelated to the farthest object.

clarity,_quality,_novelty_and_reproducibility: The paper introduced an important issue of existing point-based object detection framework as the spatial redundancy problem. Through technical novelty in the individual parts are limited, the overall framework seem to be interesting and perform well in provided experiments. Most of description seems to clear and be detailed for reproducing the proposed method.
summary_of_the_review: I mentioned all comments including reasons and suggestions in the above sections. I recommend that the author will provide all the concerns, and improve the completeness of the paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose to learn the radius in ball query with gumble-sigmoid trick. Experiments on both KITTI and Waymo show the algorithm is more efficient and effective than previous state-of-the-art point-based 3d object detectors.
strength_and_weaknesses: However, the core contribution of this paper, the dynamic ball query, is not new. In Pyramid R-CNN[1], the authors proposed an almost the same algorithm. Both of these two papers use gumble-sigmoid trick to learn the radius, and they are all applied on the 3d objection detection task.
I believe that the authors haven't seen [1] before. But based on the existence of [1], I still vote to reject this paper.

[1] Mao, J., Niu, M., Bai, H., Liang, X., Xu, H., & Xu, C. (2021). Pyramid r-cnn: Towards better performance and adaptability for 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2723-2732).

After reading the rebuttal, I still think that the idea of learning the radius of a ball query is not particularly novel, even though there are some differences in the implementation details and the position of the module. However, I missed another key point: the authors argue that the radius can be learned to reduce the latency of the network. Given this contribution, I am inclined to give a borderline recommendation.
Additionally, I noticed that the authors keep mentioning "latency" in the introduction and method sections, but only "FPS" is evaluated in the experiments. In an autonomous driving scenario, "latency" is a more important metric than "FPS", since point clouds are typically processed frame by frame with a batch size of 1. The authors should consider replacing all mentions of "FPS" with "latency" to better align with real-world applications.
clarity,_quality,_novelty_and_reproducibility: Has been discovered before.
summary_of_the_review: See above
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper points out the spatial redundancy on background points and useless group operation in MSG for the inappropriate receptive field. The authors present an algorithm named dynamic ball query, which dynamically generates gate masks for each group of MSG to process useful points and block redundant background points. The method is validated on three detection datasets, where a noticeable improvement in inference speed is observed.
strength_and_weaknesses: Strengths:

S1: The motivation of this paper is clearly presented. The paper analyzes the latency of existing point-based detectors and reveals the spatial redundancy existing in features of background points in Sec. 3. The observation may inspire studies of efficient point-based detectors.

S2: The idea of using a dynamic network to drop redundant background points is novel. In order to block redundant background points, the authors propose to predict the gating logits, which are used to select a subset of input queries for each group in MSG operation. Therefore, the method chooses optimal grouping branches to process point features and avoids useless receptive field groups.

S3: The experimental evaluations are comprehensive to support the claim. The proposed method presents an impressive improvement in inference speed over state-of-the-art and achieves comparable accuracy on KITTI, Waymo, and ONCE datasets.

Weakness:

W1: The authors claim the method achieves higher inference speed (162 vs 102). My concern is whether the setting of the baseline is reasonable enough. As shown in Fig. 3(d), no points go through the large-scale group in the fourth SA layer. The baseline is expected to achieve faster speed if it discards the useless grouping operation in SA layers.

W2: In Table. 1, the paper lists the performance of dynamic gating with different routing manners. What does the layer-wise routing mean? Please supplement a clear definition for this in the text.

W3: The visualization results on KITTI val set are not informative enough; please show local details and make an analysis about why most points are blocked at the large-scale group in fourth SA layer.

Typos:
1. Page 1, “INTRODUCATION” ⇒ “INTRODUCTION”

2. Page 1, “Yang et al. (2020b) first proposes a single stage architecture”⇒ “Yang et al. (2020b) first propose a single stage architecture”

3. Page 3, "The valuable observation motivates us to block and choose optimal grouping branches" => "The valuable observation motivates us to choose optimal grouping branches".

4. Page 6, “need to be averaged along”⇒ “needs to be averaged along”

5. Page 8, “point cloud which scale is” ⇒”point cloud whose scale is”

6. Page 9, “with with batch” ⇒ “with batch”

7. Page 9, “the inference speed of efficient the baseline”⇒ “the inference speed of the efficient baseline”

8. Page 9, “When conducting larger scale”⇒ “When conducting a larger scale”

9. Page 9, “which further deepen the understanding”⇒”which further deepens the understanding”

clarity,_quality,_novelty_and_reproducibility: The motivation of this paper is clear. The idea of using a dynamic network to drop redundant background points is novel. The results may be hard to reproduce if the authors do not open the source code.
summary_of_the_review: Unlike existing point-based detectors that focus on recalling more foreground points with efficient sampling policies, the main contribution of this paper is choosing optimal grouping branches to process point features. I appreciate the approach, but more abundant quantitative and qualitative results are necessary to demonstrate the effectiveness of the proposed method. On the other hand, the authors should correct the typos and clarify some statements.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This manuscript proposes an extension to point-cloud processing networks that use nearest neighbor queries. Instead of fixing the search radii for those queris before training, the radii are selected dynamically on a per-point basis, and the selection criteria is learned. An additional loss is used to penelize the computational cost induced by selecting multiple radii. The experiments are well-designed and show that this approach significantly improves the cost-vs-performance tradeoff compared to prior art.
strength_and_weaknesses: - The overall **idea** is novel, well motivated and well thought off. Especially the parameterization of speed vs. accuracy using an additional loss is very interesting and allows tuning that trade-off without changing the architecture. The building block is generic can can likely be introduced into any network that uses ball-like queries on 3D (or 2D) data.

- The paper is **well written**, the method description is easy to follow, all steps are well motivated and the method can likely be reproduced.

- The **experiments** are thorough and fair. They are performed on large state-of-the-art datasets and the method is compared to many prior art methods. Additional experiments shed more insight into the model's inner workings and performance, further showing the internal impact of the proposed dynamic radii selection.

- The **results** are good and show that the proposed method improves the baseline and exceeds state of the art in the cost-vs-performance tradeoff.
clarity,_quality,_novelty_and_reproducibility: See above regarding quality, clarity, and originality.

Minor points:

- Abstract: "Reduce latency by 100%" - this is somewhat ambigious, does this mean the runtime drops to zero? Which whould be a rather bold claim.

- p.4: Multiple expressions are used for different kind of nearest neighbor searches, and it is not always clear which means what. What exactly is "nearest sampling technique", "top-k", "ball gathering", and "vanilla ball query". It would help if a formal definition of those would be given, or a more common name (such as "k-NN", epsilon-search), and / or a consistent name throughout the manuscript.  
 Also, "For each query, the vanilla ball querying samples a predefined number of local points within a specific spatial radius." - isn't it *either* a kNN-query (select k nearest neighbors), OR an epsilon-search (select all points with distance < epsilon, i.e. a "specific spatial radius")?

summary_of_the_review: The paper presents a novel, well thought of and motivated idea that is fairly and thoroughly evaluated and exceeds state of the art on some metrics. Its general design has the potential to improve several 3D point cloud processing networks that use range queries.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper receives 2x marginally above the acceptance threshold, 1x accept, good paper, and 1x strong reject. 

All the three reviewers who give accepts agree that the authors explained the motivation of the paper very clearly and convincingly through the analysis of the various aspects in the existing point-based object detection model. The idea of using a dynamic network to drop redundant background points is novel. The experimental evaluations are comprehensive to support the claim. The experiments are thorough and fair. The results are good and show that the proposed method improves the baseline and exceeds state of the art in the cost-vs-performance tradeoff. The weaknesses mentioned by these reviewers are mostly additional experimental results and further clarifications that are addressed in the authors responses. 

The metareviewer carefully considers the 1x strong reject, where the main reason is the proposed method is similar to pyramid RCNN. Both papers propose the ball query method. However, the authors' response managed to clarify the issue. Specifically, the authors the clarifications that they have different objectives, propose different architectures and methods, and show different experiments and results compared to pyramid RCNN. The metareviewer is convinced that this paper is different from pyramid RCNN. 
justification_for_why_not_higher_score: This paper receives 2x marginally above the acceptance threshold and 1x strong reject.
justification_for_why_not_lower_score: This paper has 1x accept, good paper.
Decision: Accept: poster