Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a novel unsupervised method SNAKE to detect 3D keypoints from point clouds based on implicit neural representations. The key idea is to combine shape reconstruction and keypoint detection during training. Experiments show that jointly learning 3D shapes and key points improves semantical consistency, better repeatability under disturbance, and accurate geometric registration under zero-shot settings. 

strengths_and_weaknesses: + Though the idea of combining reconstruction and saliency prediction is not new (UKPGAN also reconstructs shape), this paper takes advantage of implicit representation and shows advantages over the GAN-based method. The proposed method is simple but effective. The proposed four loss functions are intuitive and ablations show that they are important.
+ The experiments are mostly thorough and show good qualitative and quantitative results compared to competitive baselines. 
+ The paper is well written and the figures are easy to understand. 
- Missing comparison with UKPGAN in section 4.2. UKPGAN is a competitive baseline and its code is publicly available. 

Post-rebuttal:
My final rating is weak accept. Thanks to the authors and reviewers for their effort. The rebuttal mostly answers my questions. I think the paper is novel and has enough difference from UKPGAN -- the method does not have a GAN and the experiment results are better. However, I do think D3Feat descriptor may not be the best way to evaluate the proposed method in experiments since it is designed for D3Feat detector and may not be discriminative enough for different feature detectors. Since the prior works are using this protocol, I won't criticize too much for following it.

questions: - L224 states that UKPGAN is not involved in the experiments due to the absence of pretrained model. Since their code is publicly available, I wonder if training from scratch is possible? If not, I wonder if it is possible to compare alternative datasets? 
- The registration experiments (sec 4.3) rely on D3Feat descriptors for the detected keypoints. I am aware that this descriptor is commonly used in the UKPGAN’s experiment. I am interested in understanding the bottleneck of the problem. Since D3Feat detector + D3Feat descriptor still serves as an upper bound in this experiment, I wonder whether it is possible that certain detector requires specifically designed descriptors in order to work perfectly in the registration problem. D3Feat descriptor may not provide the best features for SNAKE or UKPGAN. 

limitations: Yes.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper is well written, the motivation and technical details are clearly presented. The idea of estimating saliency field from sparse keypoints seem novel, and it is shown to be effective to produce repeatable and consistent keypoint detection result.
strengths_and_weaknesses: Strength:

The paper is well written, the motivation and technical details are clearly presented. The idea of estimating saliency field from sparse keypoints seem novel, and it is shown to be effective to produce repeatable and consistent keypoint detection result.

Weakness:

Keypoint extraction in inference time requires iterative gradient descent and query the implicitly defined saliency field. Thus the computational cost is high and is not suitable for real-world applications at its current form.


questions: Additional visualization of repeatibility of keypoints under SE3 transformation as well as different sparsity of points can make the contribution of the paper more clear. The plots in Figure 5 alone is not visual enough to show the quality of the proposed method in terms of repeatibility.
limitations: Keypoint extraction during inference requires considerable computational cost, not suitable for real-time on-device applications.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper presents an unsupervised method to predict 3d keypoints from point cloud. Several novel losses are proposed to enforce repeatability, Surface Constraint, and Sparsity. They also achieve superior performance on various public benchmarks. They also use the novel two head to model occupancy and saliency separately to better disentangle these two tasks and let them serve as independently different functions.
strengths_and_weaknesses: Strengths：
I like the intuition that starts from continuous instead of discrete space. 
The proposed architectures and losses are novel as far as I see. 
The presentation is very well.
It achieves the SOTA performance on several datasets.
Weaknesses：
Lack of related work discussion: FULLY CONVOLUTIONAL MESH AUTOENCODER USING EFFICIENT SPATIALLY VARYING KERNELS


questions: Are there any other newer methods to compare with? The UKPGAN seems to be in the year 2020.
Can you elaborate more on discrete space and continuous space?
Can you elaborate more of the difference when compared with "R2d2: Reliable and repeatable detector and descriptor"?
limitations: Yes
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The authors introduced a method to detect keypoints from point clouds. They leverage a deep learning model to learn an implicit function which is a mapping to provide each location a probability of being a keypoint. To make the method shape-aware, the authors use deep learning models to learn the kepoint field along with the learning of an occupancy field to explore whether the shape information can improve the keypoint detection. The contribution lies in the way of combining the learning of implicit fields and keypoint fields. The authors evaluate the effectiveness by comparing it with the latest methods under the widely used benchmark.
strengths_and_weaknesses: Strengths:
1. The visualization is good.
2. The paper is easy to follow.
 
 
Weaknesses:
1. The motivation is not convincing at all.
2. The experimental results cannot justify the effectiveness of the method.

questions: 1. First of all, I do not think it makes any sense to learn the probability of keypoints as a field. Keypoints should be located on surfaces, they are not floating in the space. Specifically, the authors aim to detect keypoints from input point clouds, the keypoints to be determined should be some of the input points. It should be much easier to learn the probability of keypoints among the discrete input points than in the continuous whole space.
 
2. The motivation of improving the performance with surface reconstruction does not make any sense either. Surface reconstruction is a harder task than keypoint detection, since point clouds have been given as input which provides a range as a good constraint to find solutions for kepoint detection. We already know the structure of shapes represented by point clouds and why we have to learn an implicit function to reconstruct shapes. I believe it just makes this problem more complex to get resolved.
 
3. The authors claim that the proposed method is a novel unsupervised method. This is also not correct, since this method requires ground truth occupancy information as a supervision.
 
4. The proposed method is also not novel. UKPGAN has explored the feasibility of combining shape reconstruction and keypoint detection together, although UKPGAN reconstructs the input point cloud rather than learning an implicit function to represent the same shape. The minor difference here is what representation is used to represent 3D shapes.
 
5. Learning occupancy field results in using an additional surface constraint. This term should not be there due to the learning of the occupancy field. Fig.2(b) may be confusing, I think the close to 0 line should be parallel to the x axis rather than y axis.
 
6. The term of repeatability loss is hard to understand. Why do we need a term like this? For shapes, they are well aligned, why do we need to consider the rigid transformation? Another question is about the cosine similarity, the probabilities are scalars, right? Is there any reason to use cosine similarity to evaluate the difference between two numbers? 
 
7. The authors use occupancy probability to represent the inverse distance between the query and the input. I do not think it works here. Since occupancy probability always gets smaller from 1 to 0 when moving a query from inside to outside while the inverse distance gets smaller to larger before going across the surface, and then gets smaller after that.
 
8. Explicit keypoint extraction via optimization is also an operation that I do not understand. An intuitive idea of extracting keypoints is to use points of the input point clouds as queries to predict the occupancy values since keypoints can only locate on the surface. Why do we have to extract keypoints via updating the locations of randomly sampled queries in the 3D space?
 
9. In the visualization of saliency field slices, how to select the slices to visualize? 
 
10. How to determine the keypoint number? I noticed that the numbers of keypoints produced by different methods are different even in the same case, such as the visual comparison in Fig.6. Why do not use the same number of points to perform the comparison？
 
11. The reason why I do not think the reconstruction can improve the performance of keypoint detection is the results in Table 2. The results without surface reconstruction are better than the results with surface reconstruction. I do not think this is a good support for the argument.
 
12. With the ground truth occupancy supervision, it is not a fair comparison with other methods.

limitations: Yes, the authors addressed the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: metareview: Post-rebuttal, the paper had split reviews, with three reviewers in favor of acceptance (6, 6, 5 but noted as a 6 in the final comment from 2TwX) and one reviewer strongly arguing for rejection (3). The AC examined the reviews, the paper, and the discussion, and is inclined to accept the paper. The AC is persuaded by the arguments presented by the reviewers in favor of acceptance. While xHwu has raised a number of concerns, the AC believes that the authors have addressed most of these well in public discussion. The AC understands xHwu's positions, but does not see the remaining concerns as grounds for rejection in light of the more positive views of the other reviewers. Given the extensiveness of the discussion, the AC would encourage the authors to use their extra page to incorporate some of the experimental results into the final version of the paper.
confidence: Certain
Decision: Accept