Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents Neural Deformation Pyramid (NDP), a learning-based method for non-rigid point cloud registration.
Given a source point cloud and a destination point cloud, NDP learns a dense SE(3) warp field (or dense Sim(3) warp field in the case of scaling difference), that is, to learn a rigid transformation for each point in the source cloud, so that the source cloud is best aligned with the destination cloud.

The proposed NDP seeks this warp filed in a hierarchical manner. Specifically, the warp field is decomposed into multiple layers of learnable MLP's, where at each layer the input 3D coordinates are passed through sinusoidal encoding with increasing frequencies. In this way, the NDP decomposes the non-rigid transformation into rigid transformations that go from a global scale to more local refinements. I found this design to be very intuitive.

Experiments compare NDP with a pretty large set of both non-learned methods and learned methods, and demonstrates the superior performance of NDP in terms of accuracy and efficiency.
strengths_and_weaknesses: Strengths:
+ The intuition of hierarchical motion decomposition is intuitive and well motivated in the introduction.
+ The design of the network and its description is succinct and to the point.
+ Experiments are comprehensive, and support the claimed benefits of the algorithm.
+ Limitations are honestly acknowledged.

questions: Is NDP (instead of LNDP) basically solving an optimization problem without any supervision?
The Chamfer distance and regularization term do not seem to require any groundtruth data.
If so, do you find the initialization of the network weights affect the performance of the optimization problem? (in terms of quality of registration)
limitations: Limitations included.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a neural model for non-rigid point cloud registration. The key idea is to decompose the desired non-rigid deformation into a hierarchy of rigid/similarity deformations, and train a neural model to predict the deformation at each level. The proposed method performs well in the presented evaluation.

The main contribution of this paper is the idea of hierarchical deformation and the proposed neural model to predict it.
strengths_and_weaknesses: Strengths:
- The idea of hierarchical neural deformation is interesting.
- The method performs well in the evaluation.

Weaknesses:
- It is unclear why the proposed loss function is suitable for partial-to-partial registration. In particular, the chamber distance is based on sum of squared distance from each point to the other model. Minimizing this loss would attempt to reduce the distance for *every* point, including those that do not correspond to any point on the other model due to the partial overlap. In fact, existing work for robust non-rigid registration typically identifies such L2 distance as a source of poor performance for partial-to-partial registration and replaces it with other robust norms (e.g., L1) to allow for large distances on some points. It is surprising that using such a term in the training loss would work well for partial-to-partial registration in the current paper (for the unsupervised case at least).
questions: See the weaknesses above.
limitations: It is adequate.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a novel method - Neural Deformation Pyramid (NDP) for non-rigid point cloud registration. Non-rigid point cloud registration remains a challenging problem as the input 3D data often contains noise, outliers, and occlusions. The proposed method NDP deals with the above complexity by decomposing motion hierarchically. The NDP method uses coordinate-MLP as the basic structure, similar to existing works Nerfies [30] and SAPE [13]. The NDP outperforms existing methods on the 4DMatch/4DLoMatch [19] dataset and is faster than existing MLP-based methods.

strengths_and_weaknesses: ## Strengths

- The proposed method NDP is well-motivated, and the connection to existing methods is clearly stated.
- The proposed method achieves better performance on both no-learned and supervised settings quantitatively and qualitatively than previous methods and is faster than the existing method Nerfies [30]. 
- The authors compared their proposed methods with a variety of existing methods.
- This paper is well written and easy to follow.


## Weaknesses
- One of the paper's main contributions is introducing a pyramid architecture to the non-rigid motion registration task. However, there is a range of methods that have already discussed pyramid structures, as justified in Related Work (line 68 to 76). Therefore, there is a lack of technical contribution in this paper. Similarly, the proposed framework is based on coordinate-MLP structure, similar to existing methods Nerfies [30] and SAPE [13]. 
- The authors only conducted experiments on a single benchmark 4DMatch/4DLoMatch [19]. The authors should consider conducting experiments on other benchmarks, such as the 3DMatch/3DLoMatch used in previous research [19].
- In line 25, the authors claim that the proposed method alleviates the complexity of non-rigid registration task (data noise, outliers, and occlusions). However, there are no quantitative experimental results regarding the above complexities. The authors should consider conducting experiments with the proposed method to data noise, outliers, and occlusions.

questions: - The authors claim that the proposed method is faster than the existing method; however, there is no detailed data about the runtime speed of the proposed method and existing methods. The author should report the speed in detail in their paper.
- In line 53, what does 'via skinning’ mean here?
- There are not enough implementation details to reproduce the proposed method, such as the parameters of the MLP structures.
- In 3.3 (line 174), the authors propose the non-rigid registration algorithm. However, there is not enough justification for the algorithm. Such as why the authors optimize different levels in order instead of updating all levels simultaneously. Also, there is no justification why the max_iter is set to 500 and the theta to 15 iterations. 

limitations: The authors have adequately addressed the limitations and potential negative societal impact.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper proposes a non-rigid registration algorithm that also works in the partial-to-partial point clouds setting. The idea is to regularize the deformation in three ways: 1) by dividing the problem into pyramidal (hierarchical) sub-problems; 2) by considering first low-frequency deformations, and scaling to higher frequencies later; 3) by defining for each point a rigid (or affine) deformation (i.e., rotation, translation, and optional scaling). The method is tested on 4DMatch and 4DLoMatch, showing compelling results and outperforming several SOTA methods.

strengths_and_weaknesses: == STRENGHTS ==
- The method seems novel in the composition of its elements. I like the proposed regularizations on the different aspects of the problem. The approach seems relatively straightforward, while probably to be implementable by scratch, further details are required (e.g., a deeper explanation of the architecture implementation, fewer pointers to different papers for details)
- The shown results are convincing on a significant variety of shapes. The attached video gives a nice insight into the low-to-high frequencies deformation.

== WEAKNESSES ==
- The method is tested on various shapes, but the quantitative results are only on near-isometric deformations of synthetic data. The only experiment between significantly different shapes (Figure 6) is not convincing in terms of structural preservation of the shapes (the left-most example shows that A's chest is mapped to B's belly).
- No analysis of noise or clutter is provided. This makes unclear the applicability of the method in real-case scenarios. Also, it mentions the possibility of including the scale factor, but no quantitative results are provided in this case. No analysis of different amounts of point clouds partiality or computational scaling at the different number of points.

Minor fixes:
- line 125: Fig.3 -> Fig. 1
- line 137: 'registration of of multiple scans'
- line 168: 'this regularization help to preserve' -> helps
- [14] is not introduced in the previous works; [a] would be a better alternative than [6] since it overcome some LBO limitations in the point coluds setting

[a]: Correspondence learning via linearly-invariant embedding, Marin et al., 2020

== AFTER REBUTTAL ==

After the rebuttal and the discussion phase, I am prone to keep my initial rating and vote for acceptance. For the final version, I suggest including in the main manuscript the feedback about the method limitations.
questions: - How much the method is sensitive to the introduction of noise, clutter, and point-cloud density?
- How does the method computation timing scale at a different number of points?
limitations: Both societal impact and limitations are fairly discussed.
review_title: null

Meta-review: metareview: All reviewers agree this work is a creative approach to nonrigid registration, which is particularly hard in the mesh-free point cloud setting.  The discussion between the authors and reviewers was extremely productive and addressed most of the major concerns about this work.

In preparing the camera ready, the authors are encouraged to incorporate the new tables of results that appear in the discussion below into the paper and/or supplemental materials, to make sure it is archived.
confidence: Certain
Decision: Accept