Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper describes a method to compute correspondences from shapes with application for shape registration, shape interpolation and unsupervised Segmentation Transfer. The authors use an implicit (meshless) shape reconstruction based on the moving least squares as a shape representation. An autoencoder provides a way to create a compact representation of shapes which is then used to solve correspondence problems.
strengths_and_weaknesses: Strengths : 
i) Novel shape representation for shape encoding inspired from meshless representations.
ii) extensive experiments showing the ability of the proposed approach to solve  shape registration, shape interpolation and unsupervised segmentation transfer problems.
iii) The paper is fairly well written with a lot of implementation details in the supplementary material

weaknesses :
i) the considered problem assumes that the correspondences between shapes are known at training time 
ii) Only displacement fields are estimated that may lead to non diffeomorphic deformations. This may be a limitation 
iii) the main concept of the paper is borrowed from 3D-CODED but the shape representation is original 
questions: - The choice of the control points seems to be essential to obtain a compact representation that captures the important details. Explain how to properly choose the number K of those points ? Why not make it as large as possible ? How the local radius r_i are chosen ?

- It is not clear where the section "Extending sparse to dense Correspondence" and Eq(12) is used in the result section. Make this more explicit  or remove the section.

- In the test time refinement, it is difficult to understand what is being done without reading the 3D-CODED paper. Improve this section. Why is the optimization taking place only for Z_y and not for Z_X ? This breaks the symmetry between the 2 shapes.

limitations: - The authors have cited 2 limitations (non diffeomorphic deformations + optimization of control points) which are relevant
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents an efficient approximation of deformation fields using nodes plus local weighted least-square fitting. The proposed representation enables easy computation of Jacobian and thus facilitates first-order regularizations. Experiments on multiple tasks show that it performs well compared to recent existing methods.
strengths_and_weaknesses: Strengths:
- The paper introduces a deformation field representation for more efficient learning of shape correspondences. It also enables easy computations of multiple regularization losses based on its accessibility to the Jacobian.
- Good experimental results show the effectiveness of the proposed method compared to prior works. The authors also presented multiple applications such as interpolation and unsupervised segmentation.

Weaknesses:
- I feel the point of preserving "high-frequency components" (claimed in line 32-37) is not well addressed in the experiments. Apart from the generic tasks, specific experiments demonstrating the method's superiority on this can make the argument stronger.
questions: Questions:
- In the node sampling in Section 4.4 it says the method "exclude(s) a node that exerts its influence in semantically different regions" (line 223), does it mean that semantic segmentations/annotations are implicitly used? -- although it seems to be a minor issue as the sampling may not be the core of the proposed method, using this information still sounds improper as they can be strong hints to correspondences (or in other words, if you already have access to this information, why not also using them in the correspondence computation...). I wonder how the method works if the node samplings involve no semantics? Or correct me with how "semantically different regions" are defined -- my understanding is that the segmentations are from SMPL[30] ground-truth annotations.

Suggestions:
- In Section 3.2, it can be made clearer which contributions are from prior works and which are proposed in this paper. For instance, the Moving Least Squares are from [16], but for the following shape representations, are they simply existing formulations, or are there any adaptations to the specific settings here?

Other minor suggestions:
- (line 3) "Differently from" -> "Different from"
- The plot in Figure 5 left is too small. It's a bit hard to read the caption and legends.
limitations: The authors have addressed the limitations of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work addresses the issue of attaining correspondence between non-rigid objects. 

Conventionally, methods using intrinsic representation require accurate mesh for each object, whereas extrinsic methods require various annotated data for training to achieve accurate correspondences. The authors propose to take an intermediate approach and use ''nodes'', or fixed control points that determine the deformation of the object.

To achieve this goal, they define the influence of each nodes using a shape function based on moving least squares, an established deformation model, and propose to learn the corresponding deformation parameters that determine the shape. One of the shapes is selected as a template shape, and the deformation parameters required from the template to each target is learnt.

To constrain the learnt deformation, the paper proposes to add various constraints using the Jacobian of the deformation field, which can be precomputed for the template shape, making the calculation significantly efficient. Furthermore, they add a constraint on the latent space so that intermediate shapes between the template and the target can be generated by traversing a continuous path in the resulting latent space.
strengths_and_weaknesses: ### Strengths

I was very intrigued by how the authors utilized well-established algorithms and models in conventional non-rigid shape analysis works, such as moving least squares and constraints using the Jacobian of deformation fields, and focus the learning on the corresponding deformation parameters, and not the point coordinates themselves. This enables the proposed method to effectively recover deformation parameters around the fixed nodes, which can be diffused to the surrounding regions to estimate the deformation field around the entire shape. Various experiments show that the resulting correspondence is highly accurate, indicating that the deformations were properly conducted.

The overall accuracy, especially with respect to achieving proper correspondence among different poses is very impressive. I presumed that data-driven extrinsic methods would perform rather well, however, according to the manuscript, the proposal is able to achieve highly accurate correspondence while using less data for training. This approach seems useful in practice, as it does not require accurate meshes.

The continuous constraint on the latent space is also interesting, as it enabled smooth transition between each of the source pose and the template. The intermediate poses in Fig. 5 demonstrate the effectiveness of such constraint. The authors also thoroughly demonstrated the role of each constraint in the supplementary material, which further supported the necessity of each element.

### Weaknesses

The node sampling strategy in Section 4.4 seems to imply that the deformation is mainly articulation-based, rather than a fully non-rigid one. I was actually curious about why this segmentation-based sampling is required, as I presumed that the deformation function based on moving least squares would nicely blend the influence from each node based on the distance away from it. I think this results in a rather curved corners, as can be seen from the purple figure in Fig. 5. The elbows seem rather rubbery, as the deformation parameters in those regions presumably had to be approximated from afar.

Experiments using shapes that have topological differences, e.g. in Fig. 4, seem to be random and do not support the true contributions of the proposed method. As the constraints and losses are designed to match corresponding shapes there is not much point in applying the method to shapes where only semantic correspondence exist, despite seemingly providing robust correspondence where available.

The detail about the order of the polynomial basis $p$ used in the experiments seems to be missing.
questions: What kind of poses would be difficult to be used as the template?

I am curious as to why the data-heavy baselines such as 3D-CODED cannot achieve high correspondence accuracy compared to the proposal. The authors mention deformation requiring less parameters, but the deformation model used in the method is rather conventional and presumably less flexible compared to the learnt one. What was the key difference? Is the lack of constraints in the prior methods the main source of weakness?

In Section 4.2, the authors mention that correspondence between unknown shapes $(\mathcal{X}, \mathcal{Y})$ is found by extracting mappings $(\mathcal{D}_\mathcal{X},\mathcal{D}_\mathcal{Y})$ and then finding the closest neighbors. However, if I understood correctly, doesn't Dx and Dy represent deformation required for template $\mathcal{T}$ to deform to $\mathcal{X}$ and  $\mathcal{Y}$? If so, the end poses are different and nearest point would unlikely be true correspondences. I think the inverse transformation is required to establish correspondence. I would like some clarification regarding the test phase. Or, does this mean that $\mathcal{D}_\mathcal{X}$and $\mathcal{D}_\mathcal{Y}$ are templates transformed to \mathcal{X} and \mathcal{Y}, and closest points corresponding to each point on the deformed templates in $\mathcal{D}_\mathcal{X}$and $\mathcal{D}_\mathcal{Y}$ are considered correspondences? This would make sense but is not very obvious from the text.

In many cases involving human figures, the head does not seem to be turning. Is there any particular reason for this phenomenon?

What would happen if the nodes were more evenly distributed, without removing any from near the segment boundaries? Would the experimental results be worse?
limitations: The authors do disucss the limitation of the proposal. One is the bijectivity of the correspondence, and the other deals with the fixation of the nodes. 

I would also like to see what would happen for shapes whose parts are difficult to determine (e.g. human organs, etc.), as the term ''non-rigid'' includes such objects as well. In other words, the title seems to suggest broader application, whereas the paper rather mainly targets articulated shapes.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper computes the correspondences between two non-rigid 3d objects with point cloud or mesh representations. From given 3D inputs with only limited supervision, the proposed method effectively understands the correspondences and matches suitable parts between two different 3D objects. The authors proposed how to represent a parametrized compact deformation in the problem of 3D matching. In the experimental section, the authors have shown the effectiveness of their proposed method in terms of non-rigid shape matching, registration, unsupervised part segmentation, and interpolation while comparing to each baseline method. The paper is also positive to publicly share their codes and models proposed in the paper.

strengths_and_weaknesses: [Strengths]
S1) The MLP-based representation approximates fixed-length parameters. The compact 3D representation matches a high-level matching between two different objects, despite the small number of training samples are worthy of evaluation. By using the proposed method, the authors have achieved good performance rates in comparison with conventional methods and is expected to become a good baseline in this field.

S2) Experimental comparisons have been made with the respective baseline methods in diverse tasks (non-rigid shape matching, registration, unsupervised part segmentation, and interpolation) to demonstrate their superiority. These comparisons show the effectiveness of the proposed method in wide-range of tasks.

S3) The code distribution in itself has a positive impact to the 3D matching community. In terms of reproducing a 3D matching method that is compact yet achieves high performance, it is easier for readers to try the proposed method. Additionally, the code would make the paper more understandable.

[Weaknesses]
W1) While the proposed paper has very good properties - a compact feature representation, small amount of training samples, and higher matching performance - there exists weaknesses that the detailed parameters are not well demonstrated through experimental section. 

W2) Although the number of training samples are suppressed which is a good aspect in the paper, the lack of a relationship between the #training-sample and their performance is insufficient. There is no guarantee as to how much training sample is required in order to reach a sufficient level of 3D matching performance. The authors can appeal to the research community if the proposed method enables to train a well-deserved 3D matcher from extremely limited number of training samples.

W3) Eq. (6) shows the objective functions, however, the mathematical equation is not validated in terms of their combination and contribution to the training at each loss function. It would be better if the loss functions of importance are clarified.

questions: - In the end, does the matching be done by 3D Point Cloud instead of mesh? In case of mesh input, will the conversion must be processed in order to translate 3D Point Cloud?

- Why does the compact representation reduce the number of training samples? It would be better to have a discussion in the experimental section, even if it is just a citation of related studies and their discussion.

limitations: - I can understand the paper limitation due to 9-page restriction, however, the discussion part is not sufficient in order to judge the effectiveness in the proposed method. At the current revision, the authors list the fact of results from performance rates and qualitative results in figures. More detailed discussion should be required such as "why is the proposed method effective in the category 'twisted'", and "why can the proposed method train with a limited labeled 3D data". I believe that a more detailed discussion improves the paper quality.
review_title: null

Meta-review: metareview: This paper presents a creative approach to shape registration that incorporates a new parameterization of the deformation field.  All authors (and the AC) agree the results are convincing and that the method presents novel and interesting ideas.

The only (borderline) negative review by reviewer 86v4 seems to be well-addressed by the rebuttal and new ablation experiments; although reviewer 86v4 did not engage during the rebuttal discussion, the AC checked these results and found them reasonable.

Hence, a recommendation of "accept" is suitable here.  The final version of the paper should be sure to incorporate any new experimental results that appear in the rebuttal discussion.
confidence: Certain
Decision: Accept