Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes an architecture and training scheme for 3D shape reconstruction that combines the generalizability of models that parameterize an entire shape family with the level of detail of models overfit to a single shape. To this end, a generalizable part-based model is trained for the shape family and another model with the same architecture is overfit to a specific shape. The two models are then combined in order to take advantage of edits via the latent space of the former while keeping the details captured by the latter. As a result, it is possible to perform part-based edits where novel geometry is hallucinated where necessary (i.e., around part joints) while existing geometry (the parts themselves) are faithfully reconstructed.

The authors show experimental results on sheep reconstruction, shape editing, and shape mixing, and they compare to state-of-the-art part-based deep shape reconstruction methods. 
strengths_and_weaknesses: I think this paper is very clearly written and proposes a reasonable and fairly simple extension of previous shape reconstruction methods. I like that it makes a step towards neural implicit reconstructions that are more usable in practice (thank to the ability to accommodate higher levels of detail) while still retaining the benefits of a data driven approach. The experimental results are pretty convincing, and there is a clear advantage in the comparisons.

One drawback is the requirement to overfit a model (which takes ~30 min) for each new target set of parts. Additionally, the method is currently not always able to extrapolate local detail when parts are edited, since the overfit model is limited to the known regions of the parts. This detracts a bit from the claim of being able to make semantically reasonable edits at a high level of detail.
questions: Looking at certain results, such as the first row of Figure 3, it seems that the model is sometimes unable to extrapolate local part texture/style when parts are edited (e.g., the slate on the chair end halfway up the seat on the edited model). 

Can this method be leveraged for compression in a superior way than previous similar models? Is there any way to significantly shrink the generalizable model, since much of the burden of representation is now carried by the overfit model?

Is it possible to improve the time required to adapt this approach to every new 3D model (e.g., retraining the overfit model)? This seems to be a bottleneck compared to using a purely generalizable model; could some initialization or fine-tuning alleviate this?

Minor typos:

L2: "form" -> "from"

L139: $P$ -> $\mathcal P$

L188: $\overline P$ -> $\overline {\mathcal P}$

L232: $1e-4$ -> "1e-4" or "0.0001"
limitations: Limitations and directions for future work are adequately discussed.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents NEUFORM, a neural representation that combines the benefits of detail preservation of overfitted representations and edit-ability of generalizable representations. In the generalizable representation the shape is represented as a set of part parameters and an occupancy function models the occupancy field of the shape at any query location. The model consists of a part mixing network, a part query network, and a global occupancy network. An overfitted occupancy function with the same architecture as the generalizable representation is used to capture geometric details. The part latent vectors are frozen before overfitting and only the query network and the occupancy network are updated. A scalar blending field defines a blending factor between the generalizable and overfitted representations at each query point. The two representations are fused by blending between the parameters, weights and the features of the networks given the blending factor. NEUFORM is evaluated on reconstruction, part based shape editing and shape mixing. Qualitative and quantitative experiments demonstrate superiority of NEUFORM over SPAGHETTI [17] and COALESCE [39] as the baselines.
  
strengths_and_weaknesses: **Originality**:
- The proposed idea of combining overfitted and generalizable representation by adaptively blending corresponding network weights is novel and interesting. 
- The authors notice that the problem of shape editing needs both detail preserving and generalizable representations, and it lends itself well to the proposed adaptive model.
- The authors carefully design the generalizable and overfitted networks so that they can be combined well in a hybrid model. 
- They use the idea of blending network weights which is trending topic in machine learning but less studied in 3D deep learning.  
 
**Quality**:
- The paper is well-motivated and well-written.  
- Experiments are thorough and convincing and sufficient details are provided for the experiments to be reproducible. 

**Clarity**: 
- The proposed approach is described clearly. The authors provide intuitive explanation of their choices as well as mathematical abstraction of the proposed model. 

**Significance**:
- The problem of shape reconstruction and editing is of great practical value and the current state-of-art methods create noticeable artifacts and are far from perfect. 
- The paper provides significant value to the research community as it shows considerable quantitative and qualitative improvements over the state-of-the-art across tasks such as reconstruction, shape editing and shape mixing. 
questions: - As commonly done in prior work, the authors train separate models on each category, and they do not discuss whether training a generalizable model jointly on multiple categories help or hurt the performance. The model architecture is not directly tied to a fixed category, and shapes from different categories can share similar-looking parts. Is it possible to share knowledge across categories to further improve reconstruction and editing?  
- It would be good to have the same surface color for shapes from NEUFORM vs SPAGHETTI in Figures 5 and 6. 
limitations: - There is no potential negative societal impact. 
- The authors note limitations of their work in terms of limited explicit knowledge-sharing between over-fitted and generalizable models and non-data-driven blending field.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a shape editing method which works by combining generalized and overfit occupancy models. By applying the generalized model to edited regions, the overfit model to unedited regions and smoothly interpolating between them they naturally achieve more attractive shape predictions. With this method they demonstrate superior shape reconstructions, shape edits and and shape mixing. 
strengths_and_weaknesses: The strengths of this paper is that the idea is presented and works well. I have no qualms with the methodology, presentation or results. 

My main concern is the time scale needed to begin editing a new shape. It is indicated that it takes 25 mins to train an overfitting model, which if I understand will be required for each new shape you wish to edit. This seems prohibitively long and I think should be addressed in the paper somewhere, with a table comparing the time to edit for the various methods you compare to. At the very least it should be mentioned in the limitations. 

With that said the editing and mixing results are impressive, and the idea and results are presented very well.  
questions: Why is the overfit model trained without joint ground truth in the reconstruction task? I am not sure it makes sense to evaluate the overfit  or the NEUFORM models in this setting as reconstruction is a generalization task and you are overfitting to shape in both situations. I may be miss-understanding something here though.
limitations: Yes , though I mention above a possible addition.
review_title: null

Meta-review: metareview: All the reviewers appreciated this work combining generalization capabilities of shape-space models and the fidelity achieved in a single shape overfit/optimization. 
confidence: Certain
Decision: Accept