Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: Summary of the paper:
This paper proposes a shape generation methods where shapes are represented by a set of parametric patches. Sets of patches from the same category will follow certain constrained defined by templates. The templates are generated from some abstract representation of the category of interested (i.e. sets of cubbies). 
The authors evaluate the method qualitatively on sketch-to-shape task, and show that the methods allows generating meshes that’s editable, interpretable, and with certain level of details.

Strength:
1. The representation of parametric patches have several advantages compared to prior works (whose patches are represented by neural networks). It can create sparse, compact, interpretable, and editable shapes that can be directly imported into many industrial design software.
2. The parametric patches allow computing CD without uniformly sampling from the surface.

Weakness:

1. The paper’s evaluation of the method is very limited. Even though the representation has many inherit advantages, I still expect the authors to demonstrate the method’s advantage in different tasks (not only sketch-modeling), different datasets (not just limited to rather small categories like Mugs), and across baselines with different representations (such as implicit representation, etc)
2.  A major weakness of the methods comes from the template generation. It occurs to me that the paper requires human user to pre-defined templates using a set of cuboids for each set of shapes before it can start generate. This makes it a little bit hard for the method to capture all topological variation of the category (which the author also mentioned in the last section). With this regard, the method requires additional information compared many prior methods (for example, implicit methods that can handle many topology and good details, and it doesn’t take a template). 
3. There are potentially missing related works. Some of which I think the paper should compare with as baselines:

[A] Nash, Charlie, et al. "PolyGen: An autoregressive generative model of 3D meshes." arXiv preprint arXiv:2002.10880 (2020).
[B] Deng, Zhantao, et al. "Better Patch Stitching for Parametric Surface Reconstruction." arXiv preprint arXiv:2010.07021 (2020).

While the paper’s idea has its value as the representation could be useful for editing in many industrial software, but it requires more sophisticated evaluation (quantitative evaluation, more baselines, more tasks) in order to show-case the effectiveness of the representation. With that, I do not recommend the paper for ICLR.
review_title: Good idea with potential novelty, but definitely needs more evaluation

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper presents a method that leverages parametric surface patches as the fundamental representation in the task of shape modeling and reconstruction. This method requires a pre-generated template for each shape category. Several losses are specially designed to regularize the generation of the surface patches. Empirical results have demonstrated the performance of the proposed method in sketch-based shape reconstruction and 3D shape interpolation.

Pros:
- The paper shows some good results on reconstructing some simple shapes, such as bottles, mugs, bathtubs, etc.
- The idea of using parametric surface patches may reduce the parameter space that a network has to search and could potentially lead to smoother surfaces.

Cons:
- It is not clear to me why such representation is advantageous over conventional polygonal mesh-based representation. 
1) Though the parametric surface patch can typically lead to smoother reconstruction, it may also suffer from the incapability of capturing fine geometric details.  
2) Similar to the mesh-based approach,  the patch-based method still has to fight similar challenges, e.g. avoiding the intersections, undesirable shape distortions, etc. 
3) Editing-wise, the reconstructed polygonal mesh can be easily converted to parametric surface patches to ease the following editing workflow, by registering the pre-generated template of coon patches to the reconstructed shape. 

- There are no quantitative evaluations in the paper.  Also, the visual comparisons with other approaches are very limited -- only one or two examples are presented. Due to the lack of evaluations, it is difficult to determine the effectiveness of the proposed approach.

---- Final Rating ----

The revised version of the paper with additional experiments has addressed my concern on the limited advantage over prior deep 3D representation.  Fig.6 in the revised version has shown the potential of the proposed approach in generating directly usable representation in downstream applications, including CAD design and manufacturing. Hence, I would change my rating to positive.

However, the paper still lacks quantitative and qualitative evaluations. I hope it could be properly addressed in the final version.


review_title: Motivation is not clear, lack of quantitative evaluation

final_score: rating: 7: Good paper, accept
review_content: The paper proposes a self-supervised method to fit a template (represented as a union of Coons patches) to a certain 2D sketch. It derives a way to build a proper template, uses a network to predict the patches' parameters, and proposes a union of different losses. The qualitative results of the method are shown in several different objects. 

PROS
- Losses: I appreciated the effort of the authors in formulating an ensemble of losses for this specific representation. I think they will be useful for future works with similar representations.
- Versatility: this representation permits working with a large variety of different classes of objects (also disconnected ones, but not topological changes), at an arbitrary resolution and predicts a small set of values. 

The paper does not clearly state if the code and the data will be made publicly available, but since the code is attached to the submission I assume this is in the will of the authors, and this is also a contribution.

Finally, the paper is well organized and I like the presentations.

CONS
- Quantitative analysis: the only quantitative evaluation is reported in the supplementary material. I think this is the major weakness of the paper. I would suggest having more experiments to analyze the performance of the method. In particular, the ablation is a critical part since many losses are involved; I would suggest providing some quantitative measures, for example showing how each loss improves a particular aspect (e.g. the distance from the ground truth, the difference between normals, the intersection area, ...).

I would also suggest highlighting the output patches on some qualitative results.

Minor fixes:
- Equation 4, fix the pedix of $\mathcal{U}_{\[0,1\]^2}$ 
- Equation 13 (Sup.Mat), fix the Integral limits locations
- I think "P2P-NET: Bidirectional Point Displacement Net for Shape Transform"(Yin et al., 2018) worth a mention since solves (among the others) a similar problem


PRE-REBUTTAL RATING
I am favorable to accept the paper because it provides several tools that will be useful for future works, and I think it has interest points for the Computer Graphics community. For the acceptance I would recommend adding some more quantitative analysis, both for ablation and for the shown results.
----------
FINAL-RATING

I read the other reviews and authors' replies carefully.

First of all, I would acknowledge the effort of the authors in replying to reviewers' concerns.

About my points, I agree with other reviewers that the quantitative analysis is a bit limited (I think it is also the main criticism), but the introduced ablation and the new Figure 6 (numerically comparing against other SOTA methods) are convincing. The authors solved also my other concerns, and so I raise my score; I think the study of this representation is interesting, and well fit the audience of ICLR.

Thanks to the author for their availability, and best of luck with their work!

review_title: A nice study, with flaws in quantitative analysis

final_score: rating: 7: Good paper, accept
review_content: I like the idea of this paper and think that finding new representations of 3D shapes in deep learning approaches is important and interesting. However I have a number of concern with respect the claims of this paper, and the lack of sufficient evidence backing them up: 

-> What does "sampling uniformly from patches is difficult" mean? Can you clarify this statement both here, and in the paper? 

-> You provide all these detail about how you make the chamfer distance and normal alignment work for patches, but then say you can triangulate the patches anyway. If you can triangulate the patches why can you not directly apply the chamfer distances and normal alignment to points uniformly sampled over these triangles?  See: "GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects" or "Point2Mesh: A Self-Prior for Deformable Meshes" for detail on how this is done. Would this not be easier? This would probably also be true for the patch flatness regularizer, where instead you could condition adjacent faces to have low curvature such as in the smoothness loss of "Neural 3D Mesh Renderer". This is not to say that the approaches you have taken are wrong, but you are proposing alternatives to solutions that already exist, perform well, and are naturally applicable to your setting. I think is it very important that you compare your approaches to these in some way in the paper. An experiment to this effect would go a long way to increasing my score.


-> The ablation you provide in the supplemental is not sufficient. Demonstrating the difference over a single model does not provide me a good understanding of the differences. You should provide a visual ablation over a larger number of  models, and in addition provide a quantitative ablation of your metrics( chamfer distance, normal alignment) over your whole test set. This is absolutely necessary. Again showing this here and in the paper will go a long way to increasing my score. 

-> You provide an explicit definition of patches, but no intuition as to what they are. A reader can go and look up coons patches independently, but it would be helpful to provide some help here. A diagram might be helpful to explain this. Patches seem to be quite similar to charts as referenced in "AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation
" and "3D Shape Reconstruction from Vision and Touch". It might be helpful to disambiguate this in the related work. I realize you touch lightly on this in the results section, though this it too late in the paper to establish the understanding on your approach. 

-> There is a lack of quantitative comparison in this paper. I see that you compare visually to Pixel2Mesh and AtlasNet, but this is only over a small number of models. A reader has no way to determine if these results have been hand picked to make your approach look superior. You need to provide a comparison to contemporary methods for 3D reconstruction from images. Here Pixel2Mesh and AtlasNet are probably not sufficient either as better performing methods have since been released. The comparison in the supplemental is not sufficient, It should be over a large number of categories, and methods. 

-> You write: "Unlike meshes or voxel occupancy functions, this representation can easily be edited and tuned after 3D reconstruction". Meshes are easily edited in 3D design software and also can be converted to NURBS. In addition voxels and occupancy functions can be converted into mesh representations and so also editable in these software as well. In fact I believe 3D deep learning frameworks exists which handle this conversion natively. I think you should be clear about these claims and make them less strong. 

I want to be convinced that the proposed approach works better that what is already available , or at least better in some specific domain. The paper in its current form does not do this due to the concerns raised above. I am willing to revise my score if they are properly addressed. 

---------------------------------------------------------------------------------------------------------------------------

I believe that the authors have adequately addressed my concerns, and have made a good effort to improve the quality of their work and so I am raising my score. 

review_title: Claims need to be backed up. 

Meta-review: Description:
The paper presents a patch-based 3D representation of man-made shapes that can be computed with deep learning and used directly in existing CAD applications. This representation is based off a deformable parametric template with Coons patches. Results in sketch-based modelling tasks shows comparable results with STOA

Strengths:
- The patch-based representation provide several advantages: compact, sparse, interpretable, consistent and easily editable.
- Can infer the right template, and thus does not require manually created templates

Weaknesses
- Limited evaluation restrained to mostly sketch-based modelling, and missing evaluation against a few STOA methods

The paper has introduced a very impactful new representation for 3D shapes and has strong technical novelty. I recommend, as reviewers have suggested, more in-depth quantitative evaluation (against other work and ablation studies)
Decision: Accept (Poster)