Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: 
This paper presents a simple (and convincing) idea that neural networks can be employed to "memorize" the surface geometry of 3D objects. Such memorized networks -- dubbed neural implicits -- exhibit significantly lower memory requirements, while allowing for better surface approximation compared to alternative prior approaches.

## Strengths

**S1** This paper pursues a very timely research direction: several interesting approaches over the past year [A, B, C] (to list a few) have been proposed that leverage the overparameterized nature of neural nets to "memorize" the surface geometry of objects. While prior approaches require the neural network to learn a "distribution" over multiple shapes of a category and "generalize" to unseen shapes at test time, this paper poses a different question: can neural nets learn to reproduce one specific shape (by overfitting). This essentially turns a bug (i.e., overfitting) into a feature (i.e., neural implicits), which is a neat idea!

**S2** The paper is extremely well-written, and a thoroughly enjoyable read. Most design choices are well-motivated, and where applicable, empirically/qualitatively justified.

**S3** I find the discussion about various sampling strategies insightful. This section (2.1.2) makes apparent several intricacies that usually fly under the radar. Although I do have a few reservations (see discussion under "Weaknesses"), I think the paper does a good job of explaining how sampling strategies such as [A] result in unintended bias. It is worth noting that there isn't consensus in the implicit-representation-learning community about what sampling strategies work better than the others (eg. [A] and [B] present conflicting accounts on this matter).

**S4** In general, I find the experiment setup convincing. The code and/or data release might see a lot of community interest in following up on this work.

## Weaknesses

I do think this work can benefit from a number of additional discussions and/or explorations, which I list below.

**W1** It appears that the focus of this work is in reconstructing the geometry of the surface. Often 3D meshes have other important attributes that are essential for downstream tasks (face normals, color and/or texture information, material properties). This feels like an important limitation that could benefit from discussion. (Do we expect learning textures, normals, etc. to be "trivial"? In which case it'd greatly strengthen the paper. If not, explicitly highlighting this could result in interesting follow-up work).

**W2** While the paper paints the rosy side of neural implicits, it might benefit from quantifying the "ineffectiveness of weight-encoded neural implicit 3D shapes". How well do surface continuity properties hold (C0, C1, and higher) hold? What can possibly be done to mitigate such ill-effects (if any)? While Sec. 4 talks about these questions being interesting future directions, it might add value to the current submission to quantify the gap (eg. Eikonal constraint violation, watertight-ness or manifold-ness violations) to be comprehensive.

**W3** Traditionally, multiple representations have flourished in the computer graphics community, primarily because each of them possesses a complementary set of strengths and weaknesses. I'd assume that to hold true for neural implicits too. For instance, rendering neural implicits, computing higher order information (curvature? derivatives?) and above all, interactive editing applications might be more inefficient with neural implicits?

**W4** I feel the discussion about sampling strategies could greatly benefit from an accompanying empirical analysis. While most arguments in this section (2.1.2) appear intuitive, they're often hand-wavy.

**W5** Since one of the primary goals of neural implicits is to reduce memory requirements for storing and processing polygon meshes, I hoped to see more discussion on "classical" data compression techniques used in the computer graphics community. A seminal reference is [D].

**W6** Discussion of a few other closely related papers, such as [E, F] can be added.

## Minor remarks

**M1** The following remarks have had zero-impact on my overall score, and as such, I do not expect the authors to respond on these issues.

**M2** The abstract contains a few (minor) grammatical errors that cound benefit by a proofread.

**M3** A concurrent recent work [G] seems to follow a fairly similar idea, and it would be nice to discuss and contrast the current submission wrt this (particularly for the camera-ready version).


References
==========

[A] Mescheder, Lars, et al. "Occupancy networks: Learning 3d reconstruction in function space." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.

[B] Park, Jeong Joon, et al. "Deepsdf: Learning continuous signed distance functions for shape representation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.

[C] Sitzmann, Vincent, et al. "Implicit neural representations with periodic activation functions." arXiv preprint arXiv:2006.09661 (2020).

[D] Pierre Alliez and Craig Gotsman. Recent Advances in Compression of 3D Meshes. Advances in multiresolution for geometric modelling, 2005.

[E] Hanocka, Rana, et al. "Point2Mesh: A Self-Prior for Deformable Meshes." arXiv preprint arXiv:2005.11084 (2020).

[F] Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. Matthew Tancik*, Pratul Srinivasan*, Ben Mildenhall*, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, Ren Ng.

[G] Overfit Neural Networks as a Compact Shape Representation. arXiv. 2020.
review_title: Enjoyable read

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper discusses implicit SDF representations encoded by neural networks. 

The paper is written partly as an opinion paper, partly as if it was an enormous contribution and completely new idea to overfit a network to a simple shape (thus not requiring a latent vector) compared to using a latent code to obtain a common latent space for a set of shapes. It is thus constantly written in opposition to Park et al. 2019, which I found useless and tiring to read, takes far too much space to my taste and I think is detrimental to the overall quality and interest. The idea of encoding an implicit function in the weights of a network without latent code is not new at all, and in particular common practice in all the NERF literature. The fact that it will lead to better reconstructions than learning a common latent space is obvious.

To me, the most interesting parts of the paper is the tools it proposes to deal with implicit SDFs, that are, independently of this paper, becoming more important in 3D vision: the use of generalized winding numbers to encode general meshes and efficient visualization tools. Admittedly, these are known in the graphics community, which may make the paper hard to publish in a top tier conference, but I believe modern tools compatible with recent DL libraries would be valuable and a paper introducing them could have a good impact. Similarly, a pipeline allowing CSG operations could have been presented and analyzed (further than a few qualitative examples)  The current version of the paper however provides too little details and evaluation on these and on the contrary spends a lot of time on discussion which I felt were not any real contribution.

In more details, I from my point of view
- sections discussing relation to deepSDF (1 and 2.1.1) could be drastically compressed (and section 1 could be rewritten completely)
- the paper is missing a related work 
- the interest of 2.1.2 is very limited. Yes, sampling is equivalent to weighting, this is more or less importance sampling. Some papers prefer to define the sampling, this one prefers to define the weight. The results to support this strategy seem anecdotical and insufficient (one would have to carefully cross-validate the parameters in both to draw any conclusion) Moreover, it is far more costly since need to start by sampling a much larger set of points.
- I find 2.2 and 2.3 of potential interest, but they are not much developed nor evaluated
- 3.1 is missing the crucial evaluation of the rendering quality
- 3.2 focusses on comparison with deepSDF which I think does not bring much since it is a weak baseline/not designed for the same purpose + should be quantitative, not qualitative
- I did not find 3.3 to be very informative
-  the paper is missing clear evaluations and quantitative analysis (which is related to the fact its contribution is unclear)
- the conclusion (4) finally discusses the issue of normals, which I believe should have been addressed earlier



Some more comments: 
- I think the "neural implicit" term introduced near the first inset is not well chosen (implicit is an adjective) but in any case it should not be used so much in the intro before its introduction
- the quality of the reconstructions is actually not great, this is not very visible in a printed version because all images are small, but zooming on the bunny for example actually shows it is quite coarse. This is an important practical problem, and one could imagine how to jointly work efficiently with representation of different quality (even if that sounds more like a graphics paper than an ICLR paper) Also, from the video it seems training is not super stable (which is surprising, maybe decreasing the LR after a number of epochs would lead to better results)
- the MLP encodes a linear by part function, the zero-level-set   (which does not change after the tanh) is thus a set of planes, resulting in a planar by part approximation of the shape, which is very visible in the qualitative results and should be discussed. (this also implies that the same quality for a similar budget can be obtained with a mesh)
- 90 seconds to obtain a representation is still very expensive. Indeed, if one were, as suggested in the paper, to use the representation to perform CSG, one would need to either save the full construction process or convert often into the implicit representation. It's also unclear how errors would accumulate when re-encoding several time a given shape. 
review_title: some interesting idea, but meeds more work on presentation and evaluations

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper proposes a weight-encoded neural implicit representation for 3D shapes. The idea is to encode every shape in the network weights of its own designated small MLP network, instead of trying to learn a latent space of shapes. This leads to a really compact shape representation based on signed distance fields that could be interesting for many applications. The approach uses importance sampling to speed up training and robust losses.

The paper evaluates the representation in terms of visualization efficiently, stability, and the compactness of the representation on a large dataset of shapes. The approach outperforms latent encoded shape representations such as DeepSDF in terms of compression and accuracy.

I believe the idea of encoding shapes using signed distance fields and MLPs for compression is really exciting. Such implicit representations of shapes are becoming really popular at the moment and have huge potential in the future. In terms of novelty, I have seen many shape representation approaches based on MLPs lately, but none so far focussed on the compression aspect.

In summary, I believe that the compression aspect of this work is really interesting and will inspire follow up works along those lines. The method achieves impressive compression ratios and can decompress/render the shapes still at high speeds.
review_title: Weight-encoded neural implicit representation for 3D shapes

Meta-review: The paper proposes how weight-encoded neural implicit can be strong 3D shape representations. A neural network is trained such that it overfits over a single shape, and the weights of such network is a great representation for the 3D shape. Results are shown on signed distance field (SDF) generation from meshes.

Strengths:
- an interesting idea for generating compact representations of 3D shapes
- Will further foster several conversations within the deep learning community

Weaknesses:
- Very limited evaluation to support the authors  claims, particularly against other traditional learnable 3D representations
Decision: Reject