Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a method for representing vector graphic fonts with multi-implicit functions.
The main goal is to allow the reconstruction of fonts at any resolution without loss of discontinuities such as corners and edges.
Instead of representing a font shape as one SDF and then optimising the training process against this SDF, the paper proposes to represent a shape using multi-SDFs, and then differentiably rendering a shape against this multi-SDF representation and optimizing the input shape against this rendering. This allows the method to bypass the need for collecting a vast amount of vector graphics data for training. 
The paper shows that this kind of multi-SDF representation helps in preserving discontinuities at corners and edges.
main_review: Originality: 
The problem being addressed in the paper is an important one. The proposed approach of multi-SDF representation of fonts is interesting and helpful for the problem (as evidenced by the experiments in the paper).

Quality: 
The paper is a good read. It is well articulated and the goals are clearly stated. The paper compares exhaustively to prior works (both image-based and curve-based approaches) on different tasks such as high-resolution reconstruction, font style interpolation, font family completion, and glyph completion. Important ablation on the need for multi-SDF's to recover/preserve sharp corners when rendering fonts in high-resolution has been demonstrated using two resampling techniques -- implicit resampling (which densely resamples the implicit filed) and bilateral resampling (which upsamples the SDF prediction from the implicit model to the target resolution). The experiments show the effectiveness of the proposed approach in modeling vector graphic fonts, both in absolute terms and relatively.

Clarity: 
Except for a few typos, the paper is well-written.
L36 - "resultant" to "resulting"
L62 - "evaluated" to "evaluate"
L97 - "as mentioned above" (instead of "as aforementioned")
L124 - same comment as above

Significance:
I think the problem being addressed is getting attention from the community for the last year. The paper follows this trend and makes a modest contribution to this line of research (which is okay).

limitations_and_societal_impact: Yes, the authors have made a comment about the positive societal aspect of the proposed approach.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper deals with the problem of glyph generation in fontset design. Whereas previous works used raster-based or vector-based representations (and combinations of the two) to learn generative models for fonts, this paper proposes the use of what they call “multi-implicit representations”. In the proposed method, implicit functions (in the style of “NeRF”) are learned to represent subregions of a glyph by combining multiple simple shapes (eg two half-planes to create a corner). These subregion-implicit models are learned with local supervision, and a multi-Signed Distance Field model learns to compose them into the global glyph. 

main_review: # Major Comments
The paper presents impressive results, including great handling of heavily stylized fonts (Fig 6).

The method requires differential rendering, as well as sampling from the individual regions. Can the authors comment on the computational complexity of training this model? Does the individual-glyph training complicate the computational expense further? Is sampling slow, since it has to sample the subregions and compose them? Does sampling scale well with resolution?

I am unsure on the utility of this method for designers. The output of the model is a combination of SDFs which need to be rendered in a custom renderer (within the model), and not a set of commands that can be manipulated in traditional font design software. This is in contrast with the papers that use vector graphics directly, which output a standard format. If this is true I believe it’s important to discuss the method’s limitations, including appropriate citations of methods that don’t have this limitation.

NeRF is trained for a single scene. Is that also true for each glyph/font? It seems like the paper uses a z latent space that was trained with the method in [17]. If that’s correct, it might be worth citing it on line 280.

# Minor Comments
How does it compare to “Learning implicit glyph shape representation?” I realize this is a new paper, but I’d be interested in reading a discussion on the differences of the two approaches
https://arxiv.org/abs/2106.08573v1

I am not as familiar with SDFs and implicit models as I am with font-generation models, so I defer the review of technical aspects of the implicit models to other reviewers

Overall, the method is really interesting and the results seem really good. If my major comments are resolved I would consider increasing the score by 1.

limitations_and_societal_impact: Yes, although I discuss some missing limitations above
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper aims to learn a representation for rasterized fonts that allow for re-scaling, reconstruction, interpolation, and generation by preserving local features such as sharp corners or smooth curves. The proposed method composes multiple implicit representations (i.e., signed distance fields) to model complex details observed in neutral and stylistic fonts. The core idea is to represent corners as an intersection of multiple curves, enabling the construction of sharp corners in arbitrary resolutions. This assumption is further supported by a local supervision objective for the corners. In the experiments, the authors present that their model is able to preserve sharp corners. The proposed model outperforms baselines both qualitatively and quantitatively.
main_review: **Originality**
The idea of representing fonts as a multi-channel signed distance field (SDF) is proposed in [4]. This paper incorporates it into learned SDFs by using the recent techniques in neural implicit surfaces. While the core idea is not novel, this combination enables learning canonical representations across several font families and hence allows for interesting applications such as super-resolution, interpolation, and exemplar-based font generation. 

**Quality**
The paper motivates the problem and the proposed method well. The paper focuses on learning a representation that can preserve the sharp corners in the downstream tasks. At the first glance, it does not sound like a challenging problem. The authors are able to illustrate the problem in the existing methods. The proposed approach for this problem is technically sound and its formulation seems correct. Experiments provide enough evidence showing the proposed method’s good performance compared to the selected baselines. The font completion results in Fig. 7 and the textured font results provided in the supplementary look impressive.

In my opinion, the main limitation of the submission is the baselines. The authors focus on font models and ignore other relevant baselines which leave some of the claims unsupported. Several SDF methods such as DeepSDF [1] and SIREN [2] could be used in the baselines to provide evidence for why multi-channel SDF is necessary. SDFs are very powerful and shown to be very effective in modeling complex 3D shapes. However, the authors claim otherwise in #41, “… implicit representations (e.g., signed distance fields) for fonts are often too complex to be represented accurately …”. I think the DeepSDF method could be a perfect baseline for this task. I am also wondering why the actual multi-channel SDF [4] is not considered in the reconstruction experiments. This would provide a piece of evidence for using a learning-based SDF.
In the ablations, it would be useful to see the contribution of design choices namely the corner loss, the sampling strategy, and using three channels instead of one (i.e., standard SDF with the corner loss and sampling).

Some claims are requiring further discussion or evidence. 
- Related work Ln. 90-92: The proposed approach can also be categorized as a transformation from a raster to a continuous domain. How does it alleviate the problem of lack of finer details in the rasterized images?
- Related work Ln. 82-85: is there evidence for the inferior performance of sequential models compared to CNN-based models?
- Ln. 134 requires citation: “There are many works for estimating d(x; y) from scene parameters but most of them are constrained by the choice of the parameterization.” 

**Clarity**
The overall organization of the paper is good. It is well-written. The illustration provided in Fig. 4 is very helpful for understanding the core idea. 

**Significance**
It is an interesting problem, and the proposed method is very effective. It outperforms the baselines. However, in my opinion, the main problem is the selection of the baselines. It is not able to convince me if the proposed approach is really necessary. I am wondering whether a standard SDF could achieve similar results or not. 

[1] Park et al., DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

[2] Sitzmann et al., Implicit Neural Representations with Periodic Activation Functions

-----------------------------------------------
**Updated:**
I thank the authors for their rebuttal and additional results supporting their submission. The clarifications and the new experiments show the advantage of the proposed multi-channel setup and the training objective in the raster domain. The local supervision seems to be useful in learning a canonical representation space, which is particularly helpful in modeling a large and diverse set of fonts. I raised my score by 1.
limitations_and_societal_impact: The broader impact statement highlights the positive impacts. There may exist potential misuse of the proposed method. For example, learning and synthesizing a person’s handwriting for forgery. Would it be possible?
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes multi-implicit neural representation for fonts given the observation that it is hard to represent complex fonts with single neural representation. The author introduces a local supervision method that can train multi-implicit functions without knowing ground truth channels. Experimental results in various settings, e.g. reconstruction, interpolation, and generation, show that it outperformed the previous methods such as rasterized representation VAE and a couple of neural vectorized representations.
main_review: 1. My main concern is that there is no clear experimental evidence that ‘multi-implicit’ functions are really necessary. Based on recent successful applications of neural implicit representations, e,g, NeRF and SIREN for 2d images, 3d shapes, etc, I do not see the reason why single implicit representation is not capable of generating complex fonts, and the author did not present the comparison against those. They mentioned the reason was SIREN and fourier feature papers do not learn a continuous latent space in the supplementary (L11-13). I do not think this is a valid reason, and in my opinion, it is very straightforward to just apply fourier features and sine activation function in the proposed method.

2. In my opinion, the proposed training techniques, such as local supervision, are specifically designed for this particular task (edge and corner aware sampling). Although it might not be a good reason to underestimate the work, but I think it is a limitation in terms of broader impact.

3. I encourage the authors to show ‘multi-implicit’ channels qualitative results in the main text. I could find a couple of examples in the supplementary, but it would also be great if the authors could explain more details about the results.

4. Multi-curve representation seems to be an interesting approach, but as I mentioned earlier, I do not think there are experimental results that support the effectiveness of the proposed method.

[minor comments]
1. L276-277: Did you train like GAN? random sampling ‘z’ during training? I think more explanation is required.
2. L213: edge/corner-award sampling -> edge/corner-aware sampling?
3. L186: what would be the format of glyph label ‘l’? it was mentioned in L276, but better to explain when it shows up for the first time.
4. How does interpolation work in experiment sections? linear interpolation between each ‘z’? then how did you get corresponding ‘z’ given the image.

[Update]
Post-Rebuttal: change the score 3 -> 4
limitations_and_societal_impact: Author addressed the limitations in the supplementary, my comments and suggestions are included in the main review.

I do not see any negative societal impact from this submission.
review_title: null

Meta-review: Meta-review of "A Multi-Implicit Neural Representation for Fonts"

The paper proposes representing vector fonts with multi-implicit functions. They were able to reconstruct vector fonts (at arbitrarily high resolution) while keeping specific font-discontinuities such as edges and corners, and also showcased their method to synthesize an entire font given one character only.

Reviewers were initially mixed. While reviewers generally agree that the proposed method works (and based on the results, seems to work really well), the concerns raised (esp by iAuP) is whether multi-implicit functions are indeed needed. DnKK raised a similar concern about whether standard SDF could attain similar results or not. The authors have responded to these concerns, and even produced some newer results in their response (They have also written a detailed general response with baseline comparisons). Reviewers were generally satisfied with their response, new results, and improvements made, and have increased their scores, and even persuaded the reviewer iAuP to improve their initial assessment.

In reviewer discussions, most reviewers are in favour of accepting the work. Reviewer iAuP strongly suggested that the authors clarify many issues they discussed in a detailed thread in the camera ready version. For instance, as they noted: *Given the author's additional experiments, it turns out that 'single implicts' w/ fourier features can reconstruct fonts as well as 'multi implicits', better in low resolution worse in high resolution, but roughly about the same.* So I would like to see the text clarified to the best of the authors' abilities (as they have pledged in the discussion thread) the text, and make sure the claims are accurate and clear, as to not mislead readers.

Given the generally high quality of the work, and fantastic results, which will be useful to the NeurIPS community and ML practitioners, I recommend accepting this work as a poster.

Decision: Accept (Poster)