Character Generation through Self-Supervised Vectorization
Authors: Gokcen Gokceoglu
Emre Akbas
Venue: ICLR 2022 Submitted (2022)
Keywords: character generation
parsing
reconstruction
self-supervised
omniglot

Abstract: The prevalent approach in self-supervised image generation is to operate on pixel level representations. While this approach can produce high quality images, it cannot benefit from the simplicity and innate quality of vectorization. Here we present  a drawing agent that operates on stroke-level representation of images. At each time step, the agent first assesses the current canvas and decides whether to stop or keep drawing. When a `draw’ decision is made, the agent outputs a program indicating the stroke to be drawn. As a result, it produces a final raster image by drawing the strokes on a canvas, using a minimal number of strokes and dynamically deciding when to stop. We train our agent through reinforcement learning on MNIST and Omniglot  datasets for unconditional generation and parsing (reconstruction) tasks. We utilize our parsing agent for exemplar generation and type conditioned concept generation in Omniglot challenge without any further training. We present successful results on all three generation tasks and the parsing task. Crucially, we do not need  any stroke-level or vector supervision; we only use raster images for training.  Code will be made available upon acceptance. 


Under review as a conference paper at ICLR 2022
CHARACTER GENERATION THROUGH SELF-SUPERVISED VECTORIZATION
Anonymous authors Paper under double-blind review
ABSTRACT
The prevalent approach in self- image generation is to operate on pixel level representations. While this approach can produce high quality images, it cannot benefit from the simplicity and innate quality of vectorization. Here we present a drawing agent that operates on stroke-level representation of images. At each time step, the agent first assesses the current canvas and decides whether to stop or keep drawing. When a 'draw' decision is made, the agent outputs a program indicating the stroke to drawn. As result, it produces final raster image by drawing the strokes on canvas, using minimal number of strokes and dynamically deciding when to stop. We train our agent through reinforcement learning on MNIST and Omniglot datasets for unconditional generation and pars- ing (reconstruction) tasks. We utilize our parsing agent for exemplar generation and type conditioned concept generation in Omniglot challenge without any fur- ther training. We present successful results on all three generation tasks and the parsing task. Crucially, we do not need any stroke-level vector supervision; we only use raster images for training. Code will be made available upon acceptance.
INTRODUCTION
While, innately, humans sketch or write through strokes, this type of visual depiction is a more difficult task for machines. Image generation problems typically addressed by raster-based al- gorithms. The introduction of generative adversarial networks (GAN) (Goodfellow et al. 2014), variational autoencoders (VAE) (Kingma & Welling) 2013) and autoregressive models (Van Oord ctt al. 2016) has led to a variety of applications. Style transfer (Gatys et al.) 2015) Isola et al. 2017 photo realistic image generation (Brock et al. 2018 Karras et 20109), and super resolution (Ledig et al. 2017 Bin et al., 2017) are some of the significant instances of the advancing field. Addition- ally, Hierarchical Bayesian models formulated by deep neural networks are able to use the same generative model for multiple tasks such as classification, conditional and unconditional generation (Hewitt al., 2018, Edwards Storkey 2016). These raster- raster-based algorithms can produce high quality images, yet they cannot benefit from the leverage that higher level abstractions bring about.
Vector-level image representation intrinsically prevents models from generating blurry samples and allows for compositional image generation which eventually may contribute to our understanding of how humans create or replicate images (Lake et al. (017). This idea, with the introduction of sketch-based datasets such Omniglot (Lake et al. 2012), Sketchy (Sangkloy et al., (016)) and QuickDraw (Ha & Eck] 2017) has triggered a significant body of work in recent years. Stroke based image generation and parsing has been addressed with both vector supervised models and supervupe generation. Of these, one prominent algorithm is Bayesian Program Learning (Lake et al. 2015), where single model can be utilized for 5 tasks in the Omniglot challenge: (i) parsing, (ii) unconditional generation or, (iii) generating exemplars of given concept, (iv) generating novel concepts of type, and (v) one-shot classification. This approach is also shown to be scalable when supported by the representative capabilities of neural networks (Feinman & Lake 2020b a) however, it requires stroke-level
vector supervision, which costly to obtain or simply non- existent. VAE/RNN (Ha & Eck, 2017 Cao et al. 2019 Chen et al.) 2017] Aksan et al.) 2020 and Transformer based models (Ribeiro et al. 2020 Lin al., 2020) are other common methods applied to vector based image generation. Although impressive results have been presented, stroke-level supervision is required to train these models.

---

Under review as a conference paper at ICLR 2022
(i.e. the absence of stroke-
Recently, self-super
6
8
D 2 4 T
X
1
A
9 d
m
a
level supervision) stroke-
4
9 2 T
/>
/
6
^ 5 m
d
based image generation
8
π
4
4
S
0
has been addressed with
2
10
0
b
b
4
4
6
d
d
7 p
Reinforcement Learning
(RL) (Ganin et al. 2018 Mellor et al. 2019: Huang al. 2019] Schaldenbrand
Figure 1: Our drawing agent can accomplish four different tasks. From left to right: it can generate novel characters, parse a given char- acter into its strokes, generate new exemplars for a given character, and generate novel concepts (i.e. characters) given a type (i.e. al- phabet). Ours is the first stroke-based method to tackle all of the generation and parsing tasks in the Omniglot Challenge, without requiring any stroke-level supervision.
& Oh 2020). We call this approach self-supervised vectorization, since the vectorization of images is learned using only raster- images as supervision. These methods mostly focus image reconstruction and their exploration in generation is limited. For example, none of them address the conditional generation problem, or, they need the number of actions (i.e. strokes) input.
In this paper, we propose a self-supervised reinforcement learning approach where we train a draw- ing agent for character generation and parsing. Our drawing agent operates on the stroke-level (i.e. vector) representation of images. At each time step, our agent takes the current canvas as in- put and dynamically decides whether continue drawing or stop. When 'continue' decision is made, the agent outputs program specifying the stroke to be drawn, A non-differentiable renderer takes this program and draws it on the current canvas. Consequently, a raster image is produced btroke-b--stroke. We first train this agent for two tasks by formulating appropriate loss functions: (i) unconditional character generation and (ii) parsing.
Unconditional character generation is the task of generating novel concep (i.e. character) given a dataset of concepts. For this task, our loss function includes the following components: an adver- sarial loss produced by a discriminator to make generated characters "real" possible, and two data fidelity losses assessing the conformity of the current canvas with the statistical properties of the overall dataset. We also use additional entropy loss to prevent mode collapse.
In the parsing task, the goal for our agent is to reconstruct a given character (in master-imagel by drawing it through strokes using as few of them as possible. We utilize the same action space and environment as in the unconditional generation model, only difference being the input fed to the policy is complete canvas be reconstructed. Our reward function in this task has two compo- nents: a fidelity reward that indicates how much of a stroke is consistent with the target image and penalty that increases with every 'continue' action being taken. This model explicitly learns the vectorization of the input raster-image in self-supervised manner.
Next, we show that our parsing model can be exploited for exemplar generation (i.e. a novel drawing of a given character) and novel concept generation from type (i.e. novel character generation given an alphabet of 10 characters) without any further training. Given a character, the policy network of our parsing model outputs distribution over the action space where likelihood of actions at each time step eventually allows us to generate variations of the input image. For novel concept generation conditioned on type (i.e. alphabet), we compose a stroke library by parsing the provided inputs. As we sample strokes from this library, we observe novel samples forming, in coherence with the overall structure of the alphabet. To the best of our knowledge, are the first to tackle these tasks with a superviue approach that operates on stroke space.
Through experiments show that agent successfully generate novel characters in all three ways (unconditionally, conditioned on a given alphabet, conditioned on a given character), and parse and reconstruct input characters. For both exemplar generation and type conditioned novel concept generation, we provide LPIPS (Zhang et al. 2018), L2 and SSIM measures between input samples and generated images.
Our contributions in this paper are two-fold: (i) we present a drawing agent that can successfully handle all of the generation and parsing tasks in the Omniglot challenge a self-supervised, stroke-
Omniglot challenge terminology.
2

---

Under review as a conference paper at ICLR 2022
based manner such model did not exist (ii) we provide for the first time perceptual similarity based quantitative benchmarks for the "exemplar generation" and 'type conditioned novel concept generation' tasks.
2 RELATED WORK
The main purpose of this work is to present self-supervised approach in order to solve the gener- ation and parsing tasks in the Omniglot Challenge (Lake et al. 2013). by capturing the stroke-level representation of images. Here we initially examine the supervised and self-supervised approaches to Omniglot challenge. Then, we review the work on image vectorization. And lastly, we touch upon the research program synthesis in the context of this study.
Omniglot Challenge Omniglot dataset of world alphabets was released with a set of challenges: parsing given letter, one shot classification, generating a new letter given an alphabet, generating a novel sample of a character, and unconditional generation. Omniglot letters have samples that are conditionally independent based on the alphabet-character hierarchy, hence, distinctive approach to achieve all these tasks is Hierarchical Bayesian modeling (Lake et al. 2015). (Lake et al. 2013). As the Omniglot letters included human strokes as labels, the compositional and causal nature of letters are leveraged to model the generation process. Later, neurosymbolic models are also shown to be successful for unconditional generation (Feinman & Lake (2020a) and conceptual compression for multiple tasks presented within the Omniglot Challenge (Feinman & Lake 2020b).
However, without the stroke set that generated concept, these tasks become more difficult. The idea of sequential image generation is examined by recurrent VAE models (Rezende et al.) 2016), Gregor al. 2015), Gregor al.) 2016). DRAW (Gregor et al., 2015) and Convolutional DRAW Gregor et al. 2016) were able to generate quality unconditional samples from MNIST and Om- niglot datasets respectively. DRAW is proposed as an algorithm to generate images recurrently. The network is able to iteratively generate a given image by attending to certain parts of the input at each time step. Convolutional DRAW improved the idea with an RNN/VAE based algorithm that can capture the global structure and low-level details of an image separately in order to increase the quality of generations. Later, it is shown that Hierarchical Bayesian Modeling can be improved by the representational power of deep learning and attentional mechanisms in order to achieve three of the five Omniglot challenges (Rezende et al. 2016). Another novel idea to leverage Bayesian modeling to tackle Omniglot Challenge was performing modifications on the VAE architecture to represent hierarchical datasets (Edwards & Storkey] (2016) (Hewitt et al., 2018). The significance of these studies is that they were able obtain latent variables describe class-level features effec- tively. Despite the ability to utilize the same model for different problems (one-shot classification, unconditional and conditional generation), raster-based one-step generative models have two disad- vantages we want to address. First, they cannot leverage the higher level abstraction and quality comes with working on vector space. Secondly, one-step generation does not provide an inter- pretable compositional and causal process describing how character is generated. In this work, we combine the advantages of two groups of aforementioned models with an agent operating on stroke representation of images that uses only raster images during training. Thus, we aim to solve all three generative and the parsing (reconstruction) tasks of the Omniglot challenge. We show that the model trained for reconstruction can also be adopted as tool that captures the compositional structure of a given character. Without any further training, our agent can solve exemplar generation and type conditioned novel concept generation problems.
Image Generation by Vectorization With Stroke Supervision Sketch-RNN (Ha & Eck 2017) is the first LSTM/VAE based sketch generation algorithm. It is later improved to gener- ate multiclass samples (Cao et al., 2019) and increase the quality of generations by representing strokes as Bezier curves (Song) 2020). The idea of obtaining a generalizable latent space by image- stroke mapping is studied by many (Aksan et al. 2020 Das et al. 2021 Bhunia et al. 202 Wang et al. 2020). In CoSE (Aksan al. 2020), the problem is articulated as 'completion of partially drawn sketch'. They achieved state of the art reconstruction performance by utilizing variable-length strokes and novel relational model that is able to capture the global structure of the sketch. The progress in stroke representation is continued with incorporation of variable-degree Bezier curves (Das al. 202T), and capturing Gestalt structure of partially occluded sketches (Lin al. 2020
3

---

Under review as a conference paper at ICLR 2022
Self Supervised Vectorization Self vector-based image generation problem has been approached by RL based frameworks (Zhou et al. 2018)) (Ganin et al. 2018). (Mellor et al.) 2019). (Huang et al., 2019), (Schaldenbrand & Oh| 2020), and (Zou et al. (020). In SPIRAL (Ganin al. 2018). unconditional generation and reconstruction tasks are tackled with adversarially trained RI. agents. Succeeding research enhanced the reconstruction process by a differentiable renderer, mak- ing it possible for agents to operate on a contínuous space (Huang et al. 2019; Schaldenbrand & Oh 2020). In order to avert the computational expense of RL based algorithms. end-to-end differen- tiable models are developed through altering the rendering process (Nakano, (2019) or formulating the generation process as parameter search (Zou et al., 2020). More recently, differentiable ren- derer and compositor is utilized for generating closed Bezier paths and the final image respectively (Reddy et al. 202T). This method led to successful interpolation, reconstruction, and sampling pro- cesses. Most related to our work is SPIRAL where both reconstruction and unconditional generation is studied through self-supervised deep reinforcement learning. However, our approach has some significant differences. First, in SPIRAL each stroke is also represented as a Bezier curve, yet, the starting point of each curve is set as the final point of the previous curve. In our model, all control points of the Bezier curve predicted by the agent each time step. Hence, the agent has to learn the continuity and the compositionality of the given character in order to produce quality samples. Secondly, SPIRAL provides generative model that works through a graphics renderer without ad- dressing the conditional generation problem. They show impressive results on both natural images and handwritten characters. While we provide a solution for multiple generative tasks, we have not explored our model in the context of natural images. Another approach that presents a similar scheme to the reconstruction problem is "Learning to Paint" (Huang et al.) 2019). In Learning to Paint, the proposed model is utilized specifically for reconstruction. When reconstruction is consid- ered, the main difference of our model is that since we try to model a human-like generation process, our agent outputs a single stroke at each time step with the environment being altered throughout this process while in Learning to Paint, 5 strokes are predicted by the agent at each time step. As a major difference from previous studies, our agent decides whether to stop keep drawing before generating a stroke. This enables the agent to synthesize an image with as few actions as possible when motivated with our reward formulations.
Self Supervised Program Synthesis Our method essentially outputs a visual program that de- pends only the rastered data. In that sense, studies Constructive Solid Geometry (CSG)are also related. Different RL frameworks for reconstruction of a given CSG image, that is essentially composition of geometric shapes, are proposed (Ellis al.. 2019; Zhou et al. 2020). The for- mer considered parsing as a search problem that is solved by using Markov Decision Process. The latter adopted Tree-LSTM model to eliminate invalid programs and the reward is considered to be the Chamfer distance between the target image and current canvas.
read-eval-print-loe
within a
METHOD
Sample stroke S,
Next canvas Ce
Current canvas C
Our model consists of a policy network and a (non- differentiable) renderer. At time step f, the policy net- work takes the current can- vas, C.- a raster-image, as input and outputs two dis- tributions, TB and Ts. The first distribution, TB. is for stroke (i.e. Bezier curve)- parameters and the second one, Ts. is for the continue/stop decision. From the first distribution, we randomly sample a stroke defined by its 7 parameters (x-y coordinates of start, end, control points of the quadratic Bezier curve, and a width). From the second distribution, we randomly sample a decision. If the decision happens to be 'continue', we add the newly sampled stroke to the current canvas. C₁, in- crement time (i.e. +- t + 1) and restart. If the decision was to 'stop', then C, is returned as the final output. Our model is able handle parsing and different generation tasks, and the processing
Policy network
Renderer
V
Contenac
Return 8-11
Sample a decision
Stop >
Figure 2: Generator model. At each time step, the policy network receives a canvas and outputs two distributions for Bezier curve pa- rameters and stop/continue decision. When the 'continue' decision is sampled, the resulting stroke is rendered and added to the final output.
4

---

Under review as a conference paper at ICLR 2022
pipeline we just described is common in all these tasks. What changes among tasks is the reward functions and/or training procedures, which we explain below.
Unconditional Generation The task of "generating new concepts' as dubbed in Omniglot chal- lenge, is essentially unconditional sampling from a distribution obtained from the whole Omniglot training set. Here, the model is asked to generate completely novel samples (i.e. characters) without any constraints. For this task, at each time step I, we calculate an instantaneous reward, Yt, that has three components:
Tt D(C,) + λ align(C,, + (2
(I)
The first term is a reward based on a discriminator to make generated characters as 'real' as possible. D(-) a discriminator that outputs the "realness" score of its input canvas. We train in an adver- sarial manner by using the generated examples as negatives and the elements of the input dataset as positives. The second term is a clustering measures the alignment between the current canvas C₁ and another canvas I, which is randomly selected cluster center at the beginning of each episode. The cluster centers are obtained by apply- ing k-means on all characters in the input dataset. align basically counts the number of intersecting on-pixels (between the two canvases) minus the number of non-intersecting on-pixels in C., and divides this quantity by the number of on-pixels in I. The final term assesses the conformity of the current canvas with the dataset in terms of the number of on-pixels. .N[[ p, σ) evaluates a normal distribution with (n, o) at [C,] which is the number of on-pixels in the current canvas. We obtain (n, r) by fitting normal distribution to the on-pixel counts of characters in the training set. We observed that the second and third terms accelerate learning as they guide the exploration within the vicinity of real characters. During training, instead of using the instantancous reward, rt, we use the difference of successive rewards, i.c. Tt- Tt-1.
data fidelity reward. The function align
In order to encourage exploration and avoid mode collapse, we use an entropy penalty term as
a (L TS], U) -)
Here, KL indicates KL-divergence and U is the uniform distribution. This term first measures the divergence between the uniform distribution and и., Ts. the distributions output by the policy net- work. Then, through the hinge function, if the divergence exceeds a threshold (7), this term activates and increases the penalty. The policy network and the discriminator D are updated alternatingyy af- ter 256 images are generated at each iteration. We employ the REINFORCE algorithm (Williams) 1992) to update the weights of the policy network. Discriminator is trained using hinge loss. In order stabilize the discriminator and keep the Lipschitz constant for the whole network equal to 1, Spectral Normalization is applied at each layer (Miyato et al. 2018). Throughout the training, we kept the balance ratio between generated and real samples at 3.
Image Reconstruction by Parsing In the "parsing" task, the goal is to reconstruct the given input image by re-drawing it through strokes as accurately as possible. To this end, we formulate a new reward function with two terms: a fidelity reward that indicates how much of stroke is consistent with the input image (using the "align" function introduced above) and penalty that is added with every time increment represented by 'continue' decisions being made:
r: = al, C.) - Ant,
where S, is the newly sampled stroke and C2 the current canvas (input). Second term simply acts as penalty for every 'continuc' action. The first term ensures the sampled stroke to be well-aligned with the input and the second term forces the model to use as few strokes as possible. There is no need for discriminator. This model explicitly learns the vectorization of the input raster-image in a self-supervised manner.
Apart from the different reward function, another crucial difference between the training of the unconditional generation model and the parsing model is how the input and output are handled. In unconditional generation, the newly-sampled stroke is added to the current canvas, whereas in
5

---

Under review as a conference paper at ICLR 2022
parsing, do the opposite: the sampled stroke is removed (masked out) from the current canvas, and the returned final canvas is the combination of all sampled strokes until the 'stop' decision. X, a and in Equations I and 3 are hyperparameters adjusted experimentally. (see "Training Details' in Appendix B
Generating New Exemplars In this task, a model is required to generate a new exemplar (i.e. variation) of an unseen concept (i.e. character). To the best of our knowledge, we are the first to tackle this task in a self-supervised stroke-based setting. Most importantly, we do not require any training to achieve this task. We utilize our parsing network described in the previous section to capture the overall structure of given letter. In order to produce new exemplars, we randomly sample different parsings (a set of strokes) from the distribution generated by the agent. In order to eliminate 'unlikely' samples, we compute the likelihood of the parsing given the resulting policy, and apply a threshold.
Generating Novel Concepts from Type In this task, the goal is to to generate a novel concept (i.e. character) given a previously unseen type (i.e. alphabet) consisting of 10 concepts. The novel concepts should conform to the overall structure, that is, the stroke formulation and composition of the given type (alphabet). We, again, tackle this challenge using our parsing network without any further training. To do so, we first parse all input images into its strokes. For each input image, we sample five stroke sets from the stroke-pant paraee-aasseters distribution output by the policy network. During the sampling process, we again use the likelihood-based quality function described in the previous section. We add all the strokes sampled during this process form a stroke library. Here the strokes are stored with the time steps they are generated. Noting that the number of strokes sampled for a given character not constant, we approximate a distribution for stopping actions. This process provides a stroke set representing the structure of letters and the way they are composed, that is, can exploit the compositionality and causality of an alphabet. Throughout the character generation process, stroke is sampled each time step belonging to that particular group of the library. The sampled strokes are summed together to obtain the final canvas.
4 EXPERIMENTS
Datasets and Implementation Details We report generation and reconstruction (parsing) results on the Omniglot dataset (Lake et al. 2015), which includes 1623 characters from 50 different alpha- bets, with 20 samples for each character. 30 alphabets are used for training and the remaining 20 used for evaluation. For unconditional generation and reconstruction,
also report results on the MNIST dataset (LoCun) [998). For both datasets, we rescale input images to 32x32 in order for them to conform with model.
Our policy network is composed of a ResNet feature extraction backbone and three MLP branches for computing the distributions over the action space. Architectural details can be found in Appendix A For the Omniglot dataset, we take brush width as a constant and omit the corresponding MLP branch.
We tune the learning rate and weight decay of the generator, hyperparameters in Equation| [ and Equation 3 a and - hyperparameters in Equa- tion 2 using the Tree-structured Parzen Estimator algorithm (Bergstra et al. 2011) in the RayTune library (Liaw al.) 2018).
FID
LSGAN
MMa al. 2017]
9.8 0.9
NSGAN
Goodiellow el al.
2014
6.8 0
WGAN
Arjovsky et al.
2011
0.4
WGAN GP (Gulrajami et al.
2017
2.0
DRAGAN
(Kodali et al.
0.4
For unconditional generation, we use the dis- criminator architecture proposed by Miyato al. (0018). In order to stabilize the discriminator and keep the Lipschitz constant for the whole network equal to 1, Spectral Normalization is applied at each layer. Discriminator is trained using the hinge loss. Throughout the training, we set the balance ratio between fake and real samples as 3. convergence during this process.
VAE
23.8 2
Ours
17.3
Table 1: Comparison of the FID scores for dif- ferent models on the MNIST dataset. We report the mean and the variance of FID scores from simulations with different weight initializations.
We performed hard- negatiee mining to speed up

---

Under review as a conference paper at ICLR 2022
W
8
5
3
6
4 2
6 6
6
Q
4
m
4
6
2
2
7 4
2
I
9 8
6
7
R
R
+
f
Figure 3: Quality of generated MNIST characters training progresses (i.e. policy is updated) from left to right.
7
2
6
2
P M
&
&
A
3
X
X
Y
Y
K
m
m
9
9
7
X
X
7
K
b
b
PN
ST
0
Q
O
A
57
V
VI
X
&
B
8
P
P
Y
V
T
7
3
4
1
D
Q
O
Q
Figure 4: Omniglot unconditional samples. For randomly sampled generations, four closest samples (in terms of pixelwise L2 distance) from the training dataset are presented.
4.1 UNCONDITIONAL GENERATION
We initially tested our approach on the MNIST dataset. Figure presents the improvement in the quality of samples generated throughout the policy network updates. At the beginning, generated characters are mostly random scribbles. Towards the end, they start to look like real digits. Table shows that method achieves acceptable FID score (Heusel et al., 2017) given the scores of other prominent GAN and VAE methods. Presented FID values are taken from Lucie et al. (2017).
Figure shows sample generations for the Omniglot dataset. To demonstrate that our generations are not du- plicates of the characters the training set, we present the four most similar characters from the training set to our generations. Similarity is computed using pixelwise I.2 presents distance. Finally, presentss more generated char- acters, which demonstrate the variability and the quality of generated concepts. The agent was able to capture the type of strokes, number of strokes a character has and let- ter structures without any stroke supervision.
6
K
8
P
d
U
N
X
Q
V
0
T
1
3
&
J
D
R
T
8
9
4
3
A
X
4 R 9 R 16 n
h
P
X
T
>
a
27
C
X
4
3 X A
P 1 D Y
0
8
D
8
r
C
V
r
2
R
2
4.2 IMAGE RECONSTRUCTION BY PARSING
6
U
T
b
8
T
m
X
Figure presents sample parsing and reconstruction re- sults on MNIST. Our agent can reconstruct character from the test set in minimal number of actions within the abilities of quadratic Bezier curves. Selected brush widths also conform with the stroke heterogeneity of the dataset.
4
9
[
C
Y
3
2
X
4
5
J
O
A
Randomly sampled uncon- ditional generations for the Omniglot dataset.
Figure
Then, we train our model with the characters in the Om- niglot training set. For evaluation, we utilize the evaluation set with completely novel characters from unseen alphabets. Thereby, we can see that our agent has learned how to parse a given char- acter. Due the penalty term that increases with the number of strokes, there is a tradeoff for the agent to replicate a character exactly and replicate it in a small number of actions. This indirectly demotivates the agent from retouching the image with small strokes minimize the difference the

---

Under review as a conference paper at ICLR 2022
2
3.
3
3 3
4
4
5
9
7
7
7
8
9
0
2
3
5
6
6
7
7
9 9
0
2
3
3
4
4
6
6
7
7
8
9
Figure 6: MNIST reconstructions. For each sample on the left hand side of the columns, parsing processes are demonstrated. Colors represent the order of the strokes. (pink: first stroke, green: second stroke, blue: third stroke)
F
6
9
4
9
P J,
X
H
D
6
H
144
7
7h
FA
e
M/U
9
[
P
1/6
4
3
3
<]
//
ay
2 2 1 7 9
7
R
7
>
1
by
a4
3
22
10
A
21
2]
X
0 Q
4
R
Q2
P
F
p
V
Figure 7: Omniglot reconstruction. For each sample on the left hand side the columns, resulting reconstructions demonstrated.
target. Results in Figure 7 show that overall structure of the target images are preserved, however, small details are lacking some of the examples. This reflects on the distance measures (Table
4.3 GENERATING NEW EXEMPLARS
the evaluation set of the Omniglot dataset. For each character in the test set, we sample 500 different parses from the policy. In Figure it can observed that given an unseen letter from novel alphabet, agent can sample from the resulting distribution, and output quality variations. The major indications of variation are structures of the strokes, number of actions to generate sample and the fine details of certain characters. We compare each produced character with its corresponding input image using LPIPS, SSIM and L2 distance values. The mean and standard deviation of these values the whole evaluation set 0.078 = 0.002, 0.616 + 0.018 and 0.08 + 0.016, respectively. Results per alphabet can be found in Appendix C.3
For this task,
4.4 GENERATING NOVEL CONCEPTS FROM TYPE
In order to generate a concept that is likely to belong in a given alphabet, we again leverage our reconstruction model. Given 10 different characters of unseen alphabet, are able generate novel images with similar structural features. Results presented in Figure/]shor that our algorithm model the compositional pattern of an alphabet in stroke space. In order to obtain quantitative results, (e.g. LPIPS, L2 and SSIM), we produce 10000 images conditioned on each input set and randomly sample characters by utilizing the discriminator trained for the unconditional generation model, assuming it has learned what features of given input imply a real character. We generate
Method
MNIST
Omniglot
Image VAE
0.0033
N/A
Im2Vec (Reddy et al. 2021)
0.0036
N/A
Learning to Paint (Huang et al. 2019)
0.006
N/A
SPIRAL (Training distance) (Ganin et al. 2018)
0.01
0.02
StrokeNet (Training distance) Zheng al. 20188)
0.015
0.02
Ours
0.04
0.06
Table 2: Reconstruction quality. L2 distance between target and the reconstructed image. (Im-
age VAE is taken from Reddy et al. 2021 it indicates purely raster-based autoencoder.
8

---

Under review as a conference paper at ICLR 2022
mn
1 1
n m m
fb
e
h
h
h
bg
1 1 1
m M m
B
7
&
1 7
m ( m
t
8
In
h
H
n
A
1
X
T
9 9 9
(1 a 0
A 6
/
X x x
7
7 7
7
nnn
4
x
x
T
n
4
A
A
x
d
mL
x イ
x
7'
7
7
9
9
Figure 8: New exemplar generation. Given an unseen character from a new alphabet (highlighted in red boxes), the model generated 9 exemplars.
X
9
0
X
P
0 1
6
9
H
Q
s
2
9 D
7
Hb
5 h 3 4 &
3
9
I
2
7
H 4 4 9 7 4 :J A
VT rl
V
9
U
2
1
&
+ 7
5
] B
@
8 2
R
2
X
7 K / O 7
1
3
9
50
70
4 4
J
7
3
$
J
3
2
999
11 A
p
3
40
G
4
0 TT
x)
5
A
&
7 6
A
5
:<
50
7
JI
b
4
3
5
$
2
3/
K.
4 y
d
5
7
2 44
Figure 9: Novel sample generation conditioned on type. Given 10 characters from an alphabet, model produced 20 new samples.
sampling distribution according to the discriminator scores of generated samples and repeat the sampling process multiple times for each input to obtain set of outputs to considered. For sample generated, calculate performance metrics with respect to all characters in the input. In order to report final metrics presented in supplemental figures [2b and 12a, we consider the most similar onput-output pairs. The mean and standard deviation of LPIPS, SSIM and L2 values for the whole evaluation set are 0.0801 + 0.003, 0.502 0.068 and 0.1263 + 0.00086 respectively.
5 CONCLUSION
We proposed a self-supervised reinforcement learning approach for stroke based image generation. We trained model for unconditional generation and parsing handwritten character datasets by defining a single action space and environment. Through experiments, we showed that, given the whole training set, our agent is able to capture the overall distribution and generate quality novel samples for the challenging Omniglot dataset. Then, we trained our agent for the parsing task; given raster image, the goal is reconstruct through as few strokes as possible. We demonstrated that the parsing agent can be utilized for generating exemplars of concept and creating novel samples conditioned on type, without any further training, only difference being how
called among tasks. To the best of our knowledge, we are the first to tackle these tasks with a self-supervised approach that operates on a stroke level. this work, we used quadratic Bezier curves as the smallest unit of sketching. However, for human-level generations, the stroke representations should be enhanced to capture more complex structures. We anticipate that this will improve the overall performance.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Emre Aksan, Thomas Deselaers, Andrea Tagliasacchi, and Otmar Hilliges. Cose: Compositional stroke embeddings. arXiv preprint arXiv:2006. 00t0:00000.0990, 2020.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214 PMLR, 2017.
James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for pprammerreeeter optimization. Advances in neural information processing systems, 24, 2011.
Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Yongxin Yang, Timothy M Hospedales, Tao Xiang, and Yi-Zhe Song. Vectorization and rasterization: celf-supervied learning for sketch and hand- writing. arXiv preprint arXiv:2103.13716 2021.
Huang Bin, Chen Weihai, Wu Xingming, and Lin Chun-Liang. High-quality face image using conditional generative adversarial networks. arXiv preprint arXiv:1707.00737, 2017.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: /096, 2018.
Nan Cao. Xin Yan, Yang Shi, and Chaoran Chen. Ai-sketcher: A deep generative model for pro- ducing high-quality sketches. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp.
2019.
Yajing Chen, Shikui Tu, Yuqi Yi. and Lei Xu. Sketch-pix2seq: a model to generate sketches of multiple categories. arXiv preprint arXiv: 179.04170. 2017.
Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, and Yi-Zhe Song. Cloud2curve: Gen- cration and vectorization of parametric sketches. arXiv preprint arXiv:2103.15536, 2021.
Harrison Edwards and Amos Storkey.
Towards neural statistician.
arXiv preprint
arXiv:1606.02185. 2016.
Kevin Ellis, Maxwell Nye, Yewen Pu, Felix Sosa, Josh Tenenbaum, and Armando Solar-Lezama. Write, execute, assess: Program synthesis with repl. arXiv preprint arXiv: (ee../99600044 2019.
Reuben Feinman and Brenden M Lake. Generating new concepts with hybrid seuro-symbolic mod- els. arXiv preprini arXiv:2003. 08978, 2020a.
Reuben Feinman and Brenden M Lake. Learning task general representations with generative neuro- symbolic modeling. arXiv preprint 1444240000.1444,, 2020b.
Yaroslav Ganin, Tejas Kulkarni, Igor Babuschkin, SM Ali Eslami, and Oriol Vinyals. Synthesiz- ing programs for images using reinforced adversarial learning. In International Conference on Machine Learning. pp. 1666-1675. PMLR, 2018.
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.065 06576, 2015.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farke Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Rezende, and Daan Wierstra. Draw: A recurrent neural network for image generation. In International Conference on Machine Learning, pp- 1462-1471. PMLR. 2015.
Karol Gregor, Frederic Besse, Danilo Jimenez Rezende, Ivo Danihelka, and Daan Wierstra. Towards conceptual compression. arXiv preprint arXiv:1604.08777 2016
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Im- proved training of wasserstein gans. arXiv preprint 1rXiv: 2017.
10

---

Under review as a conference paper at ICLR 2022
David Ha and Douglas Eck. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Luke B Hewitt, Maxwell I Nye, Andreca Gane, Tommi Jaakkola, and Joshua B Tenenbaum. The variational homoencoder: Learning to learn high capacity generative models from few examples. arXiv preprint arXiv: 00880800009999 2018.
Zhewei Huang, Wen Heng, and Shuchang Zhou. Learning paint with model-based deep reinforce- ment learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8709-8718, 2019.
Phillip Isola, Jun Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125-1134, 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
Diederik P Kingma and Max Welling. 13Xiv:1312.6114.22 2013.
Auto-encoding variational
bayes.
arXiv preprint
Naveen Kodali, Jacob Abemethy, James Hays, and Zsolt Kira. On convergence and stability of gans. arXiv preprint arXivv 2017.
Brenden Lake, Ruslan Salakhutdinov, and Joshua Tenenbaum. Concept learning as motor program induction: A large-scale empirical study. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 34, 2012.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. One-shot learning by inverting a compositional causal process. 2013.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science. 35066266) 2015.
Brenden M Lake, Tomer D Ullman, Joshua Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40, 2017.
Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/nmist// 1998.
Christian Ledig, Lucas Theis, Ferene Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang. et al. Photo-realistic sin- gle image super-resolution using generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681-4690, 2017.
Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Sto- ica. Tune: research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118, 2018.
Hangyu Lin, Yanwei Fu, Xiangyang Xue, and Yu-Gang Jiang. Sketch-bert: Learning sketch bidi- rectional encoder representation from transformers by self-supervised learning of sketch gestalt. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6768-6767, 2020.
Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet. Are gans created equal? large-scale study. arXiv preprint arXiv: 2017.

---

Under review as a conference paper at ICLR 2022
Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. Proceedings of the IEEE international confer- on computer vision, pp. 2794-2802, 2017.
John FJ Mellor, Eunbyung Park, Yaroslav Ganin, Igor Babuschkin, Tejas Kulkarni, Dan Rosenbaum, Andy Ballard, Theophane Weber, Oriol Vinyals, and SM Eslami. Unsupervised doodling and painting with improved spiral. arXiv preprint arXiv:1910.01007, 2019.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprini arXiv:/802.0590 2018.
Relichiro Nakano. Neural painters: learned differentiable constraint for generating brushstroke paintings. arXiv preprint arXiv: 1904.08410, 2019.
Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J Mitra. Im2vec: Synthesizing vector graphics without vector supervision. arXiv preprint 0727:2200..02798, 2021.
Danilo Rezende, Ivo Danihelka, Karol Gregor, Daan Wierstra, et al. One-shot generalization deep generative models. In International Conference on Machine Learning, pp. 1521-1529. PMLR, 2016.
Leo Sampaio Ferraz Ribeiro, Tu Bui, John Collomosse, and Moacir Ponti. Sketchformer: Transformer-based representation for sketched structure. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Patlern Recognition, pp. 14153-14162, 2020.
Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphies (TOG), 35 2016.
Peter Schaldenbrand and Jean Oh. Content masked loss: Human-like brush stroke planning reinforcement learning painting agent. arXiv preprint arXiv:2012.1004:: 2020.
Yi-Zhe Song. Béziersketch: A generative model for scalable vector sketches. ECCV 2020, 2020.
Computer Vision-
Aaron Van Oord. Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In International Conference Machine Learning, pp. 1747-1756. PMLR, 2016.
Alexander Wang, Mengye Ren, and Richard Zemel. Sketchembednet: Learning novel concepts by imitating drawings. arXiv preprint N0Nv::20090040066 2020.
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229-22) 226.26,, 1992.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, 2018.
Ningyuan Zheng, Yifan Jiang, and Dingjiang Huang. Strokenet: neural painting environment. In International Conference Learning Representations, 2018.
Chenghui Zhou, Chun-Liang Li, and Barnabas Poczos. Unsupervised program synthesis for images using tree-structured Istm. arXiv preprint arXiv:2001. 2020.
Tao Zhou, Chen Fang, Zhaowen Wang, Jimei Yang, Byungmoon Kim, Zhili Chen, Jonathan Brandt, and Demetri Terzopoulos. Learning sketch with deep q networks and demonstrated strokes. arXiv preprint arXiv:1810.05977, 2018.
Zhengxia Zou, Tianyang Shi, Shuang Qiu, Yi Yuan, and Zhenwei Shi. Stylized neural painting. arXiv preprint arXiv:2011.08114 2020.
12

---

Under review as a conference paper at ICLR 2022
A NETWORK ARCHITECTURE
The backbone ResNet with 3 convolutional layers and 8 residual layers. The first convolutional layer has 32 filters of size 5x5. The second and third convolutional layers have 32 filters of size 4x4 and stride of resulting
tensor with dimensions 8x8x32. Then, we use standard residual layers described in (He et al. 2016). Each convolutional layer is followed by a Batch Normalization process and ReLU activation. The output of the final residual layer flattened be processed by the MIPss. The first MLP outputs set of distributions for each control point of the Bezier curve. has fully connected layer that outputs 192x1 vector. This vector reshaped to 32x6 matrix where each 32x1 vector defines distribution over the possible coordinates. The MLPs used for selecting the brush width and sampling the stop/continue decision consist layers with 64 and 2 neurons.
2048x vector to
B TRAINING DETAILS
The hyperparamaters used for unconditional generation and reconstruction are presented in Table and 4 respectively.
A1 A2
1.016
0.336
0.415
Policy network optimizer Policy network learning rate Policy network weight decay Discriminator learning rate Batch size
Adam
39.96e - 05
0.0064
0.0001
256
Table 3: Hyperparameters for unconditional generation. A1 and λ refer to the respective hyper- parameters in Equation a and tau refer to the respective hyperparameters entropy penalty in Equation 2
A
0.089
0.59
2.72
Policy network optimizer Policy network learning rate Policy network weight decay Batch size
Adam W
1.5e
1.6e 5
256
Table 4: Hyperparameters for reconstruction (parsing). A refers the 'number of action' penalty in Equation 3 and tau refer to the respective hyperparameters of entropy penalty in Equation 2
C EXPERIMENTS: SUPPLEMENTAL FIGURES
C.1 UNCONDITIONAL GENERATION
In Figure T0 present the FID values for the generated images along the training Omniglot
dataset.
C.2 PARSING
In Table 5 we present the mean number strokes our agent used to parse the characters for each alphabet the test set.
13

---

Under review as a conference paper at ICLR 2022
80
50
40
80
100
120
140
Epoch
Figure 10: FID values for unconditional generations of Omniglot dataset throughout the training process. The experiment is repeated over 3 seeds.
Number of Strokes
Alphabet
Sample Image
Human Labeled Data
Our Model
Angelic
3.935
4.49
X
Atemayar Qelisayer
10.15
3.571
17
Atlantean
6.209
2.078
@
3
V
3
Aurek-Besh
7.6
2.565
K
+
G
Avesta
9.511
1.52
76
Ge_ez
10.112
1.984
Glagolitic
5.24
2.88
SP 9
3
Gurmukhi
6.080
3.09
3
Kannada
4.217
2.33
Keble
8.573
4.140
Malayalam
7.215
1.453
Manipuri
10.676
2.82
Mongolian
8.93
2.405
Old Church Slavonic
5.171
2.954
3
K
Oriya
3.59
2.82
Sylheti
11.38
2.84
H
4
Syriac
6.35
2.206
Tengwar
8.088
2.492
9 d
&
Tibetan
11.69
3.62
ULOG
6.417
3.253
1
Table 5: For each alphabet in the Omniglot evaluation set, we present the number strokes our agent used to reconstruct the given image vs. mean number of strokes obtained from human-labeles data. The stroke count for human-labeled data is calculated using the labels within the Omniglot dataset.
C.3 EXEMPLAR GENERATION
In Figure 11a we demonstrate LPIPS metrics calculated by using different backbones (AlexNet, VGG, and SqueczeNet). In Figure ITb present L2 and SSIM values. These metrics are calcu- lated over all examples generated for the test set.
C.4 GENERATING NOVEL CONCEPTS FROM TYPE
In Figure 12a demonstrate I.PPS metrics calculated by using different backbones (AlexNet, VGG, and SqueezeNet). In Figure 12b we present L2 and SSIM values.
14

---

Under review as a conference paper at ICLR 2022
AlexVel
U10G
ULOG
SqueczeNett
Tibetan
Tengwar
Tengwar
Syrac
Syriec
Sylheti
Sylheli
Drlya
Orlya
Cynilic
Cyrilli
Mongolian
Mongolian
Na Vanipun
Manipun
Malayalam
Malayalam
Keble
Keble
Kannada
Kannada
Gurmukhi
Gurmukhi
Glagolitic
Glagolitic
Ge_ez
Ge_ez
Avesla
Avesta
AurekBBssh
Allatten
Allantan
Atamayar
Atemayar
Angelic
SSIM
0.00
0.10
0.15
0.25
0.30
0.35
0.4
0.7
(a)
(b)
Figure 11: LPIPS values for each alphahet in the test set calculated from sampled exemplars (a), SSIM and L2 values for each alphabet in the test set calculated from sampled exemplars (b).
ULOG
VGG
U1OG
SqueezeNet
Tibetan
Tengwar
Tengwar
Syriac
Syriac
Driya
Oriya
Cyrillic
Mongolian
Mongolian
Manipuri
Manipuri
Mai ayalam
Malayalam
Koble
Koble
Kannade
Gumukki
Gurmukhi
Glagolitic
Glagolitcc
Ge cz
02
Avesta
Avesta
Aurek
Aurek Besh
Atlantean
Atlantean
Atemayar
Atemayar
Angelic
Angelic
SSIM
0.6
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Mean
Mean LPIPS
(b)
(a)
Figure 12: LPIPS values for each alphabet in the test set calculated from novel samples produced (a), L2-SSIM values for each alphabet in the test calculated from novel samples produced (b).
15


Reviews:
[{"review_id": "wkr6ikN7MxJ", "reviewer": "Reviewer_hg9W", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, I think this submission is OK due to the following points:\n\n1. The problem that this submission aims to solve, how to introduce high-level image information without the data limitation, is important.\n2. The proposed method is simple and easy to implement.\n3. The writting is good and I can follow it easily.\n\nHowever, I have the following concerns, mainly about the experiment part:\n\n1. I can understand that using the stroke information in the neural network is better than the pixel-based alternatives but the authors do not mark this advantages empirically. For example, as for me, I think there should be some experiments about the comparisons between the pixel-based methods and the stroke-based methods (at least the proposed one).\n2. Incorporating higher level abstration of image information into the deep framework is widely studied, which is also mentioend by the authros in the related work part. So I wonder what's the improvement between the developed method and those methods (like DRAW). Specifically, I am really curious about the difference when comparing with DRAW, which also uses one single image as training data. Additionally, the paper states that DRAW cannot provide an interpretable compositional and causal process describing how a character is generated, which is not true.\n3. I am confused about the functionality of each term in Eq. (1). Can the authors conduct some ablation studies to support the role of the last two terms, like why using the align operation can accelerate the training stage.\n4. In terms of the baseline comparisons, the chosen algorithms are old (all before 2018). Even though, the proposed method cannot enjoy a quantitative evaluation merit in Tab. 1-2.\n5. After reading the submission, I find the technical contribution minor. Except the two terms for speeding up the training, I cannot see any specific strategy (insights) to tackle the self-supervised image generation problem. Can the authors explain a bit here?", "summary_of_the_paper": "In this paper, the authors present a method for character generation using the self-supervised technology. Different from existing approaches, it can leverage the benefits from higher level abstration (due to the used strokes) and get rid of stroke supervision. In this way, high-quality images are generated while the supervised training data requirements are relieved. Although some comparisons are made on Omniglot dataset, the advantages of this method are not very clear.", "summary_of_the_review": "Generally speaking, I am not an expert in the field of stroke-based image generation and remain lukewarm about this project. Due to the strengths and weakness I mentioned above, I rate this submission as weak reject.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "Ed5IhLuxBjn", "reviewer": "Reviewer_aFpU", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Paper weakness]  \nModeling:  \n- The rewards are designed based on a discriminator. As we know, generative adversarial networks are not easy to train since generative networks and discriminative networks are trained alternatively. In the proposed method, the policy network and the discriminator are trained alternatively. I doubt if it is easy to train the model. I would like to see the training curves for rewards value. \n-\tThe detailed alignment function used in Eq. (1) and Eq. (3) need to be provided.\n\n\nExperiment:\n-\tThe results are not satisfying. In the experiment, the generation quality of the proposed method is not good as traditional generative networks in terms of FID. In the image parsing part, the results are far behind the compared methods. \n-\tSince the results are not comparable to the existing methods, there seems not too much significance for the proposed methods. \n", "summary_of_the_paper": "The paper proposes a self-supervised reinforcement learning approach to train a drawing agent for character generation and parsing. The drawing agent operates on the stroke-level (i.e. vector) representation of images. Different from the one-step generative model, the proposed method can capture the compositional structure of a given character. It is interesting to tackle these tasks with a self-supervised approach and reinforcement learning that operates on stroke space.\n", "summary_of_the_review": "Overall, the paper proposes a self-supervised character generation and parsing by reinforcement learning. However, there are still some issues on modeling and training details that need to be clarified. Also, since the results are not comparable to the existing methods, it seems not significant for the proposed methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "-NFJeFLnZ1", "reviewer": "Reviewer_c9oL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positives\n---------\n\nThe overall approach proposed by the paper seems sound, and the results look visually reasonable in most cases.\n\n\nConcerns\n--------\n\nMy biggest concern is that the approach and results do not feel well situated in the context of advancements in self supervised vectorisation over the last 11 or so months. For example, these papers both propose differentiable approaches to drawing stroked curves (without the requirement that they be closed):\n\n- Tzu-Mao Li, Michal Lukáč, Michaël Gharbi, and Jonathan Ragan-Kelley. 2020. Differentiable vector graphics rasterization for editing and learning. ACM Trans. Graph. 39, 6, Article 193 (December 2020), 15 pages. DOI:https://doi.org/10.1145/3414685.3417871\n- Daniela Mihai, and Jonathon Hare. 2021. Differentiable Drawing and Sketching, arXiv preprint arXiv:2103.16194, https://arxiv.org/abs/2103.16194\n\nIn both those papers experiments were performed with character reconstruction, and the former demonstrates vector generation with a VAE. In both papers, like this one, the training of the models is self-supervised utilising only the rasters. Whilst I acknowledge that this paper explores more of the OmniGlot tasks, it's not obvious to me why the approaches in these other papers could not do the same thing (and certainly could be compared for unconditional parsing/reconstruction tasks). \n\nMy secondary concern would be that whilst many of the qualitative results look okay, many of the quantitative results using the proposed model seem not to be so good; for example in terms of reconstruction quality in table 2, the proposed model is significantly worse performing than all the compared approaches.\n\nAs a side note, it's not obvious to me why one would necessarily use perceptual models like SSIM or LPIPS for measuring performance in this particular task. I'd at least like to see some kind of justification for that choice.", "summary_of_the_paper": "This paper presents an approach using reinforcement learning to parse and generate characters. Experiments are performed both with the omniglot challenge and MNIST datasets. In the context of omniglot, in addition to unconditional generation and parsing, results are also shown on the exemplar generation and type conditioned generation tasks. The proposed approach only uses pixel/raster level information during training and does not use stroke or vector data.", "summary_of_the_review": "Overall, I think there are potentially some interesting ideas within this work, but there is a significant lack of context with respect to recent works in this space that would need to be addressed. Further, the results suggest in some cases the proposed methodology performs significantly worse than other approaches.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies the problem of character generation using reinforcement learning for generation/parsing. All the reviewers recommended reject due to insufficient experimental investigation to support the ideas. The authors did not provide a rebuttal. Hence, the reviewers' opinion still remains the same. AC agrees with the reviewers and believes that the paper is not yet ready for publication.
Decision: Reject