Paper:
STROKENET: A NEURAL PAINTING ENVIRONMENT
We've seen tremendous success of image generating models these years. Generating images through a neural network is usually pixel-based, which is fundamentally different from how humans create artwork using brushes. To imitate human drawing, interactions between the environment and the agent is required to allow trials. However, the environment is usually non-differentiable, leading to slow convergence and massive computation. In this paper we try to address the discrete nature of software environment with an intermediate, differentiable simulation. We present StrokeNet, a novel model where the agent is trained upon a wellcrafted neural approximation of the painting environment. With this approach, our agent was able to learn to write characters such as MNIST digits faster than reinforcement learning approaches in an unsupervised manner. Our primary contribution is the neural simulation of a real-world environment. Furthermore, the agent trained with the emulated environment is able to directly transfer its skills to real-world software.

Review:
Revision:
The authors have taken my advice and addressed my concerns wholeheartedly. It is clear to me that they have taken efforts to make notable progress during the rebuttal period. Summary of their improvements:
- They have extended their methodology to handle multiple strokes
- The model has been converted to a latent-space generative model (similar to Sketch-RNN, where the latent space is from a seq2seq VAE, and SPIRAL where the latent space is used by an adversarial framework)
- They have ran addition experiments on a diverse set of datasets (now includes Kanji and QuickDraw), in addition to omniglot and mnist.
- Newer version is better written, and I like how they are also honest to admit limitations of their model rather than hide them.
I think this work is a great companion to existing work such as Sketch-RNN and SPIRAL. As mentioned in my original review, the main advantage of this is the ability to train with very limited compute resources, due to the model-based learning inspired by model-based RL work (they cited some work on world models). Taking important concepts from various different (sub) research areas and synthesizing them into this nice work should be an inspiration to the broader community. The release of their code to reproduce results of all the experiments will also facilitate future research into this exciting topic of vector-drawing models.
I have revised my score to 8, since I believe this to be at least in the better half of accepted papers at ICLR based on my experience of publishing and attending the conference in the past few years. I hope the other reviewers can have some time to reevaluate the revision.
Original review:
Summary: they propose a differentiable learning algorithm that can output a brush stroke that can approximate a pixel image input, such as MNIST or Omniglot. Unlike sketch-pix2seq[3] (which is a pixel input -> sketch output model based on sketch-rnn[2]), their method trains in an unsupervised manner and does not require paired image/stroke data. They do this via training a "world model" to approximate brush painting software and emulate it. Since this emulation model is differentiable, they can easily train an algorithm to output a stroke to approximate the drawing via back propagation, and avoid using RL and costly compute such in earlier works such as [1].
The main strength of this paper is the original thought that went into it. From reading the paper, my guess is the authors came from a background that is not pure ML research (for instance, they are experts in Javascript, WebGL, and their writing style is easy to read), and it's great to see new ideas into our field. While research from big labs [1] have the advantage of having access to massive compute so that they can run large scale RL experiments to train an agent to "sketch" something that looks like MNIST or Omniglot, the authors probably had limited resources, and had to be more creative to come up with a solution to do the same thing that trains in a couple of hours using a single P40 GPU. Unlike [1] that used an actual software rendering package that is controlled by a stroke-drawing agent, their creative approach here is to train a generator network to learn to approximate a painting package they had built, and then freeze the weights of this generator to efficiently train an agent to draw. The results for MNIST and Omniglot look comparable to [1] but achieved with much fewer resources. I find this work refreshing, and I think it can be potentially much more impactful than [1] since people can actually use it with limited compute resources, and without using RL.
That being said, things are not all rosy, and I feel there are things that need to be done for this work to be ready for publication in a good venue like ICLR. Below are a few of my suggestions that I hope will help the authors improve their work, for either this conference, or if it gets rejected, I encourage the authors to try the next conference with these improvements:
1) multiple strokes, longe strokes. I don't think having a model that can output only a single stroke is scalable to other (simple) datasets such pixel versions of KangiVG [4] or QuickDraw [5]. The authors mentioned the need for an RNN, but couldn't the encoder just output the stroke in a format that contains the pen-down / pen-up event, like the stroke format suggested in [2]? Maybe, maybe not, but in either case, for this work to matter, multiple stroke generation is needed. Most datasets are also longer than 16 points, so you will need to show that your method works for say 80-120 points for this method to be comparable to existing work. If you can't scale up 16 points, would like to see a detailed discussion as to why.
2) While I like this method and approach, to play devil's advocate, what if I simply use an off the shelf bmp-to-svg converter that is fast and efficient (like [6]), and just build a set of stroke data from a dataset of pixel data, and train a sketch-rnn type model described in [3] to convert from pixel to stroke? What does this method offer that my description fails to offer? Would like to see some discussion there.
3) I'll give a hint for as to what I think for (2). I think the value in this method is that it can be converted to a full generative model with latent variables (like a VAE, GAN, sketch-rnn) where you can feed in a random vector (gaussian or uniform), and get a sketch as an output, and do things like interpolate between two sketches. Correct me if I'm wrong, but I don't think the encoder here in the first figure outputs an embedding that has a Gaussian prior (like a VAE), so it fails to be a generative model (check out [1], even that is a latent variable model). I think the model can be easily converted to one though to address this issue, and I strongly encourage the authors to try enforcing a Gaussian prior to an embedding space (that can fit right between the 16x16x128 average pooling op to the fully connected 1024 sized layer), and show results where we can interpolate between two latent variables and see how the vector sketches are interpolated. This has also been done in [2]. If the authors need space, I suggest putting the loss diagrams near the end into the appendix, since those are not too interesting to look at.
4) As mentioned earlier, I would love to see experimental results on [4] KangiVG and [5] QuickDraw datasets, even subsets of them. An interesting result would be to compare the stroke order of this algorithm with the natural stroke order for human doodles / Chinese characters.
Minor points:
a) The figures look like they are bitmap, pixel images, but for a paper advocating stroke/vector images, I recommend exporting the diagrams in SVG format and convert them to PDF so they like crisp in the paper.
b) Write style: There are some terms like "huge" dataset that is subjective and relative. While I'm happy about the writing style of this paper, maybe some reviewers who are more academic types might not like it and have a negative bias against this work. If things don't work out this time, I recommend the authors asking some friends who have published (successfully) at good ML conferences to proof read this paper for content and style.
c) It's great to see that the implementation is open sourced, and put it on github. Next time, I recommend uploading it to an anonymous github profile/repo, although personally (and for the record, in case area chairs are looking), I don't mind at all in this case, and I don't think the author's github address revealed any real identity (I haven't tried digging deeper). Some other reviewers / area chairs might not like to see a github link that is not anonymized though.
So in the end, even though I really like this paper, I can only give a score of 6 (edit: this has since been revised upward to 8). If the authors are able to address points 1-4, please do what you can in the next few weeks and give it your best shot. I'll look at the paper again and will revise the score upwards by a point or two if I think the improvements are there. If not, and this work ends up getting rejected, please consider improving the work later on and submitting to the next venue. Good luck!
[1] SPIRAL https://arxiv.org/abs/1804.01118
[2] sketch-rnn https://arxiv.org/abs/1704.03477
[3] sketch-pix2seq https://arxiv.org/abs/1709.04121
[4] http://kanjivg.tagaini.net/
[5] https://quickdraw.withgoogle.com/data
[6] https://vectormagic.com/