Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a simple technique for speeding up generation from Convolutional Autoregressive Models (e.g., WaveNet and PixelCNN). The key observation is that if one naively generates each output from scratch without re-using any computation, then it is wasteful. The paper instead proposes to cache hidden state values across the generation of all the outputs that share the intermediate results. Experimentally the paper shows large speedups over the naive approach when the depth of a WaveNet is increased to 13+ layers and PixelCNN++ when the batch size is large.

Overall the paper is clear, and the approach is a clear improvement over the naive version. One question I have, though, is if it wouldn't be simpler to just build a TensorFlow model that generates an entire output at once. That is, instead of building a TensorFlow model that generates the next pixel and then calling this model repeatedly, would it be possible to define a TensorFlow model that outputs a full image?  (To deal with having to sample output values, the Gumbel-max trick could be used with all of the randomness needed supplied as an input). Then presumably the TensorFlow execution model would take care of all the necessary caching.

A second question is about the relevance of the technique in the WaveNet experiments. The headline improvement of 21x doesn't happen until there are 15 layers in the WaveNet. Is this a useful parameter regime for the model?

Pros:
- This is a clearly better method than the naive approach, and the naive approach does appear to have been used before
- The idea is simple and clearly explained
- The authors are open-sourcing their implementation, which will likely be used by a number of people in the ICLR audience

Cons:
- It's not obvious to me that this is the simplest way to implement the idea
- The idea is very simple, effectively being "cache in the obvious way"

Overall I'd lean towards accepting, but I wouldn't fight strongly for it.
review_title: Very simple idea, but likely to be used

final_score: rating: 7: Good paper, accept
review_content: This is a nice workshop paper.  its a simple idea but people will be interested in it.  If nothing else, the released code is valuable, and having the poster to advertise it is a good use of workshop poster space.
review_title: simple and good

Decision: Accept