instruction:
In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
question:
Summary: This paper on autoregressive generative models explores various extensions of PixelCNNs. The proposed changes are to replace the softmax function with a logistic mixture model, to use dropout for regularization, to use downsampling to increase receptive field size, and the introduction of particular skip connections. The authors find that this allows the PixelCNN to outperform a PixelRNN on CIFAR-10, the previous state-of-the-art model. The authors further explore the performance of PixelCNNs with smaller receptive field sizes.  Review: This is a useful contribution towards better tractable image models. In particular, autoregressive models can be quite slow at test time, and the more efficient architectures described here should help with that.  My main criticism regards the severe neglect of related work. Mixture models have been used a lot in autoregressive image modeling, including for multivariate conditional densities and including downsampling to increase receptive field size, albeit in a different manner: Domke (2008), Hosseini et al. (2010), Theis et al. (2012), Uria et al. (2013), Theis et al. (2015). Note that the logistic distribution is a special case of the Gaussian scale mixture (West, 1978).  The main difference seems to be the integration of the density to model integers. While this is clearly a good idea and the right way forward, the authors claim but do not support that not doing this has “proved to be a problem for earlier models based on continuous distributions”. Please elaborate, add a reference, or ideally report the performance achieved by PixelCNN++ without integration (and instead adding uniform noise to make the variables continuous).  60,000 images are not a lot in a high-dimensional space. While I can see the usefulness of regularization for specialized content – and this can serve as a good example to demonstrate the usefulness of dropout – why not use “80 million tiny images” (superset of CIFAR-10) for natural images? Semi-supervised learning should be fairly trivial here (because the model’s likelihood is tractable), so this data could even be used in the class-conditional case.  It would be interesting to know how fast the different models are at test time (i.e., when generating images).
answer:
en


question:
The paper describes a novel technique to improve the efficiency of computation graphs in deep learning frameworks. An impressive speedup can be observed in their implementation within TensorFlow. The content is presented with sufficient clarity, although some more graphical illustrations could be useful. This work is relevant in order to achieve highest performance in neural network training.   Pros:  - significant speed improvements through dynamic batching - source code provided   Cons:  - the effect on a large real-world (ASR, SMT) would allow the reader to put the improvements better into context - presentation/vizualisation can be improved
answer:
en


question:
This paper tackles the problem of compressing trained convnets with the goal of reducing memory overhead and speeding up the forward pass. As I understand it, the main contribution of this work is to develop fast convolution routines for sparse conv weights int he case of general sparsity (as compared with structured sparsity). They evaluate their method on both AlexNet and GoogLeNet as well as on various platforms. The authors make code available online. The paper is well written and does a good job of putting this work in the context of past model reduction techniques.  My main request of the authors would be to provide a concise summary of the speedup/memory gains achievable with this new work compared with previously published work. The authors do show the various sparsity level obtained with various methods of pruning but it is unclear to me how to translate the information given in the paper into an understanding of gains relative to other methods.
answer:

en