Task: Language Identification
Input (Spanish, English): Summary: This paper on autoregressive generative models explores various extensions of PixelCNNs. The proposed changes are to replace the softmax function with a logistic mixture model, to use dropout for regularization, to use downsampling to increase receptive field size, and the introduction of particular skip connections. The authors find that this allows the PixelCNN to outperform a PixelRNN on CIFAR-10, the previous state-of-the-art model. The authors further explore the performance of PixelCNNs with smaller receptive field sizes.  Review: This is a useful contribution towards better tractable image models. In particular, autoregressive models can be quite slow at test time, and the more efficient architectures described here should help with that.  My main criticism regards the severe neglect of related work. Mixture models have been used a lot in autoregressive image modeling, including for multivariate conditional densities and including downsampling to increase receptive field size, albeit in a different manner: Domke (2008), Hosseini et al. (2010), Theis et al. (2012), Uria et al. (2013), Theis et al. (2015). Note that the logistic distribution is a special case of the Gaussian scale mixture (West, 1978).  The main difference seems to be the integration of the density to model integers. While this is clearly a good idea and the right way forward, the authors claim but do not support that not doing this has “proved to be a problem for earlier models based on continuous distributions”. Please elaborate, add a reference, or ideally report the performance achieved by PixelCNN++ without integration (and instead adding uniform noise to make the variables continuous).  60,000 images are not a lot in a high-dimensional space. While I can see the usefulness of regularization for specialized content – and this can serve as a good example to demonstrate the usefulness of dropout – why not use “80 million tiny images” (superset of CIFAR-10) for natural images? Semi-supervised learning should be fairly trivial here (because the model’s likelihood is tractable), so this data could even be used in the class-conditional case.  It would be interesting to know how fast the different models are at test time (i.e., when generating images).
Output (English): en