Task: Language Identification
Input (Spanish, English): Summary: This paper introduces a heuristic approach for training a deep directed generative model, where similar to the transition operator of a Markov chain each layer samples from the same conditional distribution. Similar to optimizing a variational lower bound, the approach is to approximate the gradient by replacing the posterior over latents with an alternative distribution. However, the approximating distribution is not updated to improve the lower bound but heuristically constructed in each step. A further difference to variational optimization is that the conditional distributions are optimized greedily rather than following the gradient of the joint log-likelihood.  Review: The proposed approach is interesting and to me seems worth exploring more. Given that there are approaches for training the same class of models which are 1) theoretically more sound, 2) of similar computational complexity, and 3) work well in practice (e.g. Rezende & Mohamed, 2015), I am nevertheless not sure of its potential to generate impact. My bigger concern, however, is that the empirical evaluation is still quite limited.  I appreciate the authors included proper estimates of the log-likelihood. This will enable and encourage future comparisons with this method on continuous MNIST. However, the authors should point out that the numbers taken from Wu et al. (2016) are not representative of the performance of a VAE. (From the paper: “Therefore, the log-likelihood values we report should not be compared directly against networks which have a more flexible observation model.” “Such observation models can easily achieve much higher log-likelihood scores, […].”)  Comparisons with inpainting results using other methods would have been nice. How practical is the proposed approach compared to other approaches? Similar to the diffusion approach by Sohl-Dickstein et al. (2015), the proposed approach seems to be both efficient and effective for inpainting. Not making this a bigger point and performing the proper evaluations seems like a missed opportunity.  Minor: – I am missing citations for “ordered visible dimension sampling” – Typos and frequent incorrect use of \citet and \citep
Output (English): en