Task: Language Identification
Input (Spanish, English): This paper introduces a variant of the semi-supervised variational auto-encoder (VAE) framework. The authors present a way of introducing structure (observed variables) inside the recognition network.  I find that the presentation of the inference with auxiliary variables could be avoided, as it actually makes the presentation unnecessarily complicated. Specifically, the expressions with auxiliary variables are helpful for devising a unified implementation, but modeling-wise one can get the same model without these auxiliary variables and recover a minimal extension of VAE where part of the generating space is actually observed. The observed variables mean that the posterior needs to also condition on those, so as to incorporate the information they convey. The way this is done in this paper is actually not very different from Kingma et al. 2014, and I am surprised that the experiments show a large deviation in these two methods' results. Given the similarity of the models, it'd be useful if the authors could give a possible explanation on the superiority of their method compared to Kingma et al. 2014. By the way, I was wondering if the experimental setup is the same as in Kingma et al. 2014 for the results of Fig. 5 (bottom) - the authors mention that they use CNNs for feature extraction but from the paper it's not clear if Kingma et al. do the same.   On a related note, I was wondering the same for the comparison with Jampani et al. 2015. In particular, is that model also using the same rate of supervision for a fair comparison?  The experiment in section 4.3 is interesting and demonstrates a useful property of the approach.  The discussion of the supervision rate (and the pre-review answer) is helpful in giving some insight about what is a successful training protocol to use in semi-supervised learning.  Overall, the paper is interesting but the title and introduction made me expect something more from it. From the title I expected a method for interpreting general deep generative models, instead the described approach was about a semi-supervised variant of VAE - naturally including labelled examples disentangles the latent space, but this is a general property of any semi-supervised probabilistic model and not unique to the approach described here. Moreover, from the intro I expected to see a more general approximation scheme for the variational posterior (similar to Ranganath et al. 2015  which trully allows very flexible distributions), however this is not the case here.  Given the above, the contributions of this paper are in defining a slight variant of the semi-supervised VAE, and (perhaps more importantly) formulating it in a way that is amendable to easier automation in terms of software. But methodologically there is not much contribution to the current literature. The authors mention that they plan to extend the framework in the probabilistic programming setting. It seems indeed that this would be a very promising and useful extension.   Minor note: three of Kingma's papers are all cited in the main text as Kingma et al. 2014, causing confusion. I suggest using Kingma et al. 2014a etc.
Output (English): en