Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: This is an interesting and mathematically rather dense paper. I am slowly making my way through it. No specific questions at this time. 
review_title: Pre-review

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: This paper presents a way of training deep generative models with discrete hidden variables using the reparameterization trick. It then applies it to a particular DBN-like architecture, and shows that this architecture achieves state-of-the-art density modeling performance on MNIST and similar datasets. 

The paper is well written, and the exposition is both thorough and precise. There are several appendices which justify various design decisions in detail. I wish more papers in our field would take this degree of care with the exposition!

The log-likelihood results are quite strong, especially given that most of the competitive algorithms are based on continuous latent variables. Probably the main thing missing from the experiments is some way to separate out the contributions of the architecture and the inference algorithm. (E.g., what if a comparable architecture is trained with VIMCO, or if the algorithm is applied to a previously published discrete architecture?)

I’m a bit concerned about the variance of the gradients in the general formulation of the algorithm. See my comment “variance of the derivatives of F^{-1}” below. I think the response is convincing, but the problem (as well as “engineering principles” for the smoothing distribution) are probably worth pointing out in the paper itself, since the problem seems likely to occur unless the user is aware of it. (E.g., my proposal of widely separated normals would be a natural distribution to consider until one actually works through the gradients — something not commonly done in the age of autodiff frameworks.)

Another concern is how many sequential operations are needed for inference in the RBM model. (Note: is this actually an RBM, or a general Boltzmann machine?) The q distribution takes the form of an autoregressive model where the variables are processed one at a time. Section 3 mentions the possibility of grouping together variables in the q distribution, and this is elaborated in detail in Appendix A. But the solution requires decomposing the joint into a product of conditionals and applying the CDFs sequentially. So either way, it seems like we’re stuck handling all the variables sequentially, which might get expensive. 

Minor: the second paragraph of Section 3 needs a reference to Appendix A.

review_title: clever and useful contribution; clear and thorough exposition

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Paper proposes a novel Variational Encoder architecture that contains discrete variables. Model contains an undirected discrete component that captures distribution over disconnected manifolds and a directed hierarchical continuous component that models the actual manifolds (induced by the discrete variables). In essence the model clusters the data and at the same time learns a continuous manifold representation for the clusters. The training procedure for such models is also presented and is quite involved. Experiments illustrate state-of-the-art performance on public datasets (including MNIST, Omniglot, Caltech-101). 

Overall the model is interesting and could be useful in a variety of applications and domains. The approach is complex and somewhat mathematically involved. It's not exactly clear how the model compares or relates to other RBM formulations, particularly those that contain discrete latent variables and continuous outputs. As a prime example:

Graham Taylor and Geoffrey Hinton. Factored conditional restricted Boltzmann machines for modeling motion style. In Proc. of the 26th International Conference on Machine Learning (ICML), 1025–1032, 2009.

Discussion of this should certainly be added. 

review_title: Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This is an interesting paper on how to handle reparameterization in VAEs when you have discrete variables. The idea is to introduce a smoothing transformation that is shared between the generative model and the recognition model (leading to cancellations). 
A second contribution is to introduce an RBM as the prior model P(z) and to use autoregressive connections in generative and recognition models. The whole package becomes a bit entangled and complex and it is hard to figure out what causes the claimed good performance. Experiments that study these contributions separately would have been nice. 
The framework does become a little complex but this should not be a problem if nice software is delivered that can be used in a plug and play mode.
Overall, the paper is very rich with ideas so I think it would be a great contribution to the conference. 

review_title: Rich set of ideas on how to make VAEs work better. 

final_score: rating: null
review_content: It seems like the derivatives of F^{-1} could have high variance in a multimodal setting. For instance, consider this simplified example. We let z be a Bernoulli random variable with mean parameter \theta, and

\zeta | z=0 ~ N(0, 1)
\zeta | z=0 ~ N(10, 1)

Suppose \theta = 0.5 and \rho = 0.5, so that \zeta = 5 is in between the two modes. By my calculation, dF^{-1}(\rho) / d\theta is approximately 1/r(\zeta), which is 6.72e5. Since the derivative in this region is about 1/r(\zeta), the derivatives get large as fast as the probabilities get small, so maybe these low probability regions could still contribute significantly to the variance.

Because derivatives of dF^{-1} appear in the backprop computations, this could lead to high variance stochastic gradients in general. Is there some reason (either theoretical or empirical) to believe this isn't a problem?

review_title: variance of the derivatives of F^{-1}

Meta-review: The authors present a novel reparameterization framework for VAEs with discrete random variables. The idea is to carry out symmetric projections of the approximate posterior and the prior into a continuous space and evaluating the autoencoder term in that space by marginalizing out the discrete variables. They consider the KL divergence between the approximating posterior and the true prior in the original discrete space and show that due to the symmetry of the projection into the continuous space, it does not
 contribute to the KL term. 
 
 One question that warrants further investigation is whether this framework can be extended to GANs and what empirical success they would have.
 
 The reviewers have presented a strong case for the acceptance of the paper and I go with their recommendation.
Decision: Accept (Poster)