Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: How does this relate to dirichlet distribution?
review_title: dirichlet distribution

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose a method for reparameterization gradients with categorical distributions. This is done by using the Gumbel-Softmax distribution, a smoothened version of the Gumbel-Max trick for sampling from a multinomial.

The paper is well-written and clear. The application to the semi-supervised model in Kingma et al. (2014) makes sense for large classes, as well as its application to general stochastic computation graphs (Schulman et al., 2015).

One disconcerting point is that (from my understanding at least), this does not actually perform variational inference for discrete latent variable models. Rather, it changes the probability model itself and performs approximate inference on the modified (continuous relaxed) version of the model. This is fine in practice given that it's all approximate inference, but unlike previous variational inference advances either in more expressive approximations or faster computation (as noted by the different gradient estimators they compare to), the probability model is fundamentally changed.

Two critical points seem key: the sensitivity of the temperature, and whether this applies for non-one hot encodings of the categorical distribution (and thus sufficiently scale to high dimensions). Comments by the authors on this are welcome.

There is a related work by Rolfe (2016) on discrete VAEs, who also consider a continuous relaxed approach. This is worth citing and comparing to (or at least mentioning) in the paper.

References

Rolfe, J. T. (2016). Discrete Variational Autoencoders. arXiv.org.
review_title: Review: Categorical Reparameterization with Gumbel-Softmax

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper combines Gumbel distribution with the popular softmax function to obtain a continuous distribution on the simplex that can approximate categorical samples. It is not surprising that Gumbel softmax outperforms other single sample gradient estimators. However, I am curious about how Gumbel compares with Dirichlet experimentally. 

The computational efficiency of the estimator when training semi-supervised models is nice. However, the advantage will be greater when the number of classes are huge, which doesn't seem to be the case in a simple dataset like MNIST. I am wondering why the experiments are not done on a richer dataset. 

The presentation of the paper is neat and clean. The experiments settings are clearly explained and the analysis appears to be complete. 

The only concern I have is the novelty of this work. I consider this work as a nice but may be incremental (relatively small) contribution to our community. 
review_title: The paper is well written but the novelty of the paper is less clear

final_score: rating: 7: Good paper, accept
review_content: This paper introduces a continuous relaxation of categorical distribution,  namely the the Gumbel-Softmax distribution, such that generative models with categorical random variables can be trained using reparameterization (path-derivative) gradients. The method is shown to improve upon other methods in terms of the achieved log-likelihoods of the resulting models. The main contribution, namely the method itself, is simple yet nontrivial and worth publishing, and seems effective in experiments. The paper is well-written, and I applaud the details provided in the appendix. The main application seems to be semi-supervised situations where you really want categorical variables.

 - P1: "differentiable sampling mechanism for softmax". "sampling" => "approximate sampling", since it's technically sampling from the Gumbal-softmax.
 
 - P3: "backpropagtion"
 
 - Section 4.1: Interesting experiments.
 
 - It would be interesting to report whether there is any discrepancy between the relaxed and non-relaxed models in terms of log-likelihood. Currently, only the likelihoods under the non-relaxed models are reported.
 
 - It is slightly discouraging that the temperature (a nuisance parameter) is used differently across experiments. It would be nice to give more details on whether you were succesful in learning the temperature, instead of annealing it; it would be interesting if that hyper-parameter could be eliminated.
review_title: Interesting idea, encouraging results

Meta-review: This paper proposes a neat general method for relaxing models with discrete softmax choices into closely-related models with continuous random variables. The method is designed to work well with the reparameterization trick used in stochastic variational inference. This work is likely to have wide impact.
 
 Related submissions at ICLR:
 "The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables" by Maddison et al. contains the same core idea. "Discrete variational autoencoders", by Rolfe, contains an alternative relaxation for autoencoders with discrete latents, which I personally find harder to follow.
Decision: Accept (Poster)