Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper introduces a new gradient estimator for discrete variables. The estimator is based on a conditionally marginalized control variate scheme. The control variate is based on a combination of the REINFORCE estimator and the recently proposed Concrete distribution, a continuous relaxation of the discrete disribution. 

An important improvement upon the estimator of the Concrete distribution, is the unbiasedness. The authors show in experiments that theirs indeed result in better solutions. Unfortunately, it is not reported what the performance is at convergence.
review_title: Introduces new control variate for REINFORCE gradient

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper introduces a way of training models with discrete latent variables using a low-variance unbiased stochastic estimate of the true gradient. The trick is to take the recently proposed Gumbel/softmax (or concrete) distribution, but use it to construct a control variate for the stochastic gradient.

Overall, I find the idea both clever and convincing. Since the concrete distribution was already shown in previous work to be directly convertible into a discrete model, it ought to give a good approximation to the gradient for the discrete model. Therefore, the stochasticity in the REINFORCE gradient for the continuous model ought to be a strong control variate for the REINFORCE gradient in the discrete one. 

The experiments show that REBAR is able to reduce the variance of the gradient estimates by almost an order of magnitude on binary MNIST (still a challenging benchmark for discrete latent variable models). The method still appears to underperform the concrete distribution itself; I expect with a bit more effort, one should be able to find cases where the discrete model pays off. But the idea seems sound, and overall I think this is a strong workshop paper.

review_title: Solid idea for variance reduction in latent variable models

Decision: Accept