Venue: ICLR 2017 conference

Reviews:

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: The authors describe the concrete distribution, a continuous approximation to
discrete distributions parameterized by a vector of continuous positive numbers
proportional to the probability of each discrete result. The concrete
distribution is obtained by using the softmax function to approximate the
argmax operator. The paper is clearly written, original and significant.
The experiments clearly illustrate the advantages of the proposed method.

Some minor questions:

"for the general n-ary case the Gumbel is a crucial 1 and the Gumbel-Max trick cannot be generalized
for other additive noise distributions"

What do you mean by this? Can you be more specific?

What is the temperature values used to obtain Table 1 and the table in Figure 4.

review_title: A clearly relevant paper that should be accepted 

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Thank you for an interesting read.

I think this paper has proposed a very useful method, which significantly simplifies the implementation of gradients for discrete random variables. Using this trick quite a lot of discrete variable-based methods will be significantly easier to implement, e.g. a GAN-style generator for text (see the recent arxiv preprint arXiv:1611.04051).

I've got one suggestion to make the paper even better, but maybe the authors want to leave it to future work. I think compared to lots of variance reduction techniques such as NVIL and VIMCO, this relaxation trick has smaller variance (from empirical observation of the reparameterisation trick), but in the price of introducing biases. It would be fantastic if the authors can discuss the bias-variance trade-off, either in theoretical or experimental way. My bet will be that here the variance dominates the stochastic estimation error of the gradient estimation, but it would be great if the authors can confirm this.

**to area chair: concurrent paper by Jang et al. 2016**
It seems there's a concurrent submission by Jang et al. I havent' read that paper in detail, but maybe the conference should accept or reject both?
review_title: Very important development for implementing stochastic networks with discrete random variables

final_score: rating: null
review_content: Is there any explanation about why VIMCO performs better in linear models in Table 1? Similarly, why is VIMCO failing in the Table shown in Figure 4?
review_title: Reasoning for the results of VIMCO vs. Concrete

final_score: rating: null
review_content: Thank you for an interesting read. 

How does your method compare with the reparameterisation through inverse CDF? For example, see https://arxiv.org/abs/1609.02200
review_title: Comparison to inverse CDF

final_score: rating: 7: Good paper, accept
review_content: The authors of the paper present a novel distribution for discrete variables called the "concrete distribution". The distribution can be seen as a continuous relaxation for a distribution over discrete random variables. The main motivation for introduction of the concrete distribution is the possibility to compute the gradient of discrete stochastic nodes in Stochastic Computational Graphs.

I think the paper is well written and sound, definitely of interest for the conference program.

As to the experimental part, the authors have results which support some kind of consistent superior performance for VIMCO for linear models and for concrete relaxations for non-linear models. Any explanation for that? Is this confirmed over different models and maybe datasets?
Similarly, it looks like VIMCO outperforms (in Figure 4) Concrete for large m, on the test NLL. I would encourage to try with other values of m to see if this dependence on large m is confirmed or not.

I believe the paper should be accepted to the conference, however please consider that I'm not an expert in this field.

Some minor observations/comments/issues:
-Section 2.1: there is a repetition "be be" in the first paragraph.
-Section 2.4: I would add a reference for the "multi-sample variational objective"
-Section 3.1, just before Section 3.2: "the Gumbel is a crucial 1". Why 1 and not "one"?
-Section 3.3, last paragraph: "Thus, in addition to relaxing the sampling pass of a SCG the log..." I would add a comma after "SCG". More in general, the second part of the paragraph is very dense and not easy to "absorb". I don't think it's an issue with the presentation: the concepts themselves are just dense. However, maybe the authors could find a way to make the paragraph easier to assimilate for a less experienced reader.
-Section 5.1, second paragraph: "All our models are neural networks with layers of n-ary discrete stochastic nodes with log_2(n)-dimensional states on the corners of the hypercube {-1,1}^log_2(n). The distribution of the nodes are parametrized by n real values log alpha_k". It is not clear to me, where does the log_2(n) come from. Similarly for the {-1,1}.
-Section 5.2: After "this distribution." and "We will" there is an extra space.
-If a compare the last formula in Section 5.3 with Eq. 8, I don't see exactly why the former is a special case of the latter. Is it because q(Z^i | x) is always one?
review_title: Nice paper, should be accepted

Meta-review: This paper proposes a neat general method for relaxing models with discrete softmax choices into closely-related models with continous random variables. The method is designed to work well with the reparameterization trick used in stochastic variational inference. This work is likely to have wide impact.
 
 Related submissions at ICLR:
 "Categorical Reparameterization with Gumbel-Softmax" by Jang et al. contains the same core idea. "Discrete variational autoencoders", by Rolfe, contains an alternative relaxation for autoencoders with discrete latents, which I personally find harder to follow.
Decision: Accept (Poster)