Paper:
Concrete Dropout
Dropout is used as a practical tool to obtain uncertainty estimates in large vision models and reinforcement learning (RL) tasks. But to obtain well-calibrated uncertainty estimates, a grid-search over the dropout probabilities is necessarya prohibitive operation with large models, and an impossible one with RL. We propose a new dropout variant which gives improved performance and better calibrated uncertainties. Relying on recent developments in Bayesian deep learning, we use a continuous relaxation of dropout's discrete masks. Together with a principled optimisation objective, this allows for automatic tuning of the dropout probability in large models, and as a result faster experimentation cycles. In RL this allows the agent to adapt its uncertainty dynamically as more data is observed. We analyse the proposed variant extensively on a range of tasks, and give insights into common practice in the field where larger dropout probabilities are often used in deeper model layers.

Review:
This paper proposes an approach for automatic tuning of dropout
probabilities under the variational interpretation of dropout as a Bayesian approximation. Since the dropout probability characterizes the overall
posterior uncertainty, such tuning is necessary for calibrated inferences, 
but has thus far required expensive manual tuning, e.g., via offline grid 
search. The proposed method replaces the fixed Bernoulli variational distribution 
at training time with a concrete/gumbel-softmax distribution, allowing use of the
reparameterization trick to compute low-variance stochastic gradients
of the ELBO wrt dropout probabilities. This is evaluated on a range of tasks
ranging from toy synthetic datasets to capturing uncertainty in semantic 
segmentation and model-based RL.
The motivation seems clear and the proposed solution is simple and
natural, combining dropout Bayesian inference with the concrete
distribution in the obvious way. The technical contributions of the
paper seem relatively minimal, but this is not necessarily a problem;
it's great when a simple, straightforward method turns out to
give impressive results. 
That said, given the straightforwardness of the approach, the
significance of this paper rests heavily on the evaluation, and here I
am not really convinced. Although there are a lot of experiments, none
seems to show a compelling advantage and collectively they fail to
answer many natural questions about the approach. For example:
 - what is the training time overhead versus standard dropout? (line
   281 says 'negligible' but this is never quantified, naively I'd
   expect differentiating through the gumbel-softmax to add
   significant overhead)
 - do the learned dropout probabilities match the (presumably optimal)
   probabilities obtained via grid search?
 - how to set the temperature of the concrete distribution (and does
   this matter)? The code in the appendix seems to used a fixed t=0.1
   but this is never mentioned or discussed in the paper.
 - how does this method compare to other reasonable baselines, eg
   explicit mean-field BNNs, or Gaussian dropout with tuned alpha? The
   latter seems like a highly relevant comparison, since the Gaussian
   approach seems simpler both theoretically and in implementation (is
   directly reparameterizable without the concrete approximation), and
   to dismiss it by reference to a brief workshop paper [28] claiming
   the approach 'underperforms' (I don't see such a conclusion in
   that paper even for the tasks it considers, let alone in any
   more general sense) is lazy.
 - there is a fair amount of sloppiness in the writing, including
   multiple typos and cases where text and figures don't agree (some
   detailed below). This does not build confidence in the
   meticulousness of the experiments.
More specific comments: 
Synthetic data (sec 4.1): I am not sure what
to conclude from this experiment. It reads as simply presenting the
behavior of the proposed approach, without any critical evaluation of
whether that behavior is good. Eg, Fig 1b describes aleatoric
uncertainty as following an 'increasing trend' (a bit of a stretch
given that the majority of steps are decreasing), and 1c describes
predictive uncertainty as 'mostly constant' (though the overall shape
actually looks very similar to 1b) -- but why is it obvious that
either of these behaviors is correct or desirable? Even if this
behavior qualitatively matches what you'd get from a more principled
Bayesian approach (I was expecting the paper to argue this but it
doesn't), a simple synthetic task should be an opportunity for a real
quantitative comparison: how closely do the dropout uncertainties
match what you'd get from, e.g., HMC, or even exact
Bayesian linear regression (feasible here since the true function is
linear -- so concrete dropout on a linear model should be a fair
comparison).
UCI (sec 4.2): the experimental setup is not clear to me -- is the
'dropout' baseline using some fixed probability (in which case, what
is it?) or is it tuned by grid search (how?). More fundamentally,
given that the goal of CDropout is to produce calibrated uncertainties,
why are its NLL scores (fig 2, a reasonable measure of uncertainty)
*worse* than standard dropout in the majority of cases?
MNIST (sec 4.3): again no real comparison to other approaches, no
evaluation of uncertainty quality. The accuracy 'matches' hand-tuned
dropout -- does it find the same probabilities? If so this is worth
saying. If not, seems strange to refer to the converged probabilities
as 'optimal' and analyze them for insights.
Computer vision (sec 4.4): this is to me the most promising
experiment, showing that concrete dropout matches or slightly beats
hand-tuned dropout on a real task. It's especially nice to see that
the learned probabilities are robust to initialization since this is a
natural worry in non-convex optimization. Some things that seem
confusing here:
 - what is the metric "IoU"? presumably Intersection over Union but
   this is never spelled out or explained.
 - the differences in calibration between methods are
   relatively small compared to the overall miscalibration of all the
   methods - why is this? 
 - text says "middle layers are largely deterministic" which seems
   inconsistent with Fig 8c showing convergence to around p=0.2?
(nitpick: "Table" 2 is actually a figure)
Model-based RL (sec 4.5): It's nice that the model matches our
intuition that dynamics uncertainty should decrease with
experience. But it sounds like this tuning doesn't actually improve
performance (only matches the cumulative reward of a fixed 0.1
dropout), so what's the point? Presumably there should be some
advantages to better-calibrated dynamics uncertainty; why do they not
show up in this task?
The method proposed in this paper seems promising and potentially
quite useful. If so it should be possible to put together a careful
and clear evaluation showing compelling advantages (and minor costs)
over relevant baselines. I'd like to read that paper! But as currently
submitted I don't think this meets the NIPS bar. 
UPDATE: increasing my rating slightly post rebuttal. I still think this paper is marginal, but the clarifications promised in the rebuttal (describing the tuning of dropout baselines, and the problems with Gaussian dropout, among others) will make it stronger. It's worth noting that recent work (Molchanov et al., ICML 2017 https://arxiv.org/abs/1701.05369) claims to fix the variance and degeneracy issues with Gaussian dropout; this would be an interesting comparison for future work.