Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: Thank you for the interesting paper!

Summary

The authors focus on the important problem of efficient uncertainty quantification. More specifically, they propose a methodology that approximates the variance across samples from an MC dropout model with a single forward pass. To do this, they define analogs to existing layers such that they analytically (and often approximately) propagate variance from the input to the output of the layer. By repeating this for all layers, they construct a "variance-propagation BNN" (VPBNN) model that otherwise has the same architecture as the existing MC dropout model but is able to effectively output sample variance with a single forward pass. They demonstrate the effectiveness of their approach on (1) a simple synthetic problem, (2) a language modeling task, and (3) OOD detection.

Strengths

- Efficiently (from a FLOPS standpoint) propagating model uncertainty in a BNN is an important research area, particularly for compute-constrained use cases.
- The authors focus on a relevant set of experiments to demonstrate both the ability of their method to approximate MC dropout, and the ability to perform at, or better than, existing methods on downstream tasks.


Weaknesses

As noted below, I have concerns around the experimental results. More specifically, I feel that there is a relative lack of discussion around the (somewhat surprising) outperformance of baselines that VPBNN is aiming to approximate, and I feel that the experiments are missing what I see as key VPBNN results that otherwise leave the reader with questions. Additionally, I think the current paper would benefit from including measurements and discussion around the specifics of computational and memory costs of their method.

Recommendation

In general, I think this could be a great paper. However, given the above concerns, I'm currently inclined to suggest rejection of the paper in its current state. I would highly recommend that authors push further on the noted areas!

Additional comments

- p. 1: "The uncertainty is defined based on the posterior distribution." For more clarity it could be helpful to update this to say that the epistemic model uncertainty is represented in the prior distribution, and upon observing data, those beliefs can be updated in the form of a posterior distribution, which yields model uncertainty conditioned on observed data.
- p. 2: "The MC dropout requires a number of repeated feed-forward calculations with randomly sampled weight parameters in order to obtain the predictive distribution." This should be updated to indicate that in MC dropout, dropout is used (in an otherwise deterministic model) at test time with "a number of repeated feed-forward calculations" to effectively sample from the approximate posterior, but not directly via different weight samples (as in a variational BNN). With variational dropout, this ends up having a nice interpretation as a variational Bayes method, though no weight distributions are typically directly used with direct MC dropout.
- p. 2: Lakshminarayanan et al. (2017) presented random seed ensembles, not bootstrap ensembles (see p. 4 of their work for more info). They used the full dataset, and trained M ensemble members with different random seeds, rather than resampled data.
- p. 4: For variance propagation in a dropout layer with stochastic input, it's not exactly clear from the text how variance from the inputs and dropout is being combined into an output Gaussian. I believe using a Gaussian is an approximation, and while that would be fine, I think it would be informative to indicate that. The same issue comes up with local reparameterization for BNNs with parameter distributions, where they can be reparameterized exactly as output distributions (for, say, mean-field Gaussian weight dists) so long as the inputs are deterministic. Otherwise, the product of, say, two Gaussian RVs is non-Gaussian.
- p. 7: Figure 1 is too small.
- p. 7: "Estimation of ρ is possible by observing the outputs of middle layers several times under the approximate predictive distribution. The additional computation cost is still kept quite small compared to MC dropout." How exactly is $\rho$ estimated? Is it a one-time cost irregardless of data that can then be used for all predictions from the trained model? Without details, this seems like a key component that can yield arbitrary amounts of uncertainty.
- p. 7, 8: For the language modeling experiment, why do you think VPBNN was able to achieve lower perplexity values than MC dropout? The text generally focuses on VPBNN as an approximation to MC dropout, and yet it outperforms it. The text would greatly benefit from more discussion around this point. 
- p. 8: For the OOD detection experiment, I'm surprised that $\rho = 0$ was the only VPBNN model used, since Section 5.1 and Figure 1 indicated that it led to overconfident models. Can you include results with other settings of $\rho$? Moreover, from Figure 1 we see that (for that model) VPBNN with $\rho = 0$ qualitatively yielded the same amount of predictive variance as the Taylor approximation. However, in Table 2, we see VPBNN with $\rho = 0$ outperform MC dropout (with 100 or 2000 samples) and the Taylor approximation. Why do you think this is the case, particularly if the standard deviation was used as the uncertainty signal for the OOD decision. I see that "This is because the approximation accuracy of the Taylor approximation is not necessarily high as shown in Section B", but I did not find Section B or Figure 3 to be clear. I think the text would benefit from more discussion here, and from the additional experiments for $\rho$.
- Can you include a discussion and measurements for FLOPS and memory usage for VPBNN? Specifically, given the discussion around efficiency and the implementation that doubles the dimensionality of the intermediates throughout the model, I believe it would be informative to have theoretical and possibly runtime measurements.

Minor

- p. 1: s/using the dropout/using dropout/
- p. 1: s/of the language modeling/of language modeling/
- p. 2: s/is the representative of/is representative of/
- p. 2: s/In the deep learning/In deep learning/
- p. 2: s/This relations/This relation/
- p. 5: Need to define $s$ as the sigmoid function in the LSTM cell equations.
review_title: Review

final_score: rating: 3: Clear rejection
review_content: This paper proposes a sampling free technique based on variance propagation to model predictive distributions of deep learning models. Estimating uncertainty of deep learning models is an important line of research for understanding the reliability of predictions and ensuring robustness to out-of-distribution data. Results are shown using synthetic data, perplexity analysis for a language modeling task and out-of-distribution detection performance using a convolutional network.

Overall I vote for rejecting the paper. The paper proposes an upper bound to the variance estimate of predictive distributions. However, the paper does not explain how the upper bound can be ensured. Furthermore, the experiments based on real data are conducted not using the upper bound approach but assuming strong independence assumptions (\rho = 0). In my opinion, the independence assumption needs extensive experiments to validate its performance under different scenarios. Furthermore, for further adoption of the upper bound approach, I think the authors need to provide extensive experiments to showcase the tradeoffs. For instance, as a reader of the paper I would like to get a sense of how much I will be overestimating the variance under typical scenarios. For this, I would recommend a formal analysis of the uncertainty estimates by inspecting the confidence intervals through coverage properties. 

My specific comments related to this approach:
-	The authors propose two approaches in Section 3.1 for estimating the variance in predictions and the remaining subsections in Section 3 build on this. 
o	The first approach (a) assumes independence in the input. In my opinion, this is a very strong assumption. For instance, multi-collinearity in features in DNN’s is a very common usage pattern. In domains like images, by construction of the problem, you expect a spatial correlation structure. Furthermore, the multi-layer perceptron architecture by construction is susceptible to correlation among variables. I would highly recommend adding a discussion on the validity independence assumption in general.
o	The second approach (b) will be necessarily true if `\rho >= \max_{j,j’:j!=j’} \rho_{j,j’}`. Only then this will guarantee that you will obtain an upper bound to the variance but it is very likely that you will overestimate the variance since your estimate is going to be bound by the highest correlation in your system. Again, in my opinion, this is a significant limitation of this approach and I recommend that authors highlight these points. Another minor thing to note in the manuscript is that when \rho=0, option (b) reduces to option (a).
o	I would suggest adding another summation term in Var(y_i) related equations to denote the double summation (indexed by j’) happening over the covariance terms (in Section 3.1).
o	Related to the above point, I could not follow the mathematical derivation from line (1) to line (2) in Equation (1). Could the authors provide an explicit derivation to ensure that the derivation is correct?
o	Furthermore, author’s claim that “distribution of y_i is well approximated by the univariate Gaussian distribution if the correlation among x is small” (Section 3.1) needs justification. To my knowledge, Wang & Manning (2013)’s Gaussian approximation holds due to central limit theorem as the number of samples approaches infinity but I do not think this will be applicable in this sample-free setting.

-	Regarding the results:
o	The results shown in Figure 1 demonstrates that choosing an appropriate `\rho can be challenging. We see underestimation of uncertainty with \rho <=0.15 and overestimation with \rho = 1. However, since these are based on synthetic data, I would suggest that the authors formally assess the fit using metrics like confidence interval widths and coverage.
o	The authors note that “Estimation of ρ is possible by observing the outputs of middle layers several times under the approximate predictive distribution.”. I am not convinced that this is an easy problem, I would recommend that the authors provide an example and elaborate this in depth.

Minor comments:
-	Please indicate the best results in the tables by highlighting them. 
-	The uncertainty in deep learning literature typically employs distinction between aleatory and epistemic uncertainties. I think the manuscript can benefit how this proposed approach maps to the different sources of uncertainties. 
review_title: Recommendation to Reject

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes a sampling free approach for estimating predictive uncertainties in Bayesian neural networks trained via Monte-Carlo dropout. In particular,  given a dropout trained neural network, the paper develops a deterministic approximation to the test time predictive distribution that is otherwise approximated through Monte-Carlo simulations.

The paper is clearly written and proposes a solution to an important problem. Extracting fast and reliable predictive uncertainties from BNNs, trained via MC-dropout or other approximate inference techniques, is crucial for deploying BNNs in resource constrained and/or real time applications. Deterministic approximations like the one presented here are promising. 

That being said, I have several concerns with the paper that revolve around differentiation from previous work,  missing empirical comparisons, and some curious modeling choices. 

* The primary contribution of the paper appears to be the amortization of the MC-dropout posterior predictive density. This is achieved by propagating  uncertainty (first and second moments of the inputs) through the network.  The technical details of such uncertainty propagation have been worked out by several authors in the past (see 1, 2, 3, and the papers cited by the authors Wu et al., 2019 and Shekhovtsov & Flach 2019). While the application to amortizing MC-dropout is interesting, it appears to be a direct application of previous work.
* The notion of upper bounding the variance during the propagation is indeed distinct from previous work, but is neither principled nor empirically vetted to be consistently useful.  The experiments provide limited evidence in support of using the upper-bound. The derived uncertainty seems to be crucially dependent on the weighting factor $\rho$ and it is unclear how one would select this weighting factor. In general, injecting additional noise does not guarantee better calibrated uncertainties. Neither the language modeling task nor the OOD detection task make use of the upper bound.
* Amortization of the posterior predictive distribution is not a new idea. A popular approach is to use distillation [4, 5] based techniques to approximate the posterior predictive distribution with a second neural network. This work needs to be both discussed (advantages / disadvantages of the proposed approach over distillation)  and empirically compared against. 
* Since the goal of the paper is to sufficiently well approximate the posterior predictive density, the experimental section would benefit from including calibration metrics (ECE / Brier score; see 6). 
* (Minor) The paper at several places claims that the proposed approach “Unlike various kinds of probabilistic NNs, we do not need any specialized training procedure to evaluate the uncertainty”. While this is technically true, this is also true for other distillation based amortization techniques. Which arguably are simpler because one can just use an off the shelf network and not have to worry about propagating moments correctly. 
* (Minor) Since the categorical Gaussian integral is intractable, the authors replace the softmax layer with a series of independent sigmoids and use an approximation for the sigmoid Gaussian convolution. This is strange. If we are going down this path, why not use independent probits instead of sigmoids? The probit-Gaussian integral is exactly computable and requires no further approximations (see equation 3.82 in http://www.gaussianprocess.org/gpml/chapters/RW3.pdf). 

[1] https://papers.nips.cc/paper/5269-expectation-backpropagation-parameter-free-training-of-multilayer-neural-networks-with-continuous-or-discrete-weights.pdf

[2] https://arxiv.org/abs/1502.05336

[3] https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12391

[4] https://arxiv.org/abs/1506.04416

[5] https://arxiv.org/abs/2005.08110

[6] Tilmann Gneiting, Fadoua Balabdaoui, and Adrian E Raftery. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 69(2):243–268, 2007. 
review_title: The paper doesn't differentiate itself well from existing work and misses key comparisons. 

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a variational-approximation of Neural Network uncertainty dependent on the 'bayesian interpretation' of dropout. Using this approach uncertainty measurements are provided using 'variance propagation.' Validation is provided on synthetic data, and MNIST/FMNIST.

Although a genuine effort is made towards deriving uncertainty through 'variance propagation,' I think the proposed approach is rather ad-hoc. The proposed approach relies on a 'bayesian interpretation,' of dropout, and is of limited applicability requiring hard coded rules for propagating through each layer. Currently, rules are derived for a small set of neural network layers.

The chief issue with this work is the 'generality' of the work. The promised approach here is applicable to specifically neural networks trained with SGD using dropout. Thus although the proposed approach claims to evaluate uncertainty, it is rather estimating the uncertainty introduced by dropout using a variational approximation, and assuming this is the same as uncertainty. I find this assertion unfair to other works which attempt to estimate various types of uncertainties (e.g. epistemic) in a principled manner.

The proposed approach depends on the hyperparameter \rho, which requires additional tuning to allow variance propagation through affine (e.g., fully connected) layers. This additional hyperparameter gives some doubt whether the proposed approach is something which can be easily integrated into existing models. 

In Table 1, how do the authors address that their approach (which is meant to be an approximation to MC) appears to outperform MC? Shouldn't an approximation perform worse?

Although the proposed approach outperforms MC sampling in detecting out of distribution sampling, I think this benchmark is rather unfair. I'd like to see comparisons to other works which quantify uncertainty. The issue is that this only compares against model uncertainty assuming the 'bayesian interpretation' of MC is valid. I would strongly like to view comparisons to other works in quantifying network uncertainty.

On the whole, the lack of generality of the proposed approach, as well as the limited scope of 'variational approximation of model uncertainty assuming bayesian interpretation of dropout,' causes me to be highly skeptical of this work.


review_title: Limited scope and novelty; broader implications unclear.

Meta-review: This paper proposes an approach to estimating uncertainty in deep neural network models that avoids the need to make multiple forward passes through a network or through multiple individual models in a posterior ensemble. In terms of strengths, this is an important and timely topic that is of significant interest. The paper is clearly written for the most part. In terms of weaknesses, the significance of the work is low. As the reviewers note, there are multiple questions around the experimental evaluation that remain unresolved following the author feedback and discussion. In particular, the authors do not compare to baseline MCMC methods like HMC/SGHMC that can yield gold standard estimates of posterior predictive uncertainty. While not feasible for large-scale models, MCMC methods provides crucial sanity checks for uncertainty estimation on small-scale (e.g., MNIST-scale) models. Posterior distillation methods like Bayesian Dark Knowledge are also not considered in the evaluation and should be compared to where the distillation computation is feasible. There are also foundational technical correctness issues with respect to uncertainty quantification due to the fact that the paper is approximating the measure of uncertainty produced by MC Dropout, which itself only approximates the true Bayesian posterior predictive distribution under additional assumptions. This makes empirical comparissons to MCMC methods all the more important. Following the discussion, the reviewers agree that the paper is not yet ready for publication.
Decision: Reject