Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper shows that the MSE of a deep network trained to match fixed random network is a conservative estimate of uncertainty in expectation over many such network pairs. An experiment compares this previously proposed method to other approaches for uncertainty estimation on CIFAR 10. 

Strengths:
- Obtaining uncertainty estimates for predictions of deep neural networks is an important and open research question.
- Proposition 1 is an interesting result, although the paper does not seem to discuss its significance and implications enough.

Weaknesses:
- Proposition 1 is described as "our uncertainty estimates are never too small". However, as the name of the proposition suggests, it seems to only hold in expectation over models trained, which is a quite different statement.
- Proposition 2 seems to simply state that a small network can be distilled into a large network. Maybe I missed part of the reasoning here? Otherwise, it should probably not be highlighted as a major contribution.
- The experimental evaluation is very limited, training only on CIFAR 10. While the experiments add little value to the community, this may be acceptable for a mostly theoretical paper.

Clarity:
- The paper was difficult to read and unclear in explanations. It could help to define notation upfront instead of introducing shorthands on the way.
- In Figure 3, the Y axis limits should be fixed across seen and unseen histograms for the same method. The current presentation is a bit misleading here, as the presented method seems to have moved the most mass under this chart scaling.

Comments:
- As found in prior work cited in the submission, the method tends to perform well with just one network pair. This raises the question whether the contribution of the paper that holds in expectation over many pairs and the empirical success of the approach are connected.
- The marginal posterior variance \sigma^2(x_\star) appears in various forms with hat, tilde, and different subscripts. It maybe worth assigning different letters to these to avoid confusion.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Overview:
This paper introduces a new method for uncertainty estimation which utilizes randomly initialized networks. Essentially, instead of training a single predictor that outputs means and uncertainty estimates together, authors propose to have two separate models: one that outputs means, and one that outputs uncertainties. The later one consists of two networks: a randomly initialized “prior” which is fixed and is not trained, and a “predictor”, which is then trained to predict the output of the randomly initialized “prior” applied to the training samples.  
Authors show that under some reasonable assumptions the resulting estimates are conservative and concentrated (i.e. bounded and converge to zero with more data).

Writing quality: 
Overall, the paper is relatively well-written, although it might be at times hard to follow, especially for someone who is not familiar with the original work that used randomized prior functions (Burda’18, Osband ‘18, ‘19). 

Evaluation:
The method is experimentally evaluated on a task of out-of-distribution detection on CIFAR+SVHN, and seems to perform on-par or better than the baselines (including “standard” deep ensembles and dropout networks). In addition, there are experiments that demonstrate that the model is performing relatively well in terms of calibration (whether the model predictive behaviour makes sense as the model confidence changes).

Decision:
I find the core idea behind the paper quite interesting, however, as indicated by authors themselves, it has already been studied in a slightly different context (RL, works by Burda et. al, Osband et. al). That said, authors do provide additional insides for the supervised settings, and also analyse theoretically the behaviour of uncertainty estimates. 
Overall, I cannot say I am fully convinced that the paper should be accepted as is (also see questions below), but generally I am positive about this work, and hence the final score: “weak accept”.

Additional comments / questions:
(somewhat minor) p1: “While deep ensembles …, where the individual ensembles are trained on different data“ - here and related text, it should probably be “individual models” / “individual networks”. Generally, I am not convinced that these are strong arguments against deep ensembles.

(minor) p2-p3: “2. Preliminaries” - I am not sure if this section adds much to the understanding, it would seem more natural to spend more time explaining the intuitions behind the net

(kind of major) p3. “prior” - The explanation of why using a randomly initialized network makes sense is not very strict. I kind of get the general idea, but it is not clear to me why not use something less expensive, e.g. just random projections, and why do we actually need a full network. Intuitively it seems quite strange to waste a lot of capacity to fit to essentially fit a set of random weights: is it something that allows the network to avoid easily learning the “random prior”? And, more generally, can this also be considered as a “trick” to de-correlate individual predictors? I believe these points should be discussed in more detail.

<update>
I would like to thank authors for verbose response and the revised version: it is a bit more clear. 
I stand by my original rating.
</update>







review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This work introduces a simple technique to obtain uncertainty estimates for deep neural networks. This is achieved by having a set of random networks (i.e. neural networks where their parameters are randomly initialized) and then computing an uncertainty value based on the difference in the predictions between those random networks and networks that are trained to mimic them on a finite collection of points. The authors further show that this method results into uncertainties that are conservative, meaning that they are higher than the uncertainty of a hypothetical posterior, and concentrate, i.e. they converge towards zero when we get more and more data. The authors further draw connections to ensemble methods and discuss how such a method can be effectively realized in practice. They then evaluate their approach on an out-of-distribution detection task, they measure the calibration of their uncertainty estimates and finally perform a small ablation study for their concentration result. 

This work is interesting as it seems to provide a simple way to obtain reasonable uncertainty estimates. For this reason it can potentially serve as a strong baseline for this field. The theoretical considerations also help in providing some guarantees about such an approach. Having said that, in my opinion the writing could use some more work in order to make things more clear as some critical experimental details and baselines are missing and thus do not make the method as convincing. Furthermore, I also believe that some clarifications on the theoretical aspects of this work, will help in boosting its quality. More specifically:

- How exactly do you apply your method on the classification scenario? Do you select an arbitrary hidden layer of the classification model for the prior and predictor network architectures or the output logits / softmax probabilities?  In appendix A you mention the architecture but not precisely how it is employed. I believe this can be an important piece of information in order to decipher the importance of e.g. the output dimensionality on the uncertainty quality, as higher dimensional outputs might be harder to approximate thus could induce a larger squared error and hence uncertainty.
- What is the average training error of the predictor networks for the out-of-distribution task and subsampling ablation task, i.e. how far away from concentration were the priors? 
- An effect that I found weird is the following: what happens for the out-of-distribution examples when the predictor networks can perfectly predict the prior network outputs? Wouldn’t that then imply that the uncertainty would be zero for any input (even an out-of-distribution one), as the prior network and predictor network always agree? One could imagine that for e.g. simple priors and with sufficiently dense sampling of the domain of the function this can happen in practice.
- For the conservatism you show that your uncertainty estimate is higher, on average, than the posterior variance when you sample points from the model itself. In a sense this guarantee translates to the actual data when the prior is “correct”. How do those conservatism guarantees translate to the case when there is model misspecification, i.e. when the prior is not correct? Perhaps a small toy example would be informative.
- For the predictor networks as described in figure 2; do you train both the green and red parts of the network or only the red parts and keep the green part fixed to the values you used for the prior f? (This helps in understanding how easy / difficult is the task of the predictor network).
- What is the accuracy on the actual in-distribution prediction task for the RP and baselines? What did “B” correspond to for the dropout networks? Was it the number of dropout samples you averaged over to get the final predictive?
- How sensitive are the results on the actual initialization strategy of the prior network? It would be good to see e.g. some form of performance / init variance curve in order to decipher the sensitivity. 

Other comments
- It is worth pointing out that [1] showed that Monte-Carlo dropout performs approximate MAP inference, which seems more plausible than the approximate Bayesian inference perspective of [2].
- In the introduction you argue that Bayesian neural networks rely on procedures different from standard supervised learning and thus most ML pipelines are not optimized for them in practice. Could you elaborate a bit about this statement? Variationally trained BNNs with e.g. the reparametrization trick [3, 4] are straightforward since you can just use backpropagation to update their (variational) parameters.
- What is the x-axis for Figure 3 for the baselines? (I take it that for RP it is the \hat{sigma}^2(x)).
- I believe that a comparison against a simple variationally trained BNN would make the results more convincing.

Misc
- Second page, “Figure 1, top two plota” -> “Figure 1, top two plots”
- Third page, “[…] introduced in equation 2 denotes the posterior covariance [….]” -> “[…] introduced in equation 2 denotes the posterior variance […]“
- Fifth page, “[…] this makes it is reasonable for W large enough […]” -> “[…] this makes it reasonable for W large enough […]”
- Sixth page, “Corollary 1 and proposition 2”; where is corollary 1? Do you mean Proposition 1?
- Seventh page, “[…] inspired by, an builds on, […]” -> “inspired by, and builds on, […]”
- Ninth page “montonicity” -> “monotonicity”

Overall, I tend to accept this work, although, depending on the author rebuttal and other discussions, I am willing to change my rating accordingly.

[1] Eric Nalisnick, José Miguel Hernández-Lobato, Padhraic Smyth, Dropout as a Structured Shrinkage Prior, 2019
[2] Yarin Gal, Zoubin Ghahramani, Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning, 2016
[3] Diederik P. Kingma, Max Welling, Auto-Encoding Variational Bayes, 2014
[4] Danilo Jimenez Rezende, Shakir Mohamed, Daan Wierstra, Stochastic Backpropagation and Approximate Inference in Deep Generative Models, 2014
review_title: Official Blind Review #1

Meta-review: The paper provides theoretical justification for a previously proposed method for uncertainty estimation based on sampling from a prior distribution (Osband et al., Burda et al.).

The reviewers initially raised concerns about significance, clarity and experimental evaluation, but the author rebuttal addressed most of these concerns.

In the end, all the reviewers agreed that the paper deserves to be accepted.
Decision: Accept (Poster)