Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a post-hoc KDE-based method (NUQ) for single-pass uncertainty estimation. It reports improvements in robustness to data perturbation and OOD detection over deterministic networks on MNIST-based benchmarks, comparable performance to DDU (Mukhoti et al., 2021) on CIFAR100-based out-of-distribution detection and improvements on ImageNet.
main_review: **Relevance**: The paper proposes a deterministic uncertainty estimation method for deep learning. Various other methods in that direction have been proposed recently (SNGP, DDU, DUE, etc) to avoid computing an expensive model average over multiple forward passes. The submission is therefore timely and relevant.

**Novelty**: My understanding is that the paper mostly combines results from the literature (the upper bound on the Bayes risk, the variance of the excess risk). This is of course a worthwhile contribution, but arguably at the lower end of the novelty spectrum. Prior work is clearly referenced.

**Clarity**: The high-level flow of the paper is clear and easy to follow. In some places I feel like the text could be expanded a bit to not only re-state results from prior works, but give the reader some intuition forwhy they are true and how they are derived. More importantly, I find the paper fairly unclear regarding the method that is ultimately implemented, section 2 ends fairly abruptly for my taste. I would suggest adding an algorithm or pseudo-code box to make it clear what exactly is implemented. This would also help to substantiate the statement that the paper presents a “fast and scalable method”.

**Empirical evaluation**: Overall the choice of problems and metrics is standard. My biggest concern is that the comparison to reasonable baselines is fairly lackluster. The small scale experiments (toy-classification and MNIST) only compare to deterministic networks which are well-known to not provide useful uncertainty estimates. So I find it hard to conclude much from these. An equivalent of Fig. 1 should be provided for DDU and the MNIST benchmarks from sections 4.2.1 and 4.2.2 should show results for ensembles and DDU (although I could imagine that the former won’t work very well out-of-the-box since MNIST is too simple, perhaps using something like FashionMNIST would be more interesting).

The ImageNet results look promising. It is a bit surprising, however, that NUQ is not able to outperform DDU on CIFAR100 (my impression is that usually if anything performance gains don’t translate from the smaller CIFAR datasets to ImageNet). The argument that the number of classes is too high for DDU is quite speculative. It seems to me like including another baseline method such as [SNGP](https://arxiv.org/abs/2006.10108) or [DUE](https://arxiv.org/abs/2102.11409) would solidify these results.

**Other notes/questions**:
* Just under eq (1): “In our experiments, we consider one dimensional kernel function”, but (2) has a power to dimensionality -- what exactly is happening here? Is the KDE fit jointly on all features or independent across dimensions?
* The statement in the 3rd point at the end of the introduction that recent work “suffered from a lack of a principled method to quantify the uncertainty” is extremely vague and should be substantiated (what do you mean by principled and how is prior work not principled?) or removed.
* “Thus, it might be beneficial to tune the bandwidth to optimize the quality of OOD detection if some set of OOD points is available at the training time”. I would ideally like to see a small ablation results on this, e.g. with the SVHN validation and test sets. It would be useful to know how much performance could be gained by tuning the bandwidth on a known OOD dataset.
* The captions of Fig 2 and 3 are too close to each other and should be spaced evenly with Fig 4.
* Broken link to supplement at the beginning of 4.2.1.
* The references are formatted extremely inconsistently. Please don’t just copy-paste bibtex entries from google scholar, but make capitalization, initials/full first names, venue names etc consistent.

summary_of_the_review: Overall, this is an interesting, but not groundbreaking paper with a few issues around clarity and the empirical evaluation, so I am currently **leaning towards a reject**.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a nonparametric uncertainty quantification method for deep neural networks. Using a kernel-based estimator of the conditional density (i.e., the predictive distribution $p(y|x)$) in the feature space of a pre-trained neural network, the epistemic and the aleatoric uncertainty can be obtained separately from an upper bound of the excess risk and Bayes risk. For practical feasibility, several subproblems are approached with existing methods, such as kernel density estimation/GMM for $p(x)$, Improved Sheather-Jones algorithm for kernel bandwidth selection, HMSW for fast nearest neighbors retrieval. Finally, good OOD detection performance is demonstrated on several image datasets including MNIST, CIFAR, and ImageNet.
main_review: 

Reasons for score:

Although the idea is interesting, my main concern is about the significance of the proposed method. See my detailed comments below.

---

Pros:

- The paper uses the excess risk and Bayes risk to quantify the epistemic uncertainty and the aleatoric uncertainty, which is interesting.
- Practical method for large datasets is described, which is very important for nonparametric methods.
- Substantial experimental results are provided on several different image datasets.

---

Concerns:

My first concern is about the significance of the proposed method. 

1) The method is actually only nonparametric in the feature space of a neural network. Therefore, the feature extracting process, which involves a large number of parameters, is extremely important. Although the uncertainty is defined rigorously, it is not clear whether it is still informative in the feature space of an arbitrary neural network. Previous work (Van Amersfoort et al., 2020;2021, Liu et al., 2020) shows that a crucial difficulty of uncertainty estimation in deterministic neural networks is feature collapse, i.e., the network maps both data samples and OOD samples to a very narrow region in the feature space, resulting in unexpected/uncalibrated predictive uncertainty. It seems the proposed method can also suffer from the feature collapse issue, while no theoretical analysis can be found in the paper (except that the issue is alleviated using spectral normalization, which is also proposed in previous work).

2) The authors claim that the proposed method complements recent works that suffer from a lack of a principled method to quantify the uncertainty. However, there are several other methods that can produce principled uncertainty. For example, being Bayesian in the last layer (Kristiadi et al., 2020), which is equivalent to using a Bayesian linear/logistic regression in the last layer, or using a Gaussian process in the feature space (Liu et al., 2020), which is also kernel-based and nonparametric. Given the fact that these methods can also distinguish epistemic uncertainty and aleatoric uncertainty, what are the advantages of the proposed method compared with these methods?

3) I also have a concern about efficiency. The authors described several existing methods as *still require setting additional hyperparameters and/or increased computational cost*. However, the proposed method requires extra GMM modeling, nearest neighbor search, and kernel bandwidth selection. I suggest the authors provide a detailed discussion and justification about the efficiency.


My second concern is about the experiments. As mentioned above, I think more methods should be compared, such as Kristiadi et al., 2020, DUQ and SNGP. The author has explained that the compared methods do not require significant modifications to model architecture and training procedures. However, given the fact that SN regularization is also adopted, the training procedure in the proposed method is actually modified and therefore the mentioned methods seem like fair competitors.

---

Reference:
Agustinus Kristiadi et al., 2020, Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks



summary_of_the_review: In summary, I raised several concerns about the theoretical and empirical significance of the proposed method, which currently prevents me from rating this paper higher. I will consider raising my evaluation if the authors address my concerns or point out my misunderstandings.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 
This paper proposes NUQ, a kernel-based estimation of epistemic uncertainty regarding a kernel-based probabilistic classifier applied on the input or on the top hidden layer on on output logits of a pre-trained deep net classifier.

Experiments on OOD vs in-distribution binary classification based on the uncertainty score suggest that NUQ works better than several other methods.

main_review: Using the logits as input to the kernel machinery may be problematic, as highlighted for example by Postels et al 2021 (arXiv:2107.00649), because these logits will be invariant to many directions in input space (especially if the input is high-dimensional, like an image, compared to the number of classes). It means that the output logits may be invariant to a large volume of changes that may occur when going from in-distribution to out-of-distribution (in fact to volume of output-sensitive directions may be of null measure), making ANY method unable to detect changes in those directions. Hence I would advise against using the output logits.

It is not clear to what extent NUQ is fundamentally different from DUQ (yes, using a sum of kernels rather than a mixture of Gaussians, depending on how many Gaussians are fitted, of course) and the results seem to be very similar between the two in table 1.

For MNIST vs greyscale SVHN, I see in fig 4 that NUQ does the same as the stupid entropy, which is interesting to observe and suggests that in this case it is easy to detect these OOD examples. I also notice that none of the more 'serious' competitive baselines (like Ensembles and DDU) were included in the evaluation. Why?

How were the other methods optimized? have you optimized the number of networks in the Ensemble, for example? How about MC-dropout: how are its hyper-parameters optimized? It does not look like the authors did a serious job of optimizing the hyper-parameters of the other methods.

How accurate is the proposed method in terms of estimating the TRUE epistemic uncertainty? In cases where the generative distribution is known, this should be easy to compute. And how would that compare with PROPERLY OPTIMIZED alternatives?

summary_of_the_review: 
The proposed approach is suspicious on theoretical grounds, in that it relies essentially on the ratio of some estimation of training uncertainty and density in representation-space in order to estimate epistemic uncertainty.

The experimental results are also worrisome because it does not look like appropriate efforts were made to tune the hyper-parameters of baseline methods.

Finally, there is no attempt to actually compare the estimated epistemic uncertainty with the ground truth epistemic uncertainty.
review_title: null

Meta-review: The paper proposes a simple approach to quantify uncertainty in "deterministic" neural networks, not unlike the works of SNGP, DDU, and DUE, where one only performs one forward pass rather than in an ensemble or Monte Carlo sample. In particular, they propose a kernel-based method on a network's logits to estimate uncertainty, obtaining data and model uncertainty estimates separately using a bound on Bayes risk.

While I agree with the relevance of the problem, there's a shared concern among reviewers across both technical novelty and experimental validation---particularly compared to prior work that can be difficult to understand the key distinguishing factor. I recommend the authors use the reviewers' feedback to enhance their preprint should they aim to submit to a later venue.
Decision: Reject