Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: In this paper, the authors propose a method for computing the aleatoric and epistemic uncertainty estimates using a kernel-based estimator for the conditional label distribution. The method is flexible as it can use any embeddings generated from a NN model. The authors also do empirical evaluation on two different tasks of OOD detection and classification with rejection to show the utility of their method. 
strengths_and_weaknesses: Strengths 

- The overall method is very flexible in the sense that it can utilize any existing neural network model. 

- The experiments provided by the authors cover a large set of baselines and models. 


Weakness

- The paper is not very easy to follow, especially around methodology. I've added more comments on this in the next section. 

- Although the authors have used deep ensembles, the authors haven't used the traditional model uncertainty as a metric for epistemic 
uncertainty (also known as mutual information) in the comparison. The authors haven't used their distilled version either for comparison.
questions: - Lines 24-26 "MaxProb represents only aleatoric uncertainty..." I think this is incorrect. MaxProb in my point of view captures the total uncertainty, including aleatoric and epistemic. I'd be happy to hear the counterpoint.

- Line 33 "Such methods are very perspective.." Seems like a typo

- Lines 42-43 "...some methods require computing covariance...". Please mention the approaches and cite them 

- Eq 2, what's u exactly and how is this thing practically computed? 

- Line 129-130  "standard deviation of data label" - what's this std dev exactly? 

- There's additional work [1, 2] that looks at distilling uncertainty estimates as well as looks at classification with rejection using Bayesian decision theory framework. This should be discussed in the related work, and potentially the distillation method should be compared in the experimental section [2, 3]. 

- It'll be nice to see what the additional runtime looks like for generating the approximation to conditional label distribution and how does it scale wrt models/datasets.

References

[1] Vadera, Meet, Brian Jalaian, and Benjamin Marlin. "Generalized bayesian posterior expectation distillation for deep neural networks." UAI (2020).

[2] Vadera, Meet, Soumya Ghosh, Kenney Ng, and Benjamin M. Marlin. "Post-hoc loss-calibration for Bayesian neural networks." In UAI (2021).

[3] Malinin, Andrey, Bruno Mlodozeniec, and Mark Gales. "Ensemble distribution distillation." ICLR (2020).
limitations: I've highlighted the shortcomings in the previous sections.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes Nonparametric Uncertainty Quantification (NUQ), a simple approach to estimate predictive uncertainty using kernel density estimators. The model uses kernel density estimators for prediction (Eq 1) and squared kernel magnitude (Eq 2) for predictive variance. The method adopts a simple decomposition between aleatoric and epistemic uncertainty.  A consistency proof for the method's ability in approximating the optimal learning-with-abstention risk is also provided. Experiments are conducted on large-scale vision benchmark (ImageNet) and several standard NLP tasks, where NUQ illustrated consistent improvement over other methods. 
strengths_and_weaknesses: ** Strength **
  * A simple and novel approach to deep uncertainty quantification grounded on kernel density theory. 
  * The method comes with optimal guarantees towards minimizing certain learning-under-abstention loss.
  * The method was extensively evaluated on vision and language benchmarks, showing good performance.

** Weakness **
I only have some minor suggestions about technical clarity

 (1) Some of the technical details are missing or buried in the SM. For example, on line 129, what are $p(x)$ and $\mathbf{u}$? Please give brief explanation in the main text and then point to appropriate section / table in the SM.
 (2) In the experiments, how are $\sigma^2$ and $p(x)$ defined and estimated? 
 (3) In the experiments, how are hyper-parameters such as $\sigma^2$ and $h$ estimated? Specifically, what objective are you minimizing? Also in practice, when the test data / OOD data are not available, how should practitioner set the hyper-parameters? The paper can benefit a section (maybe in SM) that collects all hyperparameters of the method and discuss practical strategies in setting them. 
questions: Please see **Weakness** (1)-(3)
limitations: The method's technical clarity can use some improvement. Please see **Weakness**
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper is about epistemic/aleatoric uncertainty quantification in single deterministic neural networks using density estimation in the feature space to measure epistemic uncertainty. In particular this work uses a non-parametric density estimation method (kernel density estimation) which makes less assumptions than other methods like DUQ and similar.

I believe the contributions are:
- The use of non-parametric kernel density estimation to model the density for epistemic uncertainty quantification in single deterministic neural networks.
- Experimental improvements in out of distribution detection on CIFAR100 and ImageNet benchmarks, and text classification with rejection.
- A good statistical theory argumentation on how the method is built, and proofs for the consistency of the proposed method in the case of classification with rejection.

strengths_and_weaknesses: Strengths
- The proposed method, first perform density estimation in feature space, and then uses statistical properties to derive estimates of aleatoric and epistemic uncertainty, overall these make sense, but I have not went into the statistical details in depth.
- There are good improvements in out of distribution performance as measured by ROC AUC both in CIFAR100 and ImageNet, and also in text classification and good comparisons for classification with rejection.
- I like that a non-parametric density estimation is used to estimate epistemic uncertainty in feature space, it is a simple change, previous method usually made implicit assumptions of Gaussianity or other shapes in the feature space, and it makes sense that performance improves when taking less assumptions. I think the authors implicitly make this argument in this paper.
- The paper seems to be correctly evaluated, there are OOD detection results on ImageNet (hard) and CIFAR100 (also hard), and for classification with rejection on text datasets on a variety of datasets.

Weaknesses
- I think the paper is hard to follow and read, in particular there are a lot of statistical details and some of the machine learning details can be lost in there. There is excessive use of alternative notation, for example $\eta$ is used for the conditional density p(Y = 1 | X = x), I think it is better to use probabilistic notation directly and not introduce additional greek letters as makes the notation hard to follow. Same can be said for risk definitions.
- Overall about readability, I think the paper should be rewritten to put the terms that are usable for implementation from a machine learning perspective (like binary and multi-class uncertainty components), and leave derivations and proofs for the supplementary. The audience of NeurIPS is largely machine learning people, and the paper notation and arguments could be simplified so it is easier to follow for this audience. The statistical details are important but after reading this paper, if I want to implement this method, to me it is not clear what parts I should use or directly how this method works conceptually.

~~- The results in the paper only present out of distribution detection results as main result, and classification with rejection on text datasets. There are additional results on the supplementary but I believe these are minor. Overall I think there are missed opportunities for evaluation, for example, calibration error of epistemic uncertainty (Figure 2 could be a start point).~~

~~- The authors argue that one advantage of this method is that it can disentangle aleatoric and epistemic uncertainty, but this is only vaguely evaluated (only in a toy example in the supplementary), and not in the main paper. I think this is a missed opportunity as this is an important topic that is often overlooked, and there are opportunities for comparison with Kendall and Gall. 2017, which is a well known baseline for uncertainty disentanglement. I recommend that the authors also consider how to evaluate the aleatoric and epistemic uncertainty separately and show that they behave as expected.~~

~~- Since density estimation is computationally expensive, the authors resort to an approximation of the kernel density estimate with nearest neighbors in feature space. I think this is fine, but there is no evaluation of approximation quality or ablations on how to select an approximation for nearest neighbors and kernel density estimates, the authors just use HNSW for this purpose.~~

Minor issues
- I think a diagram showing how the method works and its training/inference process would be an easy way to understand the overall proposed approach.

~~- I think the paper is missing references to the text datasets (CoLA, SST-2, and MRPC) and other datasets used in the text classification experiments.~~

~~- I am not sure what score is used for OOD detection, my guess is that it is the epistemic uncertainty $U_e$, and this should be explicit in the evaluation section.~~
questions: - In Line 127, how do choose the "properly chosen bandwidth" to make sure that the difference of eta's converges to the desired distribution? I think there has not been much discussion in the paper about how to select the bandwidth for the KDE part of this method.
- Have you evaluated other possible approximations for the kernel density estimate?
- Would you consider rewriting the paper to include binary/multi-class details in the paper, and leave derivations in the supplementary?

limitations:  As I mentioned in weaknesses, the paper does not mention that it uses an approximation for the kernel density estimate, and this could be a limitation that could be addressed in the future. 

There are no other limitations mentioned in the paper about the proposed method, even as the checklist says the paper does discuss limitations (I do not see where, I might be wrong).

Overall I see no negative societal impact to discuss.
review_title: null

Meta-review: metareview: Decision: Accept

This paper propose a method for computing the epistemic/aleatoric uncertainties using a kernel-based estimator. Empirical evaluations on OOD detection & "classification with rejection" demonstrate the benefits of the proposed approach.

Reviewers commended that the proposed method is flexible, easy to be adapted to any neural network architecture. However, in initial reviews, concerns were raised about (a) clarity, (b) uncertainty evaluations in experiments (e.g., calibration, epistemic vs aleatoric uncertainty).

In author feedback, authors provided more explanations and some additional experimental results, which addressed many of the reviewers' concerns. Still the question regarding calibration metrics is there. 

After reviewer-AC discussions, it is concluded that we can accept this paper to recommend its contribution in OOD detection research domains. Still I'd suggest in revision the paper will benefit from an edit to improve clarity, and I'd encourage the authors to consider including calibration metrics. As a side note, for the "classification with rejection" task, I'd suggest the authors to include a baseline regarding selective classification, e.g., the method of https://arxiv.org/abs/1705.08500.
confidence: Certain
Decision: Accept