Nonparametric Uncertainty Quantification for Single Deterministic Neural Network
Authors: Nikita Yurevich Kotelevskii
Aleksandr Artemenkov
Kirill Fedyanin
Fedor Noskov
Alexander Fishkov
Artem Shelmanov
Artem Vazhentsev
Aleksandr Petiushko
Maxim Panov
Venue: NeurIPS 2022 Accept (2022)
Keywords: Uncertainty Quantification
Nonparametric Methods
Deep Learning
Machine Learning

Abstract:   This paper proposes a fast and scalable method for uncertainty quantification of machine learning models' predictions. First, we show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. Importantly, the approach allows to disentangle explicitly \textit{aleatoric} and \textit{epistemic} uncertainties. The resulting method works directly in the feature space. However, one can apply it to any neural network by considering an embedding of the data induced by the network. We demonstrate the strong performance of the method in uncertainty estimation tasks on text classification problems and a variety of real-world image datasets, such as MNIST, SVHN, CIFAR-100 and several versions of ImageNet.

Nonparametric Uncertainty Quantification for Single Deterministic Neural Network
Nikita Kotelevskiil, Aleksandr Artemenkov's Kirill Fedyanin', Fedor Noskoy Alexander Fishkov!. Artem Shelmanoy Artem Vazhentsev Aleksandr Petiushko 5.6 and Maxim Panov
Skolkovo Institute of Science and Technology, Moscow, Russia "Technology Innovation Institute, Abu Dhabi, UAE
AIRI, Moscow, Russia
'HSE University, Moscow, Russia
Lomonosov Moscow State University, Moscow, Russia Nuro, Inc.
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Abstract
This paper proposes a fast and scalable method for uncertainty quantification of ma- chine learning models' predictions. First, show the principled way measure the uncertainty predictions for classifier based on Nadaraya- Watson's 's nonpara- metric estimate of the conditional label distribution. Importantly, the proposed approach allows disentangle explicitly aleatoric and epistemic uncertainties. The resulting method works directly to any neural network by considering an embedding of the daia induced by the network. We demonstrate the strong performance of the method uncertainty estimation tasks on text classification problems and variely of real-world image datasets, such as MNIST, SVHN, CIFAR- 100 and several versions of ImageNet.
feature space. However, one can apply
1 Introduction
In many machine learning applications, it crucial complement model predictions with uncertainty scores that reflect the degree of trust in these predictions. The total uncertainty of a prediction surms from two uncertainty types, arising from different sources: aleatoric and epistemic [TE] [R] The former reflects the irreducible noise ambiguity the data due class overlap, while the latter is related to the lack of knowledge about model parameters. and can be reduced by expanding the training dataset. Disentangling epistemic uncertainty can help identify out-of-distribution (OOD) data or to spot instances important for annotation during active learning [22]. The areas with high aleatoric uncertainty might contain some incorrectly labeled instances. If we quantify both types of uncertainty well. can effectively abstain from predictions unreliable areas and address a decision to human ex expert [[]], which is imporiant in safe-critical fields, such as medicine [[33]
autonomous driving
20 and finance [6]
There is universally accepted uncertainty measure, and diverse, often heuristic treatments used in practice. For this purpose, one simply could use maximum softmax probabilities of deep neural network (NN). However, MaxProb represents only alcatoric uncertainty, and the resulting measure is notorious to overconfident data areas the model did not see during training [59]. Methods hased ensembling 37 or Bayesian techniques 21 can capture both types of uncertainty and yield more
Currespondence to: Kotel Kot.aaerkkkkiiikkkoeoeec....... and Maw.im.aaaootttti..... Panovotii.
36th Conference Neural Information Processing Systems (NeurIPS 2022).

---

Raw data (200 samples)
MaxProb
(Total)
Raw data (2000 samples)
MaxProb
DDU
NUQ (Total)
Figure 1: The first column on the left shows the raw data from the Two Moons dataset. The second column illustrates neural network prediction uncertainty obtained using MaxProb. The last two columns illustrate the results of DDU and the of aleatoric and epistemic uncertainties obtained using NUQ. The lighter color indicates higher uncertainty.
reliable uncertainty estimates (UEs) However, along with that, they introduce large computational overhead and might require big modifications to a model architecture and training procedure.
Recently, series of UE methods based single deterministic neural network model has been developed 41 69 46 70]. The main idea behind these methods is to leverage geometrical prox- imity of instances in a vector space of neural network hidden representations that capture semantic relationships between instances in the input space. Such methods are very promising since they are computationally efficient and usually do not require big changes network architectures and training procedures, which gives them versatility.
Many of these methods rely the assumption that (conditionally the class) training instances distributed normally in the latent space of a trained neural network (70 MI However, this assumption does not always hold. For example, a border between classes might be more complex, as well as the border between the in-domain and the out-of-domain region. This happens when training is accomplished in a low-resource regime with small amounts labeled instances. Another problem is that some methods hard to train and the model performance substantially deteriorate without proper hyperparameter tuning. Finally, some methods require computing the covariance matrix of training data [41] This operation might be computationally unstable if the training dataset is not large enough.
In this work, we propose new principled approach to UE that overcomes these limitations and demonstrates very robust performance both in low-resource and in high-resource regimes. We suggest to look at the pointwise Bayes risk (probability of the wrong prediction) the natural measure model prediction uncertainty at a particular data point. Then we consider the Nadaraya- Watson estimator of the conditional label distribution. Its asymptotic Gaussian approximation allows deriving uncertainty estimate based on the upper bound for the risk. The resulting Nonparametric Uncertainty Quantification (NUQ) method is amenable for uncertainty disentanglement (into aleatoric and epistemic) and is implemented in scalable manner, which allows it to be used on large datasets such ImageNet.
illustrate the strong points of the proposed method, we suggest to look at toy example Figure I Here, we compare NUQ with MaxProb and DDU 555, one of the strong baselines in deterministic uncertainty estimation aimed at epistemic uncertainty. MaxProb gives high values only in- ne-between classes while being very confident far from the data. We also see that for larger data set of 2000 points NUQ and DDU show similar results. However, when the neural network model is trained only on 200 labeled instances. DDU scores become very rough. It cannot spot unreliable area in-between classes, while the NUQ-based total uncertainty measure Importantly, the presented total uncertainty for NUQ
measure of aleatoric uncertainty
does precisely with high uncertainty can also disentangled into alcatoric and epistemic ones.
'Terms uncertainty quantification and uncertainty estimation often used in literature interchangeably.

---

Below, we give summary of contributions of this paper.
We develop new and theoretically grounded method for nonparametric uncertainty quantifi- cation (see Section which is (a) applicable to any deterministic neural network model, (b) has efficient implementation. (c) allows to disentangle aleatoric and epistemic uncertainty, (d) outperforms other techniques on image and text classification tasks (see Section (6)
2. We formally prove the consistency of the NUQ-based estimator when applied the problem of classification with the reject option, see Section
3. We conduct a vast empirical investigation of NUQ and other UE techniques on image and text classification tasks that supports our claims, see Section b
2 Nonparametric Uncertainty Quantification
2.1 Classification under Covariate Shift
In this section and below, for the sake clarity, we provide derivations for the binary case. We derive generalization to multi-class setting in Supplementary Material (SM), Section A
Let consider the standard binary classification setup (X,Y) Y) E R x {0. 1) with (X.Y) P. where P a joint distribution objects and the corresponding labels. We assume that we observe the dataset 'D = (x,, 1s) of i..dd points from P = Ptrain- Here, and are realizations of random variables X and Y.
The classical problem statisties and machine learning using the dataset D, find rule 9. which approximates the optimal one:
min a(g()) + YY.
Here, 9: Rd + f0, 1 is any classifier. and the probability of wrong classification P(g(X) # Y) is usually called risk. The rule g is given by the Bayes optimal classifier:
1, n)x)
g (x)
0,
n)())
where rxx) = P x) which the conditional probability of Y given X under the distribution P.
In this work, we consider situation when the distribution of the test samples Ptest is different from the one for the training dataset Purain, i.c. Pust * Firain. Obviously, the rule g obtained for P Perain might no longer be optimal if the aim is to minimize the error on the test data Puest (x(X) YY.
In order formulate meaningful estimation problem, some additional assumptions needed. We assume that the conditional label distribution vx() = P(Y | X = x) is the same under both Praaiin and Prest. This assumption has two important consequences:
The entire difference between PPtrain and PPest is due to the difference between marginal distributions of X: (Xrain(X) and Prest()). The situation when Prest (X) * Pursin is known covariate shift.
2. The rule g" is still valid, i.e., optimal under Puess-
However, while the classifier g" is still optimal under covariate shift, its approximation. might be arbitrarily bad. The reason for that is that we cannot expect ğ to approximate g well the areas where we have few objects from the training set or do not have them at all. Thus, some special treatment of the covariate shift required.
2.2 Pointwise Risk and Its Estimation
We consider a classification nule (XX = (x(X) constructed based on the dataset D. Let us start from defining the pointwise risk of prediction:
R(x)= P(g(X) Y X x))).

---

where + P
under the assumptions above. The value R(x) independent of the covariate distribution Prest (X) and essentially allows to define a meaningful target estimation, which is based solely on the quantities known for the training distribution.
Pirainn
Let us note that the total risk value R(x) admits the following decomposition:
R(x) R(x) R² (x),
where R'(x) (X) + x) is the Bayes risk and R(x) P()(X) / X
x) - X = is an excess risk. Here, R' (x) corresponds to alcatoric uncertainty as it completely depends on the data distribution. The excess risk R(x) directly measures imperfectness of the model 9 and, thus, can be seen measure of epistemic uncertainty.
To proceed, we first assume that the classifier has the standard form:
1, f((x)
g(x)
0, f(()
where fi(x) p(Y 1 x) estimate of the conditional density ry(x).
For such an estimate, we can upper bound the excess risk via the following classical inequality (15)
(x) (X) * Y X x) P(g" (X) Y X x) S 2 f)) r7(x),
It allows us to obtain an upper bound for the total risk:
R(x)= S [(x) (x) 211(x)) (x)],
where Rd (xxx) = (x), } in the case of binary classification. While this upper bound
still depends on the unknown quantity nxx), will see in the next section that [(x) allows for efficient approx imation under mild assumptions.
2.3
Nonparametrie Uncertainty Quantification
2.3.1 Kernel Density Estimate and Its Asymptotic Distribution
To obtain an estimate of [(x) and, consequently, bound the risk, we need to consider some particular type of estimator fj. In this work, we choose the classical kernel-based Nadaraya- Watson estimator of the conditional label distribution it allows for simple description of its asymptotic properties.
Let denote by K₁: Rd + the multi-dimensional kernel function with bandwidth h. Typically, we consider a multi multi-dimensional Gaussian kernel, but other choices are also possible.
The conditional probability estimate expressed as (y, is either or 1):
n(x) = ΣN [y: 1] (x x;)
(1)
Σ K₁ - x;)
The difference between f(x)) nf(x) for properly chosen bandwidth h converges in distribution follows (see, c.g. (63)):
-(x) - + xx((, Ö ( p(x)
(2)
where the number of data points training set, p(x) the marginal distribution of covariates (see details in SM, Section C.3 and (x) is the standard deviation of the data label at point x. For binary classification, o² rxx n(x)). The constant C = [[K (u) where u is an integration variable, depends only the choice of the kernel K₁ and could be computed in closed form for popular kernels. See details SM, Section C.4]
Now, we are equipped with an estimate of the distribution for f)x) - 1)(x). Let us denote by r(x)
the standard deviation of a Gaussian from the equation 2
Ö x²(
72(x) N p(x)
In the following sections, we first show how to use the obtained property for uncertainty estimation, and then, we show how it can be computed.

---

2.3.2 Total, Aleatoric, and Epistemic Uncertainty and Their Estimates
In this work, we suggest particular uncertainty quantification procedure inspired by the derivation above, which we call Nonparametric Uncertainty Quantification (NUQ). More specifically, we suggest to consider the following measure of the total uncertainty:
2
(x) min(n(()) 1 -=)) + 2
This measure obtained by considering an asymptotic approximation of the expected value of the total risk upper bound: Exf(x) min (x). n7xx} + 2E (x() n7xx]
in view of and estimate upper bounds the average error of estimation at the point x and thus indeed can be used as the measure of total uncertainty.
that |R|g| = ( A for the zero-mean normal variable Ç. The resulting
We also write the corresponding measures of aleatoric and epistemic uncertainties:
U (x) = min (x)), 1 nx)) (x = 2/ = -(x).
Finally, the data-driven uncertainty estimates Úx) (x) and Ú,(x) can be obtained via plug-in (²(x) Č
using estimates fx(), (x), p(x) and, consequently, +²(x) N p(x)
We should note that despite being based on asymptotic approximation, the resulting formulas for uncertainties are very natural and make sense for finile sample size (for example, epistemic uncertainty is proportional o² /p(x)). We also should note that even known non-asymptotic decompositions for risk of the NW-estimator still contain same o² (x))p(xx) component, see Proposition in 77.. That means that the proposed estimate well captures the general uncertainty trend.
Efficient computation. We note that computation of the nonparametric estimate involves sum over the whole available data. This could be intractable practice when we are working with large datasets. However. the typical kernel Kú, quickly approaches zero with the increase of the norm of the argument: ||x x; ||- Thus, over all elements in the dataset, we consider the contribution only several nearest neighbors (see SM, Section C.2 for details). It requires a fast algorithm for finding the nearest neighbors. For this purpose, we use the approach of (5I) based on Hierarchical Navigable Small World graphs (HNSW), It provides a fast, scalable, and easy-to-use solution to the computation of nearest neighbors.
approximation of the kernel estimate: instead of the sum
Application to NN and Comparison with Existing Methods. be applied to NN in the postprocessing fashion, i.e. trained NN model. One may wonder about the difference between NUQ and other embedding based methods such as, for example, DUQ [69] or DDU [55]. The difference is twofold: (i) NUQ is based on the rigorous derivation of total, aleatoric, and epistemic uncertainties, while other methods usually consider more heuristic treatment and do not allow for uncertainty disentanglement; (ii) NUQ considers better quality in a small training data regime or for complicated data; see experimental evaluation Section
The resulting NUQ method can fit it top of the embeddings of the
flexible estimator of density in the embedding space that allows achieve
3 Detailed Algorithmic Description of NUQ Approach
In this section, we provide a detailed algorithmic description of the NUQ approach. On the training stage, NUQ uses training data embeddings obtained from the pre-trane pre-trained neural network and builds Bayesian classifier based on conditional label probabilities estimated in a non-parametric way:
Li=l 1 = = KA(x - x;). c= I...,. C.
p(Y c X x)
ΣN Kn(x-x,)
The bandwidth h is tuned via cross-validation optimizing the classification accuracy on the training data (see SM, Section C.1.

---

Algorithm 1 NUQ inference algorithm.
Input: Training set {(x, y4 (i) inference point x, bandwidth
Output: Prediction ((x) and uncertainty estimate U+(x)
(xi, 1
K nearest neighbors of among {x
A(Y X =x) Les Kn(Xi, xx] md
(2(xx -i(Y = x Σ._, (xi -x)
p( =c x=|xxx)
((x) - argmax p(Y = x
C
f(x) either KDE: N =k=1 Kn(Xi, x) GMM
+²(x) max (x)
x(x) C, where Ö f[[Kn [[Kn(u)]'dd (u))
Ú,(x) + min(1 - p( = X = x)} + 2/ fx(
On the inference stage, the new object passed through the neural network and corresponding embedding is computed. Then, this embedding is used to compute the uncertainty estimates employ- ing the estimated bandwidth h, Algorithm [ that details the computation of all the necessary intermediate quantities as well as the resulting uncertainty estimates based on the embeddingss x provided by the neural network. Algorithm I allso takes into account the usage of nearest neighbours speed up the computation of kernel-based estimates. Finally, the computation of estimate of the embeddings density p(x) can be done either via the kernel density estimate (KDE) or via the Gaussian Mixture Model (GMM). We study the relative benefits of these approaches in Section F.4
4 Consistency of NUQ-based Classification with a Reject Option
Above, we obtained uncertainty estimates that characterize the classical risk of a prediction. However, they are also helpful to solve the formal problem of classification with the reject option. In this problem, for any imput we can choose either we perform prediction reject Following [8), we assume that in the case of prediction we pay binary price depending whether the prediction was correct not, while in the case of the rejection we pay the constant price À (0, 1). For this task, the risk function is
Rx(x)= ((x)](((x)) 0] Al(a(x) 1},
where (x) is an indicator of the rejection.
The minimizer of (x)x)) is given by the optimal Bayes classifier g" (x) and the abstention function
0, R* X
a ax(x)
1, R* A.
To approximate a² (x), we utilize hypothesis testing:
Ho: R' (x) À vs. 1I: R (x) < X.
We choose the confidence level B > 0 and consider the statistic Ús(x) = min(f((x) (__x)xx)]--)) + 21-sf(x) where 21-3 is the combines the plug-in estimate the Bayes risk min ((xx, 1 - vf()) } and the term =1 accounting for the confidence of estimation. The resulting abstention rule is given by:
3 quantile of the standard normal distribution. The statistic sx
0, Úg(x) < X
(x(x)
1, (,(x) >>.
Finally, if we consider the pair of kernel classifier (xx and (x(xx) then, we can prove the consistency result for the corresponding risk (x) under standard assumptions on nonparametric densitics, see
SM, Section for details.
Theorem 4.1. Suppose that assumptions D.I D.3 hold and p(x) > the bandwidth h + and d no os N tends to infinity. Then, for any B <. 1/2:
Rx)x(x)
R2(x
0.

---

This result shows the validity of the NUQ-based abstention procedure. Interesting future work is to obtain precise convergence for the method. It should be possible based the finite sample bounds provided in SM, Section D We also experimentally illustrate the benefits of the proposed estimator in SM, Section D.1
5 Related Work
The notion of uncertainly naturally appears in Bayesian statisties (25). and, thus, Bayesian methods are often used for uncertainty quantification. The exact Bayesian inference is computationally intractable, and approximations are used. Two popular ideas are the Markov Chain Monte Carlo sampling (MCMC; (57] and the Variational Inference (VI; 44). MCMC has theoretical guarantees to be asymplotically unbiased, but has high computational cost. VI-based approaches [64] 16 60 35
are more scalable, they are biased and at least double the number of parameters. That is why some alternatives considered, such as the Bayesian treatment of Monte-Carbe dropout [21].
Deep Ensemble [37 is usually considered as quite strong yet expensive approach. A series of
papers developed ways of approximating the distribution obtained using an ensemble of models by a single probabilistic model [49] (5, 66]. These methods require changing the training procedure and need more parameters to train.
Recently, a series of uncertainty quantification approaches for a single deterministic neural network model was proposed. In DUQ [99) an RBF layer is added to the network with custom training procedure adjust the centroid points (in the emhedding space). The downside of the method is its inability to distinguish aleatoric and epistemic uncertainty. Another approach to capture epistemic uncertainty was proposed in DDU [55]. It uses Gaussian mixture model to estimate the density of objects in the embedding space of trained neural network. The density values are then used as a
confidence measure. SNGP [46] and DUE (70) are similar but use a Gaussian process as the final layer, requiring estimating covariance with the use of inducing points RFF expansion.
There is a wide range of papers discussing classification with the reject option. Most likely, the problem was firstly studied by Chow in p R Moreover, in [ he introduced a risk function used across this paper. Herbei et al. [31] studied an optimal procedure for this risk and provided plug-in rule. In the following works, empirical risk minimization among a class hypotheses (see (i
or other types of risk (see [13 19 42 were investigated. Besides, a number of practical works have
been presented, see, for example,
27 24 56).
6 Experiments
We conduct series of experiments on image and text classification datasets. In each experiment, (I) we train a parametric model neural network, which call a base model; (2) fit NUQ on the training data using the embeddings obtained from the base model. We use logits as extracted features, if not explicitly stated otherwise. However, other options are also possible; see Section E.4
Following SNGP and DDU,
spectral normalization to train the base model to achieve bi- Lipschitz property and avoid the feature collapse and non-smoothness. However, NUQ works sufficiently good even without this regularization (see Table in Section [.4).. In all experiments on OOD detection, we use (x) as measure of uncertainty. An additional illustrative experiment on detecting actual aleatoric and epistemic uncertainties is presented in SM, Section
6.1 How NUQ Affects Model Predictions?
One may ask whether the nonparametric classification method used in NUQ, trained on some embedding from the base model, has any relation to the original neural network. To reassure the reader, we provide an argument that it well approximates the predictions of the base model and NUQ-based uncertainty estimates can be used for the base model as well. Specifically, the agreement between predictions obtained from the Bayes classifier based on kernel estimate (i.c. the one used in NUQ) and base models' predictions. This metric formally can be defined as agreement(p. p) a Ei= [arg max, p(y = x;) arg max, p(y = j x:)] For CIFAR- 100 (see experiments with this dataset in Section 6.2.I) I. this metric gives us the agreement of 0.975, which shows that the approach is accurate. Additionally, we computed the aleatoric uncertainty for
compute

---

OOD dalaset
MaxProb
Eatropy
Dropout
Ensemblee
TTA
Energy*
DD0
SNGP
DDU
NUQ
SVHN
39.7
71.62.5
82.900.9
62.01.7
88.746.3
86.2-7.
89.6616
89.721.6
LSUN
81.5-20
83.0221
76845.1
86.520.8
85.0227
82.7001
90.46.7
83.7-8.6
92.140.6
92.320.6
Smooth
76,523.
77.8552
63.323.8
83.7E122
73.24103
71.524.5
91.1384
609ET23
97.13.11
96.833.
Table I: OOD detection for 100 in-distribution dataset with the ResNet-50 neural network. The top two results are shown bold and underline correspondingly. Evaluation is done for three models trained with different seeds to estimate the standard deviation. Methods requiring single pass over the data to compute uncertainty estimates are marked with *
OOD dataset MaxProb
Entropy
TTA
Energy*
Ensemble
DDU
DUQ
SNGP*
NUQ
ImageNet-R
80.4
83.6
85.8
78.34
84.4
80.1
73.3
85.0
99.5
ImageNet-O
28.2
29.1
30.5
60.0
51.9
74.1
71.4
75.8
82.4
Table 2:
ROC-AUC score for ImageNet out-ol tolooooddstibbttion detection tasks for different methods.
Methods requiring single pass over the data to compute uncertainty estimates are marked with *
all test objects and found the average percentile of the objects with disagreement is 94.94 + 4.75. Thus, disagreement appears for high uncertainty points as expected.
6.2 Image Classification
The main experiments with image classification are conducted on CIFAR-100 (36] and ImageNet [12] However, several additional experiments with other image classification datasets such as MNIST and SVHN can be found in SM, Sections
FT and F22
We compare NUQ with popular UE methods, which do not require significant modifications to model architectures and training procedures. More specifically, we consider Maximum probability (MaxProb) of softmax response of NN, entropy of the predictive distribution, the Monte-Care (MC) dropout [2I], an ensemble models trained with different random seeds (deep ensemble), the Test-Time Augmentation (TTA; [8]). DDU 551. SNGP 46.. DUQ [69] and an energy-based approach 47]
For Monte-Carbe dropout, ensembles, and TTA, we first compute average predicted class probabilities and then compute their entropy (see the ablation study in SM, Section (.8). More details can be found in SM, Section[B] For deep ensembles, fixed the number of models to 5. Additional experiments, where we changed the number of models, are presented SM, Section F55
6.2.1 CIFAR-100
In this experiment, we test UE methods on the detection binary classification (OOD/not-OOD) using only the uncertainty score. Following the setup from the recent works (69 70 65), we use SVHN, LSUN [73], and Smooth (28) as OOD datasets. The reported metric is ROC-AUC.
detection task. We treat the OOD
As base model, train Re from scratch on the CIFAR-100 dataset. NUQ applied to the features from the penultimate layer of the model, and the density estimate is given by GMM, as it provides the best results (see the results for other choices of hyperparameters in Section F.4).
The results are presented in Table We can clearly see that NUQ and DDU show close results while outperforming the competitors with significant margin.
6.2.2 ImageNet
To demonstrate the applicability of NUQ to large-scale data, we evaluate it in the OOD detection task on ImageNet [12]. As OOD data, we use the ImageNet-O (30 and ImageNet-R 2 datasets. ImageNet-O consists of images from classes not found in the standard ImageNet datasct. ImageNet-R contains different artistic renditions of ImageNet classes.
In contrast to the previous experiment, we found that for NUQ, it is more beneficial to use KDE density estimator p(x), rather than GMM. Importantly, took us approximately 5 minutes receive uncertainties over all ImageNet datasets with a CPU, i.c, our NUQ implementation readily applicable to large-scale data (see more details in SM, Section F.7)

---

about ERIIT crabs!
a) Low uncertainty
b) Medium uncertainty
c) High uncertainty
Figure 2: Typical OOD images from ImageNet-R ordered from low uncertainty (bottom 10%) (a) to high uncertainty (Lop 90%) (c) by the NUQ (x scores. We can clearly see that low-uncertainty images resemble real-world objects presented in vanilla ImageNet.
The results are summarized in Table We see that for ImageNet-O, many methods show good OOD detection quality, but NUQ achieves an almost perfect result. For ImageNet-R, simple approaches completely fail while DDU, SNGP, and DUQ perform well, and NUQ shows the best result with a large margin.
Note that unlike in the CIFAR- 100 experiment, for ImageNet, NUQ significantly outperforms DDU. We conjecture that GMM struggles to approximate density here as an embedding structure is much more complicated for ImageNet compared to CIFAR- 100 (see some visualizations in Section F.3 NUQ beneficial in this case KDE much more flexible than GMM and provides better result.
Additionally, we looked some typical samples from ImageNet-R with low, moderate, and high levels of uncertainty as assigned by NUQ, see Figure Here, low, medium, and high uncertainties correspond to 10, 50, and 90% quantiles of the epistemic uncertainty distribution for images from the ImageNet-R dataset. We observe that uncertainty values correspond well to intuitive degree of image complexity compared to the original ImageNet data. Some additional ImageNet experiments are presented in SM, Section F.
6.3 Text Classification
Experiments textual data are performed in low-resource settings, where we train models small subsamples of original datasets. This regime can be challenging for many UE methods, while NUQ is naturally adapted to it. We compare NUQ to the best performing methods image classification: DDU and deep ensemble, and to the standard baselines: MC dropout and MaxProb.
The methods are evaluated on two tasks: OOD detection and classification with reject option. Clas- sification with rejection experiments are conducted on SST-2 (67). MRPC [17]. and CoLA (7]. The evaluation metric RCC-AUC (18). Experiments with OOD detection are conducted ROSTD 23
and CLINC (39]. which originally contain instances marked as OOD, and a benchmark composed from SST-2 (in-domain). 20 News Groups
(38). TREC-10 10 44 32 WMT-16 (5). Amazon [52
(sports and outdoors categories), MNLI (72), and RTE 11 26 R where SST-2 used as an
in-domain dataset, while the rest as OOD datasets. The final score is averaged across all OOD datasets. Evaluation metric is ROC-AUC as in the image classification task. The dataset statistics are presented in SM, Scction G
We use a pre-trained ELECTRA model with 110 million parameters. The features for NUQ and DDU are taken from the penultimate classification layer. The details of the model, hyperparameter optimization, and UE methods are presented in SM, Section
6.3.1 Classification with Reject Option
Figure presents the results for classification with rejection. For the MRPC dataset, we can note that NUQ, SNGP, and MC dropout show similar results to the MaxProb baseline. DDU stands out, demonstrating substantially worse performance settings with small amount of training data.
A similar dynamics for DDU can be noted on the CoLA dataset, where works substantially worse than the baseline when only 1% of the training data is available. SNGP in this experiment manages reach other methods only when 10% of training data is unlocked. On contrary, NUQ is always on par or better than the baseline oulperforming all other computationally efficient methods and has similar performance as computationally expensive MC dropout.

---

328
DOU
204
Dropect
MAC JANn)
Mul Janr
190
RF-0 Soanr)
MRPC
b) CoLA
c) SST-2
Figure 3: RCC-AUC| of classification with rejection depending fraction of unlocked training data.
Dropost
Diacout
Eoo
NuO
NUG round)
Dropout
D0U
335
siboe pww.mriepeeen
a) ROSTD
b) SST-2
CLINC
Figure 4: ROC-AUC1 of OOD detection depending on the fraction of unlocked training data.
On SST-2, NUQ works similar to SNGP and DDU, substantially outperforming the MaxProb baseline, and MC dropout in the extremely low resource setting. Starting from 1.5%, all methods work similarly with small advantage over the baseline.
Overall, we can conclude that in the settings with a small amount of training data, NUQ can be the best choice for estimating uncertainty for the classification with a reject option: it works similar to other methods when there is much training data and does not deteriorate in the low-resource regime, demonstrating much better results than others. SNGP is able to approach NUQ on SST-2. However, its performance is not stable, which is illustrated by poor results on CoLA. DDU sometimes fails to oulperform the baseline with small amount of training data, which might be due its reliance on the assumption that training data has a Gaussian distribution, which does not hold in this setting.
6.3.2 Out of Distribution Detection
Figure presents the results of OOD detection. We see that NUQ confidently outperforms MC dropout on all datasets and ouiperforms DDU ROSTD and CLINC. This is especially notable for extremely low-resource settings. SNGP has some advantage over NUQ on ROSTD. but it substantially falls behind on CLINC. Moreover, unlike SNGP, NUQ always outperforms the MaxProb baseline, therefore, it might be better choice for OOD detection in low-resource regimes. Finally, for ROSTD and CLINC datasets, DDU works poorly, sometimes failing to oulperform the MaxProb baseline, which stems from the incorrect assumption about the Gaussian distribution of training data.
7 Conclusions
This work proposes NUQ, a new principled uncertainty estimation method that applies to wide range of neural network models. It does not require retraining the model and acts postprocessing step working in the embedding space induced by the neural network. NUQ significantly outperforms the competing approaches with only the recently proposed DDU method (55) showing comparable results. Importantly, the most practical example of OOD detection for ImageNet data, NUQ shows the best results with significant margin. NUQ is also superior to DDU and other methods on text classification datasets in both OOD detection and classification with rejection. The code reproduce the experiments is available online at lhttts:: h/git:://tttiie /ttpthb.ttttttg///////// com/ com/stat. mml/UU
We hope that our work opens a new perspective on uncertainty quantification methods for deterministic neural networks. We also believe that NUQ is suitable for in-depth theoretical investigation, which we defer future work.
A Acknowledgements. The research was supported by the Russian Science Foundation grant 20-71- 10135
10

---

References
[1] Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, and Danilo Giampiccolo. The second pascal recognising textual entailment challenge. Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, 01 2006.
[2] Peter I.. Bartlett and Marten H. Wegkamp. Classification with Reject Option using Hinge Loss. Journal of Machine Learning Research, 9(59):1833-1800 2008
[3] Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. The fifth PASCAL recognizing textual entailment challenge. In Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009. NIST, 2009.
[4] David M Blei, Alp Kucukelbir, al. Variational Inference: A Review for Statisticians. Journal of the American Statistical Association, 112(518):859-87) 2017.
[5] Ondfej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Mattco Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. Findings of the 2016 conference machine translation. In Proceedings of the First Conference Machine Translation: Volume 2, Shared Task Papers, pages 131-198, Berlin, Germany, August 2016. Association for Computational Linguistics.
[6] Axel Brando, Jose A Rodríguez-Serrann Serrano, et al. Uncertainty Modelling in Deep Networks: Forecasting Short and Noisy Series. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 920- Springer, 2018.
[7] Gaëlle Chagny and Angelina Roche. Adaptive estimation in the functional nonparametric regression model. Journal of Multivariate Analysis, 146:105-118, 2016.
[8] C. Chow. On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory, 16(1):41-46, January 1970. Conference Name: IEEE Transactions on Information Theory.
[9] C. K. Chow. An optimum character recognition system using decision functions. IRE Transac- tions on Electronic Computers, EC-6(4):277-254, December 1957. Conference Name: IRE Transactions on Electronic Computers.
[10] Corinna Cortes, Giulia DeSalvo, and M. Mohri. Learning with Rejection. In ALT, 2016.
[11] Ido Dagan, Oren Glickman, and Bernardo Magnini. The PASCAL recognising textual entailment challenge. In Joaquin Quiñonero Candela, Ido Dagan, Bernardo Magnini, and Florence d' Alché- Buc, editors, Machine Learning Challenges, Evaluating Predictive Uncertainty. Visual Object Classification and Recognizing Textual Entailment, First PASCAL Machine Learning Challenges Workshop, MI.CW 2005, Southampton, UK, April 11-13, 2005, Revised Selected Papers, volume 3944 of Lecture Notes in Computer Science, pages 177-190. Springer, 2005.
[12] Jia Deng, Wei Dong, et Imagenet: A Large-Scale Hicrarchical Image Database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255. leee, 2009.
[13] Christophe Denis and Mohamed Hebiri. Consistency plug-in confidence sets for classification in semi-supervised learning. Journal of Nonparametric Statistics, 32:42 - 72, 2015.
[14] Armen Der Kiureghian and Ove Ditlevsen. Aleatory or Epistemic? Does It Matter? Structural Safety, 115)):05-1122 2009.
[15] Lue Devroye, Lászłó Györfi, and Gáhor Lugosi. A Probabilistic Theory of Pattern Recognition, volume 31. Springer Science Business Media, 2013.
[16] Laurent Dinh, Jascha Sohl-Dickstein. al. Density Estimation using Real NVP. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.

---

[17] William B. Dolan and Chris Brockett. Automatically constructing corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005.
[18] Ran El- Yaniv and Yair Wiener. On the foundations of noise-free selective classification. J. Mach. Learn. Res., 11:1605-1641, 2010.
[19] Ran El- El-yaniv and Yair Wiener. On the Foundations of Noise-free Selective Classification. Journal of Machine Learning Research, 11533:1605-164) 2010.
[20] Angelos Filos, Panagiotis Tigkas, et al. Can Autonomous Vehicles Identify, Recover from, and Adapt to Distribution Shifts? In International Conference on Machine Learning, pages 3145-3153. PMLR, 2020.
[21] Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Maria-Florina Balcan and Kilian Q Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings. pages 1050-1050 1059. JMLR. org. 2016.
[22] Yarin Gal, Riashat Islam, et al. Deep Bayesian Active Learning with Image Data. In Interna- tional Conference on Machine Learning, pages 1183-1192. PMLR, 2017.
[23] Varun Gangal, Abhinav Arora, Arash Einolghozati, and Sonal Gupta. Likelihood ratios and generative classifiers for unsupervised out-of-domain detection task oriented dialog. The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7764-7771. AAAI Press, 2020.
[24] Yonatan Geifman and Ran EI- Yaniv. SelectiveNet: A Deep Neural Network with an Integrated Reject Option. In Proceedings of the 36th International Conference Machine Learning, pages 2151-2159 PMLR, May 2019. ISSN: 2640-3498.
[25] Andrew Gelman, John B Carlin, al. Bayesian Data Analysis. CRC press, 2013.
[26] Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. The third PASCAL recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1-9, Prague, June 2007. Association for Computational Linguistics.
[27] Yves Grandvalet, Alain Rakotomamonjy, Joseph Keshet, and Stéphane Canu. Support Vector Machines with Reject Option. In Advances in Neural Information Processing Systems, volume 21. Curran Associates, Inc., 2009.
[28] Matthias Hein, Maksym Andriushchenko, et al. Why Relu Networks Yield High-Confidene Predictions Far away from the Training Data and How to Mitigate the Problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 41 41-50, 2019.
[29] Dan Hendrycks, Steven Basart, et al. The Many Faces of Robustness: Critical Analysis of -iitt--Disttibttionn Generalization. ICCV, 2021.
[30] Dan Hendrycks, Kevin Zhao, et al. Natural Adversarial Examples. CVPR, 2021.
[31] Radu Herbei and Marten H Wegkamp. Classification with reject option. The Canadian Journal of Statistics / La Revue Canadienne de Statistique, 34(4):709-72) 2006.
[32] Eduard Hovy, Laurie Gerber, Ulf Hermjakob, Chin-Yeu Lin, and Deepak Ravichandran. Toward semantics- based answer pinpointing. In Proceedings of the First International Conference on Human Language Technology Research, 2001.
12

---

[33] Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5574-5584, 2017.
[34] Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014.
[35] Ivan Kobyzev, Simon Prince, et al. Normalizing Flows: An Introduction and Review of Current Methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[36] Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto, 2009.
[37] Balaji Lakshminarayanan, A. Pritzel, et al. Simple and Scalable Predictive Uncertainty Estima- tion using Deep Ensembles. In NIPS, 2017.
[38] Ken Lang. Newsweeder: Learning filter netnews. In Armand Prieditis and Stuart Russell, editors, Machine Learning, Proceedings of the Twelfth International Conference on Machine Learning, Tahoe City, California, USA, July 9-12, 1995, pages 331-339. Morgan Kaufmann, 1995.
[39] Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. An evaluation dataset for intent classification and out-of-scope prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing JEMNLP-IJCNLA pages 1311-1316 Hong Kong, China, November 2019. Association for Computational Linguistics.
[40] Yann LeCun, Corinna Cortes, et al. MNIST Handwrilten Digit Database. ATT Labs [Online]. Available: http://yann.lecum wwtp://www.....eedbbeeeecccc
2, 2010.
[41] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting -iitof-distribution samples and adversarial attacks. In Advances Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, volume 31, pages 7167-7177, 2018.
[42] JING LEI. Classification with confidence. Biometrika, 101(4):755-66,, 2014. Publisher: [Oxford University Press, Biometrika Trust].
[43] Jesse Levinson, Jake Askeland, et al. Towards Fully Autonomous Driving: Systems and Algorithms. In 2011 IEEE Intelligent Vehicles Symposium (IV), pages 163-168. IEEE, 2011.
[44] Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Computational Linguistics, 2002.
[45] JG Liao, Yujun Wu, and Yong Lin. Improving sheather and jones' bandwidth selector for difficult densities in kernel density estimation. Journal of nonparametric statistics, 22(1):105-114, 2010.
[46] Jeremiah Z. Liu, Zi Lin, et al. Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020.
[47] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Fnergy-based out-of-distribttion detection. Advances in Neural Information Processing Systems, 33:21464-221475 2020.
[48] Alexander Lyzhov, Yuliya Molchanova, Arsenii Ashukha, Dmitry Molchanov, and Dmitry Vetrov. Greedy policy search: A simple bascline for learnable test-time augmentation. Conference on Uncertainty in Artificial Intelligence, pages 1308-1317. PMLR, 2020.
13

---

[49] Andrey Malinin and Mark J. F. Gales. Predictive Uncertainty Estimation via Prior Networks. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 7047-7058, 2018.
[50] Andrey Malinin, Brumo Mlodozeniee, et al. Ensemble Distribution Distillation. In Inter- national Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
[51] Yu A Malkov and Dmitry A Yashunin. Efficient and Robust Approximate Nearest Neighbor Search using Hierarchical Navigable Small World Graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, (244):824-836, 2018.
[52] Julian J. McAuley and Jure Leskovec. Hidden factors and hidden topics: understanding rating dimensions with review text. Qiang Yang, Irwin King, Qing Li, Pearl Pu, and George Karypis, editors, Seventh ACM Conference on Recommender Systems, RecSys '13 Hong Kong, China, October 12-16, 2013, pages 165-172. ACM, 2013.
[53] Riccardo Miotto, Li Li, al. Deep Patient: Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records. Scientific Reports, 6(1):1-10, 2016.
[54] Takeru Miyato, Toshiki Kataoka, et al. Spectral Normalization for Generative Adversarial Networks. In 6th International Conference Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 2018, Conference Track Proceedings. OpenReview.net, 2018.
[55] Jishnu Mukhoti, Andreas Kirsch, et Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Alcatoric Uncertainty. CoRR, abs/2102. 2021.
[56] Malik Sajjad Ahmed Nadeem, Jean-Daniel Zucker, and Blaise Hanczar. Curves (ARCs) for Comparing Classification Methods with a Reject Option. In Proceedings of the third International Workshop on Machine Learning in Systems Biology. pages 65-81. PMLR, March 2009. ISSN: 1938-7228.
[57] Radford M Neal et al. MCMC using Hamiltonian Dynamics. Handbook of Markov Chain Monte Carlo, 2(11) 2011.
[58] Yuval Netzer, Tao Wang. et al. Reading Digits in Natural Images with Unsupervised Feature Learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
[59] Anh M Nguyen. J. Yosinski, et al. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images. 2015 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 427-436. 2015.
[60] George Papamakarios, Eric Nalisnick, et al. Normalizing Flows for Probabilistic Modeling and Inference. Journal of Machine Learning Research, 22(57):1-64 2021.
[61] George Papamakarios, Eric T Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for prohabilistic modeling and inference. Mach. Learn. Res., 22(57): 2021.
[62] Adam Paszke, Sam Gross, et al. PyTorch: An Imperative Style, Perfforereem Deep Learning Library. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Bun Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019,
[63] James L. Powell. Notes On Nonparametric Regression Estimation. Manuscript, 2010.
[64] Danilo Rezende and Shakir Mohamed. Variational Inference with Normalizing Flows. In International Conference on Machine Learning, pages 1530-1538. PMLR, 2015.
[65] Chandramouli Shama Sastry and Sageev Oore. Detecting Out-of-Distribution Examples with Gram Matrices. In ICML, 2020.
14

---

[66] Murat Sensoy, Lance M. Kaplan, et al. Evidential Deep Learning to Quantify Classification Uncertainty. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Gameett, editors, Advances in Neural Information Processing Sys- tems 31: Annual Conference Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 3183-3193, 2018
[67] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning. Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics.
[68] Berwin A Turlach. Bandwidth selection in kernel density estimation: A review. In CORE and Institut de Statistique. Citeseer, 1993.
[69] Joost Van Amersfoort, Lewis Smith, et al. Uncertainty Estimation using a Single Deep Deter- ministic Neural Network. In International Conference on Machine Learning, pages 9690-9700. PMLR, 2020.
[70] Joost van Amersfoort, Lewis Smith, et al. Improving Deterministic Uncertainty Estimation in Deep Learning for Classification and Regression. CoRR, abs/2102. 1409, 2021.
[71] Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625-641, 2019.
[72] Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pages 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
[73] Fisher Yu, Yinda Zhang, et al. LSUN: Construction of a Large-Scale Image Dataset using Deep Learning with Humans in the Loop. CoRR, abs/1506.03366 033666, 2015,
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information how to answer these questions. For each question, change the default [TODO] to [Yes], [No], or [N/A]. You are strongly encouraged to include a justification to your answer, either by referencing the appropriate section of your paper providing brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section ...
Did you include the license to the code and datasets? [No] The code and the data are proprietary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No] Our proposed method is widely applicable, almost any modern neural network can be equipped with this mechanism. Potential social impact will highly depend on the specific domain and task.
15

---

(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of theoretical results? [Yes]
(b) Did you include complete proof's of all theoretical results? [Yes] Proofs our results are in Supplementary Material, see Section D
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] We provide the link to our code in Conclusion.
(b) Did you specify all the training details (e-g., data splits, hyperparameters, how they were chosen)? [Yes] We discuss this in Supplementary Material, Sections []and G (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, cloud provider)? [No] It not included in the main paper. but will be addressed Supplementary material.
4. If you using existing assets (e.g., code, data, models) curating/releasng
assets... (a) If your work uses existing assets, did you cite the creators? [Yes] We provide references the datasets in the
[ section. For our comparisons with existing approaches have used the author's ' implementations that were provided in the respective original papers that we cite.
(b) Did you mention the license of the assets? [No] We use only open source code and datasets.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes] We provide the link to our code in Conclusion.
(d) Did you discuss whether and how consent was obtained from people whose data you 're using/curating? [N/A] All the assets used are free for research purposes.
(c) Did you discuss whether the data you using/curating contains personally identifiable information offensive content? [No] We use publicly available datasets benchmark our results and show only numeric values and abstract plots. To the best of our knowledge have exposed any personal information offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "ohkHXi8UJK", "reviewer": "Reviewer_RSJH", "summary": "This paper is about epistemic/aleatoric uncertainty quantification in single deterministic neural networks using density estimation in the feature space to measure epistemic uncertainty. In particular this work uses a non-parametric density estimation method (kernel density estimation) which makes less assumptions than other methods like DUQ and similar.\n\nI believe the contributions are:\n- The use of non-parametric kernel density estimation to model the density for epistemic uncertainty quantification in single deterministic neural networks.\n- Experimental improvements in out of distribution detection on CIFAR100 and ImageNet benchmarks, and text classification with rejection.\n- A good statistical theory argumentation on how the method is built, and proofs for the consistency of the proposed method in the case of classification with rejection.\n", "questions": "- In Line 127, how do choose the \"properly chosen bandwidth\" to make sure that the difference of eta's converges to the desired distribution? I think there has not been much discussion in the paper about how to select the bandwidth for the KDE part of this method.\n- Have you evaluated other possible approximations for the kernel density estimate?\n- Would you consider rewriting the paper to include binary/multi-class details in the paper, and leave derivations in the supplementary?\n", "limitations": " As I mentioned in weaknesses, the paper does not mention that it uses an approximation for the kernel density estimate, and this could be a limitation that could be addressed in the future. \n\nThere are no other limitations mentioned in the paper about the proposed method, even as the checklist says the paper does discuss limitations (I do not see where, I might be wrong).\n\nOverall I see no negative societal impact to discuss.\n", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The proposed method, first perform density estimation in feature space, and then uses statistical properties to derive estimates of aleatoric and epistemic uncertainty, overall these make sense, but I have not went into the statistical details in depth.\n- There are good improvements in out of distribution performance as measured by ROC AUC both in CIFAR100 and ImageNet, and also in text classification and good comparisons for classification with rejection.\n- I like that a non-parametric density estimation is used to estimate epistemic uncertainty in feature space, it is a simple change, previous method usually made implicit assumptions of Gaussianity or other shapes in the feature space, and it makes sense that performance improves when taking less assumptions. I think the authors implicitly make this argument in this paper.\n- The paper seems to be correctly evaluated, there are OOD detection results on ImageNet (hard) and CIFAR100 (also hard), and for classification with rejection on text datasets on a variety of datasets.\n\nWeaknesses\n- I think the paper is hard to follow and read, in particular there are a lot of statistical details and some of the machine learning details can be lost in there. There is excessive use of alternative notation, for example $\\eta$ is used for the conditional density p(Y = 1 | X = x), I think it is better to use probabilistic notation directly and not introduce additional greek letters as makes the notation hard to follow. Same can be said for risk definitions.\n- Overall about readability, I think the paper should be rewritten to put the terms that are usable for implementation from a machine learning perspective (like binary and multi-class uncertainty components), and leave derivations and proofs for the supplementary. The audience of NeurIPS is largely machine learning people, and the paper notation and arguments could be simplified so it is easier to follow for this audience. The statistical details are important but after reading this paper, if I want to implement this method, to me it is not clear what parts I should use or directly how this method works conceptually.\n\n~~- The results in the paper only present out of distribution detection results as main result, and classification with rejection on text datasets. There are additional results on the supplementary but I believe these are minor. Overall I think there are missed opportunities for evaluation, for example, calibration error of epistemic uncertainty (Figure 2 could be a start point).~~\n\n~~- The authors argue that one advantage of this method is that it can disentangle aleatoric and epistemic uncertainty, but this is only vaguely evaluated (only in a toy example in the supplementary), and not in the main paper. I think this is a missed opportunity as this is an important topic that is often overlooked, and there are opportunities for comparison with Kendall and Gall. 2017, which is a well known baseline for uncertainty disentanglement. I recommend that the authors also consider how to evaluate the aleatoric and epistemic uncertainty separately and show that they behave as expected.~~\n\n~~- Since density estimation is computationally expensive, the authors resort to an approximation of the kernel density estimate with nearest neighbors in feature space. I think this is fine, but there is no evaluation of approximation quality or ablations on how to select an approximation for nearest neighbors and kernel density estimates, the authors just use HNSW for this purpose.~~\n\nMinor issues\n- I think a diagram showing how the method works and its training/inference process would be an easy way to understand the overall proposed approach.\n\n~~- I think the paper is missing references to the text datasets (CoLA, SST-2, and MRPC) and other datasets used in the text classification experiments.~~\n\n~~- I am not sure what score is used for OOD detection, my guess is that it is the epistemic uncertainty $U_e$, and this should be explicit in the evaluation section.~~", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "j2GTQmeWeEU", "reviewer": "Reviewer_TAXi", "summary": "In this paper, the authors propose a method for computing the aleatoric and epistemic uncertainty estimates using a kernel-based estimator for the conditional label distribution. The method is flexible as it can use any embeddings generated from a NN model. The authors also do empirical evaluation on two different tasks of OOD detection and classification with rejection to show the utility of their method. ", "questions": "- Lines 24-26 \"MaxProb represents only aleatoric uncertainty...\" I think this is incorrect. MaxProb in my point of view captures the total uncertainty, including aleatoric and epistemic. I'd be happy to hear the counterpoint.\n\n- Line 33 \"Such methods are very perspective..\" Seems like a typo\n\n- Lines 42-43 \"...some methods require computing covariance...\". Please mention the approaches and cite them \n\n- Eq 2, what's u exactly and how is this thing practically computed? \n\n- Line 129-130  \"standard deviation of data label\" - what's this std dev exactly? \n\n- There's additional work [1, 2] that looks at distilling uncertainty estimates as well as looks at classification with rejection using Bayesian decision theory framework. This should be discussed in the related work, and potentially the distillation method should be compared in the experimental section [2, 3]. \n\n- It'll be nice to see what the additional runtime looks like for generating the approximation to conditional label distribution and how does it scale wrt models/datasets.\n\nReferences\n\n[1] Vadera, Meet, Brian Jalaian, and Benjamin Marlin. \"Generalized bayesian posterior expectation distillation for deep neural networks.\" UAI (2020).\n\n[2] Vadera, Meet, Soumya Ghosh, Kenney Ng, and Benjamin M. Marlin. \"Post-hoc loss-calibration for Bayesian neural networks.\" In UAI (2021).\n\n[3] Malinin, Andrey, Bruno Mlodozeniec, and Mark Gales. \"Ensemble distribution distillation.\" ICLR (2020).", "limitations": "I've highlighted the shortcomings in the previous sections. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths \n\n- The overall method is very flexible in the sense that it can utilize any existing neural network model. \n\n- The experiments provided by the authors cover a large set of baselines and models. \n\n\nWeakness\n\n- The paper is not very easy to follow, especially around methodology. I've added more comments on this in the next section. \n\n- Although the authors have used deep ensembles, the authors haven't used the traditional model uncertainty as a metric for epistemic \nuncertainty (also known as mutual information) in the comparison. The authors haven't used their distilled version either for comparison.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "VykagLgzMl3", "reviewer": "Reviewer_5BnR", "summary": "This paper proposes Nonparametric Uncertainty Quantification (NUQ), a simple approach to estimate predictive uncertainty using kernel density estimators. The model uses kernel density estimators for prediction (Eq 1) and squared kernel magnitude (Eq 2) for predictive variance. The method adopts a simple decomposition between aleatoric and epistemic uncertainty.  A consistency proof for the method's ability in approximating the optimal learning-with-abstention risk is also provided. Experiments are conducted on large-scale vision benchmark (ImageNet) and several standard NLP tasks, where NUQ illustrated consistent improvement over other methods. ", "questions": "Please see **Weakness** (1)-(3)", "limitations": "The method's technical clarity can use some improvement. Please see **Weakness**", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "** Strength **\n  * A simple and novel approach to deep uncertainty quantification grounded on kernel density theory. \n  * The method comes with optimal guarantees towards minimizing certain learning-under-abstention loss.\n  * The method was extensively evaluated on vision and language benchmarks, showing good performance.\n\n** Weakness **\nI only have some minor suggestions about technical clarity\n\n (1) Some of the technical details are missing or buried in the SM. For example, on line 129, what are $p(x)$ and $\\mathbf{u}$? Please give brief explanation in the main text and then point to appropriate section / table in the SM.\n (2) In the experiments, how are $\\sigma^2$ and $p(x)$ defined and estimated? \n (3) In the experiments, how are hyper-parameters such as $\\sigma^2$ and $h$ estimated? Specifically, what objective are you minimizing? Also in practice, when the test data / OOD data are not available, how should practitioner set the hyper-parameters? The paper can benefit a section (maybe in SM) that collects all hyperparameters of the method and discuss practical strategies in setting them. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept