NUQ: Nonparametric Uncertainty Quantiﬁcation for Deterministic Neural Networks
Abstract
This paper proposes a fast and scalable method for uncertainty quantiﬁcation of machine learning models’ predictions.
First, we show the principled way to measure the uncertainty of predictions for a classiﬁer based on Nadaraya-Watson’s nonparametric estimate of the conditional label distribution.
Importantly, the approach allows to disentangle explicitly aleatoric and epistemic uncertainties.
The resulting method works directly in the feature space.
However, one can apply it to any neural network by considering an embedding of the data induced by the network.
We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets, such as MNIST, SVHN,
CIFAR-100 and several versions of ImageNet.
Introduction
It is crucial in many applications of modern machine learning methods to complement the prediction with some sort of a “conﬁdence” score.
In particular, deep neural network models, which usually achieve state-of-the-art results in various tasks, are notorious for providing overconﬁdent predictions on data they did not see during training [19].
The community in recent years made tremendous efforts to develop different uncertainty estimation methods and approaches, including calibration [6], ensembling [12], Bayesian methods [5] and many others [20, 28].
Recently, a series of methods of uncertainty estimation based on the single deterministic neural network model was developed [27, 14, 26].
In practice it is usually important to distinguish two types of uncertainty: aleatoric and epistemic [3, 10].
The aleatoric uncertainty reﬂects the internal noise in the data due to class overlap, data markup errors, or other reasons.
This type of uncertainty can not be reduced by providing more data.
The epistemic uncertainty reﬂects the model’s ignorance of data.
We can reduce the uncertainty of this type once we get more data.
Epistemic uncertainty, thus, may be used to identify out-of-distribution OOD data .
If the model can quantify this type of uncertainty, it may abstain from prediction and address it to a human expert.
Summary of the contributions.
We develop a new and theoretically grounded method of uncertainty quantiﬁcation applicable to any deterministic neural network model.
Our contributions:
We rigorously deﬁne the uncertainty of the model prediction at a particular data point.
This is done by direct consideration of the probability of the wrong prediction.
We provide corresponding uncertainty estimate by computing the variance of the kernel estimate of conditional density with the appropriately chosen bandwidth.
We implement the method in a scalable manner, which allows it to be used in the neural network’s embedding space on large datasets such as ImageNet.
The experimental results in misclassiﬁcation detection and OOD detection tasks show the signiﬁcant potential of the proposed approach.
Nonparametric Uncertainty Quantiﬁcation 2.1 Estimation under Covariate Shift
Consider a binary classiﬁcation setup (
X, Y ) ∈
R d × { 0 , 1 } with ( X, Y ) ∼ P tr .
We assume that we observe the dataset D = (cid:8)
( X i , Y i ) } ni =1 of i.i.d.
points from P tr .
The classical problem is to ﬁnd a rule ˆ g based on the dataset D which approximates the optimal one: g ∗ =
arg min g P ( g ( X ) (cid:54) = Y ) .
Here g : R d → { 0 , 1 } is any classiﬁer and the probability of wrong classiﬁcation R g
= P ( g ( X ) (cid:54) =
Y ) is usually called risk .
The rule g ∗ is given by the Bayes optimal classiﬁer : [Equation]
where η ( x ) = p ( Y = 1 | X = x ) under the distribution P .
We consider a situation when the distribution of the test samples P test is different from the one for the training dataset P tr .
The rule g ∗ obtained for P =
P tr might no longer be optimal if we minimize the error on the test data P test ( g ( X ) (cid:54)
= Y ) .
For a meaningful estimation problem, some additional assumptions are needed.
First, we assume that the distribution P test is unknown at the model construction moment, only the dataset D is available.
Also, we assume that the distribution p ( y | x ) is the same under both P tr and P test , which means that:
1) All the difference between P tr and P test is due to the difference between marginal distributions of X : p train ( X ) and p test ( X ) .
The Bayes rule is still valid , i.e., optimal even under P test .
However, while the rule g ∗ is still optimal, its approximation ˆ g might be arbitrary bad under the covariate shift.
2.2 Problem Statement
Consider a classiﬁcation rule ˆ g ( x ) =
ˆ g D ( x ) on the dataset D .
Deﬁne pointwise risk of estimation:
[Equation]
where P (ˆ g ( X ) (cid:54) =
Y | X = x ) ≡ P tr (ˆ g ( X ) (cid:54) =
Y | X = x ) ≡ P test (ˆ g ( X ) (cid:54) =
Y | X = x ) under the assumptions above.
The value
R ( x ) is independent of covariate distribution p test ( X ) and allows to deﬁne a meaningful target of estimation which is based solely on the quantities known for the training distribution.
Let us note that the total risk value R ( x ) admits the following decomposition: R ( x ) =
˜ R ( x )+ R ∗ ( x ) , where R ∗ ( x ) =
P ( g ∗ ( X ) (cid:54) =
Y | X = x ) is Bayes risk and ˜ R ( x ) =
P (ˆ g ( X ) (cid:54) =
Y | X = x ) − P ( g ∗ ( X ) (cid:54) =
Y | X = x ) is an excess risk.
Here R ∗ ( x ) corresponds to aleatoric uncertainty as it completely depends on the data distribution.
Excess risk ˜
R ( x ) directly measures imperfectness of the model ˆ g and thus can be seen as a measure of epistemic uncertainty.
To proceed, we ﬁrst assume that the classiﬁer ˆ g has the form of optimal Bayesian classiﬁer with respect to the density
ˆ η ( x ) =
ˆ p ( Y = 1 | X = x ) , which is an estimate of the conditional density η ( x ) .
We can efﬁciently bound the excess risk via the following classical inequality [4]:
[Equation]
(cid:54) =
Y | X = x ) ≤ 2 | ˆ η ( x ) − η ( x ) | .
It allows us to obtain an upper bound for the risk: R ( x ) ≤ L ( x ) =
R ∗ ( x ) + 2 | ˆ η ( x ) − η ( x ) | , where
R ∗ ( x ) = min { η ( x ) , 1 − η ( x ) } is just the Bayes risk.
2.3 Nonparametric Uncertainty Quantiﬁcation 2.3.1 Kernel Density Estimate and Its Asymptotic Distribution
For the approach above we need to consider some particular type of estimator for ˆ g .
We consider kernel-based estimator of the conditional density because of its asymptotic properties.
For a class label c , the conditional probability estimate can be expressed as:
− 2 .0 .2 .
5 − 5 .− 2 .0 .2 .5 .
− 2 .0 .2 .
Aleatoric uncertainty − 2 .0 .2 .
Epistemic uncertainty
[Equation]
We consider 1D kernel function K : R → R + and construct the resulting kernel in R d of the form:
K
[Equation]
It could be shown, that the difference ˆ η ( x ) − η ( x ) converges in distribution as follows (see, e.g. [22]):
[Equation]
where n is the number of data points in the training set, K ( · ) is the kernel used for kernel density estimate (KDE), h is the bandwidth of the kernel; d is the dimensionality of the problem and σ 2 ( x ) is the standard deviation of the data label at point x .
Let us denote the variance term in (2) by τ 2 ( x ) .
In this work, we suggest to consider the following measure of the total uncertainty:
[Equation]
which is obtained by considering an asymptotic approximation of E D L ( x ) .
To obtain the practical estimate, the integral (cid:82)
[ K ( u )] 2 du can be computed in the closed form for various standard kernels, see Supplementary Material, Table 3.
Second, we approximate the marginal density of objects p ( x ) .
The density can be again obtained via KDE (but one can choose another estimation [17]): ˆ p ( x ) = 1 Nh d (cid:80) Ni =1 K h ( x − x i ) .
The only thing left is the variance which can be estimated as ˆ σ 2 ( x [Equation] =
[Equation] y | x ) =
ˆ η ( x )(1 − ˆ η ( x )) .
We refer our readers to Supplementary Material to for more details on computation of different uncertainties.
Experiments 3.1 Toy Example
We start this section with the application of the proposed Nonparametric Uncertainty Quantiﬁcation (NUQ) method to a toy example.
As a dataset, we use a 2-dimensional mixture of three Gaussians with centers at points [3, -2], [3, 2], [0, 10], and variance equal to 1 .
Each Gaussian is treated as a separate class (see Figure 1, the leftmost panel).
We consider the Bayes classiﬁer based on the nonparametric estimate of the conditional density (1) and compute aleatoric and epistemic uncertainty values according to equations (3).
Bandwidth was selected according to Improved Sheather–Jones (“ISJ”) rule [1] independently for each data dimension.
Classiﬁcation results and uncertainties for this toy problem are presented in Figure 1.
The ﬁrst plot shows the raw data and the result of the classiﬁcation by the Bayes rule.
Two other plots present aleatoric and epistemic uncertainty estimates obtained.
The uncertainty measures show the desired behavior: aleatoric uncertainty is large in-between the classes, while epistemic uncertainty increases with the increase of the distance to the training data.
[Table] [Table]
In this section [Table] we consider a series of experiments on image datasets.
In contrast to the toy example above
[Table]
we should ﬁrst train a model and then apply NUQ to its predictive features.
We emphasise, that NUQ is the postprocessing method, which is ﬁtted to the embeddings obtained from a given model.
In what follows, we call this model a “base model”.
In the experiments of this section, we use logits as extracted features, if not explicitly stated otherwise.
However, other options are also possible; see Supplementary Material, Section A.5.
In experiments below, we compare our method to several baselines.
See Supplementary Material to ﬁnd a brief description of them.
CIFAR-100.
To reinforce our results on simpler datasets, we further conduct experiments on more challenging CIFAR-100 [11].
We want our model to detect the unconventional samples, and thus we treat the out-of-distribution detection as a binary classiﬁcation task (OOD/not-OOD) by uncertainty score, and we report the ROC-AUC for that task.
Following the setup from the recent works [27, 26, 23], we use SVHN, LSUN [29] and Smooth [7] datasets as OOD datasets.
We trained the ResNet-50 model from scratch on CIFAR-100.
For our method and DDU, we use training with spectral normalization [16] to ensure the bi-Lipshitz constraint for mappings at each layer.
In this experiment, NUQ was applied to the features from the penultimate layer, and the density estimate is given by GMM.
See the results for other choices of hyperparameters in the Supplementary
Material, Section A.5.
The results are presented in Table 1.
The ensemble has a strong performance, which is expected.
The
TTA performs reasonably well with the quality close to the one of the ensemble.
We can clearly see that NUQ and DDU show close results while outperforming the competitors with a signiﬁcant margin.
ImageNet.
To evaluate the method’s applicability to the large-scale data, we have applied our approach to the ImageNet [2] dataset.
As OOD data we used the ImageNet-O [9] and ImageNet-R[8] datasets.
ImageNet-O consists of images from classes that are not found in the standard ImageNet-1k dataset.
ImageNet-R contains different artistic renditions of ImageNet classes.
It turned out that in these experiments, NUQ beats all the competitors with a large margin; see Table 2.
Conclusions
In this work, we propose NUQ, a new principled uncertainty estimation method that applies to a wide range of neural network models.
It does not require retraining the model and acts as a postprocessing step working in the embedding space induced by the neural network.
NUQ signiﬁcantly outperforms the competing approaches with only recently proposed DDU method [17] showing comparable results.
Importantly, in the most practical example of OOD detection for ImageNet data, NUQ shows the best results with a signiﬁcant margin.
A Appendix A.1 How to Choose Bandwidth Properly?
The choice of the optimal bandwidth parameter is well-developed in the theory of kernel density estimation.
For example, one can base on asymptotically optimal values and select the bandwidth accordingly as in Silverman’s [25] or Scott’s [24] rules.
However, such estimates [-ˆ U t ( x ) and ˆ U e ( x ) can be obtained via plug-in usingestimates ˆ η ( x ) , ˆ σ ( x ) , ˆ p ( x ) and, consequently, ˆ τ 2 ( x ) = 
-]are usually very crude in practice.
A.2 How to Compute Kernel Estimate when N is Large?
Our nonparametric method involves a sum over the whole available data to compute the estimates.
This could be intractable in practice when we are working with large datasets.
However, the typical kernel K h quickly approaches zero with the increase of the norm of the argument: (cid:107) x − x i (cid:107) .
Thus, we can use an approximation of kernel estimates: instead of the sum over all elements in the dataset, we consider the contribution of only several nearest neighbors.
It requires a fast algorithm for ﬁnding the nearest neighbors. For this purpose, we use the approach of [15] based on Hierarchical Navigable Small World graphs (HNSW).
It provides a fast, scalable, and easy-to-use solution to the computation of nearest neighbors.
A.3 Multiclass Generalization for Uncertainties
In this section we show, how our method can be generalized from binary classiﬁcation to multiclass problems.
Consider data pairs ( X, Y ) ∼ P .
Now, X ∈ R d and Y ∈ 1 , . .
, C , where C is the number of classes.
We also denote η c ( x ) =
P ( Y = c
| X = x ) .
Let us start with the Bayes risk:
[Equation]
where g ∗ ( x ) := arg max c η c ( x ) is Bayes optimal classiﬁer.
Let us further move to the excess risk.
Denote by ˆ η c ( x ) density, we approximate.
Analogously, g ( x ) := arg max c ˆ η c ( x )
[Equation]
where we used the fact that ˆ η g ∗ ( x ) ( x ) − ˆ η g ( x ) ( x ) ≤ 0 for any x .
The expectation of the right hand can be upper bounded by 2 (cid:113)
π τ ( x ) , where τ ( x ) is deﬁned below.
Total uncertainty for multiclass problem is thus
[Equation]
where
[Equation]
and σ 2 c ( x ) =
η c ( x ) (cid:0) 1 − η c ( x ) (cid:1) .
A.4 Architectures A.4.1 Base Model
For CIFAR-100 and ImageNet-like datasets, we are using ResNet50 with or without spectral normalization [16].
For the spectral normalization, we use 3 iterations of the power method.
We use a
ResNet50 architecture with implementation from PyTorch [21].
This architecture was implemented for the ImageNet dataset; thus, for the CIFAR-100, we had to adapt it.
We changed the ﬁrst convolutional layer and used kernel size 3x3 with stride 1 and padding 1 (instead of kernel size 7x7 with stride 2 and padding 3).
For CIFAR-100, we train the model for 200 epochs with an SGD optimizer, starting with a learning rate of 0.1 and decaying it 5 times on 60, 120, and 160 epoch.
For ImageNet, we train the model for 90 epochs with an SGD optimizer learning rate decaying 10 times everyepochs.
For MNIST, we train a small convolutional neural network with three convolution layers with padding of 1 and kernel size of 3.
Each of these layers is followed by a batch normalization layer.
Finally, it has a linear layer with Softmax activation.
This network achieves an accuracy of 0.99 on the holdout set.
We refer readers to our code for more speciﬁc details.
A.4.2 Ensemble
For ensemble with use a combination of 5 base models, trained with different random seeds.
A.4.3 Test-Time Augmentation (TTA)
For TTA, we use a base model with applying a transformation on the inference stage.
Images of
CIFAR-100 are randomly cropped with padding 4, randomly horizontally ﬂipped, and randomly rotated up to 15 degrees.
ImageNet is randomly cropped from 256 to 224, randomly horizontally ﬂipped, and the color was jittered (0.02).
A.4.4 Spectrally Normalized Models
For both DDU and NUQ, we need spectral normalized models to extract features.
We’re wrapping each convolutional and linear layer with spectral normalization (PyTorch implementation).
We used 3 iterations of the power method in our experiments.
A.5 Ablation Study on CIFARA.5.1 Choice of Kernel for Uncertainty Quantiﬁcation
In this section, we study the choice of a kernel for uncertainty quantiﬁcation. We consider the following choices: Kernel name
[Equation]
Gaussian
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
values
of the integral (cid:82) K ( u ) 2 du .
We need a probability density estimation for our method, and there are different options: we consider kernel method with RBF kernel and logistic kernel and Gaussian mixtures models.
There is also a question about which embeddings to use - the DDU paper proposes to take the features from the second last layer; we believe the logits from the last layer are a reasonable choice as well.
To validate the options, we conducted some ablation study on out-of-distribution detection for the CIFARdataset, similar to the main experiment.
First, we compare the DDU and NUQ on embeddings from the pre-last and last layer (Table 4) on
SVHN, LSUN, and Smooth datasets.
Secondly, we compare the NUQ method on RBF, logistic kernel, and GMM for both last and penultimate layer embeddings(Table 5).
As we can see from the tables, the optimal is the option with GMM density on the penultimate layer.
Kernel-based methods rely on the “reasonable” geometry of the embedding space, meaning that embeddings of similar images should not be too far and different images should not collapse into a single point.
Our motivation to use spectral normalization during training is to make the embedding space more smooth with respect to input images.
We have conducted an extra ablation study, comparing the result for feature extractors with and without spectral normalization, see Table 6.
The
[Table]
[Table]
results conﬁrm our hypothesis, as the spectral-normalized version performs better, though the NUQ beats the baseline even without applying the modiﬁcation to the ResNet training.
We also show here that entropy performs better than maximum probability as an uncertainty measure
[Table]
A.6 Estimates of Total, Aleatoric and Epistemic Uncertainty
Let us denote by τ ( x ) the standard deviation of a Gaussian from equation (2):
[Equation]
In this work, we suggest to consider the following measure of the total uncertainty:
[Equation]
which is obtained by considering an asymptotic approximation of
[Equation]
in a view of (2) and the fact, that E | ξ | = std ( ξ ) (cid:113) 2 π for the zero-mean normal variable ξ .
The resulting estimate upper bounds the average error of estimation at point x and thus indeed can be used as the measure of total uncertainty.
We also can write the corresponding measures of aleatoric and epistemic uncertainties:
[Equation]
{+Finally, the data-driven uncertainty estimates ˆ U t ( x ) and ˆ U e ( x ) can be obtained via plug-in using estimates ˆ η ( x ) , ˆ σ ( x ) , ˆ p ( x ) and, consequently, ˆ τ 2 ( x ) = 
+}
Nh d ˆ σ 2 ( x
[Equation]
ˆ p ( x ) (cid:20) (cid:82) (cid:2) K ( u ) (cid:3) 2 du (cid:21) d .
The generalization of the considered uncertainty measures to the case of multiple classes results in the total uncertainty given by
[Equation]
where τ 2 ( x ) = 1 Nh d max c (cid:8) σ 2 c ( x ) (cid:9) p ( x ) (cid:20) (cid:82) (cid:2) K ( u ) (cid:3) 2 du (cid:21) d and σ 2 c ( x ) =
η c ( x ) (cid:0) 1 − η c ( x ) (cid:1) .
The derivation of these formulas can be found in Supplementary Material, Section A.3.
We note that the resulting formula for aleatoric uncertainty U a ( x ) = min c (cid:8) 1 − η c ( x ) (cid:9) coincides with classical maximum probability (MaxProb) uncertainty measure.
The only remaining unspeciﬁed ingredient of the procedure is the choice of bandwidth h for KDE (see Appendix).
In this work, we consider the choice of bandwidth based on the Improved Sheather–Jones algorithm [1].
We assume that the bandwidth optimal for the primary problem (density estimation) is also helpful for OOD detection.
It is not necessarily so in practice.
Thus, it might be beneﬁcial to tune the bandwidth to optimize the quality of OOD detection if some set of OOD points is available at the training time.
However, we ﬁnd that considered estimates perform fairly well in practice, see the experimental evaluation in Section 3.
A.7 Baseline description
We compare popular measures of uncertainty which do not require signiﬁcant modiﬁcations to model architectures and training procedures.
More speciﬁcally, we consider:
Maximum probability (MaxProb): 1 − max c p ( y = c | x ) ; 2.
Entropy: − (cid:80)
Cc =1 p ( y = c
| x ) log p ( y = c
| x ) ; 3.
Monte-Carlo dropout [5]; 4.
Ensemble of models trained with different random seeds; 5.
Test-Time Augmentation (TTA) – augmentation, applied to data at inference time; 6.
DDU [17] involves Gaussian Mixture Model (GMM)-like approximation of extracted features to predict uncertainties.
For Monte-Carlo dropout, Ensembles, and TTA, we ﬁrst compute average vectors of predictions and then compute its entropy (as we noticed) among MaxProb, Standard deviation, and BALD entropy provides the best ROC-AUC results).
More details can be found in Supplementary Material,
Section A.4.
A.8 Additional experiments A.8.1 Rotated MNIST
The second example is misclassiﬁcation detection on MNIST [13].
We train a small convolutional neural network with three convolution layers, see Supplementary Material, Section ?? .
This is the base model we use to obtain logits for the input objects.
We consider a particular instance of distribution shift for evaluation by using a test set of MNIST images rotated at a random angle in the range from 45 to 90 degrees.
This set contains 10000 images.
The range of angles reassures that the data does not look like the original MNIST data, though many resulting pictures can still remind the ones from training.
In this experiment, we consider MaxProb and Entropy-based uncertainty estimates of the base model (using base model predictions, not NUQ) and compare them with NUQ-based estimate of total uncertainty ˆ U t ( x ) .
To evaluate the quality of the uncertainty estimates, we sort the objects from the test dataset in order of ascending uncertainties.
Then we obtain the model’s predictions and plot how accuracy changes with the number of objects taken into consideration; see Figure 2.
The valid uncertainty estimation method is expected to produce the plot with accuracy decreasing when more samples are taken into account.
Moreover, the higher is the plot, the better is the quality of the corresponding uncertainty estimate.
We see that the plots for all the considered methods show the expected trend, while uncertainties obtained by NUQ are more reliable.
A.8.2 MNIST vs. SVHN
To make the problem more challenging, we consider the SVHN dataset [18], convert it to grayscale, and resize it to the shape of 28 x 28.
The size of this additional SVHN-based dataset is again 10000.
We take the base model trained on MNIST from the previous section and consider the problem of
OOD detection with SVHN being the OOD dataset.
As in-distribution, we ﬁrst consider the test set of 10000 MNIST images.
We again compute uncertainties for each object of this concatenated dataset (10000 of MNIST and 10000 of SVHN) and sort them by their uncertainties in ascending order.
For NUQ we use total uncertainty ˆ U t ( x ) in this experiment.
In Figure 3 we plot the share of objects included from the SVHN dataset.
It is clearly seen that NUQ assigns higher uncertainties to objects from SVHN.
In fact, NUQ almost perfectly separates MNIST from SVHN (optimal result is also depicted on the plot).
Although NUQ is the leader in this task, competitors show good performance, and we move on to make the problem more challenging.
We consider the problem of separation between rotated MNIST (see Section A.8.1) and SVHN.
We expect that it is harder to distinguish between them as rotated MNIST images differ from those used to train the network.
However, Figure 4 shows that NUQ still does a very good job and allows for almost perfect separation.
Interestingly, other methods completely fail and perform no better than random baseline.