NUQ: N ONPARAMETRIC U NCERTAINTY Q UANTIFICATION FOR D ETERMINISTIC N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
This paper proposes a fast and scalable method for uncertainty quantiﬁcation of machine learning models’ predictions.
First, we show the principled way to measure the uncertainty of predictions for a classiﬁer based on Nadaraya-Watson’s nonparametric estimate of the conditional label distribution.
Importantly, the approach allows to disentangle explicitly aleatoric and epistemic uncertainties.
The resulting method works directly in the feature space.
However, one can apply it to any neural network by considering an embedding of the data induced by the network.
We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets, such as MNIST, SVHN,
CIFAR-100 and several versions of ImageNet
I NTRODUCTION
It is crucial in many applications of modern machine learning methods to complement the prediction with some sort of a “conﬁdence” score.
In particular, deep neural network models, which usually achieve state-of-the-art results in various tasks, are notorious for providing overconﬁdent predictions on data they did not see during training (Nguyen et al., 2015).
This issue restricts their wide usage in the ﬁelds with high costs of wrong predictions, such as medicine (Miotto et al., 2016), autonomous driving (Levinson et al., 2011; Filos et al., 2020), ﬁnance (Brando et al., 2018) and others.
Thus, developing a reliable method of quantifying uncertainty is of great interest to researchers and especially practitioners.
The community in recent years made tremendous efforts to develop different uncertainty estimation methods and approaches, including calibration Guo et al.
(2017), ensembling (Lakshminarayanan et al., 2017), Bayesian methods (Gal & Ghahramani, 2016), and many others (Ovadia et al., 2019;
Wang et al., 2019).
Recently, a series of methods of uncertainty estimation based on the single deterministic neural network model was developed (Van Amersfoort et al., 2020; Liu et al., 2020; van
Amersfoort et al., 2021).
Their primary focus is on ensuring that embeddings of the data obtained on some layer of a network capture the geometrical relationships between the data samples in the input space, which is done via different regularization strategies.
Given this property, one can apply a certain approach to capture uncertainty in the embedding space.
The crucial property of these methods is the relatively mild required change in architectures and training procedures, which allows the application of the majority of existing deep learning models.
In practice it is usually important to distinguish two types of uncertainty: aleatoric and epistemic (Der Kiureghian & Ditlevsen, 2009; Kendall & Gal, 2017).
The aleatoric uncertainty reﬂects the internal noise in the data due to class overlap, data markup errors, or other reasons.
This type of uncertainty can not be reduced by providing more data.
The epistemic uncertainty reﬂects the model’s ignorance of data.
We can reduce the uncertainty of this type once we get more data.
Epistemic uncertainty, thus, may be used to identify out-of-distribution OOD data .
If the model can quantify this type of uncertainty, it may abstain from prediction and address it to a human expert.
Also, the ability to quantify epistemic uncertainty helps in active learning (Gal et al., 2017), where lack of “knowledge” naturally indicates in which areas we should label samples.
Importantly, there is no single deﬁnition of uncertainty, and diverse, often heuristic treatments are usually used in practice.
Summary of the contributions.
In this paper, we develop a new and theoretically grounded method of uncertainty quantiﬁcation applicable to any deterministic neural network model.
More speciﬁcally, our contributions are as follows.
We rigorously deﬁne the uncertainty of the model prediction at a particular data point.
This is done by direct consideration of the probability of the wrong prediction.
We provide corresponding uncertainty estimate by computing the variance of the kernel estimate of conditional density with the appropriately chosen bandwidth.
We apply the resulting method of uncertainty estimation in the neural network’s embedding space.
Our approach complements recent works in uncertainty estimation for deterministic neural networks, which suffered from a lack of a principled method to quantify the uncertainty.
We implement the method in a scalable manner, which allows it to be used on large datasets such as ImageNet.
The experimental results in misclassiﬁcation detection and OOD detection tasks show the signiﬁcant potential of the proposed approach.
The rest of the work is organised as follows.
Section 2 describes the theoretical background and the resulting uncertainty estimation method.
Section 3 reviews the related works.
In Section 4 we describe the experimental evaluation of the proposed method.
Section 5 concludes the study.
N ONPARAMETRIC U NCERTAINTY Q UANTIFICATION 2. E STIMATION UNDER C OVARIATE S HIFT
Let’s consider the standard binary classiﬁcation setup (
X, Y ) ∈
R d × { 0 , 1 } with ( X, Y ) ∼ P tr .
We assume that we observe the dataset D = (cid:8)
( X i , Y i ) } ni =1 of i.i.d.
points from P tr .
The classical problem in statistics and machine learning is to ﬁnd a rule ˆ g based on the dataset D which approximates the optimal one:
[Equation]
Here g : R d → { 0 , 1 } is any classiﬁer and the probability of wrong classiﬁcation R g
= P ( g ( X ) (cid:54) =
Y ) is usually called risk .
The rule g ∗ is given by the Bayes optimal classiﬁer :
[Equation]
where η ( x ) = p ( Y = 1 | X = x ) which is the conditional distribution of Y given X
= x under the distribution P .
In this work, we consider a situation when the distribution of the test samples P test is different from the one for the training dataset P tr , i.e. P test (cid:54) =
P tr .
Obviously, the rule g ∗ obtained for P =
P tr might no longer be optimal if the aim is to minimize the error on the test data P test ( g ( X ) (cid:54)
= Y ) .
In order to formulate a meaningful estimation problem, some additional assumptions are needed.
First of all, we assume that the distribution P test is unknown at the model construction moment, and only the dataset D is available.
Also, we will assume that the conditional label distribution p ( y | x ) is the same under both P tr and P test .
The latter assumption has two important consequences:
All the difference between P tr and P test is due to the difference between marginal distributions of X : p train ( X ) and p test ( X ) .
The situation when p test ( X ) (cid:54) =
p train ( X ) is known as covariate shift .
The Bayes rule is still valid, i
e., optimal even under P test .
However, while Bayes rule g ∗ is still optimal, its approximation ˆ g might be arbitrary bad under the covariate shift.
The reason for that is that we can’t expect ˆ g to approximate g ∗ well in the areas where we have few samples from the training set or don’t have them at all.
Thus, some special treatment of covariate shift is required.
We will discuss the particular problem statement in the next section.
2. P ROBLEM S TATEMENT
We consider a classiﬁcation rule
ˆ g ( x ) =
ˆ g D ( x ) constructed solely based on the dataset D .
Let us start from deﬁning pointwise risk of estimation:
[Equation]
where P (ˆ g ( X ) (cid:54) =
Y | X = x ) ≡ P tr (ˆ g ( X ) (cid:54) =
Y | X = x ) ≡ P test (ˆ g ( X ) (cid:54) =
Y | X = x ) under the assumptions above.
The value R ( x ) is independent of covariate distribution p test ( X ) and essentially allows to deﬁne a meaningful target of estimation which is based solely on the quantities known for the training distribution.
Let us note that the total risk value R ( x ) admits the following decomposition:
[Equation]
where R ∗ ( x ) = P ( g ∗ ( X ) (cid:54)
=
Y | X = x ) is Bayes risk and ˜ R ( x ) =
P (ˆ g ( X ) (cid:54) =
Y | X = x ) − P ( g ∗ ( X ) (cid:54) =
Y | X = x ) is an excess risk.
Here R ∗ ( x ) corresponds to aleatoric uncertainty as it completely depends on the data distribution.
Excess risk ˜
R ( x ) directly measures imperfectness of the model ˆ g and thus can be seen as a measure of epistemic uncertainty.
To proceed, we ﬁrst assume that the classiﬁer ˆ g has the standard form:
[Equation]
where ˆ η ( x ) = ˆ p ( Y = 1 | X = x ) is an estimate of the conditional density η ( x ) .
For such an estimate we can efﬁciently bound the excess risk via the following classical inequality Devroye et al.
[Equation]
It allows us to obtain an upper bound for the risk:
[Equation]
where R ∗ ( x ) = min { η ( x ) , 1 − η ( x ) } is just the Bayes risk.
While the upper bound still depends on the unknown quantity η ( x ) , we will see in the next section that L ( x ) allows for an efﬁcient approximation under mild assumptions.
2. N ONPARAMETRIC U NCERTAINTY Q UANTIFICATION 2.3. K ERNEL D ENSITY E STIMATE AND I TS A SYMPTOTIC D ISTRIBUTION
To execute the approach above we need to consider some particular type of estimator for ˆ g .
In this work, we suggest to consider classical kernel-based estimator of the conditional density as it allows for simple description of its asymptotic properties.
For an arbitrary class label c , the conditional probability estimate can be expressed as:
[Equation]
Note that in case of c = 1 the equation above gives us ˆ η ( x ) .
In our experiments, we consider one dimensional kernel function K : R → R + and construct the resulting kernel in R d of the following form:
[Equation]
Different choices of kernels are possible, see Supplementary Material, Section A.4.1).
A well-known fact (see, e.g. Powell (2010)) is that the difference between ˆ η ( x ) − η ( x ) for properly chosen bandwidth h converges in distribution as follows:
[Equation]
where n is the number of data points in the training set, K ( · ) is the kernel used for kernel density estimate (KDE), h is the bandwidth of the kernel; d is the dimensionality of the problem and σ 2 ( x ) is the standard deviation of the data label at point x .
Note that we can efﬁciently approximate the variance term in (2).
To start with, the integral (cid:82)
[ K ( u )] 2 du can be computed in the closed form for various standard kernels, see Supplementary Material, Table 3.
Second, we approximate the marginal density of objects p ( x ) .
The density can be again obtained via KDE:
ˆ p ( x ) = 1 Nh d (cid:80) Ni =1 K h ( x − x i ) .
However, one can choose the other estimates of the density, not necessarily related to the considered kernel estimate.
For example,
Gaussian Mixture Model (GMM) can be used similarly to Mukhoti et al.
The only thing left is the variance which can be estimated as ˆ σ 2 ( x ) =
[Equation]
(cid:1) .
Now we are equipped with an estimate of the distribution for ˆ η ( x ) − η ( x ) .
Let us denote by τ ( x ) the standard deviation of a Gaussian from equation (2):
[Equation]
Based on the obtained approximation of the distribution for L ( x ) , one can construct an abstention procedure that takes into account properties of this distribution and study the theoretical properties of the resulting method similarly to (Zaoui et al., 2020).
We defer these studies to future work and focus on the practical approach for obtaining uncertainty estimates which directly follows from the derivations above.
2.3. E STIMATES OF T OTAL , A LEATORIC AND E PISTEMIC U NCERTAINTY
In this work, we suggest a particular uncertainty quantiﬁcation procedure inspired by the derivation above.
More speciﬁcally, we suggest to consider the following measure of the total uncertainty:
[Equation]
which is obtained by considering an asymptotic approximation of
[Equation]
in a view of (2) and the fact, that E | ξ | = std ( ξ ) (cid:113) 2 π for the zero-mean normal variable ξ .
The resulting estimate upper bounds the average error of estimation at point x and thus indeed can be used as the measure of total uncertainty.
We also can write the corresponding measures of aleatoric and epistemic uncertainties:
[Equation]
Finally, the data-driven uncertainty estimates ˆ U t ( x ) and ˆ U e ( x ) can be obtained via plug-in using estimates ˆ η ( x ) , ˆ σ ( x ) , ˆ p ( x ) and, consequently, ˆ τ 2 ( x ) = {+
 Nh d ˆ+}
[Equation]
The generalization of the considered uncertainty measures to the case of multiple classes results in the total uncertainty given by
[Equation]
where τ 2 ( x ) = 1 Nh d max c (cid:8) σ 2 c ( x ) (cid:9) p ( x ) (cid:82) (cid:2) K ( u ) (cid:3) 2 du and σ 2 c ( x ) =
η c ( x ) (cid:0) 1 − η c ( x ) (cid:1) .
The derivation of these formulas can be found in Supplementary Material, Section A.1.
We note that the resulting formula for aleatoric uncertainty U a ( x ) =
min c (cid:8) 1 − η c ( x ) (cid:9) coincides with classical maximum probability (MaxProb) uncertainty measure.
The only remaining unspeciﬁed ingredient of the procedure is the choice of bandwidth h for KDE.
2. H OW TO C HOOSE B ANDWIDTH P ROPERLY ?
The choice of the optimal bandwidth parameter is well-developed in the theory of kernel density estimation.
For example, one can base on asymptotically optimal values and select the bandwidth accordingly as in Silverman’s (Silverman, 2018) or Scott’s (Scott, 1979) rules.
However, such estimates are usually very crude in practice.
In this work, we consider the choice of bandwidth based on the Improved Sheather–Jones algorithm (Botev et al., 2010).
We assume that the bandwidth optimal for the primary problem (density estimation) is also helpful for OOD detection.
It is not necessarily so in practice.
Thus, it might be beneﬁcial to tune the bandwidth to optimize the quality of OOD detection if some set of OOD points is available at the training time.
However, we ﬁnd that considered estimates perform fairly well in practice, see the experimental evaluation in Section 4.
2. H OW TO C OMPUTE K ERNEL E STIMATE WHEN N IS L ARGE ?
Our nonparametric method involves a sum over the whole available data to compute the estimates.
This could be intractable in practice when we are working with large datasets.
However, the typical kernel K h quickly approaches zero with the increase of the norm of the argument: (cid:107) x − x i (cid:107) .
Thus, we can use an approximation of kernel estimates: instead of the sum over all elements in the dataset, we consider the contribution of only several nearest neighbors.
It requires a fast algorithm for ﬁnding the nearest neighbors.
For this purpose, we use the approach of Malkov & Yashunin (2018) based on
Hierarchical Navigable Small World graphs (HNSW).
It provides a fast, scalable, and easy-to-use solution to the computation of nearest neighbors.
R ELATED W ORK
The notion of uncertainty naturally shows up in Bayesian statistics (Gelman et al., 2013), and, thus,
Bayesian methods are often used for uncertainty quantiﬁcation.
The idea is to utilize posterior distribution over some hidden variables (or model parameters) to receive an uncertainty estimate.
However, exact Bayesian inference is intractable for modern architectures with many parameters, and approximations are used.
Two popular approximation ideas are Markov Chain Monte Carlo sampling (MCMC; Neal et al.)
and Variational Inference (VI; Blei et al. (2017)).
The former has theoretical guarantees to be asymptotically unbiased, but has high computational cost.
A popular alternative is VI-based mean-ﬁeld approximation with a Gaussian distribution or distributions, enhanced with normalizing ﬂows (Rezende & Mohamed, 2015;
Dinh et al., 2017; Papamakarios et al., 2021;
Kobyzev et al., 2020).
However, standard VI-based methods usually do not scale well enough to apply to really large modern neural networks.
That’s why some alternatives are considered, such as the Bayesian treatment of Monte-Carlo dropout (Gal & Ghahramani, 2016).
Despite all efforts, these approximations do not reach the state-of-the-art results, which belong to Deep
Ensembles (Lakshminarayanan et al., 2017).
The ensembles are efﬁcient but again computationally demanding.
That’s why a series of papers developed ways of approximating the distribution obtained by an ensemble of models by a single probabilistic model (Malinin & Gales, 2018;
Malinin et al., 2020; Sensoy et al., 2018).
These methods require changing the training procedure and/or at least double the number of trainable parameters.
Recently, another popular type of model for uncertainty quantiﬁcation was proposed.
Speciﬁcally, it was proposed to consider a single deterministic neural network model and only apply mild changes to the architecture and training procedure.
The crucial idea behind these methods is to ensure that an embedding space induced by the network captures the geometry of the input space.
More speciﬁcally,
− 2 .0 .2 .
5 − 5 .− 2 .0 .2 .5 .
− 2 .0 .2 .
Aleatoric uncertainty − 2 .0 .2 .
Epistemic uncertainty it was suggested to consider ResNet-type arcitectures (He et al., 2016) with some regularization, which ensures that the model becomes bi-Lipchitz (Van Amersfoort et al., 2020;
van Amersfoort et al., 2021; Mukhoti et al., 2021; Liu et al., 2020).
These regularization techniques are weight clipping, gradient penalty, and spectral normalization.
Deep neural networks trained in this way achieve results comparable with standard approaches but with some training overhead.
For uncertainty estimation with such networks variety of approaches were proposed.
In DUQ (Van Amersfoort et al., 2020), an
RBF layer is added to the network with a custom procedure to adjust the centroid points (in embedding space).
The downside of the method is its inability to distinguish aleatoric and epistemic uncertainty.
The heuristic to capture epistemic uncertainty was proposed in the DDU approach (Mukhoti et al., 2021) which uses Gaussian Discriminant Analysis in embedding space of a trained neural network for that.
SNGP (Liu et al., 2020) and DUE (van Amersfoort et al., 2021) are similar but use a Gaussian process as the ﬁnal layer, requiring estimating covariance with the use of inducing points or RFF expansion.
While aiming for simplicity and minimal changes to the existing approaches for the construction of neural network models, the majority of these methods still require setting additional hyperparameters and/or increased computational cost.
E XPERIMENTS 4. T OY E XAMPLE
We start this section with the application of the proposed Nonparametric Uncertainty Quantiﬁcation (NUQ) method to a toy example.
As a dataset, we use a 2-dimensional mixture of three Gaussians with centers at points [3, -2], [3, 2], [0, 10], and variance equal to 1 .
Each Gaussian is treated as a separate class (see Figure 1, the leftmost panel).
We consider the Bayes classiﬁer based on the nonparametric estimate of the conditional density (1) and compute aleatoric and epistemic uncertainty values according to equations (3).
Bandwidth was selected according to Improved Sheather–Jones (“ISJ”) rule (Botev et al., 2010) independently for each data dimension.
Classiﬁcation results and uncertainties for this toy problem are presented in
Figure 1.
The ﬁrst plot shows the raw data and the result of the classiﬁcation by the Bayes rule.
Two other plots present aleatoric and epistemic uncertainty estimates obtained.
The uncertainty measures show the desired behavior: aleatoric uncertainty is large in-between the classes, while epistemic uncertainty increases with the increase of the distance to the training data.
4. I MAGE C LASSIFICATION D ATASETS
In this section, we consider a series of experiments on image datasets.
In contrast to the toy example above, we should ﬁrst train a model and then apply NUQ to its predictive features.
We emphasise, that NUQ is the postprocessing method, which is ﬁtted to the embeddings obtained from a given model.
In what follows, we call this model a “base model”.
In the experiments of this section, we [Figure] use logits as extracted features, if not explicitly stated otherwise.
However, other options are also possible; see Supplementary Material, Section A.4.
We compare popular measures of uncertainty which do not require signiﬁcant modiﬁcations to model architectures and training procedures.
More speciﬁcally, we consider:
Maximum probability (MaxProb): 1 − max c p ( y = c
| x ) ; 2.
Entropy: − (cid:80)
Cc =1 p ( y = c
| x ) log p ( y = c
| x ) ; 3.
Monte-Carlo dropout (Gal & Ghahramani, 2016); 4.
Ensemble of models trained with different random seeds; 5.
Test-Time Augmentation (TTA) – augmentation, applied to data at inference time; 6.
DDU (Mukhoti et al., 2021) involves Gaussian Mixture Model (GMM)-like approximation of extracted features to predict uncertainties.
For Monte-Carlo dropout, Ensembles, and TTA, we ﬁrst compute average vectors of predictions and then compute its entropy (as we noticed) among MaxProb, Standard deviation, and BALD entropy provides the best ROC-AUC results).
More details can be found in Supplementary Material,
Section A.2.
4.2. R OTATED MNIST
The second example is misclassiﬁcation detection on MNIST (LeCun et al., 2010).
We train a small convolutional neural network with three convolution layers, see Supplementary Material, Section A.2.
This is the base model we use to obtain logits for the input objects.
We consider a particular instance of distribution shift for evaluation by using a test set of MNIST images rotated at a random angle in the range from 30 to 45 degrees.
This set contains 10000 images.
The range of angles reassures that the data does not look like the original MNIST data, though many resulting pictures can still remind the ones from training.
In this experiment, we consider MaxProb and Entropy-based uncertainty estimates of the base model (using base model predictions, not NUQ) and compare them with NUQ-based estimate of total uncertainty ˆ U t ( x ) .
To evaluate the quality of the uncertainty estimates, we sort the objects from the test dataset in order of ascending uncertainties.
Then we obtain the model’s predictions and plot how accuracy changes with the number of objects taken into consideration; see Figure 2a.
The valid uncertainty estimation method is expected to produce the plot with accuracy decreasing when more samples are taken into account.
Moreover, the higher is the plot, the better is the quality of the corresponding uncertainty estimate.
We see that the plots for all the considered methods show the expected trend, while uncertainties obtained by NUQ are more reliable. [Table]
4.2. MNIST VS . SVHN
To make the problem more challenging, we consider the SVHN dataset (Netzer et al., 2011), convert it to grayscale
[Table]
and resize it to the shape of 28 x 28. The size of this additional SVHN-based dataset is again 10000.
We take the base model trained on MNIST from the previous section and consider the problem of OOD detection with SVHN being the OOD dataset.
As in-distribution [Table] we ﬁrst consider the test set of 10000 MNIST images.
We again compute uncertainties for each object of this concatenated dataset (10000 of MNIST and 10000 of SVHN) and sort them by their uncertainties in ascending order.
For NUQ we use total uncertainty ˆ U t ( x ) in this experiment.
In Figure 2b we plot the share of objects included from the SVHN dataset.
It is clearly seen that NUQ assigns higher uncertainties to objects from SVHN.
In fact, NUQ almost perfectly separates MNIST from SVHN (optimal result is also depicted on the plot).
Although NUQ is the leader in this task, competitors show good performance, and we move on to make the problem more challenging.
We consider the problem of separation between rotated MNIST (see Section 4.2.1) and SVHN.
We expect that it is harder to distinguish between them as rotated MNIST images differ from those used to train the network.
However, Figure 2c shows that NUQ still does a very good job and allows for almost perfect separation.
Interestingly, other methods completely fail and perform no better than random baseline.
4.2. CIFAR-
To reinforce our results on simpler datasets, we further conduct experiments on more challenging
CIFAR-100 (Krizhevsky, 2009).
We want our model to detect the unconventional samples, and thus we treat the out-of-distribution detection as a binary classiﬁcation task (OOD/not-OOD) by uncertainty score, and we report the ROC-AUC for that task.
Following the setup from the recent works (Van Amersfoort et al., 2020; van Amersfoort et al., 2021; Sastry & Oore, 2020), we use
SVHN, LSUN (Yu et al., 2015) and Smooth (Hein et al., 2019) datasets as OOD datasets.
We trained the ResNet-50 model from scratch on CIFAR-100.
For our method and DDU, we use training with spectral normalization (Miyato et al., 2018) to ensure the bi-Lipshitz constraint for mappings at each layer.
In this experiment, NUQ was applied to the features from the penultimate layer, and the density estimate is given by GMM.
See the results for other choices of hyperparameters in the Supplementary Material, Section A.4.
The results are presented in Table 1.
The ensemble has a strong performance, which is expected.
The
TTA performs reasonably well with the quality close to the one of the ensemble.
We can clearly see that NUQ and DDU show close results while outperforming the competitors with a signiﬁcant margin.
One may ask whether nonparametric classiﬁcation method used in NUQ, trained on some embedding from the base model, has any relation to the original neural network.
To reassure the reader, we provide an argument that it well approximates the predictions of the base model and NUQbased uncertainty estimates can be used for the base model as well.
Speciﬁcally, we compute the agreement between predictions obtained from the Bayes classiﬁer based on kernel estimate (i.e. the one used in NUQ) and base models’ predictions.
This metric formally can be deﬁned as agreement (ˆ p, [Equation]
= [Equation] ni
[Equation] [Equation]
For CIFAR-100, this metric gives us the agreement of 0.975, which tells that the approach is accurate.
[Table]
MaxProb*
Entropy*
TTA
Ensemble
DDU*
NUQ*
[Table] [Table] [Table] [Table]
[Table] [Table] O 28.
[Figure]
To evaluate the method’s applicability to the large-scale data, we have applied our approach to the ImageNet (Deng et al., 2009) dataset.
As OOD data we used the ImageNet-O (Hendrycks et al., 2021b) and ImageNet-R(Hendrycks et al., 2021a) datasets.
ImageNet-O consists of images from classes that are not found in the standard ImageNet-1k dataset.
ImageNet-R contains different artistic renditions of ImageNet classes.
It turned out that in these experiments, NUQ beats all the competitors with a large margin; see Table 2.
The advantage over DDU can be explained by the fact that DDU involves GMM-like approximation of the feature density, one mixture component per class.
In ImageNet, there are 1000 classes, and it appears to be an issue for GMM to approximate in such a high-dimensional space efﬁciently.
Also importantly, the resulting inference time for the NUQ estimator was quite fast, with less thanminutes needed to process the whole dataset on a moderate machine.
Additionally, we looked at some typical samples from these datasets with low, moderate, and high levels of uncertainty as assigned by NUQ, see Figure 3.
We observe that the uncertainty value corresponds well to the intuitive degree of complexity for these images compared to the original
ImageNet data.
Interestingly, the images with a low degree of uncertainty look like those that can be easily classiﬁed by a typical neural network trained on ImageNet.
C ONCLUSIONS
In this work, we propose NUQ, a new principled uncertainty estimation method that applies to a wide range of neural network models.
It does not require retraining the model and acts as a postprocessing step working in the embedding space induced by the neural network.
NUQ signiﬁcantly outperforms the competing approaches with only recently proposed DDU method (Mukhoti et al., 2021) showing comparable results.
Importantly, in the most practical example of OOD detection for ImageNet data,
NUQ shows the best results with a signiﬁcant margin.
All the code to reproduce the experiments is available at http://github.com/omitted/to/preserve/anonymity .
We hope that our work opens a new perspective on model-free uncertainty quantiﬁcation methods for deterministic neural networks.
Since the usage of deep models in high-risk real-world applications is usually limited due to the inability to account for uncertainty, we believe that our ideas could help the community bridge this gap.
We also believe that NUQ is suitable for in-depth theoretical investigation, which we defer to future work
A S UPPLEMENTARY M ATERIAL A. M ULTICLASS G ENERALIZATION FOR U NCERTAINTIES
In this section we show, how our method can be generalized from binary classiﬁcation to multiclass problems.
Consider data pairs ( X, Y ) ∼ P .
Now, X ∈ R d and Y ∈ 1 , . .
, C , where C is the number of classes.
We also denote η c ( x ) =
P ( Y = c
| X = x ) .
Let us start with the Bayes risk:
[Equation]
where g ∗ ( x ) := arg max c η c ( x ) is Bayes optimal classiﬁer.
Let us further move to the excess risk.
Denote by ˆ η c ( x ) density, we approximate.
Analogously, g ( x ) := arg max c ˆ η c ( x )
[Equation]
where we used the fact that ˆ η g ∗ ( x ) ( x ) − ˆ η g ( x ) ( x ) ≤ 0 for any x .
The expectation of the right hand can be upper bounded by 2 (cid:113)
π τ ( x ) , where τ ( x ) is deﬁned below.
Total uncertainty for multiclass problem is thus
[Equation]
where
[Equation]
and σ 2 c ( x ) =
η c ( x
[Equation]
(cid:0)
[Equation]
−
[Equation]
A. A RCHITECTURES A.2. B ASE M ODEL
For CIFAR-100 and ImageNet-like datasets, we are using ResNet50 with or without spectral normalization (Miyato et al., 2018).
For the spectral normalization, we use 3 iterations of the power method.
We use a ResNet50 architecture with implementation from PyTorch (Paszke et al., 2019).
This architecture was implemented for the ImageNet dataset; thus, for the CIFAR-100, we had to adapt it.
We changed the ﬁrst convolutional layer and used kernel size 3x3 with stride 1 and padding 1 (instead of kernel size 7x7 with stride 2 and padding 3).
For CIFAR-100, we train the model for 200 epochs with an SGD optimizer, starting with a learning rate of 0.1 and decaying it 5 times on 60, 120, and 160 epoch.
For ImageNet, we train the model for 90 epochs with an SGD optimizer learning rate decaying 10 times every 30 epochs.
For MNIST, we train a small convolutional neural network with three convolution layers with padding of 1 and kernel size of 3.
Each of these layers is followed by a batch normalization layer.
Finally, it has a linear layer with Softmax activation.
This network achieves an accuracy of 0.99 on the holdout set. [Figure]
A.2. E NSEMBLE
For ensemble with use a combination of 5 base models, trained with different random seeds.
A.2. T EST -T IME A UGMENTATION (TTA)
For TTA, we use a base model with applying a transformation on the inference stage.
Images of
CIFAR-100 are randomly cropped with padding 4, randomly horizontally ﬂipped, and randomly rotated up to 15 degrees.
ImageNet is randomly cropped from 256 to 224, randomly horizontally ﬂipped, and the color was jittered (0.02).
A.2. S PECTRALLY N ORMALIZED M ODELS
For both DDU and NUQ, we need spectral normalized models to extract features.
We’re wrapping each convolutional and linear layer with spectral normalization (PyTorch implementation).
We used 3 iterations of the power method in our experiments.
A. M ORE BASELINES ON ROTATED MNIST BENCHMARK
We provide two more baselines for the rotated MNIST experiments: Deep ensemble and DDU. Overall, we consider this example being too simplistic to make general conclusions and the experiment is included in the main paper is mostly for illustrative purposed and sanity check. Nevertheless, more complete picture with additional based is presented on Figure 4.
The NUQ distinctly outperforms ensembles on all examples and is on-par with DDU on out-of-distribution test.
However, NUQ is better then DDU in the example with rotated MNIST, see Figure 4a.
The possible reason is that NUQ considers total uncertainty which is natural in this examples as rotated-MNIST has naturally the same classes as MNIST and we are in misclassiﬁcation detection scenario (not pure OOD detection).
DDU is worse as it targets only epistemic uncertainty.
A. A BLATION S TUDY ON CIFARA.4. C HOICE OF K ERNEL FOR U NCERTAINTY Q UANTIFICATION
In this section, we study the choice of a kernel for uncertainty quantiﬁcation.
We consider the following choices:
We need a probability density estimation for our method, and there are different options: we consider kernel method with RBF kernel and logistic kernel and Gaussian mixtures models.
There is also a question about which embeddings to use - the DDU paper proposes to take the features from the second last layer; we believe the logits from the last layer are a reasonable choice as well.
To validate [Table] the options, we conducted some ablation study on out-of-distribution detection for the CIFARdataset, similar to the main experiment.
First, we compare the DDU and NUQ on embeddings from the pre-last and last layer (Table 4) on
SVHN, LSUN [Table] and Smooth datasets.
Secondly, we compare the NUQ method on RBF, logistic kernel [Table] and GMM for both last and penultimate layer embeddings(Table 5).
As we can see from the tables, the optimal is the option with GMM density on the penultimate layer [Table]
[Table]
Kernel-based methods rely on the “reasonable” geometry of the embedding space, meaning that embeddings of similar images should not be too far and different images should not collapse into a single point.
Our motivation to use spectral normalization during training is to make the embedding space more smooth with respect to input images.
We have conducted an extra ablation study, comparing the result for feature extractors with and without spectral normalization, see Table 6.
The results conﬁrm our hypothesis [Table] as the spectral-normalized version performs better
[Table]
though the NUQ beats the baseline even without applying the modiﬁcation to the ResNet training.
We also show here that entropy performs better than maximum probability as an uncertainty measure
[Table]
A. P ERFORMANCE DIFFERENCE ON CIFAR-100 AND I MAGE N ET
One of the things that caught our attention is superior performance of NUQ on ImageNet, given that it has very similar results with DDU on CIFAR-100.
One of our hypotheses was that embeddings have more complex and multi-modal distribution for more complex Imagenet dataset compared to simpler CIFAR-100.
To check this, we made t-SNE based embeddings of out-of-distribution and test
[Figure]
data (see Figure 5).
While we understand the limitation of this type of visualization, the ImageNet embeddings appear to be much more irregular compared to well-shaped clusters for CIFAR-100.
Because of that the modelling of the class with single Gaussian (in DDU) might not work very well for ImageNet.
NUQ approach performs the modelling of distributions in much more ﬂexible way which is beneﬁcial for approximation of complex distributions.
We hypothesize that this is the reason of the NUQ’s superior performance.
A. T OY EXPERIMENT ON DETECTING ACTUAL ALEATORIC AND EPISTEMIC UNCERTAINTIES
In this section, we conduct a toy experiment, for which we explicitly know what should be the true probability of class one, as well as the true data density.
Let us consider a binary classiﬁcation problem.
Our dataset consists of 5000 samples from three different one-dimensional Gaussians, located so that classes are mixed (see picture).
Colors denote class label: red - 0; green - 1 (Left in Figure 6)
For this particular data model, we can compute the conditional probability of a data point x belongs to class 1: p ( y = 1 | x ) .
We build an estimate of this conditional using our Nadaraya-Watson kernel-based approach.
Further, we generate a uniform grid, and for each point of this grid, using our method, we can upper bound difference between the true conditional and our approximation.
This difference, according to our approach, is considered as an epistemic uncertainty (Middle in the Figure 6).
The green line in this plot denotes an absolute difference between the true conditional and our approximation.
The red line denotes our epistemic uncertainty.
From the picture, we can see that our epistemic uncertainty approximates the probabilities difference well.
Next, we show how our aleatoric uncertainty relates to the true class 1 conditional probability.
In the right plot of the Figure 6 we show true conditional distribution p ( y = 1 | x ) and our approximation of the aleatoric uncertainty.
We can see that our approximation is high exactly in the same regions where the true conditional is absolutely unsure about the class label.
A. P SEUDO - CODE
− 10 x 0 .0 .0 .0 .0 .0 .0 .0 .
D e n s i t y − 10 x 0 .0 .0 .0 .0 .1 .
U n ce r t a i n t y ( e p
i s t
e m i c )
Data density Epistemic Prob. diﬀ. − 10 x 0 .0 .0 .0 .0 .1 .
U n ce r t a i n t y ( a l e a t
o r i c )
Data density Class 1 prob.
Aleatoric
Input:
[Equation]
set {
[Equation]
y
[Equation]
[Equation]
inference point z Output: Prediction y ( z ) and uncertainty u ( z )
[Equation]
← K nearest neighbours of z
[Equation]
[Table]