M IX -M AX E NT : C REATING H IGH E NTROPY B ARRIERS TO I MPROVE A CCURACY AND U NCERTAINTY E STIMATES OF D ETERMINISTIC N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
We propose an extremely simple approach to regularize a single deterministic neural network to obtain improved accuracy and reliable uncertainty estimates.
Our approach, on top of the cross-entropy loss, simply puts an entropy maximization regularizer corresponding to the predictive distribution in the regions of the embedding space between the class clusters.
This is achieved by synthetically generating between-cluster samples via the convex combination of two images from different classes and maximizing the entropy on these samples.
Such a data-dependent regularization guides the maximum likelihood estimation to prefer a solution that (1) maps out-of-distribution samples to high entropy regions (creating an entropy barrier); and (2) is more robust to the superﬁcial input perturbations.
Via extensive experiments on real-world datasets (CIFAR-10 and CIFAR-100) using ResNet and Wide-ResNet architectures, we demonstrate that Mix-MaxEnt consistently provides much improved classiﬁcation accuracy, better calibrated probabilities for in-distribution data, and reliable uncertainty estimates when exposed to situations involving domain-shift and out-of-distribution samples.
I NTRODUCTION
A particularly thriving sub-ﬁeld of research in Deep Neural Networks (DNNs) concerns devising efﬁcient approaches towards obtaining reliable predictive uncertainty.
NNs are known to be overconﬁdent for both, in- and out-distribution samples (i.e. for samples coming from the same distribution from which the training distribution has been sampled (IND samples) and for samples not coming from such distribution), leading to highly unreliable uncertainty estimates.
They can be wrong with very high conﬁdence (Guo et al., 2017) not only on the test data similar to the one they have been trained on, but also when facing previously-unseen conditions (Taori et al., 2016).
The overconﬁdence problem becomes even more concerning when just slight changes in illumination, atmospheric conditions or in the image capturing process (domain-shift) can severely damage the actual accuracy of the model (Taori et al., 2016).
A desirable property of any model is to be robust to such superﬁcial changes that do not affect the label of the classiﬁed image, and to become uncertain (or indecisive) when exposed to samples from a distribution different from the training distribution.
While the literature suggests that classiﬁers whose predictive distributions increase their entropy the further away the test input gets from the training data are desirable, the implementation of models satisfying such a property is not scalable, and can only occur at the cost of performing crude approximations and by non-trivial modiﬁcations to the architecture of the neural network (Liu et al., 2020a; Kristiadi et al., 2020).
Such modiﬁcations sometimes lead to degraded accuracy.
Our motivation behind this work is based on the following observations.
(1) We ﬁnd that a trained network (DNN) do not project out-of-distribution (OOD) and domain-shifted (DS) samples arbitrarily away from the training data.
In fact, all the OOD and domain-shifted inputs that we considered (detailed discussion in Section 3) are projected within the smallest hypersphere that contains all of the IND test data.
Therefore, it might not be necessary to enforce the network to be uncertain everywhere away from the data, and perhaps, in-distribution data can be used to mimic the regions where
OOD and DS samples are being projected.
(2) DNNs tend to embed OOD and DS samples in high conﬁdence (low-entropy) regions (see Figure 1).
This contradicts the desired ideal behaviour (Liu et al., 2020a; Pereyra et al., 2017), that would require a model to map such samples in high-entropy regions.
However
MaxEnt wide high region
Details of this visualization are provided in Appendix
More exhaustive visualizations of this phenomenon are provided in
Given the observations above, we propose a simple entropy maximization regularizer (called
Mix-MaxEnt ), that induces a high entropy barrier between class clusters, while the cross-entropy optimization objective keeps the entropy low close to the class clusters.
The entropy regularizer is enforced for samples synthesized using the convex combination of a pair of samples from two different classes of the in-distribution training data.
When combined with the crossentropy loss, this regularizer prefers a maximum likelihood solution such that the uncertainty of the network increases while moving away from the embeddings of one class in the direction of another, and learns features that are robust to local input perturbations.
Through extensive experiments using
WideResNet28-10 and ResNet50 architerctures on CIFAR10 and CIFAR100 datasets, we demonstrate that our method outperforms all existing single model baselines in providing clean data accuracy.
On Domain-Shift experiments, it provides remarkably improved accuracy compared to all the baselines including Deep Ensembles (DE) (Lakshminarayanan et al., 2017).
For instance, it provides 4 .
8% and 4 .
7% improvements over the highly competitive DE and SNGP (Liu et al., 2020a), respectively, on CIFAR-10 using WideResNet.
In terms of reliable uncertainty estimates, it is either the best or, in a few cases, extremely competitive with respect to the best performing one, with only slightly inferior performance.
Overall, our experiments show that
Mix-MaxEnt is by far the best performing one compared to the existing single model approaches.
We would like to highlight that, along with its effectiveness, one of the core strengths of our approach is its simplicity.
As opposed to the recently proposed SNGP and DUQ (van Amersfoort et al., 2020), it does not require any modiﬁcations to the architectures and does not trade accuracy in order to improve uncertainty estimates.
And, as opposed to the extremely competitive DE, it is a single deterministic model, hence, extremely efﬁcient.
R ELATED W ORKS
Modern NNs have been shown to be miscalibrated (Guo et al., 2017), implying, a mismatch between model’s conﬁdence and its accuracy.
While recent approaches have been proposed to ﬁx this issue on in-distribution data (Guo et al., 2017; Mukhoti et al., 2020; Lakshminarayanan et al., 2017), such calibrated models do not necessarily behave as well under domain-shift (Ovadia et al., 2019).
Uncertainty estimation methods The current state-of-the-art in producing reliable uncertainties is the extremely expensive Deep Ensembles (Lakshminarayanan et al., 2017), whose training and inference costs scale linearly with the number of members in the ensemble.
This signiﬁcantly limits its applicability in real-world applications, where time and compute are of the essence.
But even in this case, the accuracy and calibration dramatically drop when facing data-shift.
A current research trend is to either emulate deep ensemble behaviour while avoiding its disadvantages (Havasi et al., 2021; Wen et al., 2020; Huang et al., 2017) ) or combining it with other methods (Ashukha et al., 2020; Rahaman & Thiery, 2020; Wen et al., 2021).
An alternative approach to obtain a reliable predictive distribution is to rely on Bayesian NNs (Chen et al., 2014;
Zhang et al., 2020; Durmus et al., 2016).
However, obtaining the exact posterior for
NNs is computationally infeasible, for this reason, it is necessary to rely on approximate inference schemes whose effectiveness is in question (Hron et al., 2018; Foong et al., 2020).
A recent trend in literature has tried reducing the application of Bayesian inference only to the last layers to get the beneﬁt of the effectiveness of training deterministic networks with standard training procedures, and the improved uncertainty estimation provided by Bayesian approaches ( e.g. SNGP (Liu et al., 2020a),
KFAC-LLLA (Kristiadi et al., 2020), BM (Joo et al., 2020), DUQ (van Amersfoort et al., 2020)).
Unfortunately, many of these models often trade accuracy (due to changes in architecture or training procedures) in exchange of marginal improvements in uncertainty estimation, and are hard to use in a plug-and-play fashion since they are extremely sensitive to hyperparameter tuning.
Learning better features via augmentation or normalisation It has been shown (Shah et al., 2020) that if NNs can identify simple (spurious) features to separate training data, they will prefer those to semantically meaningful features.
NNs also heavily rely on textures (Geirhos et al., 2018).
This agrees with recent literature, that has shown self-supervision (Winkens et al., 2020;
Hendrycks et al., 2019) or simple augmentation techniques like mixup (Thulasidasan et al., 2019) can produce representations that yield better uncertainty estimation.
A way to make NNs insensitive to noise is imposing constraints on their sensitivity.
An extensive literature studies how controlling Lipschitzness impacts on the robustness and generalization performance (Bartlett et al., 2017; Neyshabur et al., 2018; Yoshida & Miyato, 2017; Gouk et al., 2020; Arora et al., 2018).
A NALYSIS AND OBSERVATIONS
Note, both corrupted and OOD embeddings lie within the hypersphere S as their histograms are left shifted relative to the histogram of CIFAR-10.
Also, the SVHN histogram is more shifted towards the center than that of CIFAR-100, showing an implicit ordering.
Consider the supervised task of learning a predictive distribution p ( y | x ; θ ) , where y ∈ { 1 , · · · , K } is the set of K-classes, x ∈ X ⊂ R p is the space of input data, and θ respresents the parameters of the underlying model ( e.g. a neural network).
To obtain an optimal θ , the standard approach is to collect a training data set D
= { x i , y i } ni
=1 , and maximize the loglikelihood p ( y | x ∈ X ; θ ) under D .
Therefore, the nature of the predictive distribution heavily relies on the training data and the model under consideration.
In practice, however, because of the high dimensionality of the input space X , the training data D is normally collected from a subset X I ⊂ X , where X I denotes the in-domain data manifold.
Let us denote the out-domain data manifold as X O =
X \ X I .
Where do neural networks map X O ?
Modern softmax-based neural classiﬁers operate under two essential assumptions: (1) a closed-world assumption (Bendale & Boult, 2015); (2) an implicit clutering assumption (Hess et al., 2020; Lee et al., 2018).
The closed-world assumption implies that the classiﬁer must select one out of K classes ﬁxed at training time, even if the input belongs to none of them.
The clustering assumption depends on the presence of the softmax layer: the authors of (Hess et al., 2020) have shown that softmax neural classiﬁers perform K-means clustering (which, we recall, can be interpreted as a limit subcase of Expectation-Maximisation under Gaussian Mixture
Model (GMM) assumptions (Hastie et al., 2001; Bishop, 2006)) with K centroids at equal distance from the center of the embedding space.
The authors of (Lee et al., 2018) observed the embeddings of modern neural network do empirically adapt well to GMM assumptions 1 .
Following the above insights, we consider a WideResNet (WRN) trained on CIFAR-10 ( X I ) and obtain the smallest hypersphere S in the feature space that contains all the in-domain test samples.
Note, as already mentioned in Hess et al.
, we also found the mean of the embeddings of X I to be very close to a zero vector, therefore, S is practically centred at zero.
Given S , we would like to understand where do X O (CIFAR-100 and SVHN) and data-shifted samples (corrupted CIFAR-10 test samples denoted as CIFAR-10-C) lie with respect to S .
These are our observations:
• The NN maps all (except one) out-of-distribution ( X O ) and data-shifted samples (corrupted) inside the hypersphere S that was obtained using the test data belonging to X I .
• There exist an implicit ordering in the sense that CIFAR-100 is mapped closer to CIFAR10 than SVHN.
This is clearly shown in the histogram (refer Figure 2) of the Euclidean distances of the OOD and data-shifted embeddings from the center of the hypersphere S .
This behaviour can be explained by the fact that convolution kernels are learned using X I , therefore, will produce stronger activations if the similarity between the kernels learned on X I and the features present in the input is high.
On the other hand, dissimilarity between kernels and features present in the input will produce weaker activations, thus driving the embeddings closer to the origin.
Forcing classiﬁers to be uncertain where it truly matters Recall that an ideal classiﬁer’s predictive distribution p ( y | x ∈ X ; θ ) would assume the following form
[Equation]
Since X O is unknown during the maximum likelihood estimation (MLE) of θ , the model is completely unaware of the second part of the predictive distribution that depends on the presence of
X O .
Therefore, in order for the model to be indecisive for out-of-distribution and domain-shifted samples (assuming they are away from the in-distribution data), it is desirable that the model’s uncertainty increases proportionally to the test sample distance 2 from the training data X I .
This is the prime motivation behind the recent work called SNGP (Liu et al., 2020a) where the uncertainty estimates of a NN is being improved by taking inspirations from the classic Gaussian Processes literature (Rasmussen & Williams, 2005).
While techniques like SNGP (Liu et al., 2020a) and
KFAC-LLLA (Kristiadi et al., 2020) try to increase the entropy everywhere away from the training data, they need to resort to crude approximations of the theoretical models that provide such property (both) and also have to modify the network’s architecture (SNGP).
Most approaches also compromise with the training accuracy because of such approximations.
Given the observations from our analysis, we ﬁnd that the OOD and domain-shifted samples are mapped to a speciﬁc embedding space S where the network tends to concentrate all its embeddings.
This indicates that correcting its uncertainty estimates within this space S could be sufﬁcient to achieve better performance, rather than trying to increase entropy everywhere away from the training data (i.e. mostly in regions where the network never projects its inputs).
This is precisely the motivation behind our work.
In what follows, we present an extremely simple approach to regularize the embedding space S such that entropy barriers are being created between different classes.
M ETHODOLOGY
Mix-MaxEnt Motivated by the evidence we collected, we deﬁne a simple regularization approach to make the maximum likelihood estimate aware (increasing the uncertainty and becoming less conﬁdent) of what is unknown to it during the training by simply putting a maximum entropy regularizer (Pereyra et al., 2017) on synthetically generated samples that leverage the knowledge of X I to drift towards X O .
Speciﬁcally, since X O is unknown during the training, we take a simple approach where we synthetically create samples ¯ x ∈ X as follows:
[Equation]
Here, { x i , x j } ∈ X I , and λ 0 ∼ Beta ( α, β ) , where α and β are the parameters of the beta distribution.
Note, we only choose pair of samples belonging to two different classes ( y i (cid:54) =
y j ) , so that ¯ x will mix features mimicking an off-data manifold image that has intermediate properties between those of each class.
We further ensure this by picking α = β >> 1 as, in this case, the beta distribution will have the peak in the middle and the sharpness of the peak increases as α grows.
Let us call the collection of such synthetic intermediate samples ¯ X O .
Then, our ﬁnal objective function adds to the usual cross-entropy optimisation term, an entropy maximisation term on such samples:
[Equation]
where, H ¯ y ( . ) is the entropy deﬁned over the label support set ¯ y
=
{ y i , y j } .
It is important to observe that since the input image will contain features coming from the inputs of the two classes, it is reasonable to maximise the entropy only over such classes while keeping the probability assigned to the other classes to zero (since no input features endorses the presence of these classes).
Indeed, in our experiments we show that maximising the entropy over all the classes severely degrades the performance while making the network underconﬁdent.
Since we increase the entropy only for mixed (interpolated) samples from different classes, we call our approach Mix-MaxEnt .
Mix-MaxEnt vs mixup (Zhang et al., 2018)
Though the sample interpolation in our approach is similar to the one presented in mixup , there are fundamental differences between the two.
The mixup training goal is to reduce memorization by training around the vicinity of the samples (vicinal risk minimization).
Effectively, it slightly perturbs the data and uses its perturbed version for the loglikelihood estimation.
{+It optimises − log p (˜ y | ¯ x ) where ˜ y = λ 
+}
y i +(1 − λ 0 ) y j , ¯ x = λ 0 x i +(1 − λ 0 ) x j , and y i and y j might be the same.
Practically speaking, a small value of Beta distribution parameters ( α = β < 1 ) is typically chosen for mixup (Zhang et al., 2018; Thulasidasan et al., 2019; Wen et al., 2021), thus, λ 0 in this case is either ≈ 0 or ≈ 1 . Because of this, one of the two interpolating samples heavily dominates the interpolated sample ¯ x .
Note, this is necessary for mixup as otherwise the interpolated sample distribution would be different from the original data-distribution and, because of the absence of the log-likelihood term over the clean samples (as in Mix-MaxEnt), there will be a drop in the performance at test time.
However, Mix-MaxEnt (refer Eq (2)) uses clean unperturbed samples for the log-likelihood estimation while interpolated samples are being used to regularize the network so that it prefers a solution having high-entropy at regions in-between samples of different classes. Since the purpose of the interpolated samples here is only to guide the log-likelihood solution, thus, as opposed to mixup, a high interpolation factor can be used in this case.
Hence, we use λ 0 ≈ 0 . 5 to interpolate pair samples from different classes so that the interpolated sample mimic points from outside the data-manifold for entropy maximization.
E XPERIMENTS
Training Datasets and Network Architectures We employ WideResNet28-10 (WRN) (Zagoruyko & Komodakis, 2016) and ResNet50 (RN50)
(He et al., 2016) architectures that have been shown to produce state-of-the-art classiﬁcation accuracies on real-world datasets.
We train them on CIFAR-10 (C10) and CIFAR-100 (C100).
For Domain-Shift experiments, we resort to the widely used CIFAR10-C and CIFAR100-C, corrupted versions of C10 and C100 (Hendrycks & Dietterich, 2019).
For Out-of-Distribution detection experiments, following SNGP (Liu et al., 2020a), we use
C100 and SVHN as OOD for models trained on C10.
Similarly, for models trained on C100, we use
C10 and SVHN as OOD.
Methods considered for comparisons We consider both deterministic and Bayesian approaches for comparison.
Following (Liu et al., 2020a), we also create two additional strong and simple baselines where a ResNet is enforced to be bi-Lipschitz using Spectral Normalization (SN) (Miyato et al., 2018a) and Stable Rank Normalization (SRN)
(Sanyal et al., 2020).
Note, we are the ﬁrst to consider SRN for these experiments as it induces more compact clusters in the feature space than
SN (we provide a simple mathematical proof of this in the Appendix A).
Therefore, we compare our approach with the following baselines:
• DNN: Standard deterministic neural network trained using cross-entropy loss.
• DNN-SN:DNN with SN (Miyato et al., 2018a).
• DNN-SRN: DNN with SRN (Sanyal et al., 2020).
• SNGP:
Spectrally Normalized Gaussian Process (Liu et al., 2020a).
• DUQ: Deterministic Uncertainty Quantiﬁcation (van Amersfoort et al., 2020).
• Mixup: Standard Mixup training (Zhang et al., 2018).
• KFAC-LLLA: KFAC-Laplace Last Layer Approximation (Kristiadi et al., 2020).
A method that makes a model Bayesian at test time by taking Laplace approximation of the last layer using a Kronecker-Factored approximation (Ritter et al., 2018).
For the sake of completeness, we provide a simple outline of this approach in Appendix B.
[Table] [Table]
DE: Deep Ensembles (Lakshminarayanan et al., 2017) with 5 members.
Note,
is almost 5x slower than all other approaches mentioned above.
Code base For fair comparisons, we developed our own code base for all the approaches mentioned above (except SNGP and DUQ) and performed an extensive hyperparameter search to obtain the strongest possible baselines.
For SNGP, we used the available code and made sure that we follow exactly the same procedure as mentioned in their original paper.
For DUQ, the original paper did not perform large scale experiments similar to ours.
Unfortunately, we could not manage to make their code work on C100 as it exhibited unstable behaviour.
For this reason, we borrowed numbers for
DUQ from the SNGP paper.
Please note that the authors of SNGP performed non-trivial modiﬁcations to the original DUQ methodology to make it work on C100.
Further details provided in Appendix C.
Optimization and Hyperparameters We use SGD with Nesterov momentum 0 .
9 and a weight decay of 5 × 10 − 4 .
For WRN, we apply a dropout p = 0 .
1 at train time.
We perform extensive cross-validation of all the hyperparameters for all the baselines.
Details provided in Appendix C.
Evaluation Metrics For calibration, we employ: (1) the widely used Expected Calibration Error (ECE) (Guo et al., 2017), and (2) the recently proposed Adaptive ECE (AdaECE) (Mukhoti et al., 2020).
For all the methods, the ECE and AdaECE are computed after performing temperature scaling (Guo et al., 2017) with a cross-validated temperature parameter.
Metrics and uncertainty measures used for out-of-distribution detection are discussed in detail in Section 5.1.3.
5. E XPERIMENTAL R ESULTS 5.1. A CCURACY AND C ALIBRATION ON C LEAN D ATA
Our ﬁrst set of experiments evaluates the accuracy of the proposed method on the test sets of C100 and
C10.
As it can be observed from the ‘Clean Data’ column of Tables 1 and 2, our method provides a remarkable increase in accuracy compared to recently proposed approaches for improved uncertainty estimation.
For example, in the case of C100, it is 4 .
03% and 1 .
67% better than SNGP and KFAC, respectively.
Most of the time, while improving uncertainty estimates, such approaches degrade the accuracy.
However, Mix-MaxEnt not only provides much improved accuracy compared to these approaches, it also shows either comparable or improved accuracy compared to DE as well.
Note,
DE is computationally much more expensive than other approaches both at training and test time.
In terms of calibration, our method remarkably improves the ECE and AdaECE outperforming all the baselines.
Mixup turns out to be an extremely strong baseline in this regard.
We provide ECE and AdaECE without temperature scaling in Appendix D. Due to lack of space, results using the
ResNet50 architecture are presented in Appendix D.
5.1. A CCURACY AND C ALIBRATION ON C ORRUPTED D ATA (D OMAIN -S HIFT )
To evaluate the behaviour of various models under domain-shift, we resort to the widely used CIFAR100-C and CIFAR-10-C datsets, corrupted versions of the C10 and C100 datasets (Hendrycks &
Dietterich, 2019).
The dataset is made by applying 15 synthetically generated but realistic corruptions at 5 degrees of intensity on the test sets of C100 and C10, respectively.
The desired behaviour would be to preserve the classiﬁcation accuracy as much as possible as these corruptions do not impact the underlying label, and to have an appropriate reduction in the conﬁdence when the accuracy of the model degrades so that it is not incorrect with very high conﬁdence.
We report the expected accuracy, ECE and AdaECE, averaged over all the corruptions and degrees of intensities in the column ‘Domain-Shift’ of Tables 1 and 2.
It is evident that our approach produces a remarkable improvement in the average accuracy compared to all the baselines.
For instance, as shown in Table(for C100), our method achieves an accuracy improvement of 6.85% over DNN, of 3.8% over DE, and of 2.4% over Mixup.
Similarly, in Table 2 (for C10), our method achieves an improvement of almost 6.5% over DNN, of 4.78% over DE, of 1.42% over Mixup.
Thus, setting new state-of-the-art with signiﬁcant improvements.
In terms of calibration, Mixup outperforms all the approaches for
C10 experiments.
Mix-MaxEnt turns out to be a strong runner-up in this case performing at least as good as the expensive DE and outperforming SNGP and KFAC.
However, for C100 experiments, our approach again outperforms all the baselines.
For instance, in terms of ECE (refer Table 1),
Mix-MaxEnt obtains a 4.5% improvement over DE, 2.4% over Mixup, and 2.5% over SNGP.
5.1. P ERFORMANCE WHEN EXPOSED TO O UT -O F -D ISTRIBUTION S AMPLES
Following the standard evaluation methodology (Liu et al., 2020a), we report the performance in terms of Area Under Receiver Operating Characteristic (AUROC) and Area Under Precision-Recall (AUPR) curves for the binary classiﬁcation problem between in- and out-distribution samples.
The uncertainty of the prediction of the model is normally used to obtain these curves.
Given an uncertainty measure, it is important for models to be able to provide reliable uncertainty estimates to obtain good AUROC and AUPR.
However, there is no consensus in the literature regarding which uncertainty measure to use.
Choice of the measure itself can have signiﬁcant impact on the quality of the uncertainty estimation.
In fact, there are various such metrics and they are not yet compared and evaluated properly on large scale experiments.
To ﬁll this gap, we performed extensive experiments with the following metrics to understand their behaviour and choose the one that is stable and provides the strongest possible baseline:
• Entropy : H ( p ( x ))
= − (cid:80)
Ki =1 p i log p i .
• Dempster-Shafer
(Sensoy et al., 2018): DS ( x ) =
K / K + (cid:80)
Ki =1 exp( s i ) .
• Energy : E ( x ) = − log (cid:80) Ki =1 exp( s i ) (ignoring the temperature parameter).
This metric was used in (Liu et al., 2020b) for OOD.
• Maximum Probability Score : MPS ( x ) = max i p i .
• Feature Space Density Estimation (FSDE):
Assuming that the features of each class follow a Gaussian distribution, there are several ways one can estimate the belief of a test sample belonging to in-distribution data and treat it as a measure of uncertainty.
One such approach is to compute the Mahalanobis score arg min i ∈
y ( φ ( x ) − µ i ) T Σ − 1 i ( φ ( x ) − µ i ) , where µ i and Σ
i are class-
wise mean and the covariance matrices of the train data.
We deﬁne and use other variants as well: a detailed discussion is provided in Appendix E.
Remarks regarding various metrics: We would like to highlight a few important observations that we made regarding these metrics.
(1) DS and E are equivalent as they are both decreasing functions of (cid:80) K i =1 exp( s i ) , and since log does not modify the monotonicity, both will provide the same ordering of a set of samples.
Hence, will give the same AUPR and AUROC values.
(2) We observed DS and H to perform similarly to each other except in a few situations where DS provided improved results for all the approaches.
(3) MPS , in many situations, was worse.
(4) We found
Gaussian assumption based density estimation to be highly unstable .
Though it provided extremely competitive results for C10 experiments, sometimes slightly better than the DS based scores, it performed very poorly on C100.
We found this score to be highly unstable as it involves large matrix inversions.
We applied the well-known tricks such as perturbing the diagonal elements and the low-rank approximation with high variance-ratio, but the results were extremely sensitive to such stabilization and there is no clear way to cross-validate these hyperparameters.
We provide analyses based on this metric and its variants in Appendix E. Based on these ﬁndings, we report all the results in the main paper using DS 3 , and report the performance using other scores in Appendix E.
OOD results:
In the case of SVHN as the OOD dataset (refer Tables 1 and 2 ‘Out-of-Distribution’ column), our method either outperforms all the existing approaches ( 2 . 54% higher AUROC than the runner-up SNGP in C100 experiments) or it turns out to be a runner-up with the gap of 1 . 31% compared to the top performing and expensive DE.
Note, Mix-MaxEnt outperforms all the single model approaches in these experiments as well.
In the case of CIFAR as the OOD dataset, DE turns out to be the best performing one.
Mix-MaxEnt, along with KFAC-LLLA, is the runner-up in the case of C100 experiments (refer Table 1) while SNGP is the runner-up in C10 experiments (refer Table 2).
5.1. W HICH METHOD IS THE BEST PERFORMING ONE ?
We presented extensive results using a variety of experimental scenarios and metrics.
In most of the experiments, Mix-MaxEnt outperformed all the approaches including DE.
However, there indeed were a few cases where it was not the best performing one.
Following observations further suggest that Mix-MaxEnt is the best performing approach compared to the existing single deterministic alternatives.
Out of 20 different evaluations presented in Tables 1 and 2:
• Mix-MaxEnt outperformed all the approaches (including DE) in 11 of them.
• Mix-MaxEnt outperformed all the single model approaches in 14 scenarios.
• No existing single deterministic baseline outperformed all other approaches in more thanscenarios.
5. A NALYSING M IX -M AX E NT 5.2. M IX -M AX E NT ENCOURAGES COMPACT AND SEPARATED CLUSTERS
We use the well known Fisher criterion (Bishop, 2006, Chapter 4) to quantify the compactness and separatedness of the feature clusters for various models.
Let C k denotes the indices of samples for k -th class.
Then, the overall within-class covariance matrix is computed as S W = (cid:80) K k =1 S k , where
S k
= (cid:80) n ∈C k ( φ ( x n ) − µ k )( φ ( x n ) − µ k ) (cid:62) , µ k
= (cid:80) n ∈C k φ ( x n ) N k , and φ ( x n ) denote the feature vector.
Similarly, the between-class covariance matrix can be computed as S B = (cid:80)
Kk =1 N k ( µ k − µ )( µ k − µ ) (cid:62) ,
where µ = 1 N (cid:80) Kk =1 N k µ k , and N k is the number of samples in k -th class.
Then, the Fisher criterion is deﬁned as α = trace ( S − 1 W S B ) .
Note, α would be high when the within-class covariance is small and between-class covariance is high, thus, a high value of α is desirable.
In Figure 3, we compute α over the C10 dataset with varying degrees of domain-shift.
As the amount of corruption increases, α gradually decreases for all the models, indicating that the model is not able to differentiate different classes anymore and is projecting them too close to each other.
This also explains why the accuracy and the calibration of all the models decreases as the domain-shift increases.
However,
Mix-MaxEnt consistently provides the best α .
This also explains why Mix-MaxEnt performed so well under domain-shift.
We provide further plots and visualizations in Appendix J.
5.2. M IX -M AX E NT CREATES A HIGH ENTROPY BARRIER BETWEEN CLASSES
In Figure 3, we provide heat-maps to visually show the entropy barrier.
The heat-map is created as follows.
We randomly choose 1000 pairs of samples { x i , x j } such that y i (cid:54) =
y j .
For each pair, we synthesize 20 samples ¯ x s using Eq (1) where λ 0 is equally spaced between 0 to 1.
The heat map is then created using all the 20K samples.
The intensity of each ( λ , H) bin in the heat-map indicates the number of samples in that bin.
It is clearly visible that for standard DNN, although an entropy peak between the two classes exist, the location of the peak is scattered and its intensity is inconsistent.
For most of the interpolated samples the entropy is close to zero, thus, making the model overconﬁdent even for samples with very high degree of interpolation.
However, Mix-MaxEnt clearly creates a high-entropy barrier that spans most of the space between the two classes.
At the same time, the entropy close to the clean samples is low.
Therefore, the model is conﬁdent for clean samples and it becomes indecisive as the degree of interpolation increases.
We show additional such plots and an alternative visualization of this phenomenon in Appendix F.
5.2. A NALYSING DIFFERENT FACTORS IN THE OBJECTIVE FUNCTION OF M IX -M AX E NT
The importance of mixing samples between classes We empirically observe that not imposing the constraint y i (cid:54) =
y j does not produce signiﬁcantly different performance (Table 13 in Appendix G).
This is because of the fact that the probability of sampling pairs with the same class is extremely low (0.1 for CIFAR-10, 0.01 for CIFAR-100) compared to sampling pairs from different classes.
However, as we enforce the constraint that the pairs must belong to the same class
y i =
y j , the performance degrades signiﬁcantly (refer Table 14).
Therefore, mixing samples between classes is crucial.
The importance of having an interpolation factor close to 0.5 In Table 11, we show that MixMaxEnt performance relies on having interpolation factors close to 0.5.
Mix-MaxEnt with α <<
(i.e. making λ 0 ≈ 0 or ≈ 1 ) degrades the OOD detection performance, the data-shift robustness, and the calibration with respect to the case α >> 1 (i.e. making λ 0 ≈ 0 . 5 ).
The importance of maximising the entropy deﬁned over the support of the interpolated points
Recall that Mix-MaxEnt (refer Eq (2)) maximises the entropy on the two interpolated classes.
The rationale is that the input image will contain features from the two classes, not from any other classes.
Therefore, the classiﬁer should assign a non-zero probability only to these classes and there is no reason for it to assign a non-zero probability to other classes.
We empirically check that this is a sensitive choice as maximising the entropy over all the classes signiﬁcantly degrades the performance (refer Table 12).
We also observed that, since α >> 1 (and hence λ 0 ≈ 0 . 5 ), the entropy regularizer can be replaced by the Mixup loss with no performance degradation.
C ONCLUSION
We proposed Mix-MaxEnt, an extremely simple approach that regularizes a neural network to be uncertain in regions of the data manifold that are unknown during training.
We conducted a wide range of experiments to show that Mix-MaxEnt signiﬁcantly improves the reliability of uncertainty estimates of deep neural networks, while also providing a notable boost in the accuracy.
A potential extension of our work regards the possibility of mixing features using more sophisticated methods.
An interesting future direction would be to elaborate more sophisticated mixing techniques either based on data statistics (e.g. using PCA directions) or latent space geometry that would allow to map out-of-distribution points closer to the center of the embedding space, and to further improve the clustering for data-shift examples.
Ethics Statement.
The fact that neural networks can be wrong with high conﬁdence poses a potential threat to the deployment of deep learning systems into the real-world applications such as driverless cars and medical imaging.
Therefore, ﬁnding fast and effective methods to train models that can provide reliable uncertainty is of utmost importance.
Hence, we are not particularly concerned that the research presented in this paper can lead to harmful applications – on the contrary, we believe that it can help deploying safer machine learning applications.
Reproducibility Statement.
In this paper, we provide an extremely simple approach that can be implemented in merely a few additional lines of code.
We have provided all the details needed to reproduce our and baseline results.
All the training hyperparameters and details about the dataset (we only used publicly available datasets) are provided in the manuscript.
To facilitate further research on this topic, we will release the code to reproduce all the experiments
A T HE EFFECT OF B OUNDING L OCAL S MOOTHNESS ON M IX -M AX E NT
Let f θ = g ◦ h ( x ) be the neural network where h : x (cid:55)→ φ ( x ) is the feature extractor and g ( . )
the last linear layer.
It is desirable that the predictive distribution is robust to input perturbations that do not alter the label of the image, so that the network is able to preferentially carry over the true signal in the data.
In other words, we would like our model to be bi-Lipschitz so that, given an input perturbation, the change in the output is both upper and lower bounded.
Of course, these bounds should be reasonable so that the network neither looses its capacity (is trainable), nor it is too sensitive, which otherwise would cost its generalization ability.
It has also been shown that ResNet architectures, because of their skip connections, are by design bi-Lipschitz if all the linear operators are constrained to have a spectral norm of less than one (Behrmann et al., 2019).
Recent work (Liu et al., 2020a) built on this ﬁnding and showed that applying Spectral Normalization (SN) (Miyato et al., 2018b) to a standard neural network does improve the quality of its uncertainty estimates.
As an alternative, in this work, we enforce bi-Lipschitzness via the application of Stable Rank
Normalization (SRN) (Sanyal et al., 2020).
Let W be a linear operator with { σ 1 , · · · , σ k } as its ordered set of non-zero singular values.
Then, SRN effectively provides a new matrix whose singular values are { 1 , η σ 2 σ 1 · · · , η σ k σ 1 } , where η ∈ (0 , 1] .
Note, if η = 1 , SRN boils down to SN.
Mathematically speaking, along with controlling the largest singular value of a linear operator, the
SRN objective also allows us to minimize its stable rank (a softer version of the rank operator) 4 .
We prefer SRN for the following two reasons.
(1) It has been shown to control noise sensitivity, and provide better data-dependent empirical Lipschitz constant.
(2) It encourages learning more compact embeddings in the feature space (we prove in Proposition A.1), a property that is shown to be important to obtain improved out-domain detection performance (Winkens et al., 2020).
Proposition A.1.
Stable Rank Normalization (Sanyal et al., 2020) (SRN) encourages low volume clusters compared to Spectral Normalization (SN).
Proof.
Let { v i ∈ R n } ni =1 be a set of zero-mean vectors with empirical covariance matrix as S =n (cid:80) ni
=
v i v (cid:62) i .
Let W 0 be a linear mapping with { σ i } ki =1 as its ordered non-zero singular values ( σ 1 being the max).
Let the SN and SRN versions of W 0 be W 1 and W 2 , respectively.
Then, the volume of the covarinace matrix when each vector v i is mapped using W 0 can be obained as
[Equation]
(ignoring the multiplicative constant as all the matrices are in the same space).
Note, we abuse the notation where the determinant det ( . ) of a matrix is the product of non-zero singular values.
Similarly, the volume of the projected points when the vectors are mapped via W 1 and W 2 can be obtained as Vol ( S 1 ) = 1 σ 2 k 1 (cid:81) ki
=1 σ 2 i det( S ) and Vol ( S s ) =
η 2(
[Equation]
1) σ
[Equation] [Equation]
i det( S ) , respectively.
Since η ∈ (0 , 1] for SRN, it is trivial to note that Vol ( S 2 )
≤ Vol ( S 1 )
≤ Vol ( S 0 ) .
Hence, the clusters obtained using SRN will have the lowest volume.
A. C ONSIDERING DIFFERENT TECHNIQUES TO BOUND LOCAL SMOOTHNESS
In our experiments, we considered two techniques to bound local smoothness: (1) using Spectral
Normalization (SN) (Yoshida & Miyato, 2017), (2) using Stable Rank Normalization (SRN) (Sanyal et al., 2020).
We also considered a variant of our method without these two components.
We report the performance of these variants in Table 3.
It is important to observe that, although in a few cases
SN can outperform SRN, in most cases they do not exhibit signiﬁcantly different behaviour.
We also observe that, in the case of ResNet50, bounding local smoothness via SN/SRN does not produce improvements (as already visible in Tables 5 and 6).
This probably can be attributed to the high capacity of the network and the (relatively) small quantity of training data in the datasets considered.
A. T HE IMPACT OF SRN ON DATA - SHIFT ROBUSTNESS
In this section we report additional plots showing the impact of SRN when applied on top of the maximum entropy regularizer.
As it can be seen in Figures 4 and 5, for certain kinds of noises it signiﬁcantly improves the robustness to data-shift.
[Table]
Evaluation of our method considering a version without any regulariser to bound local smoothness (Mix-MaxEnt), using Spectral Normalization (Mix-MaxEnt-SN) and using Stable Rank
Normalization (Mix-MaxEnt-SRN).
The hyperparameters are set as in Table 4.
B K RONECKER -F ACTORED L AST L AYER L APLACE A PPROXIMATION
A structural problem of using MLE logistic regression is that the produced uncertainties depend on the decision boundary.
On the other hand, replacing the MLE logistic regression with a Bayesian logistic regression and estimating the predictive posterior employing a Laplace approximation allows to produce better uncertainties (Kristiadi et al., 2020).
However, a Bayesian training either requires a modiﬁcation in the architecture (Liu et al., 2020a) or makes the inference procedure very expensive (Kingma et al., 2015; Gal & Ghahramani, 2016).
Since the objective is to utilize the standard MLE training of neural networks, the idea of Kronecker-Factored Last Layer Laplace
Approximation (Kristiadi et al., 2020) is making the network Bayesian at test time with almost no additional cost.
Let w be the parameters of the of the last layer of a neural network, then we seek to obtain the posterior only over w .
Let p ( w | x ) be the posterior, then the predictive distribution can be written as:
[Equation]
where, s is the logit vector and softmax ( s k ) is the k -th index of the softmax output of the network.
The Laplace approximation assumes that the posterior p ( s |D ) ∼ N ( s | µ, Σ) , where µ is a mode of the posterior p ( w |D ) (found via standard optimization algorithms for NNs) and Σ is the inverse of the Hessian H − 1 = − ( ∇ 2 log p ( w |D ) | µ ) − 1 .
For the formulations and deﬁnitions, including the variants with the terms associated to the bias, we refer to (Kristiadi et al., 2020).
For our experiments, we obtain Σ using the Kronecker-factored (KF) approximation (Ritter et al., 2018).
Broadly speaking, the KF approximation allows to reduce the computational complexity of computing the Hessian by factorizing the inverse of the Hessian as H − 1 ≈ V − 1 ⊗ U − 1 , then the covariance of the posterior evaluated at a point x takes following form Σ
= ( φ ( x ) T V φ ( x ))
U .
This procedure can be easily implemented using the Backpack library (Dangel et al., 2020) to compute V and U by performing a single pass over the training set after the end of the training, as detailed in the Appendix of (Kristiadi et al., 2020) and clearly exempliﬁed in the code-base of (Hobbhahn et al., 2021).
Let Σ k be the covariance matrix of the posterior over the last linear layer parameters for the kth class obtained using the Laplace approximation around µ , then, given an input x , we obtain σ k = φ ( x ) (cid:62)
Σ k φ ( x ) representing the variance of k-th logit s k .
Once we obtain the covariance matrix, the Monte Carlo approximation of the predictive distribution (equation (3)) is obtained as:
[Equation]
where, m logit vectors s ( i ) are sampled from a distribution with mean s and a covariance matrix (depending on the approximation used).
Lu et.
al (Lu et al., 2020) showed that similar performance can be achieved via the mean-ﬁeld approximation which provides an approximate closed form solution of the integration in equation (3) involving the re-scaling of the logits and then taking the softmax of the re-scaled logit.
The re-scaling is deﬁned as follows:
[Equation]
Note, the scaling of the k-th logit depends on its variance (obtained using the Laplace approximation) and a hyperparameter λ .
This approximation is efﬁcient in the sense that it does not require multiple samples as required in the MC approximation (which can become expensive as the number of classes and samples grow).
In our experiments, we use the MC approximation, since we could not ﬁnd an obvious way to ﬁne-tune λ .
Additionally, we observe that the mean-ﬁeld approximation imposes a trade-off between calibration and OOD detection performance.
Increasing λ , indeed, ﬂattens the softmax distribution and improves OOD detection scores; although, as a consequence, harms calibration by making the network underconﬁdent.
C E XPERIMENT D ETAILS
Hyperparameters For all our experiments we set the batch size to 128 5 .
At training time, we apply standard augmentation (random cropping and horizontal ﬂipping, similarly to (Liu et al., 2020a)).
The data is appropriately normalized before being fed to the network both at train and test time.
• for DNN-SN, DNN-SRN and Mix-MaxEnt the set of SN clamping factors we considered in our experiments are c ∈ { 0 . 5 , 0 .
75 , 1 . 0 } , the target of stable rank r ∈ { 0 . 3 , 0 .
9 } (as r = 1 for SRN is the same as applying SN with c = 1 . 0 ).
• for Mixup, we considered the Beta distribution hyperparameters to be α ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } , as suggested in the literature (Thulasidasan et al., 2019).
• for Mix-MaxEnt we considered the Beta distribution hyperparameters to be α ∈ { 5 , 10 , 15 , 20 , 30 } .
In both cases we set α = β in Beta ( α, β ) .
• for the KFAC-LLLA, we took 1000 samples.
Although the number might seem quite high, we could not notice signiﬁcant improvements using a lower number of samples.
We tuned the prior variance σ 0 needed for the computation of the Laplace approximation minimising the ECE on the validation set.
We also tried using the theoretical value σ 0 = 1 /τ
(Kristiadi et al., 2020), where τ represents the weight decay, but it produced inferior results with respect to our cross-validation procedure.
• for Deep Ensembles we use 5 members.
• when temperature scaling is applied, the temperature T is tuned on the validation set, minimising the ECE (we considered values ranging from 0 .
1 to 10 , with a step size of 0 . 01 ).
For Deep Ensembles, we ﬁrst compute the mean of the logits, then scale it by the temperature parameter before passing it through the softmax.
All the hyperparameters we chose are reported in Table 4.
Such hyperparameters have been selected performing cross-validation with stratiﬁed-sampling on a 90/10 split of the training set to maximise
Accuracy 6 .
Indeed, it is important to observe that:
• optimising the hyperparameters based solely on the ECE can prefer models with lower Accuracy but better calibration.
However, Accuracy is commonly considered of primary importance, and any method improving calibration should avoid degrading it.
Optimising considering any of the corrupted experiments and OOD detection metrics would be equivalent to overﬁtting the test set (indeed, both corruptions and OOD datasets are assumed to be unknown at training time by all the considered methodologies, and so they should be during the hyperparameter selection procedure).
For all of our experiments we train the methods initialising the networks with 5 different seeds and report the average and standard deviation in percentage for all the metrics.
For the hyperparameter search, we trained approximately 350 models.
For the ﬁnal tables (ﬁve seeds), we trained approximately 250 models.
Code For the SNGP method we use the ofﬁcial code-base with the suggested hyperparameters and training procedures.
The code diverges slightly from the procedure described in their paper, hence the slight differences in the performance.
The only modiﬁcation we performed to the ofﬁcial code-base was to make the inference procedure consistent with the one described in the paper: indeed, in their code they implement a mean-ﬁeld approximation to estimate the predictive distribution (Lu et al., 2020), while in their paper they use Monte Carlo Integration with a number of samples equal to the number of members in the ensembles they use as a baseline, which provides better calibration.
The rationale is that we could not ﬁnd an obvious way to tune the mean-ﬁeld approximation hyperparameters to improve at the same time both the calibration and OOD detection performance (indeed, the mean-ﬁeld approximation imposes a trade-off between calibration and OOD detection performance).
Additionally, since the standard KFAC-LLLA uses the same Monte Carlo Integration procedure,
Except for the σ 0 of the KFAC-LLLA, as we could not observe signiﬁcant differences in Accuracy between hyperparameters optimising the Accuracy and ECE
[Table] [Table]
we opted for the latter for a fair comparison.
For the KFAC-LLLA we leverage the ofﬁcial repository 7 (Hobbhahn et al., 2021) and the Backpack library (Dangel et al., 2020) for the computation of the Kronecker-Factored Hessian.
For the SNGP ResNet50 experiments, we tried running the ofﬁcial implementation.
The ofﬁcial implementation for ResNet50 is speciﬁcally ﬁne-tuned for ImageNet, and has not been used for experiments on CIFAR.
We could not make SNGP converge to SOTA accuracy values both on CIFAR-10 and CIFAR-100.
All the other methods were implemented by us in PyTorch and the training, cross-validation and evaluation code will be made publicly available upon acceptance of the paper.
D A DDITIONAL R ESULTS D. R ES N ET 50 E XPERIMENTS
In Tables 5 and 6 we report the experimental results for ResNet50.
D. C ALIBRATION M ETRICS WITHOUT T EMPERATURE S CALING
For completeness, we report the calibration metrics over all the methods and considered datasets without temperature scaling (Guo et al., 2017) in Table 7.
We can observe that temperature scaling
[Table] [Table]
E C OMPARING D IFFERENT U NCERTAINTY M EASURES
In Section 5.1.3, we reported the OOD detection performance using the DS score, except when it damages the performance of a method (e
[Table]
g. Mixup) or when it does not yield improvements (e
[Table]
g.
KFAC-LLLA).
For completeness reasons, in Table 8 we report the performance with respect to some common uncertainty measures we deﬁned in Section 5.1.3: (1) Maximum Probability Score ( MPS ), (2) Entropy ( H ), (3) Energy Score (equivalent to Dempster-Shafer) ( DS ).
Feature Space Density Estimation scores Similar to (Winkens et al., 2020; Lee et al., 2018), we also use feature space density estimation based score as an estimate of uncertainty.
Assuming that a
Gaussian Mixture Model would estimate the density of the feature space accurately, the likelihood
[Table]
[Figure]
[Table]
DNN
[Table]
that a given sample x belongs to class c can be computed as:
[Equation]
where, µ c and Σ c are the empirical mean and the covariance-matrix of training samples of c-th class.
Using p ( y = c | x ) , following scores can be computed to estimate the uncertainty:
• Maximum Log-likelihood : m ( x ) = max c log p ( y = c | x ) .
If the score m ( x ) is low then the sample more likely belongs to OOD.
Belief : This score indicates how likely is it that the sample x belongs to IND and is computed as b ( x ) = (cid:80)
c p ( y = c | x ) .
We consider two variants of both the scores mentioned above: (1) tied-covariance ( LDA assumption ) when all the covariance matrices are tied and there is one common covariance matrix for the entire training data-set; and (2) per-class covariance matrix ( QDA assumption ) when the covariance matrix is separately computed for each class.
Note, in both the assumptions, the mean is always class-dependent.
Note, the scores deﬁned above require inverting matrices, computing logarithms and exponentials.
Practically, we observe that these computations introduce numerical instability 8 leading to unreliable uncertainty scores (although in a few cases they can achieve better performance).
In particular, the main sources of instability are caused by the inversion of the covariance matrices and by the computation of their log-determinants (hence we evaluate the scores also without considering the log-normalization term).
We apply two known procedures to make the process numerically stable:
• Diagonal perturbation by addition of (cid:15)
I for the computation of the inverse, where (cid:15) is a small positive constant.
• Low-Rank Pseudo-Inverse (LRP): preprocess the covariance matrices (see next), compute the precision as the pseudo-inverse of the pre-processed covariance (stabilised further by adding (cid:15)I ).
We preprocess the covariance matrices with the following procedure.
Let’s assume A is a positive semi-deﬁnite matrix, and call ˆ A an approximation of A that exhibits better numerical stability to inversion and log-determinant operations.
We obtain ˆ A as follows:
• compute the SVD decomposition of the A as A
= U Σ V T .
• select the top-k SVD components of A that explain the γ % of the variance.
Let’s call U k , Σ k and V k
the matrices obtained by considering only such components.
• produce an approximation ˆ A of A as ˆ A
= U k Σ k V Tk
As it can be seen from Tables 9 and 10, the metric can show outsandingly good performance in detecting OOD data in a few cases.
For instance, in Table 9, consider DNN-SRN tested on SVHN: with the LRP stabilisation and (cid:15)
= 1 e − 08 , γ = 0 .
the the belief b c can achieve an AUROC of 97.84% and an AUPR of 99.06%, which are both superior to the AUROC of 96.12% and AUPR of 98.10% reported in Table 8 with the DS metric for the same method.
However, as it is clearly visible, the behaviour of these metrics is highly inconsistent across different methods, training and test datasets and stabilisation procedures.
For instance, let’s consider some examples from Table 9.
It is evident how the m t is extremely sensitive to the choice of (cid:15) for the DNN method: the OOD performance on SVHN drops from 93.99%
AUROC and 92.21% AUPR to 73.35% AUROC and 65.90% AUPR when using (cid:15) = 1 e − 08 instead of (cid:15) =
1 e − 07 .
On the other hand, for the same variation in (cid:15) , the scores improve for DNN-SN on the same task (increasing the AUROC from 98.34% to 98.40% and the AUPR from 97.05% to 97.12%).
Another example of such extreme sensitivity and inconsistency of the metric across various stabilisation techniques and methods can be observed when comparing the b c , m c and m t performance using the diagonal perturbation with (cid:15) = 1 e − 07 or the LRP with same (cid:15) and γ = 0 .for
DNN and DNN-SN when tested on SVHN.
For DNN, all the scores increase when adding the
LRP.
For DNN-SN, the b c increases dramatically (from 76.89% AUROC to 95.35% AUROC, from 88.44% AUPR to 97.89% AUPR) but at the same time a signiﬁcant drop is observed in m c and m t (e.g. m
t drops from 98.34% AUROC to 94.99%, and from 97.05% AUPR to 92.95%).
The fact the considered stabilisation procedures can produce inconsistent improvements and degradations across various methodologies and metrics, and that such variations are extremely sensitive to the choice of the stabilisation hyperparameters makes these metrics unreliable.
Indeed, there is no stabilisation procedure with a speciﬁc set of hyperparameters that dominates the others in all the cases, and there is no obvious way to blindly (i.e. without resorting to OOD data at training time) select such procedure and hyperparameters.
[Table] [Table]
[Table]
F A DDITIONAL VISUALIZATION OF THE M AXIMUM E NTROPY EFFECT OF M IX -M AX E NT
In Figure 7 we report plots similar to the ones in Section 5.2.2.
As already explained, Mixup makes the network less conﬁdent both close and far away from the training data, for this reason it results in better calibration (as it alleviates the overconﬁdence of DNN) but worse OOD detection performance (because both IND and OOD samples have higher entropy, hence are more difﬁcult to distinguish).
An alternative visualization of the phenomenon can be observed with the following method.
We take samples from the ﬁrst three classes of C10, embed them into the feature space, and visualize them using the approach used in (Müller et al., 2019) (presented in Appendix H for completeness).
Then, we randomly select two samples x 1 and x 2 from two different classes (‘purple’ and ‘yellow’) and create a set of interpolated samples { ¯ x i =
λ x
1 + (1 − λ ) x 2 } , as λ varies from 0 to 1 with an interval of 0 .
Projecting these samples into the embedding space (Figure 6) shows that DNN projects most of the samples to either class 1 or class 2, and very few to the region of high uncertainty.
However, Mix-MaxEnt projects most of the interpolated points with λ close to 0 .
5 in high uncertainty regions.
Therefore, this plot qualitatively veriﬁes that Mix-MaxEnt creates a high entropy barrier between classes.
G R ESULTS OF ABLATIONS IN S ECTION 5.2.
In this Section, we report the results that could not be included in Section 5.2.3 for reason of space.
In particular, we report Tables 14, 13, 11 and 12.
We also discuss why Mix-MaxEnt interpolates input images and not their features.
We did experiment with interpolating IND embeddings from different class clusters and introducing a maximum entropy regularizer on such samples.
However, we observed this approach to be ineffective.
A reason could be that neural classiﬁers are notoriously not bijective and interpolations in the embedding space do not necessarily correspond to inputs that even partially resemble data coming from neither X I nor
X O (restricting X O to the manifold of natural images).
The interpolated images, instead, are bound to contain features from both classes, and hence it is reasonable for the network to be unsure about which of the two classes is represented in them.
[Table]
[Table] [Table]
= 0.
98.11 α = 0.
99.19 α = 0.
α =96. α =96.
97.37 α =96.
C100 WRN α = 0.
91.43 α = 0.
93.99 α =82.
92.89 α =82.
90.41 α =81.
86.44 α =82.
[Table]
H A LGORITHMS
In this section, we report the algorithm box for a single iteration of the training procedure of MixMaxEnt (Algorithm 1) and the visualization technique used to project the embeddings in some of the ﬁgures of the paper (Müller et al., 2019)
(Algorithm 2)
Mix-MaxEnt iteration Given a batch B , a λ 0 is sampled from Beta ( α, α ) and for each sample ( x 1 , y 1 ) our algorithm randomly selects another sample ( x 2 , y 2 )
∈ B such that y 1 (cid:54) =
y 2 and adds this sample to a batch ¯ B .
Both batches are passed through the network (with a single forward pass, with batch size 2 | B | ).
The batch B is passed through the usual cross-entropy loss, the batch ¯ B is passed through our regularizer.
The two terms are summed, and one backpropagation pass is performed.
Visualization Projection Procedure Given three classes 1 , 2 , 3 and their prototypes w 1 , w 2 , w(e.g.
the means of the embeddings of the points of each class), ﬁrst ﬁnd an orthonormal basis of the plane passing through the three prototypes by using the standard Gram-Schmidt procedure.
Build a projection matrix P ∈ R 2 × h by using the vectors of the orthogonal basis (where h is the dimensionality of the embeddings).
Use the projection matrix to project any embedding φ ( x ) ∈
R h to p x ∈ R 2 .
∅ λ 0 ∼ Beta ( α, α ) ; foreach ( x 1 , y 1 )
∈
B do
Randomly select ( x 2 , y 2 ) ∈ B | y 1 (cid:54) =
y 2 ; ¯
B ← ¯ B ∪ ( λ 0 x 1 + (1 − λ 0 ) x 2 , λ 0 y 1 + (1 − λ 0 ) y 2 ) ; end /*
Implemented with a single forward pass with double the batch size */ p
[Equation]
; p (
[Equation]
Update θ backpropagating the gradient of L ;
Algorithm 2: Computing the projection of point
( x ) given three class prototypes
2 , w
Result: p x ∈ R 2 projection of φ ( x ) ∈
R h /
* Find an orthonormal basis of the plane passing through the three prototypes
[Equation]
* Compute
[Equation]
projection matrix
[Equation]
2D
[Equation]
I E MBEDDING S PACE R ELIABILITY P LOTS
A typical technique used to analyse the calibration of a model is to inspect the reliability plot (Guo et al., 2017), i.e. a histogram that shows the relationship between accuracy and conﬁdence.
Drawing inspiration from the visualization technique of (Müller et al., 2019)
(that we described in Algorithm 2), we design a new visualization technique to analyse the calibration of a neural network.
Indeed, one of the main disadvantages of the reliability plots is they do not show which are the regions of the embedding space in which the model is overconﬁdent/underconﬁdent.
The visualization technique we develop, that we call Embedding Space Reliability Plots, leverages Algorithm 2 to compute the 2D projections of the embeddings of a test set.
For each test point x , we compute the corresponding accuracy A ( x ) and conﬁdence C ( x ) , and for each of them we compute the quantity A ( x ) − C ( x ) .
We associate to each projected point the corresponding difference between the accuracy and conﬁdence, and plot a histogram of this difference.
The histogram will reveal in which regions of the latent space covered by the test set the model is overconﬁdent ( A ( x ) − C ( x ) << 0 ) or underconﬁdent ( A ( x ) − C ( x ) >> 0 ) or well calibrated ( A ( x ) − C
( x ) ≈ 0 ).
Interesting insights can be extracted from these plots.
We report a few visualizations of this kind of plots in Figure 8.
We apply it to the test points of
CIFAR-10-C for some kinds of corruptions.
In the ﬁgures, we compare DNN versus our method.
The red regions of the space represent areas of overconﬁdence ( A ( x ) − C ( x ) < 0 ), while green regions of the space represent areas of underconﬁdence ( A ( x ) − C ( x ) > 0 ).
Yellow areas can be either calibrated or contain no data-point.
From these plots we can observe that DNN has wider over-conﬁdence regions in the space between the three classes, and shows to be underconﬁdent in some of the regions close to the means of the class clusters.
Our method alleviates this issue, making the network overall more calibrated both close and far away from the cluster centers.
We believe this to be a valuable analysis tool that can be further leveraged by other researchers to better understand the calibration behaviour of their models and to design new algorithms.
J D ETAILED ANALYSIS OF THE CORRUPTION EXPERIMENTS
In this section, we report detailed plots that break down the aggregated metric reported in Table 2 for
CIFAR-10-C and WideResNet28-10.
Speciﬁcally, in Figures 9, 10 and 11 we show how the Accuracy,
ECE and AdaECE vary across all the corruption types and intensity values (horizontal axis) overseeds.
As it can be seen, in most cases our method achieves better Accuracy than any other method.
In terms of calibration, our method is sometimes outperformed by Mixup, but we observe that Mixup exhibits lower Accuracy whenever this happens.
As the intensity of the corruption increases, all the metrics deteriorate for all the methods (as expected).
However, our method still achieves superior performance in most of the cases also in these circumstances.
To support our claim that better clustering behaviour induces better classiﬁcation performance, we report the Fisher criterion, || S W || F and || S B || F (we deﬁned in Section 5.2.1) plots for all corruptions and all intensity levels in Figures 12, 13 and 14.
The observed pattern is similar for all considered architectures and corrupted datasets, hence we report these plots only for WideResNet28-10 on CIFAR-10-C.
[Figure]