Z ERO -S HOT O UT - OF -D ISTRIBUTION D ETECTION WITH F EATURE C ORRELATIONS
Anonymous authors
Paper under double-blind review
A BSTRACT
When presented with Out-of-Distribution (OOD) examples, deep neural networks yield conﬁdent, incorrect predictions.
Detecting OOD examples is challenging, and the potential risks are high.
In this paper, we propose to detect OOD examples by identifying inconsistencies between activity patterns and class predicted.
We ﬁnd that characterizing activity patterns by feature correlations and identifying anomalies in pairwise feature correlation values can yield high OOD detection rates for far-from-distribution examples.
We identify anomalies in the pairwise feature correlations by simply comparing each pairwise correlation value with its respective range observed over the training data.
Unlike many approaches, this can be used with any pre-trained softmax classiﬁer and does not require access to OOD data for ﬁne-tuning hyperparameters, nor does it require OOD access for inferring parameters.
The method is applicable across a variety of architectures and vision datasets and, for the important and surprisingly hard task of detecting far-from-distribution outliers, it generally performs better than or equal to state-ofthe-art OOD detection methods (including those that do assume access to OOD examples).
I NTRODUCTION
Even when deep neural networks (DNNs) achieve impressive accuracy on challenging tasks, they do not always visibly falter on misclassiﬁed examples: in those cases they can often make predictions that are both very conﬁdent and completely incorrect.
Yet, predictive uncertainty is essential in real-world contexts tolerating minimal error margins such as autonomous vehicle control and medical, ﬁnancial and legal ﬁelds.
In this work, we focus on ﬂagging test examples that do not contain any of the classes modeled in the train distribution.
Such examples are often referred to as being out-of-distribution (OOD), and while their existence has been well-known for some time, the challenges of identifying them and a baseline method to do so in a variety of tasks such as image classiﬁcation, text classiﬁcation, and speech recognition were presented by Hendrycks and Gimpel (2017).
Recently, Nalisnick et al.
(2019a)
identiﬁed a similar problem with generative models: they demonstrate that ﬂow-based models, VAEs, and PixelCNNs cannot distinguish images of common objects such as dogs, trucks, and horses (i.e.
CIFAR-10) from those of house numbers (i.e. SVHN), assigning a higher likelihood to the latter when the model is trained on the former.
They report similar ﬁndings across several other pairs of popular image datasets.
While we might expect neural networks to respond differently to OOD examples than to in-distribution (ID) examples, exactly where and how to ﬁnd these differences in activity patterns is not at all clear.
Hendrycks and Gimpel (2017) and others (Nguyen et al., 2015; Yu et al., 2011) showed that looking at the maximal softmax value is insufﬁcient.
In Section 2 we describe some other recent approaches to this problem.
In this work, we ﬁnd that characterizing activity patterns by feature correlations— computed with an extension of Gram matrices that we introduce—lets us quantify anomalies to allow state-of-the-art (SOTA) detection rates on OOD examples.
Intuition.
We identify out-of-distribution examples by jointly considering the class assigned at the output layer and the activity patterns in the intermediate layers.
For example, if an image is predicted to be a dog, yet the intermediate activity patterns are somehow atypical of those seen by the network for other dog images during training, then that is a strong indicator of an OOD example.
This effectively allows us to detect incongruence between the prediction made by the network and the path by which it arrived at that prediction .
Strengths.
Unlike those previous works that assume access to OOD examples and train an auxiliary classiﬁer for identifying anomalous activity patterns, our method ﬁnds differences in activity patterns without requiring access to any OOD examples, and it works across architectures.
We hope this will also help further our understanding of how neural networks respond differently to OOD examples in general , not just how a particular network responds to examples coming from a particular distribution.
Contributions.
This work includes the following contributions:
We extend Gram matrices to compute effective feature correlations.
Using the p th -order Gram matrices, we present a new technique for computing classconditional anomalies in activity patterns.
We evaluate this technique on OOD detection, testing on • competitive architectures: DenseNet, ResNet; • benchmark OOD datasets including: CIFAR-10, CIFAR-100, SVHN, TinyImageNet, LSUN and iSUN.
Zero-shot: crucially, our method does not require access to OOD samples for tuning hyperparameters or for training auxiliary models.
Nevertheless, we report results which, for the challenging and important cases of far-fromdistribution examples, are generally better than or equal to the state-of-the-art method for OOD detection that does require access to OOD examples.
R ELATED W ORK
Previous work which aims to improve OOD detection can be roughly grouped by several themes:
Bayesian Neural Networks.
A nice early Bayesian approach (Gal and Ghahramani, 2016) estimates predictive uncertainty by using an ensemble of sub-networks instantiated by applying dropout at test time.
As opposed to implicitly learning a distribution over the predictions by learning a distribution over the weights, Chen et al.
and Malinin and Gales (2018) explicitly parameterize a Dirichlet distribution over the output class distributions using DNNs in order to obtain a better estimate of predictive uncertainty; the main differences between these methods is that Chen et al.
(2019) use ELBO, which only requires the in-distribution dataset for training whereas Malinin and
Gales (2018) use a contrastive loss which requires access to (optionally synthetic)
OOD examples.
Using any pre-trained softmax deep neural network with OOD examples.
Lee et al.
(2018b)— to the best of our knowledge, the current SOTA technique by a signiﬁcant margin—compute the
Mahalanobis distance between the test sample’s feature representations and the class-conditional gaussian distribution at each layer; they then represent each sample as a vector of the Mahalanobis distances, and ﬁnally train a logistic regression detector on these representations to identify OOD examples.
Another technique in this category is ODIN (Liang et al., 2018): they use a mix of temperature scaling at the softmax layer and input perturbations to achieve better results.
In fact, both
Lee et al.
(2018b) and Liang et al.
(2018) add small input perturbations to achieve better results; the former do so to increase the conﬁdence score, while the latter do so to increase the softmax score.
Recently, Quintanilha et al.
(2019) achieve results comparable to that of Lee et al.
(2018b) by training a logistic regression detector that looks at the means and standard deviations of various channels activations.
Unlike the previous two techniques, Quintanilha et al.
(2019) achieves comparable results even without the use of input perturbations, which allows it to be applicable to non-continuous domains.
Our work, too, does not involve input perturbations.
All of these techniques depend on OOD examples for ﬁne-tuning hyperparameters (Liang et al., 2018) or for training auxiliary OOD classiﬁers (Lee et al.
(2018b)
; Quintanilha et al.
Furthermore, these classiﬁers neither transfer between one non-training distribution and another, nor do they transfer between networks, so separate classiﬁers must be trained for each (In-Distribution, OOD,
Architecture) triplet.
In many real-world applications, this is infeasible: we cannot assume advance access to all possible OOD distributions.
Our work does not require access to OOD samples.
Alternative Training Strategies.
Lee et al.
(2018a) jointly train a classiﬁer, a generator and an adversarial discriminator such that the classiﬁer produces a more uniform distribution on the boundary examples generated by the generator; they use OOD examples to ﬁne-tune hyperparameters.
DeVries and Taylor (2018) train neural networks with a multi-task loss for jointly learning to classify and estimate conﬁdence.
Shalev et al.
(2018) use multiple semantic dense representations as the target instead of sparse one-hot vectors and use a cosine-similarity based measure for detecting OODs.
Building on the idea proposed by Lee et al.
(2018a),
Hendrycks et al.
(2019a) propose an Outlier
Exposure (OE) technique.
They regularize a softmax classiﬁer to predict uniform distribution on (any) OOD distribution and show the resulting model can identify examples from unseen OOD distributions; this differs signiﬁcantly from previous works which used the same OOD distributions for both training and testing.
Unlike other methods, they retain the architecture of the classiﬁer and introduce just one additional hyperparameter—the regularization rate—and also demonstrate that their model is quite robust to the choice of OOD examples chosen for the regularization.
However, while the OE method is able to generalize across different non-training distributions, it understandably does not achieve the rates of Lee et al.
(2018b)
on most cases.
Recently, Hendrycks et al.
(2019b) make signiﬁcant advances in detecting near-distribution outliers without having any knowledge of the exact out-of-distribution examples by using in-distribution examples in a self-supervised training setting.
Generative Models.
Ren et al.
(2019) hypothesize that stylistic factors might impact the likelihood assignment and propose to detect OOD examples by computing a likelihood ratio which depends on the semantic factors that remain after the dominant stylistic factors are cancelled out.
On the other hand, Nalisnick et al.
(2019b) argue that samples generated by a generative model reside in the typical set, which might not necessarily coincide with areas of high density.
They demonstrate empirically that OOD examples can be identiﬁed by checking if an input resides in the typical set of the generative model.
Unlike the standard experimental setting, they aim to identify distributional shift, which predicts if a batch of examples are OOD.
E XTENDING G RAM M ATRICES FOR O UT - OF -D ISTRIBUTION D ETECTION
Overview In light of the above considerations, we are interested in proposing a method that does not require access to any OOD examples, that does not introduce hyperparameters that need tuning, and that works across architectures.
Gram matrices can be used to compute pairwise feature correlations, and are often used in DNNs to encode stylistic attributes like textures and patterns (Gatys et al., 2016).
We extend these matrices as will be described below, and then use them to compute class-conditional bounds of pairwise feature correlations at multiple layers of the network.
Starting with a pre-trained network, we compute these bounds over only the training set, and can then use them to effectively discriminate between in-distribution samples and out-of-distribution samples at test time.
Unlike other SOTA algorithms, we do not need to “look” at any out-of-distribution samples to tune any parameters; the only tuning required is that of a normalizing factor, which we compute using a randomly-selected validation partition of the (in-distribution) test set.
Notation If the considered deep convolutional network has L layers and the l th layer has n l channels, we consider feature co-occurrences between the (cid:80) 1 < =
l< =
L n l ∗ ( n l +1)pairs of featuremaps.
(Note that by “layer” we refer to any set of values obtained immediately after applying convolution or activation functions.)
We use the following notation:
[Equation]
The feature map at the l -th layer for input image D ; when referring to an arbitrary image D , we just write F l .
It can be stored in a matrix of dimensions n l × p l , where n l is the number of channels at
[Table]
The set of all validation examples.
10% of the examples not used in training are randomly chosen as validation examples.
[Table]
The set of all test examples, disjoint as usual from the training and validation sets.
We assume that only the test set may contain out-of-distribution examples.
Gram Matrices and Higher order Gram Matrices We compute pairwise feature correlations between channels of the l -th layer using the Gram matrix:
[Equation]
where F l is an n l × p l matrix as deﬁned above.
In order to compute feature correlations with more prominent activations of the feature maps, we deﬁne a higher-order gram matrix , which we write G pl , to be a matrix computed identically to the regular Gram matrix, but where, instead of using a raw channel activation a , we use a p , the p th power of each activation.
G pl is therefore computed using F pl , where the power of F l is computed element-wise; in an effort to retain uniform scale across all orders of Gram matrices for a given layer, we compute the (element-wise) p -th root.
The p -th order gram matrix is thus computed as:
[Equation]
We show in Section 5 that higher p values help signiﬁcantly in improving the OOD detectability.
In our experiments, we limit the value of p to 10, as exponents beyond 10 are not worth the extra computation that is needed to avoid overﬂow errors 2 .
The ﬂattened upper (or lower) triangular matrix along with the diagonal entries is denoted as G pl .
The set of all orders of gram matrices (in our case { 1 , . . .
10 } ) to be considered is denoted by P .
The schematic diagram of the proposed algorithm is shown in Fig.
15 (in Appendix A).
Preprocessing If we compute G pl for every layer l and every order p ∈ P , we obtain a total of
N S =
(cid:80) p ∈ P (cid:80)
Ll =
1 12 n l ( n l +1) correlations for any image
D. The preprocessing involves computing the class-speciﬁc minimum and maximum values for the correlations: for every class c , the minimum and maximum values for each of the N S correlations are computed over all training examples D classiﬁed as c .
We keep track of the minimum and maximum values of the N S correlations for all the classes in 4-D arrays Mins and Maxs, each of the order
[Equation]
Since each layer has different number of channels, the 4-th dimension has been large enough to accommodate the layer with the highest number of channels.
Algorithm 1 Compute the minimum and maximum values of feature co-occurrences for each class, layer and order
[Table] [Table]
[Table]
Computing Layerwise Deviations Given the class-speciﬁc minimum and maximum values of the
N S feature correlations, we can compute the deviation of the test sample from the images seen at train time with respect to each of the layers.
In order to account for the scale of values, we compute the deviation as the percentage change with respect to the maximum or minimum values of feature co-occurrences; the deviation of an observed correlation value from the minimum and maximum correlation values observed during train time can be computed as:
[Equation]
The deviation of a test image with respect to a given layer l is the sum total of the deviations with respect to each of the (cid:80) p ∈ P 12 n l ( n l + 1) correlation values:
[Equation]
Total Deviation of a test image D ( ∆( D ) ), is computed by taking the sum total of the layerwise deviations ( δ l ( D ) ).
However, the scale of layerwise deviations ( δ l ) varies with each layer depending on the number of channels in the layer, number of pixels per channel and semantic information contained in the layer.
Therefore, we normalize the deviations by dividing it by E Va [ δ l ] , the expected deviation at layer δ l , computed using the validation data.
Note that we use the same normalizing factor irrespective of the class assigned.
[Equation]
Threshold As is standard (Lee et al., 2018b), a threshold, τ , for discriminating between out-ofdistribution data and in-distribution data is computed as the 95th percentile of the total deviations of test data ( ∆( D ) ).
In other words, the threshold is computed so that 95% of test examples have deviations lesser than the threshold τ ; the threshold-based discriminator can be formally written as:
[Equation]
Computational Complexity.
In order to reduce computational time, we can in fact compute deviations based on row-wise sums rather than individual elements.
This would mean that the variable stat , deﬁned in line 8 of Algorithm 1, would now contain row-wise sums of G pl instead of the ﬂattened upper triangular matrix; the inner loop of Eq.
4 would loop over n l elements instead of 12 n l ( n l + 1) elements while also reducing the storage required for Mins and Maxs.
In practise, we found that computing the anomalies this way yields differences of less than 0 .
5% , and usually imperceptible,
so the results described in the next section were computed in this way.
E XPERIMENTS - D ETECTING OOD
In this section, we demonstrate the effectiveness of the proposed metric using competitive deep convolutional neural network architectures such as DenseNet and ResNet on various computer vision benchmark datasets such as: CIFAR-10, CIFAR-100, SVHN, TinyImageNet, LSUN and iSUN.
For fair comparison and to aid reproducibility, we use the pretrained ResNet (He et al., 2016) and DenseNet (Huang et al., 2017) models open-sourced by Lee et al.
(2018b), i.e. ResNet and DenseNet3 models trained on CIFAR-10, CIFAR-100 and SVHN datasets.
For each of these models, we considered the corresponding test partitions as the in-distribution (positive) examples.
For
CIFAR-10 and CIFAR-100, we considered the out-of-distribution datasets used by Lee et al.
(2018b):
TinyImagenet, LSUN and SVHN.
Additionally, we also considered the iSUN dataset.
For ResNet and DenseNet models trained on SVHN, we used considered CIFAR-10 dataset as the third OOD dataset.
Details on these datasets are available in Appendix B.
[Table]
We benchmark our algorithm with the works listed in Table 1 using the following metrics:
TNR@95TPR is the probability that an OOD (negative) example is correctly identiﬁed when the true positive rate (TPR) is as high as 95%.
TPR can be computed as TPR = TP/ ( TP + FN ) , where TP and FN denote True Positive and False Negative respectively.
Detection Accuracy measures the maximum possible classiﬁcation accuracy over all possible thresholds in discriminating between in-distribution and out-of-distribution examples.
For those methods which assign a higher-score to the in-distribution examples, it can be calculated as max τ { 0 .
5 P in ( f ( x ) ≥ τ ) + 0 .
5 P out ( f ( x ) < τ ) } ; for those methods which assign a lower score to in-distribution examples, it can be calculated as max τ { 0 .
5 P in ( f ( x ) ≤ τ ) + 0 .
5 P out ( f ( x ) > τ ) } . 3.
AUROC is the measure of the area under the plot of TPR vs FPR
For example, for those methods which assign a higher score to the in-distribution examples, this measures the probability that an OOD example is assigned a lower score than an in-distribution example.
Experimental setup: We use a pre-trained network to extract class-speciﬁc minimum and maximum correlation values for all pairs of features across all orders of gram matrices.
Subsequently, the total deviation is computed for each example following Eq.
Since the total deviation values depend on the randomly selected validation examples, we repeat the experiment 10 times to get a reliable estimate of the performance.
The OOD detection performance for several combinations of model architecture, in-distribution dataset and out-of-distribution dataset are shown in Table 2.
The results for Outlier Exposure (OE) are available in Table 3; some more results for OE and the results for DPN,
Variational Dirichlet and Semantic are available in Appendix E.1 and Appendix E.2 respectively.
The results of Table 2 show that at a glance, over a total of 24 combinations of model architecture/indistribution-dataset/out-of-distribution-datasets, the proposed method outperforms the previous competing methods in 15 of them, is on par in 6 of them, and gives second highest results on 3 of them 3 .
Furthermore, it does so without requiring access to samples from the OOD dataset .
If the hyperparameters and/or parameters of Mahalanobis and ODIN algorithms are ﬁne-tuned using
FGSM adversarial examples instead of the real OOD examples, their performance decreases.
We also observe that our performance is similar for both architectures.
We also performed experiments with fully-connected networks by using three different MLP architectures trained on MNIST; Fashion-MNIST (Xiao et al., 2017) and KMNIST (Clanuwat et al., 2018) were considered as the out-of-distribution datasets (Results are provided in Appendix E.3).
[Table]
[Table]
Near-distribution Outliers
The metric does not perform well for near-distribution outliers (for example, CIFAR10 vs CIFAR100) – the detailed results can be seen in Table 4; a recent work which performs very well for near-distribution outliers and yet does not require knowledge of OOD distribution is Hendrycks et al.
(2019b).
Source of Performance
Gain The above results are obtained when we consider all elements of gram matrix (Algorithm 1), compute deviations from extrema (Eq 3) and ﬁnally, compute the total deviation with normalized layerwise deviations (Eq 5).
In order to better understand the source of performance gain, we conduct a detailed ablation study that considers alternative choices for the three steps outlined before; the alternative choices are chosen in order to answer the following questions:
Q1) What if strictly diagonal elements or strictly off-diagonal elements are considered instead of complete Gram Matrix?; Q2) What happens if the deviation is computed from the mean instead of the extrema? ; Q3)
What happens if we do not normalize the layerwise deviations?
In all, we conductexperiments: 3 choices for Q1 × 2 choices for Q2 × 2 choices for Q3.
As a broad summary, we ﬁnd that, while there is no single rule that is unbroken by an exception, our proposed combination—i.e. using the complete Gram matrix, using the min/max metric, and using normalization as in Eq 5—is generally more robust than any of the other combinations that we tried.
More details on the ablation tests and discussions are available in Appendix C.
D ISCUSSION AND C ONCLUSION
Beyond explicit OOD detection, this line of work may ultimately help better interpret neural networks’ responses to OOD examples.
With this goal in mind, and at the same time to clarify the internal mechanism of our method, we perform tests to address the following two questions:
Which representations are most useful?
In order to examine the role of the depth at which we compute G in detecting OODs, we construct detectors which make use of correlations derived from just one residual or dense block at a time; however, all orders of gram matrices are considered.
Representative results are shown in Figure 1.
For all combinations of model/in-distribution/out-of-distribution-dataset, we ﬁnd that the lower level representations are much more informative in discriminating between in-distribution and out-of-distribution datasets.
However, the difference in detective power depends on the in-distribution dataset considered: for example, the difference in detective power between higher-level representations and lower-level representations is bigger for Cifar-100 than for Cifar-10.
More graphs are available in Appendix C.1.
Which orders of gram matrices are most useful?
In order to understand which orders of gram matrices are most helpful in detecting OODs we construct detectors which make use of only one order of gram matrix at a time; however, correlations are derived from the representations of all residual and dense blocks.
Representative results are shown in Figure 2.
For all combinations of model/in-distribution/out-of-distribution-dataset, we ﬁnd that the higher order gram matrices are much more informative in discriminating between in-distribution and out-of-distribution datasets.
Ignoring the variations at orders greater than 4, we ﬁnd that the TNR @ 95TPR increases with higher orders and ﬁnally saturates.
More graphs are available in Appendix C in Figure 4 to 9.
Conclusion.
Out-of-distribution detection is a challenging and important problem.
We have proposed and reported on a relatively simple OOD detection method based on pairwise feature correlations that gives new state of the art detection results without requiring access to anything other than the training data itself.
[Figure]
[Figure]
B D ESCRIPTION OF OOD D ATASETS
The following includes the description of the out-of-distribution datasets:
TinyImagenet , a subset of ImageNet (Russakovsky et al., 2015) images, contains 10, test images from 200 different classes.
Each image is downsampled to size 32 x 32 and all 10,000 images are used, as given in the opensourced version by Liang et al.
LSUN , the Large-scale Scene UNderstanding dataset (Yu et al., 2015) has 10,000 test images from 10 different scenes.
Each image is downsampled to size 32 x 32 and all 10,000 images are used, as given in the opensourced version by Liang et al.
iSUN , a subset of SUN images (Xiao et al., 2010), consists of 8925 images.
Each image is downsampled to size 32 x 32 and is used; the downsampled version of the dataset has been opensourced by Liang et al.
SVHN , the Street View House Numbers dataset (Netzer et al., 2011), involves recognizing digits 0-9 in natural scene images.
The test partition consisting of 26,032 images is used.
C D ETAILED A BLATION R ESULTS
The results in the main paper correspond to the performance obtained when considering:
Feature Set : all gram matrix entries 2.
Metric : layerwise deviations computed with respect to the mins and maxs.
Aggregation Scheme : the total deviation is then computed using Eq 5.
In this section, detailed ablation results are reported by considering other options.
Speciﬁcally:
Alternate Feature Set :
In addition to considering all gram matrix entries, we consider a proper partition of the gram matrix: strictly diagonal elements, and strictly off-diagonal
elements.
The diagonal elements correspond to the unary features, while the off-diagonal elements correspond to pairwise features.
This can be done by appropriately changing the deﬁnition of variable stat in Line 7 of Algorithm 1.
In these experiments, we consider row-wise sums wherever the size of stat is O ( n 2 ) ; in other words, we consider row-wise sums when considering off-diagonal elements and all gram matrix entries.
Alternate Metric : An alternative formulation for computing feature-wise deviations can be to compute the deviation from the means using the one-dimensional Mahalanobis distance.
In the preprocessing stage, this would be done by storing the Means and Variances of stat (feature-wise) instead of their Mins and Maxs .
Under this new alternative, the function δ deﬁned in Eq 3 would be redeﬁned as:
[Equation]
Accordingly, the layerwise deviation δ l can be deﬁned as:
[Equation]
where G pl ( D ) would correspond to the statistic chosen in the previous step: diagonal entries only, row-wise sums of off-diagonal entries only or row-wise sums of complete gram matrix.
We thus consider 2 options for computing the deviations: the Min/Max method presented in the main paper and the Mean/Variance method (Gaussian) described above.
Alternate Aggregation Scheme :
In order to compute the total deviation ∆ from the layerwise deviations δ l , we can compute it by following 5 or taking a simple sum as shown:
[Equation]
We refer to Eq 5 as the normalized estimate and Eq 9 as the unnormalized estimate.
In all, the Table 4 reports detection rates in 12 settings: 3 choices for stat (only the diagonal entries of gram matrix G , only the off-diagonal entries of G , or all of G ) × 2 metrics for computing deviation (Min/Max or Mean/Variance) × 2 choices for computing total deviation (Normalized sum or Unnormalized sum).
All layers and all orders of gram matrix are considered in Table
[Table]
[Table] [Table]
By analysing the ablation results
[Table]
we attempt to answer the following questions:
Are pairwise features more useful than unary features?
We observe that the Min/Max metric can work equally well with both unary and pairwise features; in some cases, the unary features are marginally better (Ex: ResNet/CIFAR-10 vs SVHN) and in some cases, the pairwise features are marginally better (Ex: ResNet/CIFAR-100 vs iSUN/LSUN/TinyImgNet).
Interestingly, the behavior of the Mean/Var metric is different: the performance with pairwise features are signiﬁcantly higher than with unary features in 19 out of 28 tested cases.
For example, the TNR at TPR95 for ResNet/CIFAR-100 vs TinyImgNet is 68.0 with unary features and 84.2 with pairwise features.
We notice that using the unary features (diagonal entries) sometimes did well when pairwise features (off-diagonal entries) did not do well, and vice versa, so using both gives the kind of effect that we want in an ensemble: models that cover and work well over different parts of the space.
Therefore, an overall message of our experiments is that it is worthwhile to consider all elements of the gram matrix.
Is it neccessary to use Min/Max metric?
Except in 6 cases (ResNet: CIFAR-10 vs CIFAR-100, ResNet: CIFAR-100 vs CIFAR-10, DenseNet: CIFAR-10 vs CIFAR-100, DenseNet: CIFAR-100 vs CIFAR-10, ResNet: SVHN vs CIFAR-10 and DenseNet: SVHN vs CIFAR-10), the min/max metric consistently performs better than the mean/var metric.
Additionally, it is not clear if the Mean/Var estimate performs better with normalized sums or unnormalized sums: for example, observe that Mean/Var estimate performs very poorly with the unnormalized estimate for ResNet/CIFAR-100, while the performance of Mean/Var for DenseNet/CIFAR-100 is competitive with the performance of Min/Max only when an unnormalized estimate is computed.
One can observe that computing the one-dimensional Mahalanobis distance for each
component of the statistic derived from the Gram Matrix and later computing the total sum is equivalent to representing each input image by a big vector (say, Z ) derived from the Gram
Matrices computed across various layers, constructing class-conditional distributions of Z (assuming that each component of Z is normally distributed and independent of the other components) and subsequently computing the probability of an unseen Z .
In early research, we noticed the following problems with the Mean/Var estimate:
(a) The individual components of gram matrices do not follow normal distribution strictly and Mean/Var assigns lower probabilities to the in-distribution images as well.
(b) The total deviation ∆ – computed by simply summing across the layerwise deviations, δ l – was not able to accurately summarize the information contained in the different δ l s. Speciﬁcally, information about the layer where the input example had a higher deviation was lost when a simple sum was taken.
The proposed Min/Max idea solves problem (a) by employing a weaker metric: deviation from extrema instead of the mean.
It can also be said that the Min/Max metric considers a uniform probability density between the extrema.
Problem (b), which exists even for this newer metric, is solved by the normalization scheme described in the main paper for computing the sum total deviation.
Higher Order Gram Matrices
The Min/Max metric is a weak approximation to the true probability density.
On conducting a thorough analysis of how the OOD examples were able to fool the metric, it appeared that the intermediate features had several tiny activations that could yield innocuous correlation values.
Higher-order gram matrices as described in the main paper provide a natural way to mitigate these effects.
Notable observations from Figures 4 through 9 (all layers are considered but only one order of gram matrix is considered at a time):
• Ensemble effect : In 24/28 cases, higher order gram matrices improve detection rates.
Higher order gram matrices help both the Min/Max and the Mean/Var metrics.
In most cases, the even powers are more helpful than the odd powers; in some cases, the odd powers are more helpful (Ex: DenseNet/CIFAR-100 vs CIFAR-10).
Despite these variations, it is possible to get an ensemble effect by considering all possible powers as demonstrated in the main paper.
• In ResNet:CIFAR-10 vs CIFAR-100 and DenseNet:CIFAR-10 vs CIFAR-100, the higher order gram matrices yield lower detection rates.
We ﬁnd these exceptions interesting, and would like to understand them better in future.
Summary The unambiguous message from this ablation study is that the Gram matrix contains useful information which can be used for detecting OOD examples.
While the standard Mean/Variance metric does not always work well, the proposed Min/Max metric yields consistent performance competitive with state-of-the-art methods.
The use of higher-order Gram matrices further boosts the overall performance.
Although the Min/Max method can work very well for "far-from-distribution" examples, it does not work well when a ﬁne grained estimate is needed (for example, CIFAR-10 vs
CIFAR-100).
We hope the strong empirical proof that Gram matrices contain useful information can motivate the development of OOD detectors with powerful density estimators; eventually, such estimators might not need higher-order Gram matrices to boost their performance.
Here they allow us to propose a metric based on simple thresholding that, generally, works independently of the particular OOD examples and independently of the architecture, and we believe that the fact that this is the case—the fact that this approach scores relatively well—itself raises several interesting new questions.
We hope that answering these questions will ultimately strengthen our understanding of deep neural networks and give rise to more robust neural network models in the future.
10 Order of Gram Matrix 0.
T N R a t T P R
ResNet: CIFAR10 vs iSUN Min/MaxMean/Var    10 Order of Gram Matrix 0.
T N R a t T P R
ResNet: CIFAR10 vs LSUN Min/MaxMean/Var    10 Order of Gram Matrix 0.
[Figure] [Figure]
[Figure]
T N R a t T P R
[Figure] ResNet
[Figure]
CIFAR
[Figure] [Figure]
T N R a t T P R
ResNet: SVHN vs iSUN
[Figure]
/Var
[Figure] [Figure]
[Figure] [Figure] [Figure]
[Figure]
T
[Figure]
[Figure]
[Figure]
[Figure] [Figure]
T N
[Figure]
a t T P
[Figure]
[Figure] /MaxMean/
Var    10 Order of Gram Matrix 0. T N R a t T P R DenseNet: CIFAR100 vs TinyImgNet Min/MaxMean/Var    10 Order of Gram Matrix 0.
T N R a t T P R
DenseNet: CIFAR100 vs SVHN
Min/MaxMean/Var    10 Order of Gram Matrix 0.
T N R a t T P R
DenseNet: CIFAR100 vs CIFAR
Min/MaxMean/Var
[Figure]
N
[Figure]
[Figure]
T N
[Figure]
C. S IGNIFICANCE OF D EPTH
[Figure]
[Table]
1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 Block 0.
T N R a t T P R
DenseNet: CIFAR100 vs iSUN
Min/MaxMean/Var 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 Block 0.
T N R a t T P R
[Figure]
CIFAR
[Figure]
/Var [Figure] [Figure] [Figure]
[Figure] [Figure]
[Figure]
[Figure]
T N R a t T P R
DenseNet
[Figure]
CIFAR
[Figure]
TinyImgNet Min/MaxMean/Var 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 Block 0.
T N R a t T P R
DenseNet: CIFAR100 vs SVHN
Min/MaxMean/Var 1.00 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 Block 0.
T N R a t T P R DenseNet: CIFAR100 vs CIFAR
Min/MaxMean/Var
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR10 vs iSUN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR10 vs LSUN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR10 vs TinyImgNet
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR10 vs SVHN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR10 vs CIFAR100 Min/MaxMean/Var
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR100 vs iSUN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR100 vs LSUN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR100 vs TinyImgNet
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR100 vs SVHN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: CIFAR100 vs CIFAR
Min/MaxMean/Var
4.0 Block 0.
T N R a t T P R
ResNet: SVHN vs iSUN
Min/MaxMean/Var 1.
4.0 Block 0.
T N R a t T P R
ResNet: SVHN vs LSUN
Min/MaxMean/Var 1.
[Figure]
[Figure]
SVHN vs CIFAR
Min/MaxMean/Var
D C OMBINING OE + O URS
[Table]
CIFARiSUN 18.
96.1 LSUN 20.
97.2 TinyImgNet 20.
95.3 SVHN 14.
92.2 CIFAR 16.
E F EW MORE OOD RESULTS E. C OMPARING WITH OE In-distribution
OOD
OE (Base)
OE
Ours (Base)
[Table]
Following Hendrycks et al.
(2019a)
, we created the gaussian, rademacher, blob and bernoulli synthetic datasets.
Their descriptions are as follows:
Gaussian anomalies have each dimension i.i
[Table]
d. sampled from an isotropic Gaussian distribution.
Rademacher anomalies are images where each dimension isor 1 with equal probability, so each dimension is sampled from a symmetric Rademacher distribution.
Bernoulli images have each pixel sampled from a Bernoulli distribution if the input range is [0, 1].
[Table]
data consist of algorithmically generated amorphous shapes with deﬁnite edges.
Textures is a dataset of describable textural images (Cimpoi et al., 2014).
E.
[Table]
C OMPARING WITH DPN, VD AND S EMANTIC .
[Table]
E. R ESULTS FOR F ULLY - CONNECTED N ETWORKS Architecture
[Table]