A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks
Authors: Matan Haroush
Tzviel Frostig
Ruth Heller
Daniel Soudry
Venue: ICLR 2022 Poster (2022)
Keywords: out of distribution
DNNs
p-value
hypothesis testing
inductive conformal predictor

Abstract: Background.
Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn from a distribution similar to that of the training set. However, DNNs' predictions are brittle and unreliable when the test samples are drawn from a dissimilar distribution.
This is a major concern for deployment in real-world applications, where such behavior may come at a considerable cost, such as industrial production lines, autonomous vehicles, or healthcare applications.

Contributions.
We frame Out Of Distribution (OOD) detection in DNNs as a statistical hypothesis testing problem. Tests generated within our proposed framework combine evidence from the entire network.
Unlike previous OOD detection heuristics, this framework returns a $p$-value for each test sample. It is guaranteed to maintain the Type I Error (T1E - incorrectly predicting OOD for an actual in-distribution sample) for test data. Moreover, this allows to combine several detectors while maintaining the T1E.

Building on this framework, we suggest a novel OOD procedure based on low-order statistics. Our method achieves comparable or better results than state-of-the-art methods on well-accepted OOD benchmarks, without retraining the network parameters or assuming prior knowledge on the test distribution --- and at a fraction of the computational cost.

Published as a conference paper at ICLR 2022
A STATISTICAL FRAMEWORK FOR EFFICIENT OUT OF DISTRIBUTION DETECTION IN DEEP NEURAL NETWORKS
Matan Haroush"
Tzviel Frostig'
Electrical & Computer Engineering Department Technion, Haifa, Israel. Habana Labs An Intel company
Statistics & Operation Research Department
Tel Aviv University, Israel. t tfrostig@gmail com
mha rearrhhh@hii habana. a
Ruth Heller
Daniel Soudry
Statistics & Operation Research Department Tel Aviv University, Israel. ruhellerstauex tau. ac.
Electrical Computer Engineering Department Technion, Haifa, Israel. daniel. soudry gmail com
ABSTRACT
Background. Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn from a distribution similar to that of the training set. However, DNNs' predictions are brittle and unreliable when the test samples are drawn from dissimilar distribution. This is major concern for deployment in real- world applications, where such behavior may come at a considerable cost, such as industrial production lines, autonomous vehicles, healthcare applications.
Contributions. We frame Out Of Distribution (OOD) detection in DNNs as a statistical hypothesis testing problem. Tests generated within our proposed frame- work combine evidence from the entire network. Unlike previous OOD detection heuristics, this framework returns a p-value for each test sample. It is guaranteed maintain the Type Error (TIE incorrectly predicting OOD for an actual in-distribution sample) for test data. Moreover, this allows to combine several de- tectors while maintaining the TIE. Building on this framework, we suggest novel OOD procedure based on low-order statistics. Our method achieves comparable or better results than state-of-the of-the-art methods on well-accepted OOD benchmarks, without retraining the network parameters assuming prior knowledge the test distribution - and at a fraction of the computational cost.
INTRODUCTION
Deep Neural Networks' (DNNs) predictions shown to be overconfident and unreliable when the test samples are drawn from an unexpected distribution (e.g., Biggio et al. (2013): [Szeedyy et al. (2013); Goodfellow al. (2014); (Eyhho/yllllcc et al. (2017).. Therefore, numerous Out Of Distribution (OOD) detection methods were suggested to address this issue by informing an operator when not to trust the DNN's prediction.
In the context of DNNs, methods for OOD detection can be roughly divided into two classes. "Mutators" methods modify the network structure or loss, and depend on training its parameters to provide a confidence measure (Hendrycks et al. 2018) Malinin and Gales 2018 Winkens et al.) 2020 Zhang et al. 2020 Hsu et al.] 2020). "Observer" methods model the prediction uncertainty of pre-trained network, without modifying its architecture or parameters (Hendrycks and Gimpel
2016 Liang et al. 2017 Lee et al. 2018 Zisselman and Tamar 2020 Raghuram et al. 2020).
Mutators potentially costly, they require substituting retraining the network and subsequently repeating an exhaustive hyperparameter tuning process. Also, the resulting model quality on the
* Equual contribution

---

Published as a conference paper at ICLR 2022
original task may vary. Alteraaively,, observer methods commonly utilize auxiliary models monitor features produced by the reference network. Since this approach is decoupled from the network optimization process, the reference accuracy is retained at the cost of additional test time resources, such as compute and memory.
Another differentiating factor between methods, regardless of their class, stems from fundamental assumptions regarding prior knowledge on the test distribution. Specifically, some methods (e.g.. Hendrycks al. (2018); [ee et al. (2018); Zisselman and Tamar (2020)) rely on prior knowledge, such as access to the OOD data or its proxy. However, this prior can lead to undesirable bias and poor detection performance on other test distributions.
Notably, OOD detection can be naturally cast as statistical hypothesis testing problem (Grubbs 1969) Vovk et al. 2005 Papadopoulos 2008, Grosse al. 2017). Nevertheless, authors of contemporary
observer methods largely overlook the statistical aspects of their proposed algorithms. For example, they do not provide meaningful error guarantees nor explicitly define what hypothesis is being tested. For further details regarding related work, see Section [ In addition, these methods also tend to be computationally expensive (see Section 4.4). In this work, we aim to tackle these issues.
Contributions.
We first propose methodological innovations
We present a novel framework for OOD detection in DNNs, based on statistical hypothesis testing and phrase two relevant null hypotheses. The first is whether sample is drawn from the same distribution of the classes in the training dataset (any-class). The second is whether sample is drawn from the distribution corresponding to the predicted class (class-conditional, We construct valid tests for both hypotheses and further provide empirical evidence for these theoretical results (Section B)
Our framework can be viewed as extension of the inductive conformal predictor framework (Papadopoulos) 2008), which is a non-parametric method for conducting hypothesis testing with an arbitrary test statistic (Vovk et al. 2005). However, instead of focusing solely on the last layer of the DNN predictor, we show how to normalize and aggregate multiple non-conformity (i.e., abnormality) measures based on the entire network. Empirically, we demonstrate that this generalization is crucial for OOD detection. Namely, we find that a multi-layer detector improves accuracy by 50% average compared to single-layer variants (Section 4.2)
These innovations lead to considerable practical advantages
Importantly, unlike current methods for OOD detection in DNNs, our framework returns valid p-value for each tested sample. This guarantees maintaining a Type Error (detecting OOD as in-distribution) on test data (see Section 3.2, Propositions I and 2), and has additional advantage which do not exist in standard bhreshold-based OOD classification.
Based on this framework, we design a detection scheme denoted as MaSF Seaes-imms-FFisss Section 4.2). MaSF achieves similar or better results compared to SOTA on the benchmark proposed by Lee et al. (2018), while in our experiments, we show its test statistic inference is 35 times faster. Making MaSF appealing for real-world applications with a limited compute budget (Section 4.4).
We demonstrate potential avenues to further reduce the computational overhead via structured sparsity (i.e., channel sampling) - further reducing the computational cost by factor of 10 while degrading accuracy by no more than ~ 1% (Section 4.5)
2 BACKGROUND
In the following section, we provide a summary of Null Hypothesis Statistical Testing (NHST). We begin with single hypothesis and then further expand on Global-Null tests.
Null Hypothesis Statistical Testing, NHST is the process of choosing between two hypotheses - Ho (null) and Hi (alternative) regarding observation (sample) X. It is formalized as Ho X ~ P vs
For example, p-values enable adjusting the decision rule when more than one hypothesis is tested at time while maintaining a certain error criterion (Chapter in Breiz et a (2016). ). For instance, one can potentially employ several OOD detectors on single sample (e.g., each specialized on specific type outher distribution).
2

---

Published as a conference paper at ICLR 2022
H41 : X of P. where P is some distribution. The decision to reject No is based the critical value y and the test statistic T(X). Throughout this section, we will focus on right-tailed tests. We call test right-tailed when higher values of T(X) are more likely under Hi than under Hg. In those kinds of tests, Ho is rejected if T(X) 2 7. The critical value 7 quantifies the required evidence to determine if Ho is false.
Erroneously rejecting Ho. is called a Type Error (TIE). The significance level (i.e., probability of TIE) is given by a Pro (T) 2 7). Usually, is predetermined, and 7 is found accordingly. For a given significance level, the goal is to maximize the power of the test, PH, (X) 2 7), which is the probability of correctly rejecting Hy. Commonly, p-value is used to summarize NHST. The p-value indicates the probability of obtaining a test statistic equal or more extreme than the observed test statistic, given that Ho true. The p-value (denoted by q) is given by, g(fobs) = Pho (T(X) 2 labs), where tobs is the observed value of the test statistic.
Assuming the null is correct, and that continuous, then the p-values are uniformly distributed (Abramovich and Ritov 2013). This allows to simply reject the null if g(toba) S a. It is clear that the significance level will be maintained if the p-value distribution is stochastically larger than or equal to the uniform d
Global Null Tests, Given a set of hypotheses, combination tests are used to determine if any hypothesis is false (global null). Let denote vector of m p-values, q (41...., 9m,,, ). that is used to test the m poth-hyso Ho....... Ho.m The global null hypothesis is defined as Ho. N==1 Ho,i. The test statistic is T(q). The test power depends on the unknown distribution of the test statistic under H1.
Two ubiquitous tests of the global-null are the Fisher and Simes tests (Fisher 1992 Simes 1986). The Fisher test statistic is -2 Et= log(n:) is suitable for detecting dense signals due to the summation of p-values. The Simes test statistic is min
4(s) 7. where (i) is the index of the p-values
iE(I.....m)
after sorting, It is adept for detection of sparse signal, as it focus on a single p-value. These complementing attributes are appealing for OOD detection, as we will demonstrate in the following sections. For more detailed introduction of these tests see Section B.3
An important benefit of using p-values is that they allow maintaining meaningful error rate. There are many methods employed to maintain such error rates, which are applied on p-values (Holm) 1979 Hochberg 1988 Benjamini and Hochberg. 1995).
HYPOTHESIS TESTING IN NEURAL NETWORKS
For the sake of simplicity, we focus the next section solely on convolutional neural networks (CNNs) for image classification tasks. The method presented, however, is applicable to any DNN.
3.1 PRELIMINARIES
V:) ; E [ne], y: = c), where Ix*1= = ne. X₃ is an image and yi: di are its true and predicted class (out of i classes). We further split the observations of each class to the CNN training set Xirain. and the validation set x The training set is denoted by
Define [k] = E k} and let x== {(x,,
Xtrain = =U Xirain and its cardinality is Nurain Ed _ Nec We similarly define Xval and Nyat- Pe is the class distribution of X, where y/1 = c, E [k]. The distribution of any test statistic under Ho required to obtain p-values. For that reason we apply the empirical cumulative distribution function (cCDF) to estimate the null distribution of class function T' at x using an observations set x E {Xtrain Xeath.
Exxredex I (T(X.) S x) +1
P(x: T,x")
(I)
The +1
where I is the indicator function. The addition of is necessary to ensure that the p-values are not equal to O, as it implies they are rejected at all significance levels. The problem of determining if an
2 X stochastically lurger than Y Vz P(X x) > P(Y > x).
Whhnn it unelear if T(X) is expected to be larger or smaller under 74, one can use two-sided tests, so the p-value 2 min No (T(X) 2 barw), Pso (X)) S fab.)), which can be viewed as conducting a right and left sided tests and combining the results with Bonferroni correction.
3

---

Published as a conference paper at ICLR 2022
image Xtest is OOD, can be formalized
Hg 3c X
Pe, HG Xtest of pe Vc E [k]
By rejecting Hö. we conclude that the image is not drawn from any of the class distributions. One can also test if an image is sampled from specific class distribution.
Hig Xtest Pe, Hi Xtest o P
(3)
Since the true label is unknown at test time, we are interested in the case where the class is as- signed according to the CNN prediction, Ho Therefore, rejecting it indicates the image is either misclassified or OOD.
3.2 THE TESTING PROCEDURE
The typical CNN (F) is composed of L layers. Each layer, contains a; channels. The feature map of the j 'tth channel in I'th layer is denoted by FJ, X > R where X is the input image, hy and ui refer to the spatial dimensions the feature maps at the l'th layer. We begin by testing at the channel level. Each channel is summarized to scalar value using spatial reduction function, 7
(x) = (F)((x))
T RIXI R.
Therefore, the empirical null distribution of the fy.1 channel in layer / is PP,;; f,1, X The p-value is denoted by of, for channel j. and by gf, for the entire layer (of class c). We use two-sided p-values for the channel reduction, outliers appear in both tails of the distribution, see Section D.I. Next we aggregate the evidence from E,,, for each layer, by applying a channel reduction function, :
((x)= = (T") Thth 10, -
(5)
By employing P(x; tf Xirai)) we can recover the layer's p-values for class c: ...... L. Given .i. of. for all layers, the class conditional test statistic of an image is obtained by applying the layer reduction function, T4,
(()) =T
Th: 10, R.
(6)
The class conditional p-value for sample X, is obtained using PP(X; to; ( and denoted by (X). can be used test Hg (Eq. 3) Note, that (Xtest is not exchangeable with (x( X Xtrain) since the are dependent through the DNN. Using the validation set ensures the exchangeablity of the
test-statistics and through the validity of the proposes test. Aggorithm-Tdesereee obtaining cuus--oonditinall p-vuuue.
the procedure for
We now turn to present the two main propositions regarding the validity of the procedure described above. The procedure is considered valid if it maintains the TIE at the specified significance leve Proof for both propositions and assumptions can be found in the Appendix
Proposition 1 A level a test of Hi vs. (Xtest) S a.
Hi
is: reject Hi if max (Xusst
max (X*(st)st),,.....
Since the true class of the image is unknown, the maximum p-vuuue ensures the method rejects Hi for sample only if the evidence against all classes, i.c., against H....., H/:.
When the class is assigned according to the CNN prediction, P Prgirea (g (Xuea) S a) 2 a, since the decision which hypothesis test was made based the data. We can bound this TIE as stated in the next proposition and adjust the p-values accordingly to obtain a valid test.
Proposition 2 For testing Hftrr vs. Hirn at significance level a, the TIE probability is P Haroat (grrett 5 SaPhett = yreet) + P(heest + yhestt)
Note that Xersan can used even though it correlated with the CNN weights since allows us to leverage more observation. The procedure remains valid since Xvai used calibrate final test statistic.
4

---

Published as a conference paper at ICLR 2022
Both propositions rely on the assumption that the validation set used to estimate the CDF of the final combination test is independent of the DNN training set, in similar vein to induc- tive conformal prediction (Vovk et al., 2005 Papadopoulos 2008
Algorithm 1: Class-conditional p-values
Input
Xtest c; // The network, inga: image and class of intereat :T ,r,h, // Spatial, channel and layer reductions
Input
Input
1f, j [ai), E [T]] // eCDF per- after T
3.3 SUMMARY AND CHALLENGES
Input
: xiram) E [L]:
// eCDF
per- layer after T
To summarize, we construct our test function in 3 steps. i) Spatial reduction followed by p-value extraction per channel. ii) Channel reduction, summarizing resulting p-values for each layer. iii) Layer reduction, aggregating p-values from all layers into a final p-value. For steps (i) and (ii) we approximate the CDF using Xtrains while Xval is used for the final step. Transforming each channel's spatial features into a p-value ensures that the evidence from all channels is assessed comparably even though their distribu- tion may vary. At each layer, the aggregation of the per-channel results can cause a disparity (layers with more channels will dominate the test statistic value). Transforming the layer test statisties into p-values resolves the issue. Ulti- mately, this hierarchical approach guarantees fair comparison at each step when pooling evi- dence from the entire CNN to reject either Hg or Hij.
Input :P(.:;t, ,P(:ttxXaa)) ; // eCDF after T Output :g" (Xeeal // class-conditiona
p-value for Xome and class
for 1 [L] do
for E [ad do
(Neen) = (F),(XXaa)))) // Soatial reduction
25. (It(P(t,,((Neeet ff,1 Xfrain), 1- (Xt,a fy,, Xiram));
// p-value per-channel, assaming
two sided teat
6 (X(ea)
T Thh (Xteet). (X
1/ Channel ruduction ion of (Xuu))
min(F(((()non); (J,,, Xirain); 1
(Xeea))s)): ti, Xerain)); per- zyer
11 p-val:se
L' ("(mma) T (di((e (X(on)))) // Layer reduction
However, combining multiple tests into a single one has its challenges. On the one hand, we want the rejection
Return 1 (X tt xmal);; // Assiming a right sided p-value
to include many hypotheses of the global null can be caused by any of them. On the other hand, more true null hypotheses are included, the power may decrease. For example, consider the case where the combined p-values are independent under the null. In the Fisher test, the more test statistics from true null hypotheses are combined, the greater the noise to signal ratio. It implies that the rejection criteria will be harder to reach. In the Simes test, the sorted p-values are multiplied by the number of hypotheses, m. Again, implying that adding test statisties from true null hypotheses will lower the power of the test.
4 DESIGNING TEST STATISTICS FOR OOD DETECTION
So far, we have described our novel theoretical framework for NHST in DNNs. Before we describe our proposed OOD detection algorithm, we briefly review popular benchmark used to evaluate and motivate the rationale behind the specific design choices.
4.1 OOD DETECTION BENCHMARK
An accepted benchmark for OOD detection was introduced by Lee et al. (2018). We adhere to the same protocol to evaluate our suggested method. Therefore, we use the same pre-trained image classification CNNs (Huang et al. 2016 He et al., 2016) and datasets. In this benchmark, test samples are drawn from the alternative datasets and presented to the reference CNN. Competing methods output an abnormality score per sample by observing the CNN's ' activations. Unlike classical NHST, the rejection threshold for the predicted OOD score is determined based on the in-distribution test set, where a threshold is found to ensure a False Positive Rate (FPR). That is, results are reported as the True Positive Rate (TPR- i.e., correct OOD prediction rate) while maintaining FPR of 5% for
5

---

Published as a conference paper at ICLR 2022
in-distribution validation samples (TPR95). Adjustment to the p-value the threshold is equivalent. In the following section, to ensure a fair comparison we apply the same method in for MaSF
For comparability of results, modify our procedure to estimate the CDFs of the null distributions using only the CNN training set. This violates the statistical guarantees of maintaining TIE due to the dependency between the CNN weights and the training samples. However, since demonstrating the procedure's 8 ability to maintain TIE is of independent interest, we provide additional experiments employing a hold-out set in the Appendix B.2 There, we show OOD detection based on valid procedure does not change meaningfully from the reported results. Finally, to evaluate the overall performance, we calculate the mean TPR95 (mTPR) and standard deviation (SD), along with the minimal TPR95 (Min-TPP) observed over all test scenarios.
4.2 DESIGN CONSIDERATIONS
We now turn to discuss our OOD algorithm, which returns a class conditional p-value (i.c., Hypothesis HD from Eq. provided the trained network, input image, and the predicted class.
Choosing reductions. The choice of reduction functions can greatly impact the power of the final test statistic and incorporate domain knowl- edge into the designed detector. However, it is not trivial choice. Therefore, construct a set of detectors employing simple building blocks and evaluate their overall performance. Specifically, we use Simes and Fisher tests as the channel the layer reductions along with Max and Mean pooling as spatial reductions. Table [ portrays the mean and SD of each configuration (see Section
Table 1: Hierarchical reduction ablation.
Layer
Channel
Spatial
mTPR
SD
Fisher
Simes
Max
96.4
4.7
Fisher
Fisher
Max
95.3
4.9
Simes
Simes
Max
95.3
5.9
Fisher
Simes
Mean
91.1
8.6
Simes
Fisher
Max
90.1
10.7
Fisher
Fisher
Mcan
89.8
8.8
Simes
Simes
Mcan
87.9
9.5
Simes
Fisher
Mean
80.1
14.3
"Results are sorted according to the mean TPR95 (mTPR) OOD benchmark (Section [.]]
We observe several consistent trends across all scenarios in the benchmark. Namely, the maximum value of a given feature map is more sensitive to outliers compared to the mean value, which is expected. Additionally, we find that the Fisher combination test performs better than Simes as a layer reduction. This suggests the evidence of abnormality tends to propagate throughout the networks' layers. Finally, the Simes test appears to be a better channel reduction. Note that using Simes (or any other multiple comparison methods) alleviates the need of estimating the distribution at the channel level, as normalizes the p-value to the number of channels (see Appendix Ep. Furthermore, hints that the OOD evidence is either sparse (i.e., abnormality exists in few of the channels) or that channels are strongly correlated, rendering the Fisher test uninformative.
Moving forward, focus the best configuration: Maximum (spatial), Simes (channel), and Fisher (layer) configuration, dubbed MaSF. The MaSF algorithm is available in Appendix F
Observing multiple layers. Next, aim to investigate the impact of utilizing multiple layers versus relying on a single layer from the end of the model (as in the inductive predictor framework of Papadopoulos (2008)). We evaluate mTPR when using the baseline Maximum Softmax Probability -MSP- Hendrycks and Gimpel (2016), and the MaSF detector using features from single layer. Specifically, we consider either the inputs outputs of the linear classifier or the input feature of the penultimate layer (i.e., the final Average Pooling layer AP). This is done by eliminating the Fisher layer reduction step. The detector achieved the best results in terms of mTPR (SD) using the AP layer's inputs 64.7 (21.5). However, these are significantly lower compared to standard MaSF when tracking all layers 96.4 (4.7) as reported in Table T
Despite the potential gains from observing additional shallow layers, the test power could decrease when adding non-informative layers (see Section and the computational cost increases. Choosing which layers to include in the test is not trivial may vary depending the network architecture, downstream task, or computational budget. Thus, in all our experiments, we test the outputs of all convolution and dense layers in the network. In Appendix G we experiment with assigning higher weight to the final layer of the networks. This approach yields improved near distribution results, leave its development for future work. This is particularly interesting if we know that the OOD)

---

Published as a conference paper at ICLR 2022
samples are drawn from a distribution close to that of the training data. In this case, expect shallow layers (i.e., close to the input) to produce features that will not be discriminative.
Finally, in Appendix D provide additional analysis the correlation between the test statistics among the monitored layers. We suggest that variance reduction techniques should play a role in designing algorithms within our framework. For example, by introducing random sampling whitening strategies into the construction of the test statistics, we defer this topic for future work.
4.3 EMPIRICAL EVALUATION
In this section, we provide a breakdown of MaSF performance on the popular benchmark described in Section 4.1 In addition, Section (ectioonSSooiin contains review of the related work discussed below.
We compare our results with Deep Mahalanobis (Lee et al. 2018), ResFlow (Zisselman and Tamar 2020) and GRAM (Sastry and Oore 2019). To the best of our knowledge, these the best performing observer methods (i.e.. they do not retrain the model) to date. We also report MSP as a baseline (Hendryeks and Gimpell (016). We omit results tuned using OOD data provide fair evaluation. Such knowledge can be integrated within our framework, given the test distribution, by selecting which layers and channels monitor based on their discriminative power. Additionally, Area Under Receiver Operating Characteristics (AUROC scores and ROC) are provided in the Appendix (Section. G.3 & Fig.
The results for each method are summarized in Table 2 GRAM and MaSF outperform the competing methods in almost all scenarios. Specifically, MaSF has an advantage considering the overall quality (i.e.. mTPR, SD, and Min- However, GRAM has a slight edge in the DenseNet scenarios (e.g., CIFAR-100 vs. SVHN). We posit that the incremental residual connections of the DenseNet architecture (i.e., concatenation of input and output features) lead to a higher correlation between individual layers' test statistics (see Appendix Fig. 6). Ultimately, it leads to a Fisher test statistic with a higher variance while degrading the performance of MaSF (see discussion in Appendix D.2))
and it is much lighter, as we shall see in the following section.
In contrast, Deep Mahalanobis and ResFlow perform exceptionally well in several scenarios but fail in others. In particular, CIFAR- 100 (as in-dist) appears to be challenging in contrast to SVHN (as in-dist). Both methods rely Mahalanobis distance which involves estimating and inverting large covariance matrices. Thus, we conjecture that the performance loss can be attributed to the limited number of samples in the dataset. This is known phenomenon. For instance, Bai and Saranadasa (1996) showed a decrease in the Hotelling test power as the number of observations approaches the number of dimensions. We note that the authors use a Linear Discriminant Analysis (LDA) assumption (i.e., the covariance is identical across classes) to reduce the number of estimated parameters, however, appears to be insufficient in this case.
Notably, no method consistently outperforms the others. It is expected since a uniformly most powerful test cannot be designed without access the test distribution (Birnbaum) 1954). In essence, for each reasonable detection method, setting exists for which it is most powerful. This fact highlights the importance of our framework as a principled approach for constructing OOD detectors.
Finally, we show that MaSF generalizes well to other scenarios through an extensive evaluation in Appendix G Also, we demonstrate and analyze how to incorporate alternative test statistics from GRAM and Deep Mahalanobis within our framework (see Appendix G.2 and G.I
4.4 EVALUATION OF COMPUTATIONAL COST
Computational cost is a key factor when choosing an algorithm for a specific application that was often overlooked by prior work. A direct comparison between different methods may not be trivial. This is due to the fundamental differences between procedures and the potential for optimizing specific implementations. Thus, we suggest measuring the Test statistic Computation Time (TCT) to approximate the cost of similar methods (i.e., methods that use form of summary functions over intermediate feature maps). Table 3 presents a simple benchmark measuring the mean TCT (mTCT) and the mean global TCT time over all convolution layers in MobileNet- V2 (Sandler et al. 2019). This serves as a proxy for a typical CNN intended for edge devices, where resources are limited. In addition, report the DNN's 's compute time (measured independently from TCT) reference.
7

---

Published as a conference paper at ICLR 2022
Table 2: TPR at 95% of competing detectors on popular OOD benchmark.
Network
In-dist
Out-of-dist
MSP
Mahalanobis
ResFlow
GRAM
MaSF(ours)
SVHN
89.6
%6.1
98.4
CIFAR-10
TinyImageNet
59.4
94.9
96.1
98.8
97.8
LSUN
66.9
97.2
98.1
99.5
99.0
SVHN
26.3
62.2
48.9
89.3
83.7
DenseNet
CIFAR-100
TinylmageNet
17.5
87.2
91.5
95.7
93.9
LSUN
16.7
91.4
95.8
97.2
97.2
CIFAR- -
61.8
97.5
90.0
80.4
86.8
SVHN
TinyImageNet
80.5
99.9
99.9
99.1
99.8
LSUN
80.2
100
100.0
99.5
99.9
SVHN
27.8
75.8
91.0
97.6
99.0
CIFAR-10
TinyImageNet
42.3
95.5
98.0
98.7
98.4
LSUN
41.3
98.1
99.1
99.6
99.7
SVHN
15.1
41.9
74.1
80.8
89.7
ResNet
CIFAR-100
TinyImageNer
96.1
LSUN
56.6
70.4
96.6
98.2
CIFAR- 10
79.2
94.1
96.6
85.8
98.0
SVHN
TinylmageNet
74.7
99.2
99.9
99.3
99.9
LSUN
78.5
99.9
100.0
99.6
100.0
mTPR SD Min-TPR95
46.7
86.1
89.6
94.9
96.4
15
17,4
13.8
6,4
4.7
25.9
41.9
48.9
80.4
83.7
"Results for Mahalanobis and ResFlow when tuned using adversarial examples and input mee-rocesssing
Results include Mahalanobis distance (under a LDA assumption) as a popular baseline, GRAM deviation the lead competitor, and proposed method (MaSF). We find that MaSF mTCT smaller by factor of x35 and x2.5 when compared to the GRAM and the Mahalanobis statistics. Furthermore, Mahalanobis' TCT in reflects significantly lower compared to the full procedure from Lee et al. (2018) (as reported in Table since it does not include the costly input pre-processing strategy. This strategy involves introducing perturbations the input features at test time to increase their likelihood under the predicted class via backpropagation and second forward pass. Moreover, the MaSF statistic does not involve general -mtrix multiplication. Hence, its computation run concurrently appropriate hardware without blocking the resources required the CNN acceleration. Lastly, the abundant resources used to produce Table 3 are favor of the alternative methods. Therefore the observed speedup of MaSF is expected to increase on small devices with limited resources. The full settings for this benchmark are in Appendix
4.5 RANDOM CHINNNES SELECTION
popular trend in CNN design favors large models, where the number of parameters is greater than the number of available training samples. Naturally, the number of estimated parameters in the OOD detector is likely increases as well. This presents a challenge from aspects. First, from the statistical efficiency perspective, more samples are required to achieve the same power. Second, it leads to a substantial computational burden.
We explore dimensionality reduction via channel selection and focus on simple scenario where subset of channels randomly selected for each layer. These subsets remain fixed during the measurement and evaluation phases of the test statisties. Table 4 presents the mTPR of the MaSF variants on the same benchmark as in Table the channels are sampled uniformly for each layer, reducing the total number of channels
given rate. Results are averaged over five random seeds.
Surprisingly, the MaSF statistic's 's power is not dramatically affected by the proportion channels, indicating that the OOD signal is present across all channels. This implies that in some cases, channel sampling can be used to decrease the method test-time cost. In this case, MaSF suffers from negligible degradation (~ 1%) while using only 10% of the total number of channels, theoretically reducing the TCT cost by factor of 10. In Appendix G.I we show how sampling can be used improve the Mahalanobis test statistic.
8

---

Published as a conference paper at ICLR 2022
Table
Test statistic compute time.
Table 4: MaSF with random channels.
Single (ms)
Total (ms)
Channels
5%
10%
25%
50%
75%
100%
Method
\ /CT
Relative
Mean
SD
Mean
SD
Reduction
x20
x10
x4
x2
x1.33
x1
MaSF (ours)
0.22 0.03 11.6 0.23
0.93
mTPR
94.5
95.2
95.8
96.1
96.3
96.4
Mahalanobi
0.54 0.13
28.7
0.39
2.3
SD
6.7 6.1
5.5
5.14
4.8
4.7
GRAM
7.56 0.93 393.6 42.0
31.6
Min-TPR95 78.5 80.8 81.4 82.4
83.7
83.7
6 Total/TCT DNN (Forward) Time 12.44 1 0.32 ms. DNN
"Theoretical cost reduction for MaSF TCT is lineur in
Without pre-processing required for Table 2 results. which adds backward and forward passes.
the number of channels.
5 RELATED WORK
In the following section, we review the relevant observer methods and related work.
Hendrycks and Cimpel (2016) proposed using the Maximum Softmax Probability (MSP) as baseline method for OOD detection, positioned on the observation that a well-trained DNN tends to assign higher probability in-distribution vs. OOD examples. Liang al. (0177) presented ODIN as an improvement to the baseline method by combining Softmax temperature scaling with a costly input pre-processing strategy. The temperature and perturbations magnitude
paper-parameterss are calibrated on OOD samples. Lee et al. (2018) suggested the Deep Mahalanobis detector employing Mahalanobis distance over per-channel mean values to incorporate features from deep layers. At each layer, the minimal distance is selected over all classes. The results combined using weighted average, where the weights are determined via logistic regression on a small subset of the OOD dataset. The authors suggested replacing OOD data with adversarial examples a proxy with diminished results. Zisselman and Tamar (2020) extends the Mahalanobis detector with a learned likelihood function, using deep residual-flow models (ResFlow). Similarly to Lee et al. (2018), the method leverages the aforementioned pre-processing strategy, and the per layer scores are also combined weighted average following the same procedures. Later, [Sastry and Oore (2019) (GRAM) suggested a score based on the outer product (Gram matrix) over intermediate feature maps. The authors consider the total deviation from the minimal and maximal values observed on the training data for several Gram matrix exponent orders. Each layer contribution to the total sum is normalized by the mean deviation (of the predicted class) over portion of the validation set. Recently, [iu et al.. (2020) suggested mutator method that employs an energy-score that is maximized for OOD samples during training (similar to Hendrycks et al. (2018)) This score computation is efficient and can be used without tuning the model. However, its results deteriorate compared to SOTA observer methods.
To the best of our knowledge, Grosse et al. (2017) were the first to adopt a statistical hypothesis test to detect adversarial examples in DNNs. The proposed method is based on the Maximum Mean Discrepancy (MMD) test for equality of distributions between two groups. The test statistic distribution is estimated using permutations (i.e., shuffling the labels of the two compared groups). However, the method can only be used on groups with a significant number of OOD samples. Hence, cannot flag a single observation as OOD while it also requires extensive computing resources. Later, [unn and Lampert (2019) suggested utilizing the Kolmogorov-Sminoo test to compare the distribution of the observed percentiles of the model's Softmax scores with the uniform distribution. The test can efficiently determine if a network operates outside its specification (e.g., processed batch includes OOD samples). However, it is not suited for single sample gramularity.
A parallel line of work by Papernot and McDaniel (2018) proposed method for robust prediction by measuring the disagreement between a test sample and its K-Nearest Neighbors (K-NN) within the training data (i.e.. the number of samples whose label is different from the candidate label). The proposed algorithm adds the disagreement counts from all layers. It compares the result to the empirical distribution (measured using a holdout set in Papadopoulos (2008)), providing confidence measure that can be used for OOD detection. Recently, Raghuram et al. (2020) suggested an approach based on statistical hypothesis testing for detecting adversarial attacks. The proposed test statistics are also based on K-NN. The method obtains p- values per layer then combines them using classical combinations tests. However, the procedure's ultimate output is not valid valuue,
Raghurr

---

Published as a conference paper at ICLR 2022
as the dependency between the combined p-values is ignored (i.e., the assumption regarding the combined test statistic distribution does not hold). Moreover, a K-NN based approach has two crucial drawbacks. First, K-NN requires computing the features' distance per layer between the test sample and the entire training dataset at test time. Second, due to the use of full feature maps, this approach suffers from the phenomenon known "the curse of dimensionality" when used in conjunction with large inputs. These flaws can be associated with Raghuram et al. (2020) performance on simple OOD benchmarks compared Lee al. (2018).
is a related topic that was a subject of extensive study (Zadrozny and Elkan 2001] 2002, Platt and Karampatziakis, 2007 Nacini et al. 2015 Guo et al.) 2017p. The domain's 's objective is to assign confidence scores that correspond with the probability of the model error (e.g. by applying temperature scaling before SoftMax (Guo al., 2017) It defers from OOD detection since the test and training samples are drawn from similar distributions, and a correct prediction exists. Therefore, datasets, benchmarks, and metries designed for one space are generally irrelevant to the other.
Finally, model
6 DISCUSSION
This paper presents a novel framework for OOD detection in DNNs based on statistical hypothesis testing. Our approach does not rely prior knowledge regarding the test distribution nor changing the DNN's parameters. Next, we present detection scheme dubbed MaSF (Max-Simes- -as----iisssery Fisher). MaSF is compared to current OOD detection methods and demonstrates equivalent or better detection accuracy. Our procedure is also more efficient compared to other methods, a crucial property for real-world applications.
We suggest that without any assumptions regarding the test distribution, a uniformly most powerful test for OOD detection cannot be constructed (i.e., "no free lunch", |Birnbaum| (1954). Therefore, reasonable way to deal with OOD in the wild is to maintain a set of specialized detectors that are updated for new OOD patterns they emerge. Such detectors can be constructed and combined within our framework. Furthermore, modern combination methods can be used to improve the detection accuracy. For example, tests such as Vovk and Wang (2020) can deal with dependent p-values, or tests using random sampling of features as in Frostig and Benjamini (202T).
Munch",(irrnbaum
While our framework described in the context of classification, it can easily apply for regression (similarly to how the intermediate layers' features are handled). Class-dependent statistics could be used by assigning meta-classes to groups of inputs (e.g., via clustering) or by using a one-class approach. We intend to explore this in future work.
Moreover, the proposed framework can be used for other applications as well. One potential avenue is to test various hypotheses on the DNN itself. For instance, one can detect which channels are significant for the detection of specific classes using methods such as Benjamini and Bogomolov Hellerr et (018). This can be used for network analysis or even to reduce the computational cost of inference. Other use cases include active or continual learning as a scoring mechanism to detect novel samples. Another example is use framework to filter unwanted samples (i.e., outliers) from large unlabeled datasets in sell/semi-superiised scenarios.
22014),Hellr
REFERENCES
Felix Abramovich and Ya' acov Ritov. Staristical theory: a concise introduction. CRC Press, 2013.
Zhidong Bai and Hewa Saranadasa. Effect of high dimension: by an example of a two sample problem. Staristica Sinica, pages 311-329, 1996.
Stephen Bates, Emmanuel Candès, Lihua Lei, Yaniv Romano, and Matteo Sesia. Testing for outliers with conformal p-values. arXiv preprint arXiv:2104.08229, 2021.
Yoav Benjamini and Marina Bogomolov. Selective inference multiple families of hypotheses. Journal of the Royal Statistical Society: Series B: Statistical Methodology, pages 297-318, 2014.
Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series (Methodological), 57(1):289-300, 1995.
10

---

Published as a conference paper at ICLR 2022
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šınićć Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. Lecture Notes in Computer Science, page 387 2013. ISSN 1611-3349. doi: 10. URL httlthtp:://xx..o. do http:///xxx.oo..og///...... .tg.ioo.ggliie ott.:/.....011111110707777666660
25.
3
Allan Birnbaum. Combining independent tests of significance. Journal of the American Statistical Association, 49(267):559-574 1954.
Frank Bretz, Torsten Hothorn, and Peter Westfall. Multiple comparisons using R. CRC Press, 2016.
Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, and Alexei A. Efros. Large-scale study of curiosity-driven learning. 2018.
Lan Cheng and Xuguang Simon Sheng. Combination of "combinations of values". Empirical Economics, 53(1):329- 350, 2017.
Tarin Clanuwat, Mikel Bober- Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. CoRR, abs/1812.01718,, 2018. URL
http://arde //axivv wwtp/oggglaa//ee/e htsp//aa./11.0000
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-worf attacks deep learning models, 2017.
Ronald Aylmer Fisher. Statistical methods for research workers. In Breakthroughs in statistics, pages 66-70. Springer, 1992.
Tzviel Frostig and Yoav Benjamini. Testing the equality of multivariate means when p n by combining the hotelling and simes tests. TEST, pages 1-26, 2021.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (statistical) detection adversarial examples, 2017.
Frank E. Grubbs. Procedures for detecting outlying observations in samples. Technometrics, 11(1): 1-21, 1969.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
Ruth Heller, Nilanjan Chatterjee, Abba Krieger, and Jianxin Shi. Post-selection inference following aggregate level hypothesis testing in large-scale genomic data. Journal of the American Statistical Association, 113(524):1770-1777 2018.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks, 2016.
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure, 2018
Yosef Hochberg. A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75 (4):800-802, 1988.
Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pages 65-70, 1979.
Yen- Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized odin: Detecting out-of- distribution image without leamnnng from out-o--distribution data, 2020.

---

Published as a conference paper at ICLR 2022
Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. Densely connected convolutional networks. CoRR, abs/ 0bbs/1608.nm 06999, 2016. URLhtttp://xxxiv wttp://gga.bbooooo.o 06993
Alex Krizhevsky et al. Learning multiple layers of features from tiny images, Technical report, Citeseer, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, volume 86, pages 2278-2324, 1998. URL http://citteseex.
wwtp:ddwwwvvvweeeeeoo http::/ssuuamaaaaiyaaaoo 1.1.1............... 12....1... 7665.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks, 2018
Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks, 2017.
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection, 2020.
Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks, 2018.
Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Obtaining well calibrated prob- abilities using bayesian binning. Proceedings of the AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence, 2015:2901-2907, 2015.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 201 1.
Harris Papadopoulos. Inductive conformal prediction: Theory and application neural networks. In Tools in artificial intelligence. Citeseer, 2008
Nicolas Papernot and Patrick McDaniel. Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. 2018
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings the IEFE International Conference Computer Vision, pages 1406-1415, 2019.
John Platt and Nikos Karampatziakis. Probabilistic outputs for svms and comparisons to regularized likelihood methods. 2007.
Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, and Suman Banerjee. Detecting anomalous inputs dnn classifiers by joint statistical testing at the layers. arXiv preprint 20XX:20077.1117 2020.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chinh Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks, 2019.
Chandramouli Sastry and Sageev Oore. Zero-shot out-of-distribuutin detection with feature correlations. 2019.
Johnn Simes. An improved bonferroni procedure for multiple tests of significance. Biometrika, 73 (3):751-754, 1986.
Rémy Sun and Christoph H. Lampert. Ks(conf): A liight test if convnet operates outside of its specifications. Pattern Recognition, page 249-259, 2019. ISSN 1611 doi: 10. doi 978-3-030-12900 12939-2_188 18. htt.:p////ooeell.ooooo/000001//0
18
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2013.
Vladimir Vovk and Ruodu Wang. Combining p-values via averaging. Biometrika, 107(4):791-808,
2020.
12

---

Published as a conference paper at ICLR 2022
Vladimir Vovk, Alex Gammerman, and Glenn Shafer. Algorithmic learning in random world.
Springer Science & Business Media, 2005.
Jim Winkens, Rudy Bunel, Abhijit Guha Roy, Robert Stanforth, Vivek Natarajan, Joseph R. Ledsam, Patricia MacWilliams, Pushmeet Kohli, Alan Karthikesalingam, Simon Kohl, Taylan Cemgil, S. M. Ali Eslami, and Olaf Ronneberger. Contrastive training for improved out-of-distribution detection. 2020.
Jiayu Wu, Qixiang Zhang, and Guoxi Xu. Tiny imagenet challenge. Technical report, Technical report, Stanford University, 2017. Available online at http .... 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. Technical report, 2017.
Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Carla E. Brodley and Andrea Pohoreckyj Danyluk, editors, Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28 July 2001, pages 609-616. Morgan Kaufmann, 2001.
Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass prob- ability estimates. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 08 2002. doi: 10.1145/775047.775551
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. Procedings of the British Machine Vision Conference 2016, 2016. doi: 10.5244/c.30.87. BRLhttp:/ dx doi. owg.........222222 4
30.87
Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo. Hybrid models for open set recognition. arXiv preprint arXiv:2003. /2506, 2020.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
Ey Zisselman and Aviv Tamar. Deep residual flow for out of distribution detection, 2020.
13

---

Published as a conference paper at ICLR 2022
A
NOTATION
x
The set of all images of class in the measurement
P(a; 9, x") Empirical distribution of function g for class c, P(x:g,1))
1+ 1x (x))))))) (1++1)
pe
Class image distribution
The number of channel at layer /
F
The convlutional neural network
(X)
Class cansiconniiional p-value of image X, given class
(x)
Class-conditional p-value of channel in layer / for image X, given class c.
The maximum p-value for an image across all classes, gmax (X) = max (X)...., (X)) (X))
fmax
e
Layer reduction function, (X,) (f) (X,))
r
The layer reduction function
T
Spatial dimensions reduction function
Tcc
Channel reduction function
Trisher 9 Fisher statistic, Tisher (q) - In(g:)
Tsimes
(q)
Simes test statistic, Tsimes (q) min 9(0)
sE(.....m
Spatial reduction function value channel j layer 1, 11, = 7 (Fs,((())
Channel reduction function value at layer 1, (xx.) 7 (of (X;), Vay
X, Image
The class of image i
B
STATISTICAL PROCEDURE DETAILS
B.1 THFORETICAL VALIDITY
We are interested two distinct hypotheses to test. The first one, is to identify if the image is OOD,
H 3e:
Xtest P Hi Xtest o p Vc E [k]
ie., the goal is to find if there does not exist class distribution which the image was sampled from. The null hypothesis can be tested using (Xtest), yielding a valid test (i.c., a test which maintains the significance level, ax).
The second hypothesis of interest is Hiite
Hi -test
X o6
pireet
In this case, we are interested to know if the image sampled from the same distribution the class predicted for it.
When proving Proposition 1, we will rely on the fact that the maximum p-vuuue is always larger or equal to the true-class p-value. In Proposition 2, we will show that when the class-conditional p-value used, the accuracy of the model needs to be taken into account.
Important for both proofs, is that for sample X,y, the distribution of the test statistic gd (X() is stochastically at least as large as the uniform distribution (it is in fact larger than uniform since the test statistic is discrete) under the null hypothesis, that is. PHo (X) I a) I (see Vovk et al. (2005) Proposition 4.1).
The assumptions required for our propositions are the following:
$ For right sided tests, (X) Px(()).
14

---

Published as a conference paper at ICLR 2022
1. For each class c, the images in (X,, Vc) Xvat independent identically distributed.
2. Xtrain is independent of Xval-
3. (Yeest; ttett) are independent of Xval-
Proof of Proposition We need show that qmaar (Xtest) is stochastically at least as large as the uniform distribution when Hū is true, I.e., Jrest E [k]. Since geenn (Xrest) is valid p-value for Hffter
Vovk al. 2005 Proposition 4.1), it follows that
Pretect (g (Xtest) S Xtrain) S a.
(9)
Inequality (9) follows since to (Xtestt' Xtrain) is exchangeable with ft Xtrain) X E Xearh. Since y max (isst) Urnett (X then
Pho (X S a Xtraia) S Peg (g (Xees) S Xtrain)
Pagien (grrem (Xrest) S a Xtrain)
(10)
S a,
concluding the proof.
We now turn to find the required adjustment presented in Proposition] 2
Proof of Proposition In the following proof we will bound the probability of TIE as function of the classifier accuracy.
Prggeen Drans (Xtest) S a) Pagaess e be (Xrest) S Ok (ieest = Frest) P(irest Frost))
Pagqum (g (Xree) S aletest Ytest) P(Steet Yrest).
The equality is according to the law of total probability. Since Dtest is only function of Xtrain (and Xtest, it follows that
Paycan (g (Xrest) S alSrest ylest) ((PHooow (g (Xrest) S a x(Xtrasn)) Prest -test) S a,
where the expectation is over the distribution of Xtrain conditional on the event Prest Ylest, and the inequality follows from (9).
Moreover, Paolas (gines S a alStest * yrest) S 1. Therefore,
Pagree (a (Xeest) S a)) S P(Itest Yrest) + P(ftest F Yrest)
concluding the proof.
Assuming ftest is given by the CNN classification, then if the network does not make any mistake, the test is valid. If not, then the significance level needs to be adjusted. (For our experiments adjustment of the significance level is necessary, since we already calibrate the significance level in our experiments in order compare with the various competing methods, as detailed in Section 4.
Note, that the conformal p-values are obtained by conditioning solely on the training set, resulting p-values that are dependent on the validation set (TIE is guaranteed in expectation over all existing test samples and validation sets). Alternatively, Bates et al. (2021) suggested method in which the TIE guarantee obtained conditionally on the validation set as well. It ensures that the resulting p-values are independent but comes at a cost of decreased power. In practice, we find the conformal p-values are conservative implying the TIE is maintained (even on specific validation set, see next. section).
B.2 EMPIRICAL EVIDENCE OF METHOD VALIDITY
This section presents simulation and methods relating to the validity of the suggested framework and MaSF specifically. According to the theory presented above, the resulting p-values from
15

---

Published as a conference paper at ICLR 2022
procedure should follow uniform (or stochastically larger than uniform) distribution. We graphically assess the resemblance of the distributions using a qq-plot, in which the quantiles of the uniform distribution plotted vs. the quantiles of the p-values distribution. The identity line will represent distribution exactly matching the uniform distribution quantiles. Curves below/above represent stochstically smaller/larger than uniform distributions.
DenseNet3
DenseNet3
ResNe134
ResNet34
M&SF
MeSF
MaSF
MeSF
1.00
0.75-
0.50-
0.25
Empirical Quanties
0.00
1.00
0.75
00 00./B10 Fajo
0.50
0.25
0.00
1.00-
0.75
NHAS
0.50
25
0.00
0.00
0.25
0.50
0.75
1.00
0.00
0.26
0.60
1.00
0.75
1.00
0.00
0.26
0.50
0.75
1.00
Theoretical Quantiles
P-value Type
Maximum
Prediction
True Class
Figure 1: p-values from MaSF and McSF comparison to uniform [0.1] distribution. eCDFs are estimated using the training data.
We begin by examining the max p-values when using Lee et al. 20188) benchmark, used to report resltSr The distribution indeed appears uniform or stochastically larger, e.g. the maximum (although there are no theoretical guarantees for it). The exception is ResNet-34 in which, when inspecting quantiles away from the tails, the distribution appear stochastically smaller than uniform.
In order to assess our theoretical guarantees of the framework we abandon Lee et al. (2018) benchmark. We split our data into three sets: 1. DNN training set, Xtrain 2. OOD validation set, Xval 3. Test set. The eCDFs of the spatial and channel reduction is estimated using the training-set of the DNN. This introduces bias to eCDF estimation caused by the dependency between the DNN weights and the eCDFs. The alternative, is to use different independent set of observations, wasting valuable resources. Note, this does not harm the procedure validity. We the use set Xval to estimate the final layer reduction eCDF and Xtrain in order estimate the spatial and channel reductions eCDF. Since, Xval independent of Xtrain it ensures that we obtain independent p-values, thus our proposition hold. Intuitively, if the p-values the layer level are obtained from the same data that used estimate the channel reduction eCDF, they will tend to be larger than p-value resulting from observations independent of the eCDF. This makes new observations to appear "unusual".
We examine the resulting p-values using this procedure on the test test, and they are assessed using qq-plots (Fig. 2). In our experiments we focused on SVHN and CIFAR- 10 (CIFAR-100 has smaller validation set). We split their validation set as follows (the number of samples is per class): the DNN training-ser set used for the channels reduction eCDF, 800 samples for estimating the layers reduction eCDF, and 200 samples to inspect the p-values distribution.
We use the the simulation to demonstrate the theory. The p-vauues indeed follow uniform distribution, when splitting the data and ensuring the independence of each step. This implies that for all significance levels our framework yields valid test. The subsequent split of the validation set required in all methods in which the test statistic is function of estimated distributions. In Table 5 it can be seen that the power of the valid method does not decrease substantially compared to the experiment version. In both cases seems that the MaSF variant is the most powerful.
16

---

Published as a conference paper at ICLR 2022
DenseNet3 MaSF
DenseNat3 MeSF
ResNet34
ResNet34
MaSF
McSF
1.00
0.50
Empirccal Quantiles
0.25
0.00
1.00
0.75
NHΛS
0.2
0.00
0.00
0.26
0.50
0.5
1.00
0.00
0.28
0.50
1.00
0.00
0.76
1.00
0.00
0.25
0.50
0.75
1.00
Theoretical Quantiles
Pavalue Type
Maximum
Prediction
True Class
Figure 2: p-values from MaSF and McSF comparison to uniform [0.1] distribution. eCDFs estimated on holdout data from the validation set. CIFAR-100 was removed due to the small number of observations.
Table 5: Comparison of the suggested framework various OOD detectors. F- Fisher, S Simes, Me Mean and Ma Max, the order is first the spatial reduciton (Me / Ma), channel reduction (F S) and finally layer reduction (F S).
TPR
95%
AUROC
Network
In-dist
Out-of-dist
MaFF
MaSF
McFF
McSF
MaFF
MaSF
MeFF
MeSF
SVHN
98.3
98.4
94.5
95.3
98.4
98.6
94.7
96.1
CIFAR-10
TinylmageNet
97.7
88.4
91.2
93.3
97.7
88.4
91.3
LSUN
97.4
99
93.2
95.2
96.3
99.1
93.5
95.7
DenseNet
CIFAR-10
85.5
87.1
62.1
61.6
91
93.2
84.5
83.5
SVHN
TinylmageNet
99.6
99.8
92.6
93.7
99.8
99.9
97.9
98.6
LSUN
99.9
99.9
90.1
91.5
100
100
98
98.7
SVHN
99.2
97.2
97.4
98.8
98.4
96.9
96.3
CIFAR-10
TinyImageNet
96.5
98.5
93.3
94.3
94
97.4
91.6
93.7
LSUN
99.1
99.7
98
98.4
98.1
99.3
97
97.9
ResNet34
CIFAR-I0
97.5
95.2
93.9
95.2
96
95.4
95
SVHN
TinylmageNet
99.8
99.9
99.3
99.3
99.7
99.8
99.4
99.3
LSUN
99.8
100
98.9
98.9
99.8
100
99.4
99.3
Avg
97.3
98.1
91.9
92.6
97
98.3
94.7
95.4
SD
3.5
10.1
3
2
4.6
4.5
Min
85.5
87.1
62.1
61.6
91
93.2
84.5
83.5
B.3 GLOBAL NULL TESTS
In some cases, we care about set of hypotheses and testing whether none of these hypotheses false (the global null). Combination tests are used such cases. Let denote a vector m p-values, = (91,..., 9m)) used to test null-hypotheses, defined as Ho.. = n Ho.:- The test statistic is T(q). Every combination test summarizes differently. Hence their respective powers differ, depending the unknown distribution of the test statistic under H1-
......... Hu,m- The global null hypothesis is
Two common tests of the global-null are Fisher (Fisher 1992) and Simes (Simes) 1986). The Fisher combination test statistic is Frisher (q) 22 In(g:) If the p-values are independent and have uniform null distribution, then Trisher ~ X2m when the global null hypothesis is true. Otherwise. the null distribution of the test statistic is unknown, and its distribution needs be estimated. For
17

---

Published as a conference paper at ICLR 2022
cases, when not all hypotheses are of the same importance, the procedure can be modified to use weights, Fiihes (q: w) = >m= wi In(q,) Σ 1.
We can turn permutation-testing to test the weighted version of the Fisher combination test or the standard version when the assumptions do not apply. We will estimate the CDF of the test statistic under the null-hypothesis. When new observation arrives, one uses this eCDF to test the null-hypothesis.
Simes test (Simes 1986), involves ordering the p-values, Q() S 9(2) S... S km)), and calculating the following test statistic Tsimes (q) min 4(4) 3 If the p-values independent and have
ie(
uniform null distribution, then Tsimes (q) ~ Uniform(0, 1] when the global null is true, so rejecting the global null for Tsimes S a will maintain the TIE at level o (Simes) 1986). When these assumptions are not met, one may use the eCDF, as with the Fisher test statistic.
To demonstrate how these tests differ from one another, consider testing μt, the mean vector of m features, is zero. The evidence against the null (signal) is dense if the number of non-zero entries in is large and sparse if only a few entries in μ are non-zero. The Simes test is best suited to detect a sparse signal, small p-values will dominate the value of Simess In contrast, the Fisher combination test performs better when the signal is dense since Trisher aggregates the p- p-values of all features. See Cheng and Sheng (2017) for simulation comparing the two methods.
C EXPERIMENTAL SETTINGS
Models and in-dist datasets. In our experiments we focus on popular vision architectures: DenseNet (Huang et al. 2016) and ResNet-v1 (He et al.,22111111 2016). We follow the benchmark proposed by Lee et al. (2018), reusing the same pretrained models, datasets and evaluation code to report our results. Each architecture (DenseNet-BC & ResNet-34) is paired with set of weights, trained on CIFAR-10. CIFAR-100 (Krizhevsky et al. 2009) and SVHN (Netzer et al. (20TT) the appropriate training dataset is referred to as the in on-distribution while results are reported on the full validation split.
OOD datasets. We evaluate the OOD detection on the resized variants of LSUN (Burda al. 2018) and
(Wu et al. 2017) as processed and shared by Liang et al. (2017). We similarly include CIFAR-10 and SVHN validation splits when they not used for model training.
General calibration settings. Our proposed method requires estimating the statistics' class condi- tional distributions (i.e., the empirical CDF) over the calibration set to extract p- values.. Thus, effectively collect a set of percentiles for each statistic and every class in the in-distribution dataset. In our experiments, each sample is used once and without any typical train-time augmentations (i.e., we only apply the required manipulations for inference such as normalization and resize).
Furthermore, since our objective is rejecting Hi or HD, we can focus on estimating percentiles at the edge of the spectrum for each statistic (i.e.. the distribution tails), which improves the sensitivity to abnormal observations. For instance, when performing the Simes combination test, the output p-value is scaled proportionally to the number of channels and rank. Therefore, layers with many channels will require an extreme observation in one channel reflect abnormality for the entire layer (see Appendix E for more details).
In our experiments, we estimate percentiles between 0.1 and 0.9 at 0.1 increments. In addition, we collect smaller than 0.025 and greater than 0.975 for the two-tail test per-channel statistics. For the layer combination test, we simply use a uniform p-value resolution of 1e-3. We remind the reader that a high resolution of percentiles is not needed for testing but improves the estimation of the produced p-value distribution, which we use for the empirical validation of the method in Appendix B.2
Since any hardware has limited amount of memory, cannot always observe the entire calibration data simultaneously. For simplicity, we average percentiles over fixed size batches. Therefore, the minimal percentile resolution is batch sise Our experiments use batch size of 1000 samples of each class for models trained on CIFAR-10 and SVHN. For C1FAR-100, we use a batch-size of 500 due to the limited number of examples per class.
When the calibration set size permits (i.e., more samples can in a single batch), we also collect the top and bottom 200 values observed during the entire calibration process to improve tails quantiles
18

---

Published as a conference paper at ICLR 2022
estimation. After observing the entire calibration set, select 10 percentiles at regular intervals from the end of each tail. The resulting percentiles are determined by the total number of examples in the calibration data.
Test statistic Computation Time. Our measurements for Table include:
MaSF includes spatial max-pooling operation, followed by a sort operation required by the Simes test. MaSF p-value lookup time discounted.
Mahalanobis time includes average poollin followed by Mahalanobis distance for all classes under LDA assumption. Computation is done in matrix form to optimize execution time. Moreover, it does not include input pre-processing which was used by the original authors to produce results reported T
Gram time includes computation of the Gram deviation score over 10 matrix powers.
Total inference time for MobileNet- V2 with 1K classes and a synthetic input shape (1,3,224,224).
When calculating the mean of single and total (i.e., time per-sample) statistic execution time (TCT), we measure the wall time over the all convolution layers' outputs, that are induced using synthetic input. Global-mTCT times are measured over 10k + 1k warm-up iterations, while statisties are computed sequentially (blocking next layer statistic until current statistic compute is done). Single-mTCTT is also averaged over all monitored layers.
Hardware used to produce results is based on 2 x Intel(R) Xeon(R) Gold 6152 CPU @ 2.10GHz system with 2080ti GPU. Environment is based on Ubuntu 18.04, Py PoTch 1.6, cuda 10.2, cudnn 7.6.5. Network compute and TCT are decoupled and do not compete over compute resources.
D ADDITIONAL ANALYSIS
D.1 CHANNEL REDUCTION STATISTIC DISTRIBUTION
In Fig. 3] we illustrate the distribution of mean and max spatial reductions per-channel in ResNet34 for a single class compared to all other classes within the calibration set.
The distributions appears to be uni-modal, while abnormality be higher or lower than the target class statistic. This supports our choice of two-tail test for MaSF. Similar assumption can be easily validated for any reduction and data during the design phase of new method, given knowledge the in-distribution.
We note that did not find substantial evidence that favours prioritizing channels based such inter-class discrimination. compared to a simple random choice in our preliminary OOD detection experiments. However, this an intriguing avenue for future work.
D.2 LAYER CORRELATION ANALYSIS
The correlation between the test statistics of different layers seems to be a property of the network rather than that of a specific dataset or the test statistic used (comparing Fig. 4 and Fig. 6 B). It can be seen that for both ResNet34 and DenseNet, sequential as well as skip connection layers are strongly correlated and that the correlation tends to intensify as among layers the end of the network (Fig. f 6). Notably, the DenseNet correlation occurs even for distant layers, while in ResNet34 they tend to lessen rapidly as the layers are further apart. In addition, the correlation does vary greatly for different test statistics (see Fig. 5)
Our method uses the Fisher combination test to combine p-values from layers. Usually, it requires the assumption of independence, however, have cireumvented it by estimating the statistic eCDF (under Ho). We find that when combining the log p-values the variance of the statistic can be very large due to the strong correlations. Since the power of the test is function of the overlapping between the distributions under the null and alternative hypotheses, a reasonable strategy is to reduce the variance of the statistics.
simple method of achieving the goal is to select certain layers. However, this can lead to bias since the signal care for can be located in the omitted layers. Therefore, we use hierarchical clustering
19

---

Published as a conference paper at ICLR 2022
10
5A
6130
CM4
180
Clare 2M
On 40
1601
1.08
CN
Figure 3: Histogram visualization of max and mean spatial reductions in the ResNet34 for selected layers and channels for two in-distribution classes of CIFAR-10.
to group together layers with low correlation. Then, each cluster, combine the p-values using the Fisher combination test. The final p-value is obtained using a simple Simes test. We found that in experiments reducing the variance led to marginally improving the results, therefore they presented. We leave further study of this topic for future work.
ResN e34 SVHN Max Simes
ResNeI34 C CIFAR100 Max Simes
Inzer
Ineer
1.1
400c1
40.1
4.11
322
112
102
122
20
20.c
202
202
S0 dee Por on
342
402
4003
woo Booel
50 NNNNN goan aense ronde Saannasen iinteiee
Figure 4: Comparing the correlation layer level p-values for MaSF, between CIFAR-100 and SVHN with ResNet34. Red indicate positive correlation, and white neutral correlation.
E TESTING LAYERS WITH LARGE NUMBER OF CHANNELS
We have used the Simes test to combine all channels' p-values to represent the layer. possible alternative may consider, is to simply choose the minimum p-value instead. The issue with this approach is its ignorance of the layer size (i.e., the number channels). Suppose there are observations in calibration and p channels, assuming the channels independent, the
20

---

Published as a conference paper at ICLR 2022
Resh NVHN Max Simes
ResNet34 SVHN MMx Mahelanobis
101)
203
40.5
4.055
10.
401
300
301
101
[11
700
7.0
101
403
warg pood
Figure 5: Comparing the Max- Sime- statistic correlation with the Max Mahalannobis statistic across layers for ResNet34 and SVHN. Red indicate positive correlation, and blue negative correlation.
DenseNeet3 CIFAR10
DenseNets
Figure 6: Comparing Max-Simes statistic across layers on CIFAR100 and SVHN for DenseNet3 model. Layer names removed for eligibility. Red indicate positive correlation, and white neutral correlation.
probability of obtaining the minimal possible p-value, 1/n, under the null hypothesis, is (n 1/n) As p increase this probability tends to 1, rendering the layer uninformative. Thus, a good combination method should account for the total number of channels.
21

---

Published as a conference paper at ICLR 2022
The problem is addressed by the Bonferroni correction, which is common method for multiple comparisons corrections. It involves multiplying the channels' p-values by the number of hypotheses considered. Now, more channel than the number of available observations, n < the obtained p-value will always be 1. This is due to the conservative correction rule. Since, the p-values obtained by the Bonferroni correction, are equal or larger than those from Simes test. The Simes test is uniformly more powerful than the Bonferroni test (Simes) 1986). Other multiple hypotheses correction methods could be applicable such as Hochberg (1988) and more. One still needs to choose the how summarize the p-values into single value, example taking the minimum corrected p-value. We have chosen the Simes test, since it is powerful and simple test well suited for detecting sparse signal, with added benefit of adjusting for the number channels.
simply use the corrected minimal p-value. However, for layer that has
A different option is to simply use any reduction and simply evaluate the eCDF of the reduction again. However, as was demonstrated with using the minimum these functions can be uninformative.
F MAX-SIMES-FISHER ALGORITHM
Algorithm Srnitthtm--esccee describes the MaSF detector. assumes that the channel reduction function Toh the Simes test. Therefore, ti (Xtett) (0, 1), removing the need for another calibration. In the following sections we will experiment and analyze additional schemes by replacing the spatial and channel reductions.
Algorithm 2: MaSF: class class-conditional p-values
Input :F, Xuea,,
The network, input image and class of interest Input (., ximan) [a]]. [4]; // eCDF per-channel afzer max spatial reduct
Input :PP(:;t,, x
test statistic // Class conditionsl p-value for the input image
// sCDP
Output: :q" (X"uout (Nusn)
for
[L] do
for E [aj do
tya (ax(FF,((Xa)))
G (Xxer) tir Xirsin), 1 P(tt, (irsiii
end
Sort q. ((1)....: 4(..)
(Xeest) = max 9 (X),e
// Simes test
end
("(iest) 2 Σ
// Fisher
Return Pi( (Neest)) 1 x
0.015
Densty
0. 0.010
0.000
100
200
sóo
Fisher Combinaton
Type
Alternaivve
Nuil
Figure 7: empirical distribution of class conditional MaSF test statistic digit "0" from the SVHN dataset (bluc). The alternative distribution is based on the remaining digits.
22

---

Published as a conference paper at ICLR 2022
Table 6: Comparison of the adapted McMF Mocan-Mahllanobis-Fiii under LDA and GDA as- sumtions to Deep Mahalanobis (Lee et al. 2018). GDA suffers from the small number of samples CIFAR100. Applying random channel selection significantly improves the detector. Deep Maha- lanobis results include pre-processing. SDs are given in brackets where applicable.
Network
In-dist
Out-of-dist
Mahalnobis
McMF
McMF (GDA)
McMF 425%
McMF 025% (GDA)
SVHN
$8.7
90.1
83.1
92.5 (0.5)
88.1 (0.6)
CIFAR-10
Imagenet
88.6
94.7
96.6
93.1 (0.3)
95.6 (0.2)
LSUN
92.4
97.3
96.6 (0.2)
(0.1)
SVHN
48.7
65.8
47.6
(2.2)
60.3 (0.3)
DenseNet3
CIFAR-100
TinyImagenet
80.4
89.8
83.0
87.1 (0.8)
86.7 (0.6)
LSUN
83.8
93.3
84.6
(1)
87.6 (0.8)
CIFAR-10
92.5
78.5
87.1
(1.9
80.4
SVHN
TinyImagenet
99.1
99.1
99.6
98.6
99.3 (0.1)
LSUN
99.7
99.2
99.8
98.8 (0.3)
99.5 (0.1)
SVHN
87.5
67.4
(0.6)
81.7 (0.5)
CIFAR-I0
Iinglnugene
93.1
92.7
93.6 (0.3)
94.8
LSUN
99.9
95.9
97.9
97.0 (0.4)
98.3 (0.2)
SVHN
66.5
22.9
36.1) (1.7)
41.5 (1)
ResNer34
CIFAR-100
Tinylmagenet
56.7
87.6
88.4 (0.5)
82,9
LSUN
38.4
90.1
0,0
89.4 (0.7)
82.8 (0.7)
CIFAR- TO
95.2
97.1
98.6
(0.1)
97.9 (0)
SVHN
Tinylmugenet
99.3
99.7
99.7 (0.0)
LSUN
99.9
100.0
99.9
99.8 (0.0)
99.9
Average
83,9
86.7
74.2
88.2
87.5
SD
18.8
18.4
35.8
14.8
14.9
Min
38.4
22,9
G ADDITIONAL EXPERIMENTS
G.1 MAHALANOBIS EXAMPLE
In T present the full results of adapting the Mahalanobis statistics to framework, dubbed MeMF Sican--aaallnnobi--Fiseerr This is done by computing the Mahalanobis distance over the spatial means for all channels. The resulting distance converted into layer p-value using the appropriate eCDF. Finally, the layers are combined using the Fisher test statistic.
Our approach removes the necessity of OOD proxy for calibration, while the resulting detector is guaranteed to maintain TIE. We provide results with and without the LDA assumption (which we refer as GDA). It can be seen that by utilizing our framework, we improve Mahalanobis detector results (LDA) by 2.82%, although it does not use costly input vre-processing. When LDA assumption removed (GDA) the Mahalanobis detector severely degrades CIFAR-100 as expected due to the small sample size. We also evaluate random channel selection similarly to Section(4.5 results in Table are for the best channel sampling rate. Applying random channel selection significantly improves the detector, specifically in previously failed scenarios.
G.2 GRAM METHOD ANALYSIS
The GRAM procedure works well, as evident by the results. wish understand which elements essential to its success. To do we segment the method into 3 components, considering simple formulation based on single Gram matrix power.
Spatial reduction. The Gram matrix is effectively used to reduce the spatial dimensions to single value per-channel via row summation. Then, the min and max parameters are estimated using the observed values per-class. Finally, the statistic can be seen the per-channel deviation from the estimated extreme parameters. We note that these parameters are essentially quantiles that depend on the total number of examples per class.
Channel reduction. Each layer is summarized using the sum over all per-channel deviations.
Layer reduction. Since the number of channels impact the scale of each layer deviation. The layer deviation is divided by the expected deviation of the layer (E,,, (5,)), which is measured on 10% of the validation set. This normalizes the contribution of layers with different number of channels
23

---

Published as a conference paper at ICLR 2022
Table 7: Evaluating the importance of key components from GRAM method within our proposed framework. Replacing the Gram matrix with simple max pooling slightly improves OOD detection results. This indicates that the deviation function is more important than the Gram matrix which is the compute intensive part of the method.
Network
In-dist
Out-of-dist
GP
GP2 p6
Max+5
SVHN
0.22
95.3
94.7
CIFAR-10
TayyImageNet
91.5
97.5
LSUN
83.3
95.9
99.0
SVHN
81.3
DenseNet
CIFAR-100
TinyInageNet
77.0
85.7
94,2
LSUN
81.7
87.0
97.0
CIFAR
43.5
72.6
79,6
SVHN
TinyImageNe
67.1
97.7
99.3
LSUN
97.9
99.6
SVHN
0.29
97.9
93.5
CIFAR-10
TnyImageNet
64.3
94.8
98.0
L.SUN
67.9
98.6
99.3
SVHN
0.94
50.9
$7.0
ResNet
CIFAR-100
TinyInageNet
77.3
LSUN
81.7
80.0
97.0
CIFAR-IO
53.2
95.8
82.2
SVHN
TinIlngeeNet
68.6
99.6
94.2
LSUN
68.9
99.6
92.4
Average
91,7
SD
31.4
12.4
10.7
Min
0.20
50.9
57.0
to the total deviation metric. Finally, all normalized deviations are summed together to produce a final abnormality score per sample.
Using the above formulation, we aim to translate these components into similar counterparts within our proposed framework. This allows evaluate the importance of each component, which is difficult to perform using the original formulation.
Namely, we split the spatial reduction into separate functions. The first uses only the gram matrix (power-1) reduction denoted as GP,, while the second uses either GP1 or max pooling in conjunction with modified version of the deviation function (8*). Specifically, instead of measuring the maximum and minimum values over the training data, we choose to use less extreme quantiles (0.05 and 0.95). The new deviation function is defined follows,
quantileo.os S value V quantile value quantileq.os
8* (quantileg.os quantile value)
quantileo.og value quantilep.o osl value- quantileg qqaanille,oo os
value quantile
(11)
The channel reduction remains unchanged, i.c. channels are reduced using a simple summation over the observed values.
For the layer reduction is replaced with the fisher statistic which is better suited for the task of exposing irregularities over all layers compared simple summation. This also allows avoid using Eva (6i) normalization, by converting the observed layer deviations with their representing p-value. This leads to a uniform scale across all layers deviations.
We compare these variants on OOD detection tasks, where the main objective is to identify which elements are more dominant in their importance to the detection performance. From Table 7 we observe that 5 is crucial for OOD detection. We attribute its importance to normalizing the channels according to their magnitudes before adding them to the layer deviation score. On the other hand, when replacing the Gram matrix with a simple max-pooling, the detector is marginally better on average. Based on our evaluation, we posit that the success of the GRAM method could potentially be based more its deviation function than the specific of Gram matrix countering the original intuition linked to style transfer. However, is interesting to see GP, clearly outperforms max pooling in certain cases and vice versa.
24

---

Published as a conference paper at ICLR 2022
G.3 EXTENDED EVALUATION
We provide the results from Table including the AUROC in Table as well AUROC curve in Fig. We suggest that the AUROC metric is not well suited for OOD detection. In essence, it equally weighs TPR for all FPR, while lower TIE rates are predominantly relevant in practical applications. Furthermore, in Table 10 we report results for additional types of OOD intended to test the stability of the method (Peng et al. 2019; /Zhou et al. 2017, Clanuwat et al. [018 Xiao et al. 2017] [LeCun] et al. 1998). In these results, is, we used the entire test split of each of the additional OOD datasets.
8.
We compare our method against GRAM as the best available observer method and MSP as a baseline. Since GRAM did not report their results on these OOD datasets, we provide our best effort to reproduce their results. We rely on the authors' open-sourced code with a slight modification. The main difference is that we do not rely test data when constructing the test statistic for GRAM. This is done by splitting the training dataset into two splits. The first is a 90% split, used for extracting GRAM's s min/max values per layer. The remaining 10% split is used for estimating the expectation over the deviation score (see Sastry and Oore (2019)) In addition, we replace the cross-validation with random seeds and report the mean and (SD).
Table 10 shows GRAM and MaSF have comparable power MaSF generally seems to have an advantage when more samples available (SVHN in-dist), while GRAM seems to have slight advantage when fewer samples are available (CIFAR-100 in-dist). It can be seen that both MaSF and GRAM are relatively weak compared to MSP in the gear-distribution experiments (i.e.. CIFAR- 10-100). Conversely, MSP fails in "casy" scenarios, such as Normal random noise in TPR95 and AUROC. These results indicate that using multiple layers important to obtain greater power in many scenarios; however, it might be harmful in near-distriuutions.
(Zagoruyko and Komodakis 2016) models trained with Outlier Exposure (OE) (Hendrycks et al. 2018). OE is a training procedure that aims to reduce the Softmax over-confidence property on OOD samples. This is done by training the model parameters to maximize the entropy of the output for OOD samples. A large set of OOD data is used as a proxy for the unseen OOD test distribution. We use the models which were trained from "scratch" by the authors with Wu et al. (017) the exposed outlier dataset. MSP-OE results for CIFAR- 10-10-100 100 scenarios are better than both GRAM and MaSF. Notably, MSP-OE significantly improves some of the catastrophic failures of MSP observed on "easy" OOD samples, such as the Normal random noise scenario. An expected improvement is also visible for datasets that are similar the OE distribution, such as TinylmageNet. However, MSP-OE detection capacity is still limited compared to GRAM and MaSF for general OOD scenarios. Furthermore, we observe some of the weaknesses of MaSF. Specifically when CIFAR- is in in-distribution. We associate this failure with the disproportion between the number of channels and the number of samples available.
In Table II we report results for MSP, GRAM and MaSF using
Overall, these results expose some potential weaknesses of MaSF. However, we remind the reader that it is a relatively simple detector designed for efficient inference at fraction of the compute budget of GRAM. Moreover, the "free" MSP alternative is appealing only for near distributions, while the trend of failures on seemingly "easy" OOD samples not yet resolved. Thus, we conclude that as of date, there are uniformly powerful detectors. Detecting unknown test distributions remains a challenging task. Our framework attempts to support this effort by allowing to design and combine multiple specialized detectors, using principled procedures borrowed from classic statistical hypothesis testing- - while allowing the designer to maintain a meaningful error rate.
G.4 ASSIGNING LAYER PRIORITIES
It is clear from the experimental results that certain combinations of in-out distribution datasets (e.g.. CIFARIO vs. CIFAR 100 and vice versa) are more challenging than others. is also apparent that simple methods that only use the final layer sometimes produce superior results in this scenario while failing on seemingly simple cases (e.g.. Table I0 Normal random inputs). We suggest this failure is linked to the hierarchical processing in neural networks where the final layers target more complex features related to specific classes. In contrast, earlier layers target basic features common to the data seen during training and less class specific.
25

---

Published as a conference paper at ICLR 2022
Table 8: Full Table 2 including AUROC
In-dist
Out-of-dist
MSP
Mahalanobis
TPR 95%
Resilow
GRAM
MaSF
MSP
Malnalnobis
AUROC Resllow
GRAM
MrSF
SVIN
40.3
$9.8
86.1
96.1
98.4
59.3
97.6
97.3
59.1
99.6
CIFAR-10
Tey
59.4
94.9
96.1
98.8
91.8
94.2
97.5
99.1
99.7
99.3
LSUN
SVHN
26.2
62.2
48.9
89.3
83.7
82.6
85.6
87.9
97.3
96.1
DenseNet
CIFAR-100
TayylmageNet
17.4
87.2
91.5
98.7
93.9
71.8
92.7
98.1
99.0
98.1
LSUN
16.6
900
8014
80.8
92.1
96.7
98
97.1
SVHN
Tiny Tnageegee
80.4
99.9
99.9
99.1
99.8
95.5
99.5
99.9
99.7
99.6
LSUN
80.2
100
100
99.5
199.9
95.5
99.8
99.9
190
99.6
97.1
CIFAR-I0 10
TeylmageNet
42.2
95.5
98.7
4
90.3
97.9
99.6
99.7
99.5
LSUN
41.3
98.1
99.1
99.6
99.7
90.1
99.2
99.8
99.9
99.8
SWHN
15.0
41.3
M0.3
89.7
76.1
91.1
95.1
M6611
96.9
ResNet
TaylmageNet
17.6
77.5
LSUN
15.0
56.6
70.4
96.6
98.2
70.9
66.2
87.2
99.2
99.2
CIFAR-10
79.1
94.1
96.6
85.3
98
95.0
98.1
99
97.3
99.2
SVHN
TenylmgeeNet
75
99.6
100
91.5
99.9
100
99.8
99.8
Average
46.3
56.2
89.6
94.9
96.4
86.5
93.9
97.1
98.9
99.0
SD
26.0
16.9
13.8
6.4
4.8
9.4
9.0
4.2
1.4
1.1
Minimum
15.0
41.9
80.4
70.9
95.5
96.1
Naturally, when testing the global null, all p-values are combined with an even weight. This is since we do not know which hypothesis will be more significant without assuming any knowledge about the test distribution. We demonstrate the impact of assigning a larger weight on the final layer, reducing the contribution of the remaining layers uniformly. This can be achieved by replacing the standard Fisher test statistic s.t. Teisher_weighted
weighted (q) - -n ai In(q:) where a; 2 O, >n ag
In Table 9 we observe that assigning higher weight to the last layer improve the detector's performance. Specifically, MaSF performance improves in the near-distribution experiments (CIFAR- 100/10) while it deteriorates in the general case (i.c., assigning a high weight to the final layer TPR 95% drops to 0 for in-dist CIFAR- 100 and the ResNet, while the AUROC maintains smoother decline. This indicates that the produced p p-values resolution not sufficient to separate between OOD and in-distribution samples.
bascline from Table 2). Notably, when
This is due the discrete percentile estimation and the inherent conservativeness of p-value retrieval, which is required to maintain the statistical guarantees the method. Namely, the retrieved p-values conservative in the sense that they will always be larger than the p-value. For example, suppose we observe a realization of 4.01 for a specific random variable, where the estimated 0.04 quantiles is and the 0.05 quantile is 5, the assigned p-value (left-sided) would be 0.05 even though the realization is much closer to 4. Bounding the p-value in such a way ensures the validity of the method for any type of distribution. However, this reduces the power of the test. Even more so, when the number of observations is small since the percentiles are estimated sparsely.
In our experiments, the final layer has 100 channels (to match the number of classes), while the number of samples is per 500. Thus, the p-values resolution is insufficient to discriminate between in-dist and OOD all samples receive the lowest possible percentile. As a result, at FPR of 5%, TPR is reduced to O. AUROC behavior suggests that this gap closes for higher FPR rates. Furthermore, results suggest there seems to be a potential balance that benefits both near and general scenarios compared to vanilla MaSF We leave the topic of weight assignment to future work.
26

---

Published as a conference paper at ICLR 2022
(a) DenseNet
CIFAR1O
CIFARTO Tinylmagenet
CIFAR1D ISUN
HC7
$1.00
CIFARI00 SVHN
CIFAR100 vs. Tinylmagenet
CIFAR100 vs. LSUN
True Positive Rate
76.07
SVHIN vs. CIFARIO
SVHN vs. Tinylmagenet
SVHN LSUN
1.0
0.
1242
CRAN
False Positive Rate
(b) ResNet
Tinylmagenet
CIFAR100
CIFAR10D Tinylmagenet
CIFAR10D
True Positive Rate
HC7
$1405
SVHHN CIFAR1O
SVHN Tinylmagenet
SVHN LSUN
False Positive Rate
Figure 8: ROC plots for Table 2 MaSF typically yields better TPR for small FPR budget (<0.1).
27

---

Published as a conference paper at ICLR 2022
Table Comparison of weighted MaSF using varying weights on the final fully-connected layer. MSP as reference to the discriminative power of the networks' prediction based on this layer
TPR 95%
AUROC
Our-of-dist
Weighted
Weighted
0.05
0,1
0.25
0.5
0.75
0.05
0.1
0.25
0.5
0.75
benchmark.
SVHN
40.3
96.7
99.1
98.5 5
92.1
80
89.9
99.7
99.8
99.7
98.1
91,9
CIFAR-10
TinyImagenet
59.4
98.5
98.9
96.5
88.7
83.7
94.2
99.4
99.5
99.3
97.3
95.3
66.9
99.4
99.6
99.7
98.7
CIEAR-10
61.8
59.4
91.4
55.9
68.2
4022
92.3
97.5
97.7
97.5
95.6
955
DenseNes3
SVHN
TinyImagentt
99.5
57.5
99.6
98.4
LSUN
80.2
90.1
52.2
95.5
6
99.6
99.4
98
91,3
SVEN
26.2
34.7
85.9
S7
78.5
46.6
82.6
96.3
96.5
96.6
95.3
90
CIFAR-100
TinyImagenet
17.4
89.5
72.5
$2.2
97.77
94.4
LSUN
16.6
97.1
97.2
95.6
$1.5
39.6
99.1
99.1
98.8
96.6
92.3
SVHN
15
90.4
89.5
87.8
0
0
76.2
97.1
97.2
96
90
83.7
Tinylmagener
98.5
LSUN
15
97.3
84
0
0
70.9
99..2
9M 9
95.8
86.3
78.5
CIEAR-10
79.1
98..
91.8
91.6
X6.9
999.2
99.2
ResNet34
SVHN
TinyInagent
79.9
99.9
99
95.7
M0.S
93.5
99.8
98.9
97.4
LSUN
75
100
99.9
99
93.2
86.6
91.5
99.8
99.8
99.6
98.5
96.8
SVHN
27.9
85
54.5
39.3
99.8
99..
96.9
CIFAR-10
Tinylmagenet
42.2
944
74
90.3
99.5
96.6
94.1
LSUN
41.3
99.6
99.5
98.11
92
86.8
90.1
99.8
99.8
99.4
97.9
96.3
Avg
SD
25.3
5.3
33.1
30.7
9.1
1.4
3.8
5.8
Min
15
84.7
85.9
84
0
0
70.9
96.3
96.5
95.8
86.3
78.5
Near dittiiiiti CIFAR-10-11
10-10-
DenseNer3
CIFAR-10 CIPAR-10 LXX
CIFAR-FOI 100
40.6
23.7
30.4
40.9
46.4
46.2
89.3
76.2
80.9
85.4
85.5
85.2
ResNoc34
CHEAK-IH
17.6
6.1
4.2
0
0
76.6
06.2
00:4
73.6
74.3
CIFAR-10
CIFAR-100
33.7
34.7
37.8
41.8
40.6
39.5
86.4
83
84.4
86.1
86.2
85.8
Avg
27.6
20.2
23.2
23.1
22.6
72.6
75.6
75.8
75.4
SD
9.9
12.5
14.2
18.2
20.6
20.4
5,9
10.7
11.2
11.7
11.8
11.8
Min
55.8
57.2
28

---

Published as a conference paper at ICLR 2022
Table 10: Extended evaluation MSP, GRAM and MaSF for DenseNet and Resnet models. R, indicate resize and crop respectively.
TPR 95%
AUROC
In-dist
Out-of-dist
GRAM
CIFAR-100
40.6
26.3 (0.4)
18.9
89.3
72.2 (0.2(
69.7
Intograph
44.6
89.8 (U.4)
89,9
91
97.8 (0.1)
Quickdraw
11.4
100 (0)
100
84
100
100
Sketch
41.5
93.6 (0.3)
94.5
90.5
98.6
98.7
Fashiso-MNIST
80.5
100
97.3
99.9
LSUN (C)
52.1
87.7 (0.4)
83
93
97.3 (0.1)
96.3
(R)
99
99.9
CIFAR-10
Textues
49.1
89.9 (0.3)
88.1
91.5
97.9 (0.1)
97.6
Thy ImageNe (C)
96.3 (0.2)
93.3
93.8
99.2 (0)
98.4
TinlInagNNet (R)
59.4
98.6
97.8
94.2
99.7
99.3
K-MNIST
79.1
100 (0)
100
97
100
99.9
MNIST
(0)
100
96.8
100
Places-365
52.9
72.2 (0.6)
64.1
93.1
93.4 (0.1)
92.2
N(0, 1)
74.4
(0)
100
96.4
100
100
SVHN
40.3
95.8 (0.1)
98.4
89.9
99.1
99.6
CIFAR-IO
18.4
7.3 (0.2)
4.5
75.8
59.4 (0.2)
52.9
Infograph
37.9
74.1
67.8
84.3
93.8 (0.1)
90.7
Quickdraw
90.3
100 (0)
100
97.8
100 (0)
100
Skotch
94.6
Fashion MNIST
64.9
99.2 (0.1)
99.7
92.7
99.6 (0)
99.5
LSUN (C)
(0.4)
47.8
80.1
89,5 (0.1)
LSUN (R)
16.6
97.4 (0.1)
97.2
71
99.4
99.1
DenseNet3
CIFAR 100
Textures
23.4
(0.5)
$3.6
76.8
90.3 (0.1)
85.6
TinylmageNet (C)
24.4
87.6
80.8
76.2
97.2
TinyImageNor (R)
17.4
95.3 (0.1)
93.9
7L8
98,9 (0)
98.3
K-MNIST
99,9
MNIST
33.2
100 (0)
100
84
99.8 (0)
99.9
Places-365
28.3
35.1 (0.3)
21.7
80.7
80.1 (0.1)
68.8
NO,
100 (0)
100
36.7
100
100
SVHN
26.2
87.7 (0.4)
83.7
82.6
96.8 (0.1)
96.
CIFAR-TO
61.8
(1.66
86.8
92.3
94,,
97.3
CIFAR-100
61.4
75 (1.3)
88
91.9
93.5 (0.2)
97.4
Infograph
Quickdraw
19
100 (0)
100
76.1
100 (0)
100
Sketch
49.5
96.5 (0.1)
98.2
86.4
99.2 (U)
99.4
Fashion MNIST
99.5
100 (0)
100
99.2
100
L.UU (C)
67.1
91.7 (0.4)
95.4
92.8
98.2 (0.1)
98.9
SVHN
LSUN (R)
80.2
99.6
99.9
95.5
99.9
Textures
64.4
90.2 (0.3)
91.6
91.9
97.9 (U)
98.3
TmyllmggeNe (C)
(0.1)
TinyImageNet (R)
80.4
99.1 (0.2)
99.8
95.5
99.7 (0)
99.6
K-MNIST
100
100 (0)
100
99.1
100
100
MNIST
100
100 (0)
100
99
100
100
Places-365
68.2
84.5 (0.9)
92
93.1
96.9 (0.1)
98.2
NO,, 1)
(0)
100
97.2
100
CIFAR-100
33.7
38.3 (0.6)
32.4
86.4
82.2 (0.3)
82.1
Infograph
(0.3)
94,8
92.8
98.4 (0)
Quickdraw
71.2
100 (0)
100
92.8
100
99.9
Sketch
55.3
94 (0.3)
96.4
92.6
98.9 (0.1)
99.2
LSUN (C)
46
90.1 (0.3)
91.8
91.7
98 (0)
98.3
99.7
CIFAR-I0
Textures
37
89.7 (0.5)
91.8
88.7
98.1 (0.1)
98.5
TmyllmgeNNr (C)
96 (0.3)
96.4
91
99.2 (0)
99.1
Tay EmylmngeNNet (R)
42.2
98.3
98.4
90.3
99.7
99.5
K-MNIST
41.1
100 (0)
100
90.5
99.9 (0)
99,9
MNIST
Places 365
40.1
76.4 (0.6)
78.7
90.3
95.4 (0.1)
96
N(O, 1)
(0)
96.9
100
SVHN
27.9
97.1 (0.3)
99
89.3
99.4
99.8
CIFAR-10
17.6
9.6 (0.3)
5.7
76.6
69.3 (0.2)
Infograph
76.3 (0.1)
76.5
76.7
95.3
94.7
Quickdraw
11.8
100 (0)
100
68.1
99,9 (0)
99.7
Sketch
(0.3)
95.8
Fashion MNIST
36.9
98 (0.2)
99.6
86.4
99.3 (0)
99,4
(C)
(0.4)
59.9
74.1
92.2 (0.1)
LSUN (R)
97.5 (0)
98.2
70.9
99.4 (0)
99.2
ResNet34
CIFAR-100
Textures
17.8
66.5 (0.5)
70
75.3
92.4 (0.1)
TiyllmggeNetco (C)
21.9
88.9 (0.1)
88.8
77.2
97.7
97.4
TmyImageNet (R)
17.6
95.5 (0.1)
96.1
73.4
99 (0)
98.8
MNIST
MNIST
17.9
100 (0)
100
74.6
99.6 (0)
99.7
Places-365
19.6
(0.5)
32.2
77.8
86.6 (0.2)
(0.
100 (0)
100
77.1
100
99.7
SVIIN
80.8 (1.2)
89.7
76.1
96.1 (0.1)
96.9
29

---

Published as a conference paper at ICLR 2022
CIFAR-I0
79.1
(0.4)
98
93
(0.1
99.2
CIFAR- 100
87.4 (0.3)
97.7
92.7
97.5 97.5 (0.1)
99.1
Infograph
96.7 (0.1)
99.3
94.1
99.2
99.7
Quickdraw
95.5
100 (0)
100
97.2
100 (0)
99.9
Sketch
(0.1)
93.9
Pashioo-MNIST
98.3
100 (0)
100
99.1
100 (0)
99.9
LSUN (C)
93.6 (U.2)
98.7
93
98.6 (0)
99,5
SVII
LSUN (R)
99.6
100
91.5
99.8
99.8
Textures
81.8
95.5 (0.2)
98.4
94.4
99 (0)
99.5
TinylmageNer (C)
81.2
98
99.8
94.2
99.4
TmyImageNor (R)
79.8
99.3 (0.1)
99.9
93.5
99.7 (0)
99.8
K-MNIST
(0)
99.1
100
MNIST
96.4
100 (0)
100
98.5
100
99.9
Places-365
76.9
89.2 (0.3)
97.6
92.6
97.7 (0.1)
99.2
N 1)
85.8
100 (0)
100
96.2
100
Ave
51.4
88.3
88.5
87.7
96.8
96.3
SD
Min
0.0
7.3
4.5
36.7
59.4
52.9
Table 11; Extended evaluation with Outlirr Exposure (Hendrycks and Gimpel 2016 using Wide- ResNets. R,C. indicate resize and crop respectively.
TPR at 95%
AUROC GRAM
In-dist
Out-of-dist
MSP
GRAM
MaSF
MSP
MaSF
CIFAR-TOX 100
73.8
49 (I.3)
25.3
94.8
83.3 (0.4)
76.9
Infograph
99.1
96.5 (0.3)
93.9
99.7
99.2 (0)
98.7
Quickdraw
94.5
100 (0)
100.0
98.9
100 (0)
100.0
Sketch
98.8
97 (0.2)
96.0
99.6
99.3 (0)
99.1
Fashion-MNIE -NNIST
91.4
99.5 (0.1)
100.0
98.5
99.7 (0.1)
99.7
LSUN (C)
98.0
92.8 (0.4)
85.7
99.4
(0.1)
LSUN (R)
98.5
99.6 (0)
99.4
99.4
99.9 (0)
99.7
CIFAR-10
Textures
97.1
94.4 (0.3)
91.2
99.2
98.8 (0)
98.3
Taggem (C)
95.2
96.8 (0.1)
95.5
98.7
99.2 (0)
99.0
TmnylmageNet (R)
93.8
98.7 (0.1)
98.5
98.5
99.7
99.6
K-MNIST
100 (0)
100.0
99.1
(0)
99.9
MNIST
92.8
100 (0)
100.0
98.5
99.9 (0)
99.9
Places365
97.3
85.2 (0.9)
67.5
99.3
96.7 (0.2)
93.7
N(0, 1)
100.0
100 (0)
100.0
98.8
100 (0)
100.0
SVHN
98.0
98.1 (0.1)
98.2
99.5
99.5
99.5
CIFAR-10
20.1
8.5 (0.2)
3.5
79.5
61.8 (0.2)
51.4
Infograph
85.4
84.6 (0.3)
71.7
97.2
96.6 (0.1)
93.1
Quickdraw
98.4
100 (0)
100.0
99.2
100 (0)
99.9
Sketch
88.6 (0.4)
76.3
96.9
97.5 (0.1)
94.2
Pashion-MNIST
81.9
98.1 (0.2)
96.6
(0.1)
99.0
LSUN (C)
72.9
73.3 (0.4)
53.3
94.6
94 (0.1)
86.4
LSUN R
54.3
98.5 (0.1)
98.1
89.3
99.6 (0)
99.5
CIFAR-100
Textures
67.3
77.5 (0.2)
64.6
93.3
94.9 (0)
90.3
TinyImageNer (C)
91 (0.2)
85.0
(0.1)
96.7
TinyImageNer (R)
34.6
96.5 (0.1)
95.7
81.9
99.2
99.0
K-MNIST
82.6
99.9 (0)
100.0
96.7
99,9 (0)
99,9
MNIST
58.7
100 (0)
100.0
92.4
99.8 (0)
99,9
Places-365
70.7
51.9 (0.4)
26.3
94.5
(0.1)
74.5
N(0, 1)
71.2
100 (0)
100.0
96.0
100 (0)
99.9
SVHN
65.5
84.9 (0.3)
81.3
94.1
96.7 (0.1)
95.3
CIFAR- 10
100.0
99,9 (0)
100.0
100.0
99.8 (0.1)
99.9
CIFAR- 100
100.0
99.8 (0)
100.0
100.0
(0.1)
99.9
Infograph
100.0
100 (0)
100.0
99.9 (0)
100.0
Quickdraw
100.0
100 (0)
100.0
100.0
99.8 (0)
99.9
Sketch
100.0
99.9 (0)
100.0
100.0
99.8 (0.1)
99.9
FashionMNIST
100.0
100 (0)
100.0
100.0
99.9
100.0
LSUN (C)
100.0
100 (0)
100.0
100.0
99.9 (0)
100.0
SVHN
LSUN (R)
100.0
100 (0)
100.0
100.0
100 (0)
100.0
Textures
100.0
99.7 (0)
99.9
100.0
(0)
99.9
TinyImageNet (C)
100.0
100 (0)
100.0
100.0
100 (0)
100.0
TinyImagoNet (R)
100.0
100 (0)
100.0
100.0
100 (0)
100.0
30

---

Published as a conference paper at ICLR 2022
K-MNIST
100.0
100 (0)
100.0
100.0
(0)
100.0
MNIST
100.0
100 (0)
100.0
99.9
100 (0)
100.0
Places-365
100.0
100 (0)
100.0
100.0
99.9 (0.1)
99.9
1)
100.0
100 (0)
100.0
100.0
100 (0)
100.0
Avg
87.0
92.4
89.0
97.1
97.7
96.4
SD
19.4
17
21.9
4.7
6.3
8.7
Min
20.1
8.5
3.5
79.5
61.8
51.4
31


Reviews:
[{"review_id": "yzg4AkjudX8", "reviewer": "Reviewer_a8KZ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1. Considering OOD detection as a hypothesis test is an interesting formulation.\n\n2. Proposed approach considers an 'Observer'-style approach that does not require updating the underlying model. This is more efficient and easily accessible.\n\n3. Proposed framework achieves high efficiency while detecting OOD samples and can be deployed in real-time setups.\n\nWeaknesses:\n\n1. Formulating OOD detecting as a hypothesis test is previously explored in the following works. What are the novel contributions compared to these works. Authors are encouraged to add these in the reference section.\n\na. Haroush et al., \"Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks\"\n\nb. Cai & Koutsoukos, \"Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systems\" \n\nc. Bates et al., \"Testing for Outliers with Conformal p-values\"\n\n2. Experiments are limited in terms of comparisons with the recent state-of-the-art approaches and evaluating on benchmark datasets. Here are a few recent approaches on OOD detection and authors are encouraged to compare with these approaches.\n\na. Hendrycks et al., \"Deep anomaly detection with outlier exposure\"\n\nb. Liu et al., \"Energy-based out-of-distribution detection\"\n\nc. Lee eta al., \"Training confidence-calibrated classifiers for detecting out-of-distribution samples\"\n", "summary_of_the_paper": "This paper poses detecting out of distribution (OOD) samples as a hypothesis testing problem. Specifically, the authors propose a variant of the inductive conformal prediction framework where a 'p'-value is computed as a measure for OOD-ness. Finally, the OOD samples are detected by thresholding the 'p'-values.", "summary_of_the_review": "The approach lacks novelty and experiments are limited to justify the efficacy of the proposed approach. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gQEnR0B9Qm2", "reviewer": "Reviewer_XHJz", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Their framework is interesting and solves a relevant problem. It also seems to work well as, based on results, it outperforms baseline methods in several cases being also much more computationally light. \n\nHowever, a problem of the manuscript is that it quite extensively uses appendices as extra space (the whole paper is 28 pages long). Their framework is not completely described or cannot be understood if appendices are left out. I agree that improving this is a very hard task, but e.g. moving Algorithm listing (page 21) to the main body would already provide better understanding of the framework. \n\nSpecific comments:\n\n- There are few instances in which authors are used statistical terminology that are unclear for wider audience. For instance, \".. maintaining the Type 1 Error ..\" gives impression that the aim is to keep T1E error level in certain level. \n\n- I might have missed this point, but how was the threshold for p value for final decision (Algorithm 1 output) chosen?\n\n- page 3, \"... the p-value is given by $q(t_{obs})=P(...)$ ...\", this would be less confusing if it would be mentioned that p-values are denoted by q\n\n- Eq (1): I is not explained, I guess it's indicator function\n\n- Section 3.2: the procedure of obtaining p-values was not unclear based in this description (as I do not have too background in hypothesis testing). Perhaps the content of footnote 3 and/or Algorithm description from appendix could be moved to this section? \n\n- Could \"Related work\" be earlier? The baseline methods are now described here, but already used in the previous section. \n\n- Second paragraph of discussion mentions that OoD detector cannot be constructed without any assumptions regarding the test distribution. I am bit puzzled that how this is reflecting to the proposed framework. Is it that there is assumption that training data for the framework comes from the same distribution as the testing and results are more or less arbitrary if not? But, in the experiments, the OOD detector was trained using the same training data as CNN (i.e. in-distribution data?) without seeing any out-of-distribution samples?\n\n- page 13, \"F convlutional neural network\"\n\n- page 18, figure 3: what red and blue colors represent? Font is also quite small\n\n- page 21, Algorithm 1, the second last line: $T^L$ is taking $t_1^c$s as input, but based on (6), the input of $T^L$ is p-values. Algorithm is missing t->q calculation in this point?\n", "summary_of_the_paper": "The paper proposes a new framework to detect out of distribution samples for deep neural networks without retraining or modifying the model. The method is based on statistical hypothesis testing which is applied to activations of all or multiple layers. \n", "summary_of_the_review": "As the framework is novel and results shows good performance both in accuracy and computational complexity, I feel that the study deserves to be published. But the length might be issue.\n\nI chose \"marginally above the acceptance threshold\" as my recommendation due to the length issue. If there would not be any limits, my recommendation would be \"accept, good paper\".", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SYHo-b27vAl", "reviewer": "Reviewer_zW6V", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths\n\nS1) The paper frames the out-of-distribution (OOD) detection problem in the framework of hypothesis testing, which more directly links the problem to existing statistical tools and methods.\n\nS2) The paper proposes a new, computationally efficient method (MaSF) for performing hypothesis testing which uses multiple layers from a deep neural network. The authors report many empirical experiments which supports their method.\n\nS3) The paper is generally well-written.\n\n\n### Weaknesses\n\n**W1) Lack of comparison / discussion about \"calibration\" literature**\n\nI am admittedly not very familiar with the OOD detection literature. However, the motivation for OOD detection, as described in the paper's introduction, is that deep neural networks output prediction scores that tend to be overconfident. This sounds like a problem of \"calibration,\" which the paper does not mention. For example, a canonical paper on calibration of deep neural networks is \"[On Calibration of Modern Neural Networks](http://proceedings.mlr.press/v70/guo17a.html)\" by Guo et al. 2017. I would appreciate a discussion of how OOD detection is related too and/or different from calibration techniques, given that they are motivated by the same underlying problem of overconfident neural network predictions.\n\n**W2) Applicability to regression settings?**\n\nThe method described seems to be specifically designed for classification problems. Is there a similar formulation for regression problems? If not, I think it should be emphasized (maybe even in the paper title, e.g., \"A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Network Classifiers\") that the method is specific for classification.\n\n**W3) Needs more background / justification for Fisher and Simes tests**\n\nPerhaps this is due to my unfamiliarity with the Fisher and Simes tests, but I found the paper lacking in justification for why these were the two tests that authors use. What about other spatial reduction and channel reduction functions, such as simple mean/max reductions? What if you used a neural network for the reduction functions?\n\nOne way that the authors could help with this issue is to describe the Fisher and Simes tests in the body of the main text, instead of leaving it for the appendix.\n\n**W4) Implications for designing neural networks?**\n\nAre there any implications of the OOD detection method for designing better neural networks that are able to generalize better? For example, does this method present any natural mechanism for better calibrating neural network layers?\n\n**W5) Other clarity issues**\n\n- In Section 1.1 (Contributions), the definition of a Type I Error seems wrong, or at least poorly worded. The paper says that a Type I Error is _detecting OOD as in-distribution_, which to me is confusing. A better wording could be, for example, _incorrectly predicting \"OOD\" for an actual in-distribution sample_.\n- In Section 3.1 (Preliminaries), the definition for $N_{val}$ seems to be a typo. I think it should say $N_{train}$.\n- Why is the global null hypothesis denoted with an intersection $\\cap$ symbol? Is each $H_{0,i}$ a set? And if so, wouldn't it be a union instead of an intersection? Presumably this is notation that is standard in the literature, so pardon my ignorance. In any case, I think it would be helpful to clarify the notation in the text.", "summary_of_the_paper": "The authors formalize the out-of-distribution (OOD) detection problem in the framework of statistical hypothesis testing. The proposed method, called Max-Simes-Fisher (MaSF), is designed for deep neural networks and combines sample features extracted from multiple layers in the network. The method returns a $p$-value for every test sample at a specified significance level (_i.e._, probability of incorrectly predicting \"OOD\" for an actual in-distribution sample). Compared with existing methods for OOD detection, the proposed method achieves similar or better True Positive Rates (TPR, _i.e._, correct OOD prediction rate) with significantly lower computation cost.", "summary_of_the_review": "I preface this summary with an admission that I am not the most familiar with the OOD detection literature and global null hypothesis testing. However, I believe that formulation of OOD detection as a hypothesis testing problem is well thought out and beneficial for the ML community by rooting the problem in a concrete statistical framework. Furthermore, the proposed method seems sound (I skimmed through the math, albeit not thoroughly) and has significant speed-ups compared to existing methods, such that the method is far more likely to be deployed in real-world applications and edge devices. For these reasons, I believe this to be a valuable contribution to the ML community.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "MCbEPxffqtT", "reviewer": "Reviewer_Ni17", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n(1) Different from previous work, the paper considers the empirical distribution of each layer and channel in CNN and proposes to use global null tests with Simes and Fisher statistics to aggregate the p-values.\n\n(2) The proposed method is more computationally efficient than compared baselines while preserving comparable performances.\n\n(3) The effect of the proposed method on different layers and channels is discussed in detail. \n\n\nWeaknesses:\n\n(1) Proposition 1 and 2 are obtained by the underlying cumulative distribution without considering the gap between the estimated cumulative distribution and the real one. Equation (8) in the proof of proposition 1 is true only when q extracted is from the real distribution. It seems like it is considered as the real distribution in the following analysis, while the experiments consist of the estimated cumulative functions and corresponding p-values.\n\n(2) The proposed method uses p-values from every layer and channel in CNN, but intuition and arguments of Fisher and Simes tests as layer and channel reduction methods are not fully presented. As the core of the proposed algorithm,  it would be better to include the background of Fisher/Simes tests and why they are effective as layer and channel reduction respectively.  The current draft only includes a brief introduction with no discussion in the appendix.\n\n(3) It is great to see the computational cost is much lower than baselines such as Gram and Mahalanobis, but some baselines do not require retrain or expensive computation (e.g. Energy score (Liu et al., 2020)) are missing. It would be interesting to see the comparative performance, both computation-wise and detection accuracy-wise. \n\nSuggestions for improvement\n\n(1) The notation of t and T can be pre-defined before introducing equation (4) (rather than presented in the appendix) to facilitate better understanding.\n\n(2) Move the introduction of Fisher and Simes to the main paper.\n\n(3) Given the large OOD uncertainty space, it's desirable to also include more evaluation datasets as used in literature. \n\n(4) Add more recent baselines, especially the ones that are already computationally efficient. ", "summary_of_the_paper": "This paper constructs OOD detection as a hypothesis testing problem and extracts p-values from each layer and channel in CNN from the empirical cumulative distribution. Max, Simes, and Fisher test statistics are adopted as spatial, channel, and layer reduction methods (MaSF). Empirical results show that MaSF achieves comparable or better results than GRAM (Sastry and Oore, 2019), ResFlow (Zisselman and Tamar, 2020), and Mahalanobis (Lee et al., 2018) while significantly reducing the computational cost.", "summary_of_the_review": "Hypothesis testing using the empirical cumulative distribution is not novel enough as a \"new framework\": existing works already include such spirit of it by using FPR95 of the in-distribution samples. The theoretical part is also not strong enough without considering the gap between the true and estimated cumulative distribution. \n\nThe proposed method uses p-values from every layer and channel in CNN, but intuition and arguments of Fisher and Simes tests as layer and channel reduction methods are not fully presented. Lastly, the experimental evaluations can be further strengthened (by adding more recent baselines). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers the empirical distribution of layer/channel in CNN ,and proposes to use global null tests with Simes and Fisher statistics to aggregate the p-values. This method is competitive while computationally efficient. The underlying theoretical insights are discussed in detail.

The paper received mixed ratings, and the discussions weren't active. So, AC carefully read the paper and inspected all reviews. Reviewer a8KZ comments were factually inaccurate in listing references, and lack substantial feedback on the actual content of the paper. Hence, the review was down-weighted. 

The other negative reviewer Ni17, as an OoD expert, unfortunately did not offer more feedback to author rebuttals. From what AC comprehends, the authors should have clarified their The theoretical guarantee and compared properly with Liu et. al. 2020 energy-score (ES).

Considering the above, AC feels that the study deserves to be published.
Decision: Accept (Poster)