Provably Safeguarding a Classifier from OOD and Adversarial Samples
Authors: Nicolas Atienza
Johanne Cohen
Christophe Labreuche
Michele Sebag
Venue: ICLR 2025 Poster (2025)
Keywords: OOD detection
Adversarial detection
Extreme Value Theory
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: This paper aims to transform a trained classifier into an abstaining classifier, such
that the latter is provably protected from out-of-distribution and adversarial samples. The proposed Sample-efficient Probabilistic Detection using Extreme Value
Theory (SPADE) approach relies on a Generalized Extreme Value (GEV) model
of the training distribution in the latent space of the classifier. Under mild assumptions, this GEV model allows for formally characterizing out-of-distribution
and adversarial samples and rejecting them. Empirical validation of the approach
is conducted on various neural architectures (ResNet, VGG, and Vision Transformer) and considers medium and large-sized datasets (CIFAR-10, CIFAR-100,
and ImageNet). The results show the stability and frugality of the GEV model and
demonstrate SPADE’s efficiency compared to the state-of-the-art methods.

Published as a conference paper at ICLR 2025
PROVABLY SAFEGUARDING A CLASSIFIER FROM OOD AND ADVERSARIAL SAMPLES
Nicolas Atienza
Christophe Labreuche
Thales cortAlx Labs
Thales cortAlx Ix Labs
Industrial AT Laboratory SINCLAIR
Industrial AI Laboratory SINCLAIR
LISN CNRS-INRIA
Palaiseau, France
Paris-Saclay University
christophe. htt.tpiie...bbecchhhhhheeeeeoope
nicolas wecilasstiiizzatte www.enntttaaeeeeroouue.eeo com
Johanne Cohen
Michèle Sehag
LISN CNRS-INRIA
LISN CNRS-INRIA
Paris-Saclay University Saclay, France jcohen0 lisn. fr
Paris-Saclay University
Saclay, France sebag@1 1imnffii..ff.
ABSTRACT
This paper aims to transform a trained classifier into an abstaining classifier, such that the latter is provably protected from -it-str-iittiittoon and adversarial sam- ples. The proposed cample-efficeen Probabilistic Detection using Extreme Value Theory (SPADE) approach relies Generalized Extreme Value (GEV) model of the training distribution in the latent space of the classifier. Under mild as- sumptions, this GEV model allows for formally characterizing out-of-distribution and adversarial samples and rejecting them. Empirical validation of the approach is conducted on various neural architectures (ResNet, VGG, and Vision Trans- former) and considers medium and large-sized datasets (CIFAR-10, 10, CIFAR-100 and ImageNet). The results show the stability and frugality of the GEV model and demonstrate SPADE's efficiency compared to the state-of-the-arr methods
INTRODUCTION
key challenge in deploying learned models in real-world settings is the management of out-of- distribution (OOD) and adversarial samples. When learned model encounters data that deviates from the training distribution or is maliciously crafted to deceive the model (Szegedy et al. 2014] Goodfellow et al. 22015), it can lead to high consequence failures, especially in high-stakes applica- tions such as medical diagnosis, autonomous driving or risk analysis (Salehi al. 2022 Yang et al.) 2022).
The ultimate goal of machine learning is to achieve generalization, where the learned model encap- sulates core concepts accurately enough to effectively handle atypical but real-world samples (Ye) et al. 2021). A step towards generalization with respect to OOD samples is OOD detection, which equips the learned model with the ability to recognize atypical samples and refrain from making
risky predictions. OOD detection is approached from several directions, including methods based
on classification (Hendrycks & Gimpel 2017 Liang et al. 2018 Hsu al. (020). reconstruction
metrics (Jiang et al. 2023 Li et al. 2023). density estimation (Ren et al. 2019; et al. 2021 Du al.] (022bb and distance- based estimation (Papernot McDaniel 2018 Sun al. [022)] Dziedzic et al. 2022; Ming et al. 2023) (more in Section 6). OOD detection is complicated by the fact that, to the best our knowledge, there no universally accepted definition of an OOD sample. The dis- tinction between in-distribution and OOD data is inherently ambiguous, as different domain experts may classify the same sample differently based on their understanding and experience (Idrissi al.) 2023). Therefore, the validation of OOD detection methods relies heavily on experimental studies using well-curated datasets, such as near and far OOD datasets (Yang et al. 2022).
The code is publicly available at
xx 4/SAAD

---

Published as a conference paper at ICLR 2025
Regarding the robustness adversarial examples, many attack and defense approaches have been
designed along an ongoing arms race; see e.g. (an et al.] 2018 Zhang et al. 2019 Wang et al.
2020 Hosseini et al. 2020; Huang al., 2023; Mao et al. 2019 Liu et al. 2023 Carmon et al.
2019 Wang et al., 2023) (more in Section 66. It should be noted that human experts and models
tend to make different decisions when handling both OOD and adversarial samples, albeit in dis- tinct ways. Experts typically recognize an OOD sample as belonging to a known class, despite its atypicality, while the model assigns it random class. Conversely, experts perceive an adversarial example as typical of a specific class, yet the model confidently misclassifies it into a different class.
The approach presented in this paper, referred as Sample-efficient ProbAbilistic Detection using Extreme value theory (SPADE) and inspired by distance- approaches (Sun et al.) 2022), in- troduces an original model of the training distribution with respect to learned model (hereafter the teacher). Specifically, the distances between samples in the teacher latent space are modeled using the Extreme Value Theory (EVT) (Fisher & Tippeti, 1928). This model provides a sound and robust test for detecting and rejecting OOD samples; under mild assumptions this test also provably rejects adversarial examples with high probability, subject to a bound on their perturbation amplitude.
The contributions of the proposed approach are fourfold: i) it introduces a formal definition of OOD samples in relation to a teacher and its latent representation; ii) this definition leads to a statistically frugal OOD test based on EVT first principles; iii) this test operationally rejects OOD and adver- sarial samples with provable guarantees; iv) the effectiveness of the approach is experimentally and successfully validated against strong baselines for teachers with different neural architectures (He iet al. 2016 Dosovitskiy et al. [0221] Ming et al. 2023).
The limitation of the approach in detecting adversarial samples lies in its dependence on the Lips- chitz constant of the teacher. A relaxed characterization is therefore proposed, leveraging Lipschitz constants that are valid in the local neighborhood of the training samples. Experimental analysis of these local Lipschitz constants confirms the practical usefulness of the approach.
The paper is organized as follows. Section 2 outlines the formal background of distance- OOD detection and introduces extreme value theory. Section 3 provides an overview of the SPADE approach and presents its formal analysis. Sections 4 and respectively detail the experimental setup and the experiments conducted to validate SPADE against state-of-the-arr art methods. Section 6 discusses the contributions within the context of related work, and the paper concludes with per- spectives for future research.
Notations. Let DD = {(x1, y1),.... (Xn. yn)} denote the training set, iid drawn after the joint distribution Px,y, with X' the instance space and y fI..... .......,,, neJ the set of classes. The trained
teacher / is expressed as f = g h, where is the embedding in the latent space (h X + RRd)) and g is the mapping used to make decisions based on the latent representation.
2 FORMAL BACKGROUND
This section describes the main concepts defined by (Ye al. 2021 in the context of distance-based OOD generalization, and briefly introduces extreme value theory for completeness.
2.1 PROPERTIES OF LATENT IN- IN-DISTRIBUTION
As mentioned earlier, the complexity of OOD characterization arises from the highly complex and diverse nature of real-world data (Farquhar & Gal 2022). In the literature (Cimpoi al. 2214 Horn et al. 2018 Wang et al., 2022 Vaze et al. 2022 Bitterwolf et al. 2023), OOD characterization often relies on subjective assessment ("data that appear noticeably different from in-distribution to human observers"). A common used definition states that the OOD data are different from the known in-distribution, e.g., the training distribution (hereafter ID). However, this definition does not fully capture the specifics of OOD: OOD is not merely any distribution that differs from ID.
On the other hand, as noted by Papernot & McDaniel (2018); Sun et al. (022).. the latent space representation of in-distribution data often presents distinct characteristics, typically forming com- pact and well-separated clusters. Ye et al. (202l) formalize these properties in terms of variation and informativeness of the latent representation:
2

---

Published as a conference paper at ICLR 2025
Definition 1 Ye et al. (2021). The variation of embedding h across finite distribution DD noted Vohh, D). is defined as the maximum diameter over all classes c of the ball containing the distribu- tion h(x) for (x. y) D and y
v(h, = max cE (x.c)ED
sup
-x(')l
(1)
(x' ,c)ED
where the distance [x)(x)) h(x')ll is usually set to L2 distance across D if (h, D) <
Embedding h is said n-invariant
The variation of embedding / thus measures the maximum thickness and width of the latent mani- fold containing the image of (samples in) a class.
Definition 2 Ye et al. (2021)). The informativeness of embedding h across finite distribution DD,
noted Ih,, D), is defined as the average over all pairs (c, of distinct classes, of the minimum distance between h(x) and h(x') for (x. y) and (x',y' in DD with c # V = =d:
[(h,D) nn - 1)
min oFCeYy (x WDED y=oty =d
Σ
l1((x) h(x')ll
where the distance is usually set to Ln distance (see footnote 1). and ne denotes the number of classes. The latent embedding h is said 8-informative across if (h, D) > 8.
It is noted that informativeness and variation closely related to the compactness and dispersion metrics introduced and optimized in CIDER to achieve OOD generalization (Ming et al. 2023).
These definitions are expressed in terms of the supremum statistical stability and algorithmic exploitation (e.g., setting appropriate thresholds). The approach presented here addresses this challenge by exploiting extreme value theory, described below, and referring the reader to (Haan & Ferreira 2006) for a comprehensive introduction.
classes, raising the challenge of their
2.2 EXTREME VALUE THEORY
Dating back Fisher & Tippett (1928); Gnedenko (943). Extreme Value Theory (EVT) focuses on modeling and understanding the tail behavior of distributions. EVT is based on the premise that, under mild assumptions, the distributions of extreme events converge to a common form, even their original distributions differ. For instance, while the distributions of seismic intensities and the heights of rogue waves factors that respectively influence the design of buildings and oil rigs may differ, their extreme values (maxima) follow the same class of distributions. This limiting distribution is known as the Extreme Value Distribution (EVD):
Definition 3 (Extreme Value Distribution (EVD) (Fisher & Tippett 1928)). Let Z be random variable over the real-valued space R. Let (t) be the random variable defined as the maximum value over independent drawings of Z. When & goes to infinity, the limiting distribution of Z is
the cumulative distribution P(Z(() 2) 80000
Ge (2), expressed as one of the two parametric
models:
(1+
ifE $
Ge (2) exp
exp (=))
otherwise
with μ E R location parameter, E R4 dispersion parameter and R shape parameter referred to as extreme value index.
Overall, the EVT framework provides general parametric model for extreme events associated with a random variable Z, independent of the underlying distribution of Z itself. The universality of these models reflects the fact that modeling the extreme events associated with distribution relies only on the behavior of the tail of the distribution. This tail can take one of three forms: (i) an exponential the Gumbel distribution); (ii) heavy tail (E O, corresponding to the Fréchet distribution); or (iii) bounded tail (g O, corresponding to the Weibull distribution).
tail (E = corresponding
TThe Kullhack Leibler divergence is considered when embedding h is prohabilistic one. See Ye al.
2021 for more detail.
3

---

Published as a conference paper at ICLR 2025
Despite its broad applicabilityy, EVT has, to the best of our knowledge, seen limited use in machine learning, with notable exceptions in the area of anomaly detection (Smith et al. 2012 Siffr et al. 2017 French al. 2019).
3 SPADE OVERVIEW
SPADE is built upon the characterization of the training distribution (hereafter referred to as in- distribution, ID) with respect to the trained teacher model and its associated latent representation. This formal characterization relies on generalized extreme value (GEV) models, which allow for detecting and rejecting out-of-ID samples. For simplicity, and by abuse of language, out-of-ID samples are referred to OOD in the following. Interestingly, under mild assumptions, the GEV models also allow for detecting adversarial samples. SPADE thus defines an abstaining classifier, that behaves like the teacher all samples but those that are rejected by the GEV-based detection tests. This abstaining classifier provides probabilistic guarantees of OOD and adversarial sample rejection, subject lower bound on the magnitude of the adversarial perturbation.
3.1 EVT-BASED CHARACTERIZATION OF OOD
The setaaccebbssd OOD detection literature (see, e.g., (Papernot & McDaniel 2018 Dziedzic et al.)
2022 Sun et al., 2022) suggests that sample is likely to be an OOD sample if it suffiiinnll distant from the training samples of all (or most) classes in the latent space.
In SPADE, this intuition is formalized using generalized extreme value models, directly yiclding the probability for a sample to be OOD. Let (X, Y) denote the random variable following the joint distribution Px,y. For Y = let Ze be the random variable defined as the distance between ((X) and its k-th nearest neighbor in latent distance, belonging to D with same class c. By definition, the limiting distribution of the maxima Ze follows Generalized Extreme Value model noted G( with Pr(Ze <V) (v).
For each sample in the instance space and each class c, let me be defined [[h(x) hx I where Xknn,c is the k-thh nearest neighbor of x in latent space, such that (Ikmn,es c) belongs to D. As the true label of x is unknown at inference time, the proposed OOD test retains the lowest probability of x being OOD according to all G(o);
Definition 4 (OOD test). Let x denote an instance in X with 2, denoting its Euclidean latent dis- lance to its k-nearest neighbor of cluss in the training set D. The probability of x being an OOD sample, denoted (x), is defined as:
OOD(x) = min GO (z,) ccy
(4)
In other words, x is considered OOD if it is extreme according to each and every GEV model associated with one class. The choice to consider a separate GEV G(e) for each class c is aimed handling cases where classes exhibit different levels of variation the latent space (Def. D. In such cases, using a single GEV model for all classes could result in either erroneously rejecting samples from a class with high variation or incorrectly accepting OOD samples from class with low variation. Since the OOD test depends on the classifier's 's latent representation, one might wonder what extent different tests based on different classifiers are consistent; we shall return this issue in Section 5
3.2
ABSTAINING CLASSIFIER ON OOD SAMPLES
The SPADE abstaining classifier is built as follows:
Definition 5 (Abstaining classifier). Given teacher and confidence 1 1 with 0 < T < 1, classifier f- abstains from making predictions on a sample x if x considered to extreme with probability at least 1 w.r.t. its candidate class = f(x). With same notations as above:
f(x)
if (
fx(x)
(5)
abstain otherwise
where z. is the distance between h(xx) and its nearest neighbor of class in D.
4

---

Published as a conference paper at ICLR 2025
Note that the OOD test (Eq. is more tight than the above abstention test (Eq. 5 the latter exploits the class f(x) predicted by the teacher for x. However, both tests coincide under the common assumption that OOD sample lies closer to the samples in its class than to samples in other classes, everything else being equal.
3.3 ABSTAINING CLASSIFIER WITH PROVABLE GUARANTEES W.R.T. ADVERSARIAL EXAMPLES
By design, the SPADE abstaining classifier fr rejects the samples outside the training distribution with probability 1 r. Moreover, under the assumption that the latent embedding h admits finite Lipschitz constant, f+ also provably rejects the adversarial samples built by perturbing a training sample, subject a lower bound on the perturbation amplitude.
Formally, let us consider an adversarial example x built by perturbing a training sample noted x", with perturbation amplitude= ||x x=1 |. Let be the class of x* and let assume that the class assigned to is different from that of x": f(x) =
Let Gleee) denote the generalized extreme value model associated with the minimum latent distance among pairs of examples (x, x') respectively belonging class and c' (akin the informativeness, Def.
Glade (v) Pr M(X) M(X'II >
(X,Y) Px,,, (X', Pxy, Y c Y'=
Theorem (Global Pertubation Bound). Assume that the latent embedding is K- K-Lisschitz Let x be an adversarial sample obtained by perturbing training sample x" class with perturbation amplitude ε = ||x xl,,. and let f(x) = & c. Let x" of class d! denote the k-th nearest training either f+ abstains
sample in D of Then, with probability least following lower bound:
admits the
E 2 K
- G(6) (1-
(6)
Proof. If [1((x) h(x*)|| (G(d)) 11 7), then fr abstains x after Eq. Otherwise,
[[A(x") - ((x'))) < [1((x') 1 (x)]++ h(x''|1 KE +
Moreover, with probability 1 1- after the definition of G(ec'),
(G 7) < |h(x*) h(x'*)ll
(8)
Putting together Eqs. 7 and g concludes the proof.
A main limitation of the above result is that the lower bound on becomes trivial if embedding h is associated with a very high Lipschitz constant. This limitation is alleviated by considering instead
local Lipschitz constants, taking inspiration from (Jordan & Dimakis 2020 Shi et al. 2022), and providing local guarantees.
Theorem (Local Perturbation Bound). Let x" be a training sample of class Let r be positive real value, and let us assume that embedding h admits local Lipschitz constant K,(x*) in the vicinity of x":
Vx E X sit. |x-xx"1 < r, [|((x)) - h((x"l || Kx(x")) -xxxxxxx1| Let be an adversarial sample built by perturbation of x* such that ||x f(x) =d' &c.
x"1
r, with
Then, with probability at least T, either fr abstains on x. or satisfies the following lower bound:
2 (x+x) (1-))))(1----)))) (1-
Proof. Same proof of Thm
The guarantee thus depends on the local Lipschitz constants associated with the training samples.
The range of these constants is presented in Supplementary Material (SM, section E).
5

---

Published as a conference paper at ICLR 2025
3.4 ESTIMATING THE GEV MODELS
The estimation of the GEV models used in SPADE is detailed in the case of G(e), defining its approximation G(o). The same process is applied to learn an approximation of the Gle)) models, with the only difference being that the considered extreme values are minima instead of maxima.
After Siffer et al. (2017), a straightforward approximation of an EVD proceeds by sampling the extreme events along the block maxima method. However, This fitting process found to be sensi- tive to both the number of blocks and the block size. The proposed approach thus is the Peak Over Threshold (POT) method, which relies on the Barkaand-BBlkemaal to the second theorem of EVT (Balkema & de Haan 1974 Pickands [1775.. Formally, the POT considers the occurrences of events bypassing a given threshold t, noting that the distribution of these occurrences follows Generalized Pareto Distribution (GPD):
theorem, often referred
{
1 exp
- otherwise
(E) = Pr(Z -t
- 1 ((=-1) -1/E
(10)
Informally, for each class POT proceeds by fitting GPD model to samples the distances se that exceed a predefined threshold t.- Formally, for each sample (x;, e) in D, let 2; E IR be the distance between X; and its k-th nearest neighbor belonging to class in latent space. Let D. be the set including all such 2, for 2, > tc.
The parameters (Ne, Ce Ee) of model (()) are learned by maximum likelihood estimation (MLE) on De:
(pe, Ges Ee) arg max
Σ (n,ng(2))
(11)
H.a.E sED.
with L the log-probability density function of the GPD. MLE preferred over alternative methods, such as the method of moments, due to its higher robustness and efficiency (Siffer et al. 2017).
Eventually, model (ic) approximately characterizes whether a given sample is OOD with respect to class with confidence т. The OOD test (Def. 4 is accordingly approximated as:
OOD(x) min G( (Ex)
(12)
.cy
Overall, the SPADE abstaining classifier relies on Def. Alg.
5 where the GEV models are learned using
Algorithm SPADE. Learning EVT models
1: Input: training set D, integer k, threshold > O.
2: Output: GEV distributions (Z(o) of each class
3:
4: for E y do
5: T(c) =
6: for (x,c) do
7:
Compute normalized activation: z = 1x(x/ [[(()]]
(k)
8:
Compute distance to k-th nearest neighbor in class
min
(l|z
z'l))
(x' CDED
9:
if t: (e)) (c)) (v) end if
10: end for
Fit an Extreme Value Distribution (G(el) from extreme samples of (c))
12: end for
return (o
3.5 DISCUSSION
SPADE retains the main benefits of distance-based OOD detection approaches, as it is agnostic the structure of the OOD distribution and easy implement. Furthermore, being grounded in

---

Published as a conference paper at ICLR 2025
the first EVT principles, it enables estimating the probability for sample to be OOD. Lastly, the abstaining classifier fr also rejects adversarial samples, subject to a lower bound on the magnitude of adversarial perturbations.
A potential weakness is the complexity of approximating GEV models, quadratic with respect to the number n of samples. This raises the question of whether stable and accurate GEV approximations can be achieved when aggressively subsampling the training set. A second issue concerns the effec- tiveness of the lower bound used reject adversarial samples (Eqq. [ specifically the question is
whether (ee,')) (1 r) G(e) -1 (1-r) is strictly positive for non-trivial confidence levels 1 - Both questions will be examined experimentally.
EXPERIMENTAL SETTING
This section outlines the experimental setup used evaluate SPADE comparison state-of-the- art methods. All experiments are conducted on Tesla A100 80GB GPUs. Additional details are provided in the Supplementary Material (SM).
Goals. The experiments aim to empirically answer four questions: the accuracy of the SPADE OOD detection tests, particularly in comparison with distance-based approaches (QI); their sensi- tivity with respect the considered teacher (Q2); similarly. their accuracy of the adversarial sample detection tests, compared with state-of-the-an the-art methods (Q3); the computational complexity and stability of the approximate GEV models embedded in SPADE (Q4).
The performance of SPADE is compared to that of five established baselines: the seminal MSP (Hendrycks& & Gimpell 2017), ODIN (Liang al. 2018), MDS (Lee et al. 2018). KNN (Sun al., 2022). and its extension CIDER (Ming et al. 2023) (further discussed in Section 6).
Metrics. The comparative evaluation is conducted using the OpenOOD framework (Zhang al. 2023), with performance assessed by standard indicators: The Area Under the ROC Curve (AUC) measures the average rate of correct OOD/adversarial sample detection across all confidence levels, corresponding to the true positive rate as described in (Zhang et al. 2023 The FPR95 indicator represents the fraction of true samples misclassified as OOD (respectively, adversarial) when the detection threshold ensures 95% of OOD (resp., adversarial) samples are correctly rejected.
Benchmarks. Three medium- and large-sized datasets are considered: CIFAR-10 10, CIFAR- (Krzhevsky)) 2009) and ImageNet-IK (using the ILSVRC2012 version). Three types of neural architectures are used to assess the generality of the SPADE approach: ResNet (He et al.) 2016), VIT (Dosovitskiy al. 2021), and VGG (Simonyan & Zisserman 2015). The distance in latent space between sample and its k-th nearest neighbor is calculated using the normalized L2 distance, following (Snn al. (2022). OOD samples are sourced from near-OOD and far-OOD datasets, following (Cimpoi et al.) 2014 Vaze et al. 2022 Horn et al., 2018 Wang et al.) 2022, Bitterwolf al. 2023). as detailed in SM. Adversarial samples are generated by perturbing training samples
using FGSM (Goodfellow et al. 2015). PGD (Madry et al. 2018) and AutoAttack (Croce & Hein attacks, with perturbation amplitude e ranging from 0.001 to 0.004.
2020
EXPERIMENTAL RESULTS
OOD Detection (Q1). The performance of SPADE is reported in Table for a ResNet teacher in the representative case of ImageNet-IK. K. For the considered near-OOD datasets, the best- best-performing method is MSP, while for far-OOD datasets, KNN achieves the best results. In all cases except one, SPADE- -eseeet is slightly outperformed by KNN. In terms of ranking (determined by the average of AUC and FPR95), SPADE ranks second best on both near- and far-OOD datasets.
Sensitivity of OOD Detection (Q2). The impact of the considered teacher the performance of the SPADE OOD test is illustrated in Table comparing SPADE built on teachers ResNet, VGG,
Note that the requirement for (G( to be sufficiently large is reminiscent of the clustering assumption that underpins semi-supervised learning (Rigallet 2007
7

---

Published as a conference paper at ICLR 2025
OOD detection on ImageNet-IK: performance of SPADE with ResNet teacher compared to that of baselines MSP, ODIN, MDS and KNN in terms of AUC (the greater the better) and FPR95 (the lower the better; best performances bold). The rank, averaged over far and near OOD datasets, is computed based on the half sum of AUC and FPR95.
Table 1:
Near OOD
Far OOD
SSB Hard
NINCO
INaturalist
Textures
OpenImages-O
Rank
AUC
KPR95 4
AUC
KPR95
AUC
FPR95
AUC
FPR95
AUC
FPR95 !
MSP
72.53
74.43
80,66
57.72
87.78
44.08
82.81
59.16
85.21
49.62
3
ODIN
72.51
77.36
77.55
70.83
89,51
41.46
87.02
56.58
86.33
54.10
4
MDS
52.15
90.46
68.49
71.66
76.49
56.07
94.11
27.07
77.68
59.66
KNN
62.80
84.08
79.30
58.92
84.62
42.39
96.06
23.39
86.38
44.24
SPADE
61.91
85.27
77.99
61.04
85.26
44.84
95.86
24.63
85.79
46.33
2
Table 2: OOD detection on CIFAR-10: performance of SPADE with teachers ResNet, VGG and
ViT-B16, compared to that of baseline CIDER.
MNIST
SVHN
Textures
Places-365
AUC
FPR95
AUC
FPR95
AUC 1
FPR95
AUC
FPR95
AUC
FPR95
CIDER (d=512)
71.56
70.34
68.84
71.86
57.51
78.43
71.06
70.70
71.73
69.97
ResNet- 18 (d=512)
90.41
32.14
93.646
21.91
92.17
22.94
91.97
25.68
91.03
30.41
VGG-16 16 (d=512)
76.04
66.84
89.37
35.12
81.53
44.06
80.47
50.73
72.14
77.27
VIT-B16 (d=384)
96.27
20.18
94.52
12.23
81.78
34.83
99.97
00.06
99.67
00.51
VīT-B 16 with the CIDER baseline on CIFAR-10. These results show that the detection accuracy in- deed depends on the teacher and its latent space, with SPADE-ViT-BIL dominating ResNet (except on SVHN) and ResNet strongly dominating VGG.
Still, the performance does not merely depend on the size of the latent space. The discrepancy between the AUC and FPR95 indicators suggests that the optimal detection thresh-
EV index
0.0
old varies depending on the dataset. Notably, all SPADE OOD detection tests outperform CIDER, which is based KNN and specifically targets OOD detection. We shall come back this remark in Section G
-0.20.0
0.5
1.0
Sampling Ratio
(a) Extreme Value Index E
Location u
0.50
Detection of Adversarial Samples (Q3). Table 3 reports performance of SPADE DE with a ResNet teacher on the representative cases of CIFAR-10 10 and CIFAR-100, compared with MSP, MDS, KNN and CIDER. For all perturbation am- plitudes, SPADE ranks first w.r.t. AUC (and second w.r.t. FPR95). In terms of FPR95, KNN ranks first on CIFAR-10 10, while MSP ranks first on CIFAR-100. Overall, SPADE be- haves on par with, or better than OOD detection methods w.r.t. the detection of adversarial examples. The slight AUC improvement suggests that SPADE may capture more sub- tle differences between on-disribbution and adversarial sam- ples. Conversely, the high FPR95 values suggest that SPADE tends to be overly cautious, rejecting true samples the con- fidence level where 95% adversarial samples are rejected.
0.25
0.0
0.5 Sampling Ratio
1.0
(b) Location parameter μ
Dispersion o
0.06
0.04
0.0
0.5 Sampling Ratio
1.0
(c) Digression parameter ir
Figure 1: Stability of EVT pa- rameter estimation wrt sampling ra- tio and estimation variance for one class of CIFAR-100 ResNet-18.
Computational Frugality: Stability of GEV Models and SPADE Performances (Q4). The stability of the GEV models is analyzed by sub-sampling the training set. Fig. D reports the G(e) hyperparameters the sub-sampling rate, in the case of the ResNet teacher on CIFAR-100. The figure shows: i) a low sensitivity of the tail index (Fig. (a)); ii) a decrease in with the sub-sampling rate (Fig. (b)): iii) low sensitivity of the dispersion parameter (Fig. (c)).. Additional results are presented in SM, confirming the relevance of the lower bound on the amplitude of the adversarial perturbation (Eqs. and
8

---

Published as a conference paper at ICLR 2025
Table Adversarial samples detection on CIFAR-I0 and CIFAR-100, with perturbation amplitude from .001 to .004: comparison of SPADE (ResNet teacher) with baselines MSP, ODIN, MDS and KNN in terms of AUC (the greater the better) and FPR95 (the lower the better; best in bold).
0.001
E
0.002
E
0.003
E
0.004
Average
AUC
FPR95
AUC
FPR95
AUC
FPR95
AUC
FPR95
A
FPR95
MSP L
81.68
79.08
81.90
78.70
82.06
79.14
82.20
78.
81.96
78.76
CIFAR-11 10
MDS
81.46
69.34
81.51
68.76
81.57
69.04
81.61
69.42
81.54
69.14
KNN
85.65
54.46
85.75
54.19
85.85
53.91
85.92
54.29
85.79
54.21
CIDER
85.46
55.68
85.50
54.79
85.53
54.93
85.60
54.57
85.52
54,99
SPADE
85,96
55.02
86.06
54.51
86.15
54,40
86.24
53.78
86.10
54.43
CIEAR- 100
MSP
83.24
51.84
83.39
50.57
83.54
49.94
83.68
49.64
83,46
50.50
MDS
60.34
82.93
60.29
83.01
60.25
82.60
60.19
83.01
60.27
82.89
KNN
83.54
56.83
83.67
56.39
83.79
55.64
83.89
55.13
83.72
56.00
CIDER
82.75
63.17
82.85
63.57
82.95
63.17
83.06
62.34
82.90
63.06
SPADE
84.33
53.13
84.45
52.84
84.56
52.44
84.66
52.44
84.50
52.72
100
Method
100
Method
SPADE
SPADE
KNN
75
KNN
50
Metric
Metric
FPR95
50
FPR95
10
10-1
10
AUC
10
10-1
10
AUC
Sampling Ratio
Sampling Ratio
(a) near OOD
(b) far OOD
Figure 2: Sensitivity analysis of the OOD detection on CIFAR-100 w.r.t. the subsampling rate of the training set: AUC (dashed line) and FPR95 (solid line) performances for SPADE (in blue) and KNN (Sun et al. 2022) (in orange; better seen in color).
As expected, the stability of the ( models results in stable OOD detection performances (AUC and FPR95) when subsampling the training set (Fig. ( Quite the contrary, the OOD detection performances of KNN are significantly deteriorated when aggressively subsampling the training set, particularly so near-OOD (Fig. 2 left).
POSITION W.R.T. RELATED WORK
As said, the lack of robustness of ML models w.r.t. OOD and adversarial samples remains major obstacle for deploying them in real-world settings (aalhii et al. 2022 Yang et al.) 2024). Conse- quently, hot research topic is concerned with designing more robust learning methods.
The detection OOD samples has been tackled along different approaches, ranging from learning with rejection (Bartlett & Wegkamp 2008), to anomaly detection (Bulusu et al. (020). novelty de- tection (Markou & Singh [203a b). and open set recognition (Boult et al.] 2019). To the best of our knowledge, the OOD detection problem was first formalized by Nguyen al. (2015); the early MSP approach, observing the margin between the logits of the trained teacher and exploiting the fact that it behaves differently for in-distribution and OOD samples (Hendrycks Gimpel 2017). still is among the most effective ones (Table []. Along the same line, ODIN exploits the gradient information separate in- and out-of-dit dfiittiiuuuuio samples (Liang et 2018).
Some approaches address OOD detection
yet another supervised learning problem, handling OOD samples as if they belonged to an additional class and (re)-training the model accordingly Du et al. 2022a Zhang et al. 2022) (see also (Hsu et al. 2020)).
Quite a few other methods, based on the so-called manifold assumption, focus on identifying the manifold on which real samples lie; one option is to consider the latent representation of an auto- encoder (AE) trained solely on real samples. As empirically shown by e.g. Ren et al. (2019); Liu et al. (2021): Du et al. (2022a), the reconstruction enror of OOD samples through the AE is much higher than for real samples; an effective OOD test can thus based on this error. On the positive side, this reconstruction emrrr defines general criterion and does not depend on the teacher;

---

Published as a conference paper at ICLR 2025
the negative side, it does not leverage class information. Another option is based on modeling the behavior of true samples in the penultimate layer of the neural net, using probabilistic models (Du
ct al. 2022b
Liu al. 2021 Ren al. 2019).
Finally, the option most closely related to SPADE is to consider the latent representation of the teacher itself, done in bistance-based OOD detection approaches (Papernot & McDaniel] 2018 Lee et al. 2018), particularly in KNN (Sun et al. 2022). The difference is that KNN exploits the distance 2 of sample to its k-th nearest neighbor in the training set, whereas SPADE exploits GEV models and uses (ze) to estimate the probability for the sample to be OOD, A possible interpretation for the better performance of KNN (Table 1 is the better bias- trade-off in the empirical test based on 2 compared to the test based on (ze). While learning a parametric GEV model achieves some regularization, this model is only trained from examples in class c, thus operating with one two orders of magnitude less data.
The problem of dealing with adversarial examples differs from that of detecting OOD examples, as adversarial examples are deliberately crafted to deceive the teacher (Szegedy et al.] [2014] Goodfel- low et al., 2015 Madry et al., 2018; Croce & Hein 2020). Knowing their structure allows for the design of specific defense strategies, such as adversarial training (Goodfellow al. 2015
Madry
iet al. (018)) which incorporates adversarial examples into the training process ( (an et al. 2018
Zhang et al. 2019; Wang et al., 2020). Other notable defense strategies include adversarial archi-
tectures (Hosseini et al. 2020 Huang et al. (2223) adversarial regularization (Mao et al. 2019 Liu
ct al. 2023). and data augmentation methods (Carmon al., 2019 Wang et 2023).
In contrast, SPADE neither requires additional information nor retrains the classifier to defend itself against adversarial examples. It employs the same agnostic strategy for both adversarial and OOD
threats: it characterizes the true samples. It abstains from deciding on any sample that appears to be extreme compared to the true (ID) samples, according to the chosen confidence level.
7 CONCLUSION AND PERSPECTIVES
The main contribution of the paper, SPADE (Sample-efficient ProbAbilistic Detection using Ex- treme value theory), is a statistical test designed to detect samples appearing to be extreme w.r.t training samples, enabling the classifier to abstain from making decisions on such extreme exam- ples. The ability of this test to accurately detect OOD and adversarial samples has been empirically demonstrated, behaving on par with, or better than, the prominent state of the art approaches.
Like distance-based OOD detection approaches, this test exploits the latent distance between the given example and its nearest neighbor in the training set. Its originality lies in leveraging Extreme Value Theory (Fisher & Tippett 1928) to provide a formal characterization of the in-distribution samples. This characterization offers two key benefits: first, it yields provable guarantees for de- tecting adversarial examples, subject to the adversarial perturbation amplitude to be lower bounded; second, it provides some new hints into the key aspects governing the teacher robustness Lastly. the computational frugality and the stability of the generalized EV models underlying the test have been empirically established.
This approach opens several perspectives, aimed making classifiers more robust and better un- derstanding the key robustness factors. A short term perspective is to apply the generalized extreme value models to some of the criteria empirically used in the OOD detection literature; one such criterion is the score margin involved in MSP (Hendrycks & Gimpel 2017). Another perspective to enhance the classifier training loss to favor the robustness of the latent space, e-g. to con- sider the optimization of the Lipschitz constant of the classifier embedding besides its variation and informativeness as done CIDER (Ming et al. 2023).
Our long-term goal is to investigate whether safe example behaviors can be identified in the la- tent space and whether these behaviors can be certified, as a step toward the certification of neural networks.
'For instance, CIDER (Ming et al. 2023 involves the optimization of the variation and informativeness of the teacher latent space. The SPADE analysis suggests that, beyond these two factors, the regularity of the teacher its Lipschitz constant- also matters. A possible interpretation for why SPADE outperforms CIDER that optimizing variation and informativeness might adversely affect this Lipschitz constant.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The authors want to thank Cyril Furtlehner, TAU-INRIA, for many discussions about extreme value theory. We also thank the anonymous reviewers for their critiques, that helped improve the paper.
REFERENCES
A. A. Balkema and L. de Haan. Residual Life Time at Great Age. The Annals of Probability, 1974.
Peter L. Bartlett and Marten H. Wegkamp. Classification with reject option using hinge loss. JMLR, 2008
Julian Bitterwolf, Maximilian Mueller, and Matthias Hein. In or out? fixing imagenet out-of- distribution detection evaluation. ICML, 2023.
E. Boult, S. Cruz, A.R. Dhamija, M. Gunther, J. Henrydoss, and W.J. Scheirer. Learning and the unknown: Surveying steps toward open world recognition. AAAI, 2019.
Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K. Varshney, and Dawn Song. example detection in deep learning: A survey. IEEE Access, 2020.
Anomalous
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. Unlabeled data improves adversarial robustness. NeurIPS, 2019.
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. CVPR, 2014.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICMI,, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition scale. ICIR, 2021.
Xuefeng Du, Xin Wang, Gabriel Gozum, and Yixuan Li. Unknown-aware object detection: Learning what you don't know from videos in the wild. CVPR, 2022a.
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't know by virtual outlier synthesis. ICLR, 2022b.
Adam Dziedzic, Stephan Rabanser, Mohammad Yaghini, Armin Ale, Murat A. Erdogdu, and Nico- las Papernot. p-dknn: Out-of-distribution detection through statistical testing of deep representa- tions. ArXiv, 2022.
Sebastian Farquhar and Yarin Gal. What 'out-of-distribution" is and is not. NeurIPS ML Safety Workshop, 2022.
Ronald Aylmer Fisher and L. H. C. Tippett. Limiting forms of the frequency distribution of the largest smallest member of sample. Mathematical Proceedings of the Cambridge Philosoph- ical Society, 1928.
Stanislav Fort. Adversarial vulnerability of powerful near out-of- detection. ArXiv, 2022.
Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution detection. NeurlPS, 2021.
Joshua French, Piotr Kokoszka, Stilian A. Stoev, and Lauren Hall. Quantifying the risk of heat waves using extreme value theory and spatio-temporal functional data. Comput. Stat. Data Anal., 2019.
Boris Gnedenko. Sur la distribution limite du terme maximum luue serie alcatoire. Annals of mathematics, 1943.
11

---

Published as a conference paper at ICLR 2025
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. ICLR, 2015.
Laurens Haan and Ana Ferreira. Extreme Value Theory; An Introduction. Springer Series in Opera- tions Research and Financial Engineering. Springer, New York, 2006.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. ICLR, 2017. Alias (1) used in Table !
Grant Van Horn Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun. Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. CVPR, 2018.
Ramtin Hosseini, Xingyi Yang, and Pengtao Xie. Dsma:: Differentiable search of robust neural architectures. CVPR, 2020.
Yen-Chang Hsu, Yilin Shen, Hongxia Jin, and Zsolt Kira. Generalized ODIN: Detecting out-of-
distribution image without learning from out-of-distribution data. CVPR, 2020.
Shihua Huang, Zhichao Lu, Kalyanmoy Deb, and Vishnu Naresh Boddeti. Revisiting residual net- works for adversarial robustness. CVPR, 2023.
Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestricro, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, and Mark Ibrahim. Imagenet-x: Understanding model mistakes with factor of variation annotations. ICLR, 2023.
Wenyu Jiang, Yuxin Ge, Hao Cheng, Mingeai Chen, Shuai Feng, and Chongjun Wang. Read: Aggregating reconstruction error into duttrf-distribution detection. AAAI, 2023.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. JEEE Transactions Big Data, 2019.
Matt Jordan and Alexandros Dimakis. Exactly computing the local lipschitz constant of relu networks. NeurlPS, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report, 2009.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurlPS, 2018. Alias (3) used in Table
Jingyao Li, Pengguang Chen, Shaozuo Yu, Zexin He, Shu Liu, and Jiaya Jia. Rethinking out-of- distribution (ood) detection: Masked image modeling all you need, 2023.
Shiyu Liang, Yixuan Li, and R. Srikant. Enhaneing the reliability of out-of-distribution image detection in neural networks. ICLR, 2018. Alias (2) used in Table
Hong Liu, Zhun Zhong, Nicu Sebe, and Shin'ichi Satoh. Mitigating robust overfitting via self- culibial regularization. Artificial Intelligence, 2023.
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribution detection. NeurIPS, 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2018.
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang. Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. NeurIPS, 2019.
Markos Markou and Sameer Singh. Novelty detection: a review- part 1: statistical approaches. Signal Processing, 2003a.
12

---

Published as a conference paper at ICLR 2025
Markos Markou and Sameer Singh. Novelty detection: review -arr 2: neural network based approaches. Signal Processing, 2003b.
Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li. How to exploit hyperspherical embeddings for out-of-distribution detection? ICLR, 2023. Alias (5) used in Table
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confi- dence predictions for unrecognizable images. CVPR, 2015.
Nicolas Papernot and Patrick D. McDaniel. Deep k-nearest neighbors: Towards confident, inter- pretable and robust deep learning. ArXiv, 8.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. NeurlPS, 2019.
J. Pickands. Statistical Inference Using Extreme Order Statistics. The Annals of Statistics, 1975.
Jie Ren, Peter J. Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distribution detection. NeurIPS, 2019.
Philippe Rigollet. Generalization error bounds in semi-supervised classification under the cluster assumption. JMLR, 2007.
Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Ro- hban, and Mohammad Sabokrou. unified survey on anomaly, novelty, open-set, and out-of- distribution detection: Solutions and future challenges. TMLR, 2022.
Zhouxing Shi, Yihan Wang, Huan Zhang, Zico Kolter, and Cho-Jui Hsieh. Efficiently computing local lipschitz constants of neural networks via bound propagation. NeurlPS, 2022.
Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, and Christine Largouet. Anomaly detection in streams with extreme value theory. KDD, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogmition. ACPR, 2015.
Mark Smith, Steven Reece, Stephen J. Roberts, and lead Rezek. Online maritime abnormality detection using gaussian processes and extreme value theory. ICDM, 2012.
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. ICML, 2022. Alias (4) used in Table
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. ICLR, 2014.
Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman. Open-set recognition: good closed- set classifier is all you need? ICLR, 2022.
Haoqi Wang, Zhizhong Li, Litong Feng. and Wayne Zhang, Vim: Out-of -itlt---distitutionn with virtual- logit matching. CVPR, 2022.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. ICLR, 2020.
Zekai Wang. Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. Better diffusion models further improve adversarial training. ICML, 2023.
Ziang Yan, Yiwen Guo, and Changshui Zhang. Deep defense: Training DNNs with improved adversarial robustness. NeurIPS, 2018
13

---

Published as a conference paper at ICLR 2025
Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. Openood: Benchmarking generalized dustrf-dist detection. NeurlPS, 2022.
Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distributioo detection: A survey. IJCV, 2024.
Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. Towards theoretical framework of out-of-distribution generalization. NewrIPS, 2021.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. ICML, 2019.
Jingyang Zhang, Nathan Inkawhich, Randolph Linderman, Yiran Chen, and Hai Li. Mixture outlier exposure: Towards -iitt---isttibttionn detection in fine-grained environments. WACV, 2022.
Jingyang Zhang, Jingkang Yang. Pengyun Wang, Haoqi Wang, Yueqian Lin, Haoran Zhang, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Yixuan Li, Ziwei Liu, Yiran Chen, and Hai Li. Openood v1.5: Enhanced benchmark for di detection, ArXiv, 2023.
14

---

Published as a conference paper at ICLR 2025
SUPPLEMENTARY MATERIAL
The supplementary material is organized as follows:
Section presents additional results, including SPADE's accuracy in detecting hard OOD
samples (Fort 2022) and adversarial examples generated using PGD (Madry et al. 2018)
and AutoAAtack (Croce & Hein 2020);
Section B presents a new theoretical result, concerning the detection of adversarial OOD
samples (informed attacks);
Section provides a more comprehensive introduction to Extreme Value Theory.
Section details the experimental setting to ensure reproducibility, specifically describing the benchmark implementations and the hyperparameter configurations.
Section SectionnEiinessnn investiigaes the feasibility of local Theorem
A COMPLEMENTARY RESULTS
A.1 OOD DETECTION, ROC CURVE
The Receiver Operating Characteristic (ROC) curve, a standard indicator for evaluating the perfor-
mance of continuous model (/ X + R), measures the trade-off between the true positive rate and the false positive rate of the binary classifier defined as g(x) = (h(r) > t) for threshold t varying in R. Fig. illlustrates the ROC curves for OOD detection CIFAR-10 and CIFAR-100,
comparing the performances of SPADE and KNN. These results confirm SPADE's robustness in detecting OOD and adversary examples, with the same performance KNN.
A.2 DETECTION OF NEAR-OOD, FAR-OOD AND ADVERSARIAL EXAMPLES (FGSM)
Table 4 presents the detection performance of OOD and adversarial examples, comparing SPADE
with four baseline methods on CIFAR-10 and CIFAR- 100 using two performance indicators: AUC (higher is better) and FPR95 (lower is better). On CIFAR- 10, SPADE ranks first 4 out of 6 cases,
and second in the remaining cases, behind KNN. On CIFAR-100 100. SPADE ranks first in 3 out of 6 cases and second two times, behind KNN or MSP. These results highlight the robustness SPADE in detecting both OOD and adversarial examples.
A.3 DETECTION OF HARD NEAR--OD EXAMPIEES
As shown by Fort et al. (2021). more challenging near-OOD detection task consists in distinguish- ing between CIFAR-IO MDS, KNN and CIDER, using CIFAR10 as in-distribution vs CIFAR10 as Out-Of-Distrbuttion. and vice versa. Results are reported for two teacher models: VGG (Table left) and ViT (Table right).
CIFAR- 100. Table 3 presents the accuracy of SPADE against MSP,
For in-distribution CIFAR-IO, SPADE ranks first in 2 out cases second in the remaining cases, following MSP. For in-distribution CIFAR-100, SPADE ranks second in 3 out of 4 cases, behind either MSP or MDS. These results confirm the general robustness of SPADE.
A.4 DETECTION OF MORE CHALLENGING ADVERSARIAL EXAMPLES
Complementary results are provided in Tables 6 and 7 which respectively evaluate adversarial ex- amples generated by PGD (Madry al. 2018 and AutoAttack (Croce & Hein 2020). for an ad- versarial perturbation amplitude ranging in {10
10 "J. The comparison of SPADE (with ResNet teacher) against the baselines shows that for PGM attacks (Table 6). SPADE ranks first in out of cases on CIFAR-10 10 (second otherwise) while it ranks first out of 8 cases on CIFAR-100.
Table further shows that for AA attacks, SPADE ranks first out of 8 cases CIFAR-10 10. and first 5 out cases on CIFAR- - These results similarly confirm the robustness SPADE. While SPADE is second behind CIDER in 6 out of cases CIFAR-10, CIDER generally performs
15

---

Published as a conference paper at ICLR 2025
1.0
1.0
TPR
TPR
0.5
0.5
SPADE
SPADE
KNN
KNN
FPR@95
F
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
FPR
FPR
(a) CIFAR-10
(b) CIFAR-100
Figure 3: OOD detection: The ROC curves compare the performance of SPADE and KNN (Sun et al. 2022). Left: in-distribution CIFAR-10. Right: in-distribution CIFAR-100.
Table 4: Comparative performance of SPADE (with teacher ResNet) against MSP, ODIN, MDS and KNN: detection of near OOD, far OOD and adversarial examples on CIFAR-100 and CIFAR- 10 using AUROC (1) and FPR95 (4) indicators. Statistically significantly best performances in bold.
Near OOD
Far OOD
Adversarial
AUC
FPR95
AUC
FPR95 |
1
FPR95
MSP
87.68
53.54
91.00
31.43
81.96
78.76
CIFAR-10
ODIN
80.25
84.49
87.21
60.89
MDS
86.72
46.23
90.20
30.31
81.54
69.14
KNN
90.70
34.54
93.11
23.88
85.79
54.21
SPADE
90.85
34.74
93.44
22.97
86.10
54.43
CIFAR- 100
MSP
80.14
55.23
77.34
59.77
83.46
50.50
ODIN
79.86
58.03
78.98
59.34
MDS
58.68
84,22
70.65
72.04
60.27
82,89
KNN
80.32
61.33
82.64
53.47
83.72
56.00
SPADE
80.70
59.36
81.95
53.27
84.50
52.72
slightly worse on CIFAR- 100. Similarly, while SPADE ranks second behind MSP out of cases CIFAR-100, MSP is generally less effective on CIFAR-10.
A.5 DETECTION OF ADAPTATIVE ATTACKS
The PGD Madry et al. 2018) loss has been modified and adapted to inform the attacker of the detection method by augmenting the PGD loss with the distance between x' and its nearest neigh- bor (to be minimized). This attack, referred to as PGD-ad, generates more challenging adversarial samples for method, as it primarily targets the detection mechanism of SPADE. A performance comparaison of distance-hased techniques is presented in Table g It highlights that, for a perturba- tion amplitude of 10 MDS suffers from an AUC loss of (from 82.96 to 82.86), KNN incurs an AUC loss of .7 (from 86.75 to 86.06), and SPADE suffers from an AUC loss of .2 (from 86.91 to 86.70). For a larger perturbation amplitude of 0. MDS suffers an AUC loss of 1, KNN drops by 1.2 and SPADE SPADE decreases by 0.8 while still maintaining the highest ranking. For smaller perturbation amplitudes, the interaction between the two terms of PGD-ad less straightforward to interpret.
B DETECTION OF ADVERSARIAL OOD EXAMPLES
As shown in Fort et al. 2202 Fort (022), while latent distance metrics metrics effectively detect even hard OOD examples, their performance can be significantly degraded by white-box attacks.
16

---

Published as a conference paper at ICLR 2025
Table 5: Detection of hard-OOD examples with VGG teacher (Left) and ViT teacher (Right), for in- distribution CIFAR-10 10 vs OOD CIFAR-100, and vice-versa. Comparative performances of SPADE against baselines MSP, MDS, CIDER and KNN.
(a) VGG teacher
(b) teacher
In- Di-DDitrrbution
CIFAR-I0
CIFAR-100
In-Distribution
CIFAR-I0
CIFAR-I00
AUC
FPR95
AUC
FPR95
AUC
FPR95
AUC T FPR95
MSP (
78.63
59.90
66.89
73.98
MSP L
92.01
27.66
76.50
70.68
MDS 8
66.96
90.69
43.78
95.63
MDS 5
87.08
53.37
78.02
66.28
KNN
78.01
59.53
60.47
77.41
KNN
91.52
35.83
69.80
85.18
CIDER 60.34
90.03
60.09
89.87
CIDER
60.34
90.03
60.09
89.87
SPADE
78.41
89.34
61.86
79.97
SPADE
92.58
32.53
77.32
71.69
Table 6: Adversarial samples detection against PGD (Madry et al. 2018) attacks on CIFAR- 10
and CIFAR-100, with perturbation amplitude from 10 to 10 comparison of SPADE (ResNet teacher) with baselines MSP, MDS, CIDER and KNN in terms of AUC (higher is better) and FPR95 (lower is better). Statistically significant best results are highlighted in bold.
0.0001
E
0.001
Average
AUC1
FPR95
AUC
FPR95
AUC
FPR95
AUC
FPR95
AUC1
FPR95
MSP
81.50
79.60
81.69
79.10
83.19
73.58
88.82
46.44
83.8
69.68
CIFAR-10
MDS
70.44
81.44
81.47
69.63
82.96
62.19
90.36
35.34
81.31
62.15
KNN
52.24
92.35
28.52
87.51
48.29
CIDER
85.41
56.30
85.47
55.51
86.48
49.13
91.53
31.36
87.22
48.08
SPADE
85.67
56.02
85.66
54.50
86.91
50.89
92.49
27.94
87.68
47.34
MSP
83.11
52.49
83,25
51.38
84.36
48.98
86.43
42.72
84.29
48.89
CIFAR-100
MDS
82.69
60.38
82.82
60.34
82.77
59.95
78.51
64.49
81,70
61.29
KNN
CIDER
82.68
63.49
82.75
63.19
83.87
60.47
87.61
44.22
84.23
$7.84
SPADE
84.07
57.30
84.17
56.41
85.26
52.73
88.89
40.06
85.60
51.63
Table 7: Adversarial samples detection from AA (Croce & Hein
2020) on CIFAR-10 and comparison of SPADE
CIFAR-100, with perturbation amplitude ranging from 10-4 to 10-1 (ResNet teacher) with baselines MSP, MDS, CIDER and KNN using AUC (higher is better) and FPR95 (lower is better). The best results are highlighted in bold.
0.0001
0.001
0.01
= 0.1
Average
AUC
FPR95
AUC 1
FPR95
AUC 1
FPR95
AUC 1
FPR95
AUC 1
FPR95
MSP
82.35
74.86
82.35
74.76
82.72
73.60
87.47
58.56
83.72
70.45
CIFAR-10
MDS
71.13
81.83
KNN
85.53
59.03
85.54
58.90
85.79
57.98
91.71
31.74
87.14
51.91
CIDER
86.10
53.72
86.10
53.56
86.34
53.09
92.00
30.86
87.53
48.26
SPADE
85.85
58.62
85.85
58.01
86.13
57.28
92.03
30.53
87.47
51.11
MSP
84.16
50.24
84.16
30.61
84.42
$1.06
84.63
46.92
84.34
49.71
MDS
61.71
61.71
82.69
70.02
81.69
63.79
CIEAR-
KNN
85.19
55.12
85.18
55.58
85.45
53.98
88.18
44.36
86.00
52.26
CIDER
83.69
59.49
83.68
59.44
84.09
59.38
86.55
47.33
84,50
56.41
SPADE
85.71
52.90
85.70
53.14
85.96
51.91
88.45
43.31
86.46
50.32
Table 8: Adaptative attacks PGD-ad on CIFAR-10 with perturbation amplitudes ranging from 10-
to 10 comparison of SPADE with distance- based baselines MDS and KNN using AUC (higher better; best in bold) and FPR95 (lower is better; best in bold).
0.0001
0.001
0.01
0.1
AUC t
FPR95 1
AUC 1
FPR95 1
AUC 1
FPR95 +
AUC
FPR95
MDS 3
81.80
66.39
81.84
65.83
82.86
64.07
89.37
64.07
KNN
85.50
55.58
85.49
55.96
86.06
57.56
91.08
34.58
SPADE
86.04
55.97
86.02
57.57
86.70
55.68
91.64
32.82
17

---

Published as a conference paper at ICLR 2025
(a) Apple
(b) Truck
(c) Man
(d) Caterpillar
(c) Lion
(f)
Aquarium
Fish
Figure 4: Hard near- OaD setting, with CIFAR-10 as in-distribution and CIFAR-100 as OOD: rep-
resentative OOD samples missed by SPADE.
We establish that certain guarantees against adversarial OOD attacks can be derived in a manner similar Theorem contingent upper bound the amplitude of the adversarial attack, as follows.
Theorem 3. Let G be Generalized Extreme Value (GED) model characterizing the maximum latent distance among an ID sample and its latent nearest ID neighbor:
G(v) Pr([h(x) - h(x')ll < with ID,x x' nearest neighbor ID x)
Similarly, let G' be a GEV model characterizing minimum latent distance between an OOD sample and its nearest ID:
(u = Pr[[h((x) h(x')l o with x OOD; x'nearest neighbor ID of x)
Then let x be an OOD sample; let x' be an adversarial perturbation of of amplitude such that:
||x - x'|| =
Assume that the input space is Euclidean, and the latent embedding function is Lipschitzian with Lipschitz constant K. For
1 ( K
x' is detected with probability 1 - as an OOD (i.e. the adversarial OOD attack fails).
Proof. Let be an OOD sample, and let x' denote an adversarial pertubation of x such that ||x X'|| = c. Let x" be the nearest ID neighbor of x' in the latent space. By contradiction, if x' is not
detected as an OOD sample with probability 2 then:
[h(x') h(x"))l G- 7)
By the definition of the Lipschitz constant K,
[lh(x) h(xx)l Ke
Thus, with probability 1 1T,
1 2 -(x)) S KE G
By definition G', with probability 1 I
G (1 2 S [h(x) h(x")l
Combining the last two inequalities, it follows that with probability -.
K - -1 -a-a-
})) S
This contradiction concludes the proof.
Experimentally, the proposed upper bound is non-trivial (i.c., is positive), as shown on Fig. for in-distribution CIFAR-10. The two adversarial attacks uses SVHN (Fig. 5 left) and MNIST (Fig. [5] right) as OOD datasets. Furthermore, analyzing this bound offers valuable insights into the architectural properties that enhance the performance of SPADE. For example, Fig. iillusttssess that the bounds for a ViT model are significantly higher at higher confidence levels compared to ResNet model (on CIFAR-10).
18

---

Published as a conference paper at ICLR 2025
Confidence proba.
0.8
Confidence proba.
0.9
Sup(Ke)
sup(Ke)
0.5
0.5
GUD)
G
G 000)
G10.000)
0.0
0.0
0.00
0.10
0.20
0.30
0,40
0.50
0.00
0.10
0.20
0.30
0.40
0.50
Distance
neighbor
Distance nearest neighbor
(a) SPADE (VIT model); OOD SVHN
(b) SPADE (VIT model); OOD CIFAR-I00 100
Confidence proba
1.0
(DD)
Confidence proba,
1.0
G
GtD DODI
(10.000)
0.5
0.5
0.3
(kp(Ke)
0.00
0.10
0.20
0.30
0.50
0.00
0.10
0.20
0.30
0.40
0.50
Distance nearest neighbor
Distance to nearest neighbor
SPADE (ResNet model): OOD SVHN
(d) SPADE (ResNet model): OOD CIFAR-I00
Figure 5: Theorem SPADE Empirical upper bound on e for r = 0.1 for in-distributtion CIFAR-10, with teachers VIT (top) and ResNet (bottom). Results are presented for ut-of-distribution datasets SVHN (left) and CIFAR-100 100 (right). For clarity, plot G(D,OOD) -
C EXTREME VALUE THEORY
This section provides additional theoretical results related the Extreme Value Theory, referring the interested reader to Haan & Ferreira (200)) for a comprehensive presentation.
In the following. let Z be real random variable with cumulative distribution function (CDF) de- noted by F. We consider Z(m) as the random variable representing the maximum of independent and identically distributed (i..d.) samples of Z, i.c., z(n) m Zn). The CDF of the maxima, denoted a F", is given by (r(Z(n) <z) (=))).
C.I GENERALIZED EXTREME VALUE
The key assumption underlying Definition 3 in the main paper is the convergence in distribution of the maxima. To formalize this result, we introduce the following two definitions.
Definition 6 (Maximum Domain of Attraction). Let F. and H be non-degenerate CDFs. F is said to be in the domain of attraction of H, denoted E DA(H), if there exist sequences of constants an > 0 and by such that every point of continuity of H:
7(1) bn U
lim Fa,a + b H(x)
(13)
an
Where U is random variable whose CDF is H. Definition (Distribution Type). Let F. and G be deg-eeeeeee CDFs. F and G are said to be of the same type if there exist constants a >> O and such that
E F(z) G
(14)
These definitions play fundamental role in the Fisher-Tippett theorem, which characterizes the limiting distribution of the maxima of sequence of independent and identically distributed (i.i.d.) random variables. Informally, this theorem states that if the distribution of the normalized maxima converges, the limit belongs one of the three classes of distributions. These three classes form the so-called Generalized Extreme Value (GEV) distribution.
Theorem (Fisher- (Fisher-Tippett). Let F and G be two non-degenerate CDFs, Ff E DA(G), then of the same type than the following CDF defined for every TR.
19

---

Published as a conference paper at ICLR 2025
Gemo exp { (1 ( 1/8
#$ 0
(15)
exp -
otherwise
where μ.. E R is a location parameter, a E RR is dispersion parameter, and & E R. is the shape parameter.
C.2 GENERALIZED PARETO DISTRIBUTION
As highlighted in the main paper, the Block-Maxina technique becomes inefficient for accurately modeling the tail behavior when the data lacks natural block structure. An alternative approach to model tail behavior using threshold exceedances, where the Generalized Pareto Distribution (GPD) plays a central role.
Definition 8 (Generalised Pareto Distribution). The Generalised Pareto Distribution is defined as:
{
- exp ž-H(
= 0
(Z)
=
1//
(16)
5(2-H)
otherwise
Let us consider the distribution of the excess over threshold / E
R:
F.(2) = P(Z - is Z> F(t + F(t)
5<=
(17)
1 F(t)
where 2p < oo denotes the right endpoint of F.
The following theorem establishes that for sufficiently large thresholds i, the distribution of excesses over the threshold can be approximated by a Generalized Pareto Distribution (GPD). Theorem (Pickands-Balkwwn Balkemallkaaeeee De Haan). Let Z be random variable with non-degenerate CDF
F. There exist positive functions a<t). B(t) such that
lim
sup =F-t
F=(=) FE..(00.800))))) = FE FEAAGG)
EER
(18)
t-PF
where & is the shape parameter, and Gę is the Generalized Extreme Value distribution.
This theorem establishes formal link between Block Maxima methods and threshold exceedance approaches. Specifically, it provides a theoretical justification for using excess modeling to describe the tail behavior, which is typically captured by Block Maxima methods, that estimate the distribu- tion of extreme values.
D EXPERIMENTAL SETTING
All experiments are implemented using PyTorch (aaske age Zhang et al. 2023
2019 and the OpenOOD pack-
Teachers. All teacher models trained using stochastic gradient backpropagation, with the learning rate adjusted by the Adam optimizer Kingma & Ba (2015). The models, initially pre- trained on ImageNet, are sourced from the PyTorch hub and fine-tuned (if necessary) on the con- sidered datasets (CIFAR-10, CIFAR-100), This fine-tuning process consists of adding new fully connected head layer with ReLU activation functions, placed on top of the frozen latent space. The head is trained with dropout rate of 0.5.
Datasets. OOD samples are categorized into near-OOD and far-OOD distributions. For CIFAR-10, 10. near-OOD samples are from CIFAR-100, while far-OOD samples originate from MNIST, SVNH, Texture, and Places365. For CIFAR-100. near-OOD samples come from ImageNet- 1K and Tiny-ImageNNe,, whereas far-OOD samples are taken from MNIST, SVNH, Texture and Places365. For ImageNet-IK, near-OOD samples are from SSB-Hard (Vaze et al., 2022 and NINCO (Bitterwolf et al.) 2023), while far-OOD samples come from iNaturalist (Horn 2018). Texture (Cimpoi et al. 2014), and OpenImage-OO (Wang al. 2022) datasets.
et al.
https://gii https://gtthbb..coo//gggagg/g///
20

---

Published as a conference paper at ICLR 2025
180
(
00
(b) r=0.5
r=)
(d) r=2
Figure 6: ResNet model: Histogram of local Lipschitz constant (x)x)) for different radiuses
CIFAR-10.
1.6e+01
Local Perturbation Bound
1.8e-03
1.6e+01
1.7e-03
1.5e+01
1.7e-03
1.5e+01
1.6e-03
1.4e+01
2
2
Figure 7:
ResNet model, for representative x: local Lipschitz constant vs radius and upper bound on the adversarial perturbation (Theorem 2).
Nearest Neighbors. The nearest neighbor search is performed in the latent space of the model, where the latent representation corresponds to the penultimate layer. Following (un al. (2022), normalization step is applied to the latent features before computing the Euclidean distance, defined (((x)= /xx) [[)())]). The distance is then computed as ||}{(x)
=(xx)]
Following the approach in Sun et al. (2022). we employ Faiss (Johnson et al. 2019), a li- brary optimized for fast and efficient nearest neighbor search. Specifically, we utilize the faiss. IndexFlat lat L2 index, which operates using Euclidean distance. In practice, we precom- pute the embeddings for all images and store them key-value map facilitate efficient KNN
queries.
Hy perrpaaraeesess. The performance comparison with state-of-the-art (SOTA) methods was con- ducted using the OpenOOD framework (Zhang et al. 2023). Each SOTA method involves or more nyper-parametent which are tuned by performing a grid search over the default values pro- vided by OpenOOD. The optimal curve (AUC) related to the ID-OOD detection, following the common practice (Zhang et al, 2023).
selected by maximizing the under the
E LOCAL PERTURBATION BOUND
The relevance of Theorem (main paper) is illustrated on Figs. and for a ResNet model. Fig.
6 depicts the histogram of the local Lipchitz K, for ranging in (-1, .1,,5, 1, Fig. shows the local Lipschitz. constant K,(x) for a representative sample x, and the associated lower bound on the adversarial perturbation. Both figures confirm that the lower bound consistently remains smaller than r, demonstrating the relevance of Thcorem 2.
21


Reviews:
[{"review_id": "ip7IqL6S5o", "reviewer": "Reviewer_3aNg", "summary": "The paper presents SPADE (Sample-efficient Probabilistic Detection using Extreme Value Theory) = a new method for detecting out-of-distribution (OOD) and adversarial samples in neural network classifiers. The key contributions are:\na. A novel method to transform a classifier into an \"abstaining classifier\" that can refuse to make predictions on suspicious inputs\nb. Use of Extreme Value Theory (EVT) to model the training distribution in the classifier's latent space\nc. Mathematical guarantees for detecting both OOD and adversarial samples\nd. Experimental validation across multiple architectures (ResNet, VGG, ViT) and datasets\n\nI am pretty unfamiliar with Extreme Value Theory, so please flag if if I'm misunderstanding something basic in my review. But I'm very familiar with OOD detection and adversarial attacks, so it could compensate for it.", "questions": "Included in the weaknesses section.\n\nI am primarily interested in the authors answers to:\na. the CIFAR-100 <-> CIFAR-10 challenging near OOD task(s)\nb. the white-box attack on latent metrics point", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Theoretical Foundation: The approach is grounded in statistical theory (EVT)\n2. Sample Efficiency: Good performance even with strong subsampling of the train set\n3. Versatility: Works with different architectures and can handle both OOD and adversarial inputs\n4. Evaluation: Tested against strong baselines on multiple datasets", "weaknesses": "# 1. Missing challenging near-OOD evaluation\nThe evaluations in the Table 2 and Table 3 are missing a very simple, yet challenging near OOD detection task which is CIFAR-100 vs CIFAR-10 (and the other way round). [1] shows strong results for large models and also provides a human score to benchmark against. I would be very curious to see the AUROC, AUC and FPR95 on that. I believe that this would be a very worthwhile and easy evaluation to add and I think the authors should try it. The AUROC in particular is something I would really like to see. Also a set of examples of where the model fails. **I will consider increasing my score if the authors successfully address this point.**\n\n# 2. A collection of weakness that do not need to be addressed in the rebuttal:\n2.1 Computational Complexity: The GEV model is quadratic in the number of samples\n2.2 Performance Trade-off: While SPADE performs well in general, it's sometimes outperformed by simpler methods like KNN for specific tasks\n2.3. Parameter Sensitivity: The effectiveness depends on the choice of teacher model and its latent space characteristics\n2.4. High FPR95 Values: Shows tendency to be overly cautious. Perhaps rejecting valid samples?\n\n# 3. Weak adversarial attacks used only\nThe paper's evaluation of adversarial robustness is limited. They only test against the FGSM attack (=very basic) with small perturbation magnitudes (epsilon from 0.001 to 0.004). Given that SPADE is positioned as providing security guarantees against adversarial examples, a more thorough evaluation against SOTA attacks would be necessary to support the claims. Additionally, the authors should clarify whether their evaluation considers attacks targeting just the classifier or the full system including the OOD detector, and include both targeted and untargeted attack scenarios. **This is especially relevant to the next weak point.**\n\n# 4. White-box attacks\n\nI have the following concern that I would like you to address directly:\n\n1. SPADE offers an elegant theoretical framework for OOD detection based on a distance metric in the latent space\n2. [1] shows that distance metrics in the latent space can be amazing at even hard, near-OOD detection\n3. [2] shows that white-box directly targeting such a metric can *still completely destroy it* with a simple adversarial attack\n\nGiven SPADE's reliance on distance metrics in latent space and GEV models, it may be vulnerable to white-box attacks that:\n\n4. Directly optimize perturbations to minimize latent space distances to k-nearest neighbors while maintaining misclassification\n5. Exploit knowledge of the GEV model parameters to generate samples that appear in-distribution\n6. Target the gap between separate class-specific GEV models to find adversarial blind spots\n\nThe authors should consider and discuss these potential vulnerabilities, particularly since they claim SPADE as providing security guarantees. While Theorem 1 provides a lower bound on adversarial perturbation magnitude, an analysis of potential attack strategies and additional defensive measures (such as ensemble approaches or adversarial training) would strengthen the paper's security claims.\n\n# References\n\n[1] Exploring the Limits of Out-of-Distribution Detection. Stanislav Fort, Jie Ren, Balaji Lakshminarayanan (https://arxiv.org/abs/2106.03004)\n\n[2] Adversarial vulnerability of powerful near out-of-distribution detection. Stanislav Fort (https://arxiv.org/abs/2201.07012)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VUIboSyZ8x", "reviewer": "Reviewer_EMSd", "summary": "This paper uses extreme value theory as a means to detect out-of-distribution (OOD) samples. By abstaining to predict on samples that are determined to be OOD, the proposed approach is shown to yield high-probability robustness guarantees against adversarial attacks up to a certain perturbation magnitude. Experiments are conducted on CIFAR-10, CIFAR-100, and ImageNet to assess the performance of the proposed method compared to prior OOD detection schemes.", "questions": "1. Line 90: You use the acronym ID before defining it.\n2. Line 102: You should de-italicize the term $\\eta$-invariant that you are definining.\n3. Definition 3: The role of the parameters $\\mu,\\sigma,\\zeta$ are unclear in how you wrote Definition 3. It seems to me like you need to re-word things to somehow say that \"there exist $\\mu,\\sigma,\\zeta$ such that $P(Z^{(\\ell)} < z) \\to G_{\\zeta,\\mu,\\sigma}(z)$.\" Otherwise, the reader might be inclined to think that the choice of parameters is somehow \"up to us.\"\n4. Line 150: Extreme value theory has been utilized by the ML community in popular robustness verification works such as [1]. I encourage you to take a look at [1] as well as the references therein.\n5. Line 169: \"For $Y = c$, let $Z_c$ be the random variable defined as the distance between $h(X)$ and its $k$-th nearest neighbor in latent distance, belonging to $\\mathcal{D}$ with same class $c$.\" It would help readability if you write out the mathematical expression defining the random variable $Z_c$.\n6. Line 170: What is $k$? Is this a user-specified hyperparameter of your method?\n7. Line 171: The notation here is inconsistent; you should be writing $Z^{(\\ell)}$, not $Z^{(l)}$. Also, before, you used $P$ for probability, not $Pr$.\n8. Line 215: Where is the minimum coming into this equation defining the extreme value distribution $G^{(c,c')}$? It seems like without the minimum inside the probability expression on the right, you aren't really considering extreme values.\n9. Theorem 1: It looks like now you are assuming that the instance space $\\mathcal{X}$ is a Euclidean space $\\mathbb{R}^d$ (so that you can define norms on the instance space as well as Lipschitzness of the embedding map), which was not explicitly assumed before. Therefore, you should make this clear in the theorem statement.\n10. Theorem 1: Do you mean to say $f_\\tau$ instead of $f_\\epsilon$ in the statement of the theorem?\n11. Experimental Setting: You state that you generate attacks using FGSM. At this point FGSM, as introduced in Goodfellow et al., 2015, is not really considered by the adversarial robustness community to be a strong attack. At the very least, I'd expect you to use projected gradient descent (PGD) [2], or something stronger like AutoAttack [3] which is used in the RobustBench benchmarks. Have you tested your methods against these stronger attacks?\n12. There are a handful of distinct conceptual discussions and innovations leading up to your experiments in Sections 4 and 5. However, there is no explicit description of SPADE where you bring everything together into a single model/algorithm. I think it would greatly benefit readability if you were to conclude your conceptual discussions by writing out SPADE as an explicit algorithm (including any robustness certification steps), immediately before moving into experiments.\n13. Line 318: I'm assuming by \"KNN\" you are referring to $k$-nearest neighbors? You should clearly define this acronym before using it. Similarly, MSP is not defined. Please define it, or at the very least, associate a reference to it.\n14. Line 190 and Experiments: On line 190 you said that you would tease out the effect of the latent embeddings on the OOD tests in Section 5 (since your proposed OOD test in Definition 4 depends on the embedding from instance space to latent space). However, after reading through the experiments of Section 5, I do not see how you are answering this concern. Could you clarify?\n\n[1] \"Evaluating the robustness of neural networks: An extreme value theory approach,\" Weng et al., ICLR 2018\n\n[2] \"Towards deep learning models resistant to adversarial attacks,\" Madry et al., ICLR 2018\n\n[3] \"Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,\" Croce and Hein, ICML 2020", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The problem considered is interesting, and fits well within the scope of the ICLR community. The paper is easy to follow, and does a good job at clearly yet concisely introducing the main tools being used (e.g., extreme value theory). There is a nice blend of conceptual/theoretical development, and empirical evaluation.", "weaknesses": "See my specific Questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5lUw5FInFz", "reviewer": "Reviewer_QvKb", "summary": "In this paper, the authors propose sample-efficient probAbilistic detection using extreme value theory (SPADE), which models the training distribution using Extreme Value Theory to create a statistically efficient test that identifies and rejects both OOD and adversarial samples with high probability. Their contributions include a formal OOD definition in relation to a model's latent space, a frugal OOD detection test grounded in EVT, and demonstrated effectiveness against strong baselines across diverse model architectures.", "questions": "See weaknesses", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces SPADE, a novel approach leveraging Extreme Value Theory (EVT) for OOD detection, which provides a statistically grounded method to detect and reject OOD samples effectively.\n\n2. SPADE not only detects OOD samples but also offers provable guarantees for rejecting adversarial examples, making it robust against potential adversarial attacks.\n\n3. The approach is experimentally validated across multiple model architectures, demonstrating its versatility and effectiveness compared to strong baselines, enhancing its relevance for various real-world applications.", "weaknesses": "1. The provable guarantees only hold on **strong** assumptions. In theorem 1, the authors assume the embedding network is $K$-Lipschitz. Firstly, a $K$-Lipschitz network can already provide provable robustness on adversarial examples. Secondly, due to the complexity of the neural network, it is not possible to calculate the exact Lipschitzness empirically. Thus, this theorem offers limited utility for practical OOD detection scenarios.\n\n2. The experimental setup largely follows standard OOD detection settings and does not address the \"provable protection of OOD examples,\" which is claimed as the main contribution of this work. This raises doubts for the reviewer about whether SPADE can indeed provide provable protection for OOD examples, given the limitations outlined in Weakness 1.\n\n3. SPADE performs poorly on near OOD examples, with Table 1 showing that it is outperformed by the MSP baseline in 3 out of 5 cases. Additionally, the average rank is not an ideal metric. To demonstrate the superiority of the proposed method, SPADE should outperform the baselines across a majority of the datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pStdaSyDpJ", "reviewer": "Reviewer_KzFE", "summary": "This paper studies OOD detection. The authors study OOD in terms of GEV, thus enabling a characterization of the probability/confidence of OOD samples. Based on this framework, they propose a method to detect OOD and adversarial examples.", "questions": "Could the authors provide more convincing experimental results about the benefit of the proposed method?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The method is well-motivated, and the theory is sound. I love the first-principle design. Paper is well-written and easy to follow. The studied problem is important. Introducing GEV to OOD in this way is novel.", "weaknesses": "The main weakness is the weak experimental results. In almost all result tables, the proposed method does not achieve significant benefit over the baselines. In particular, Table 1 shows the proposed method is uniformly dominated by the baseline methods. Table 2 seems to suggest ViT is a particularly strong teacher model for the proposed method, maybe revisiting Table 1 with ViT could bring more benefits.\n\nThe authors claim that the proposed methods help to detect adversarial examples. However, the common practice of a defense is to endure adaptive defense, which is aware of the detection and tries to find an adversarial example that surpasses the detection. This highly depends on the claim that the authors would like to make, but overall such experiments would help to establish the significance of the defense.\n\nIn summary, while I like the design and the theories, the experimental results do not suggest sufficient improvements, questioning the significance of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)