A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection
Authors: Marine Picot
Federica Granese
Guillaume Staerman
Marco Romanelli
Francisco Messina
Pablo Piantanida
Pierre Colombo
Venue: Accepted by TMLR ()

Abstract: Despite the widespread use of deep learning algorithms, vulnerability to adversarial attacks is still an issue limiting their use in critical applications. Detecting these attacks is thus crucial to build reliable algorithms and has received increasing attention in the last few years.
In this paper, we introduce the  HalfspAce Mass dePth dEtectoR (HAMPER), a new method to detect adversarial examples by leveraging the concept of data depths, a statistical notion that provides center-outward ordering of points with respect to (w.r.t.) a probability distribution. In particular, the halfspace-mass (HM) depth exhibits attractive properties such as computational efficiency, which makes it a natural candidate for adversarial attack detection in high-dimensional spaces. Additionally, HM is non differentiable making it harder for attackers to directly attack HAMPER via gradient based-methods. We evaluate HAMPER in the context of supervised adversarial attacks detection across four benchmark datasets. 
Overall, we empirically show that HAMPER consistently outperforms SOTA methods. In particular, the gains are 13.1% (29.0%) in terms of AUROC (resp. FPR) on SVHN, 14.6% (25.7%) on CIFAR10 and 22.6% (49.0%) on CIFAR100 compared to the best performing method.

Published in Transactions on Machine Learning Research (03/2023)
A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection
Marine Picot"
morine. parine.picot:. wwwp:pooooooeoetreeeeeeeeeeeeee fcentrolesupeele..fy moil.meilll.c ca)
Laboratoire des Signaux et Systèmes (L2S), Université Paris-Saclay CNRS CentroleSupélec
Deportment of Electrical and Computer Science McGill University, QC, Canada
Federica Grancse"
Liz, Inria, Institute Polylechnique de Paris, Sapienza University of Rome
Guillaume Staerman
guillaame..taermmanniiiii
Université Paris-Suclay, Inria, CEA
Marco Romanelli
mr668520myu.co edu
Laboratoire des Signauz et Systèmes (L2S), Université Paris-Suclay CNRS CentraleSupélec
Francisco Messina
fmessina wba
School of Engineering, Universidad de Buenas Aires CSC- USC-CONICET, Buenos Aires, Argentina
Pablo Piantanida
peblo.pianlanidaeerrs
International Laboratory on Learning Systems (ILLS), CNRS, CentraleSupélec
Pierre Colombo
pierre. www.omoooeeoooooooooeaoeeoaao
MICS, SeptraleSupélec
Reviewed OpenReview: https: opennevieeu net/ pormm/ id= YtUOnDb55e
Abstract
Despite the widespread use of deep learning algorithms, vulnerability to adversarial attacks is still an issue limiting their in critical applications. Detecting these attacks is thus crucial to build reliable algorithms and has received increasing attention in the last few years. In this paper, we introduce the HalfspAce Mass dePth dEtectoR (HAMPER), a new method to detect adversarial examples by leveraging the concept of data depths, a statistical notion that provides center-outward ordering points with respect to (w.r.t.) a probability distribution. In particular, the halfspaco-mass (HM) depth cxhibits attractive propertics which makes it natural candidate for adversarial attack detection in high-dimensional spaces. Additionally, HM is non differentiable making it harder for attackers to directly attack HAMPER via gradient based-methods. We evaluate HAMPER in the context of supervised adversarial attacks detection across four benchmark datasets. Overall, we empirically show that HAMPER consistently outperforms SOTA methods. In particular, the gains are 13. 13.1% (29.0%) in terms of AUROC* (resp. FPR 195%) on SVHN, 14.6% (25.7%) on CIFAR10 and 22.6% (49.0%) on CIFAR100 compared the best performing method.
1 Introduction
most machine learning applications, deep models have achieved state-of-the-att performance. However, an important limitation their widespread use in critical systems is their vulnerability adversarial attacks (Szegedy al., 2014), i.e., the introduction of maliciously designed data crafted through minor adversarial perturbations to deceive trained model. This phenomenon may lead to disastrous consequences

---

Published in Transactions on Machine Learning Research (03/2023)
in sensitive applications such autonomous driving, aviation safety management, health monitoring systems (Geifman & El-Yaniv, 2019; Geifman et al., 2019; Guo et al., 2017; Meinke & Hein, 2020).
Over time, a vast literature has been produced on defense methods against adversarial examples (Croce & Hein, 2020; Athalye et al., 2018b; Zheng et al., 2019; Aldahdooh et al., 2021b). On the one hand, techniques to train models with improved robustness to upcoming attacks have been proposed in Zheng al. (2016); Madry et al. (2018) or Picot al. (2021). On the other hand, effective methods to detect adversarial examples given pre-trained model were reported in Kherchouche et al. (2020); Meng & Chen (2017) or Ma et al. (2019). Detection methods for adversarial examples can be mainly grouped into two categories (Aldahdooh et al., 2021b): supervised and unsupervised ones. In the supervised detection setting, the detector is trained on features extracted from adversarial examples generated according to one or multiple attacks. In particular, the network invariant model approach consists of features that are derived from the activation values of the network layers (cf. Lu et al., 2017; Carrara et al., 2018 or Metzen et al., 2017); in the statistical approach (e.g., maximum mean discrepancy (Grosse el al., 2017), PCA (Li & Li, 2017), kernel density estimation (Feinman al., 2017), local intrinsic dimensionality (Ma et al., 2018), latent graph neighbors (Abusnaina et al., 2021), the Mahanalobis distance (Loc et al., 2018); in the auriliary model approach, the features are instead
the features are linked to in-training or out-of-training data piteributioommmaiitin
derived from monitoring clean and adversarial charac- teristics (e.g., model uncertainty (Feinman et al,, 2017), natural scene statistics (Kherchouche et al., 2020)). In the unsupervised detection setting, the detector does not rely on the prior knowledge of the attacks, and it only learns from the clean data at training time. Different techniques are usod to extract the mcaning- ful features (e.g., feature squeezing (Xu et al., 2018; Liang et al., 2021), denoiser approach (Meng & Chen, 2017), network invariant (Ma et al., 2019), sensitivily to noise (Hu et al., 2019), auziliary model (Sotgiu et al., 2020; Aldahdooh et al., 2021a; Zheng& & Hong, 2018), k- Nearest Neighbors (Raghuram et al., 2021)). Detection methods of adversarial examples differ as well accord- ing to whether the underlying classifier is assumed to be pre-trained not: when further training of the classifier is allowed, the methods present a novel truin- ing procedure (e.g., with reverse cross-entred entropy (Pang et al., 2018); with the rejection option (Sotgiu et al., 2020; Aldahdooh et al., 2021a)) and a thresholding test strategy. Finally, the learning lask of the underlying network also impacts the adversarial examples detec- tion methods (c.g., detection of adversarial examples for human recognition tasks (Tаo et al., 2018)).
KSS
LID
HAMPER4A
KD-BU
HAMPER
AUROCT
100
50
FPR
50
SVHN (95.57)
CIFAR10 (93.33)
CIFAR100 (74.12)
Figure 1: Average performances of our method (i.c., HAMPER) in attack-aware and single delector set- ting, along with the performances of state-of-the the- art detection mechanisms (i.e., NSS, LID, KD-BU), on three classically considered datasets (i.e., SVHN, CI- FAR10 and CIFAR100). Below the dataset names is the accuracy of their underlying classifiers. In ad- dition to outperforming other methods all three considered datasets, our method, contrary to the others, does not losc performances as the classifier's accuracy decreases.
Adversarial detection can be related to the anomaly detection problem. Indeed, anomaly detection aims to identify abnormal observations without previously knowing them, possibly including adversarial attacks. A plethora of techniques has boen designed to address this problem ranging from machine learning algorithms
such as Isolation Forest (Liu et al., 2008; Stacrman et al., 2019), Local Outlier Factor (Breunig et al., 2000) or One-Class SVM (Schölkopf et al., 2001) to statistical tools such as kernel density estimation (Feinman et al., 2017) or Data Depth (Zuo & Serfling, 2000) (see Chandola. et al., 2009 for an extensive review of anomaly detection methods). In particular, data depth stands out as a natural candidate to detect anomalies (Chen et al., 2009).
The idea of statistical depth has grown in popularity in multivariate data analysis since its introduction by John Tukey (Tukey, 1975). For a distribution on dd with d > 1, by transporting the natural order on the real
2

---

Published in Transactions on Machine Learning Research (03/2023)
line to RR, depth function provides center- center-outward ordering of points w.r.t. the distribution. The higher the point depth scove, the deeper the point is in the distribution. In addition to anomaly detection (Serfling, 2006; Staerman et al., 2020; Chen et al., 2009; Staerman et al., 2021b; 2022), the notion of depth has been used to extend the notions of (signed) rank or order statistics to multivariate data, which find mumerous applications statistics and machine learning (e.g. robust inference (Cuevas et al., 2007), classification (Lange et al., 2014), hypothesis testing (Oja, 1983), clustering (Jörnsten, 2004; Stacrman et al., 2021a). To the best of our knowledge, it has not been investigated yet through the lens of adversarial attack detection. This paper aims to leverage this overlooked notion to build an adversarial attack detector.
Contributions. Our contribution is threefold:
1. We propose applying the halfspace-mass depth notion in the context of the adversarial detection problem. To the best of our knowledge, we are the first to both explore and successfully apply data depth for adversarial detection.
2. Through an analysis of the classifier's behavior under threat, we show how to leverage the halfspace-mass depth to build an anomaly score. To that end, we introduce HAMPER, a simple supervised method to detect adversarial examples given a trained model. Given an input sample, HAMPER relies on a linear combination of the halfspace- mass depth score. These depth scores are computed w.r.t. a reference distribution corresponding to the training data conditioned per-class and per-layer.
3. We extensively evaluate HAMPER's performance across popular attack strategies and computer vision benchmark datasets (e.g., SVHN, CIFAR1O, and CIFAR100). As shown by Fig. 1, HAMPER largely outperforms SOTA detection methods and consistently detects attacks that SOTA approaches fail to identify.
The paper is organized as follows. First, in Sec. 2, we describe the adversarial detection problem and provide a detailed overview of the SOTA supervised detection methods and the attack mechanisms considered throughout the paper. In Sec. 3, after rocalling the concept of data depth by focusing on the halfspaco-mass depth, we introduce HAMPER, our proposed supervised detector method based on the HM depth. In Sec. 4, provide insights the underlying classifier's behavior under threats. In Sec. 5, we extensively evaluate HAMPER through numerical experiments on benchmarks on visual datasets and compare it to SOTA methods. Finally, concluding remarks are gathered in Sec. 6.
2
Background
After defining the problem formulation, we present the SOTA detection methods and the allack mechanisms that we will consider throughout this paper.
2.1 Problem Formulation
Let (X,Y) be random tuple of variables valued in X x y with unknown data distribution pxy: X C Rd represents the feature space and y = .l, C} represents the labels attached to elements in X, where CEN, C2 2. The training dataset D = {(x,, ());%, is defined 2 1 independent identically distributed (i.idd) realizations of pxy- Subsets of the feature space associated with label c E ) are denoted by
S. = {I.
E S :.:y cf with S = {
Given a parametric model with L 2 layers; let fg : X + Rd with E fl...., L}, denotes the output of the f-th layer of the deep neural network (DNN) parametrized by E where the dimension of the latent space induced by the f-th layer is dz. The class prediction is obtained from the L-th layer softmax output as follows:
f/ (o: D) e arg max qe (c/z) with qe (-1z) softmax(f% (x; D)).
eEy
The adversarial problem. Given a E X and p 2 1, the adversarial generation problem can be defined producing x' such as (Szegedy et al., 2014):
2' =
arg min d'ERd pll
|| - x|l s.t. (b(; + y.
(1)
3

---

Published in Transactions on Machine Learning Research (03/2023)
where y is the true label associated to the sample x, and 11-1lp is the p-norm operator. Since this problem is computationally infeasible in general, it is commonly relaxed as follows:
x' arg max Z'ERd Z'ER":IZ'' =llp<
2(x,2';0),
(2)
where C(x, 0) is the objective of the attacker, representing surrogate of the constraint to fool the classifier, i.e., 16 (a'; DD # y. The variety of attacks differs with the choice of the norm (e-g.: p 2, 00) and the value of e.
Crafting a detector. Given a new observation x E IRd, detecting adversarial attacks boils down to build a binary rule : 12 + {0, 1}. Namely: a new observation I E R is considered as 'normal' (or 'natural', 'clean"), i.e. generated by pxy, when g(x) = a with a E {0, 1}, and x is considered as an adversarial example when g(z) a. For given scoring function s : Rd + R, and a threshold y e RR we have
1 if s(x) > 7 (()) 57
= =
> >
(3)
2.2 Supervised Detection Methods
Supervised detection mothods, when the defender has access to the future threats that is going to face, can be separated into two main groups: atlack-aware and blind-to-attack methods.
Attack-aware methods. In the attack-aware setting, the methods are going to face single threat, and they have full knowledge about them. Therefore, it is possible to train one detector per altack. In this setting fall two detection methods: KD-BU (Feinman et al., 2017) and LID (Ma et al., 2018). KD-BU is based on the intuition that the adversarial examples lie off the data manifold. To train the detector, a kernel density estimation in the feature space of the last hidden layer is performed, followed by an estimation of the buyesian uncertainty of the input sample. LID extracts the locol intrinsic dimensionality features for natural and attacked samples for each layer of the classifier and trains a detector on them.
Soind-to-Attack setting. In the blind-to-atlack setting, the defender has knowledge about the fact that it is going to be attacked, but do not know exactly how. In that case, a single detector is trained, deployed and tested against all possible threats. NSS (Kherchouche et al., 2020) falls into that category. It is based on the extraction of the natural scene statistics from the clean and adversarial samples from different thrcats models, later used to train a delector to discriminate between natural inputs and adversarial examples. Natural scene statistics are regular statistical properties that are altered by adversarial perturbations.
2.3 A Brief Review of Attack Mechanisms
Multiple methods to generate adversarial examples have been developed in recent years. The attack mechanisms can be divided into two main categories: whitcbox, where the attacker has complete knowl- edge about the targeted classifier, and blackbox, where the attacker does not know about the targeted classifier.
Whitebox attacks. The simplest one is Fast Gradient Sign Method (FGSM), introduced by Goodfellow el al. (Goodfellow et al., 2015). It consists in modifying the examples in the direction of the gradient a specific objective, w.r.t. the input on the targeted classifier. Two iterative versions of FGSM have been proposed: Ba- sic Iterative Method (BIM; Kurakin al., 2018) and Projected Gradient Descent (PGD; Madry et al., 2018). The main difference is that BIM initializes the adversarial example to the natural sample while PGD initializes it to the natural example plus random noise. Although PGD was initially created under an Loe constraint, it is possible to extend the method to any Lp-norm constraint. Later, Moosavi-Dezfooli et al. (Moosavi-Dezfooll et al., 2016) introduced DeepFool (DF), an iterative method based, at each step, on a local linearization of the model, resulting in a simplified problem. Finally, Carlini-Wagner (Carlini & Wagner, 2017) presents the CW method to find the smallest noise solving the original adversarial problem. They proposed a new re- laxed version of the adversarial problem that optimizes an attack objective, chosen according to a specific task.
4

---

Published in Transactions on Machine Learning Research (03/2023)
Blackbox attacks. Without any knowledge about the targeted classifier or its gradients, blackbox attacks are expected to rely on different. mechanisms. Square Attack (SA; Andriushchenko et al., 2020) employs a random search for perturbations that maximize a given objective, Spatial Transformation Attack (STA; Engstrom et al., 2019) applics small translations and rotations to the original image while Hop Skip Jump (HOP; Chen et al., 2020) estimates the gradient-baser direction to perturb through a query on the targeted classifier.
Adaptive attacks. There exists third type of attacks called Adaptive Attacks (Athalye et al., 2018a; Tramer et al., 2020; Carlini & Wagner, 2017; Yao et al., 2021). Adaptive attacks have full knowledge about not only the underlying classifier to attack, but also the defense mechanisms one may have deployed. To build efficient adaptive attacks, it is therefore crucial to understand the mechanisms involved into the defense, and finding ways to bypass them. For examples, the Backward- Differentiable Attack (BPDA; Athalye et al., 2018a) has been developed to overcome the non-differeniiabiiiy of the defense mechanisms by finding a suitable surrogate to the non-differential parts of them.
3 A Depth-Based Based Detector
After presenting the data depth in Sec. 3.1, with an emphasis on the halfspace-maar mass depth, we introduce our depth-based detector in Sec. 3.2.
3.1 Background on Data- Depth
A data depth function D(., Id + [0, measures the centrality of any element in x E Rd w.r.t. a probability distribution P (respectively, a data set). It provides a center-outward ordering of points in the support of P and can be straighuforwardly used to extend the notions of rank or order statistics to multivariate data. The higher D(x, P), the deeper x Ed is in P. The earliest proposal is the halfspace depth introduced by John Tukey in 1975 (Tukey, 1975). This depth is very popular due to its appealing properties and ease of interpretation. Assume that P is defined on arbitrary subset K. c Rd and denote by P(H) & P(H KC) the probability mass of the closed halfspace H. The halfspace depth of point x E Rd with respect to a probability distribution P on Idd is defined as the smallest probability mass that can be contained in a closed halfspace containing z:
DH(I, inf PH)). HeH(x)
where H(x) is the set of all closed halfspaces containing
However, the halfspace depth suffers from three critical issues: (i) finding the direction achieving the minimum to assign a score induces a significant sensitivity to noisy directions, (it) assigning the zero score to cach new data point located on the outside of the convex hull of the support of P makes the score of these points indistinguishable, and (tii) as the dimension of data increases, an increasing percentage of points will appear at the edge of the convex hull covering the data set leading to have low scores to every points.
To remedy those drawbacks, alternative depth functions have been independently introduced in Chen el al. (2015) and Ramsay et al. (2019). In this regard, the extension of Tukey's halfspace depth, recently introduced and referred to as the halfspace-mas mass (HM) depth (Chen et al., 2015) (see also Ramsay et al., 2019 and Staerman et al., 2021b), offers many advantages.
Authors proposed to replace the infimum by an expectation over all possible closed halfspaces containing x, following in the footsteps of Cuevas & Fraiman (2009). More precisely, given a random variable X following distribution P and probability measure Q on H(x),, it is defined as follows:
DHM(2, P) = Eh-Q [P(H)]
(4)
In addition to basic propertics a depth function should satisfy, the halfspaco-mass depth possesses robustness properties: it has a unique (depth-indueed) median with an optimal breakdown point equal to 0.5 (Chen et al., 2015) which means that the halfspace-mass depth provides a stable ordering of the 'normal' data
5

---

Published in Transactions on Machine Learning Research (03/2023)
even when polluted data belong to the training set. In addition, it has been successfully applied to anomaly detection in Chen et al. (2015) making it natural choice to adversarial attack detection. When a training set {ai...., Inf is given, Eq. 4 boils down to:
a I H}
DHM (z, Pn) EQ
(5)
where Pr. is the empirical measure defined by E dz.- The expectation can be conveniently approximated with Monte-Carlo scheme in contrast to several depth functions that are defined solutions of optimization problems, possibly unfeasible in high dimension. The aim is then to approximate Eq. 5 by drawing a finite number of closed halfspaces containing z (see Appendix A for the approximation algorithm for training and testing).
3.2 Our Sapth-Based Detector
The methodology we propose here is based on the halfspace-maa mass depth that exhibits attractive mathematical and computational properties, as described in the previous section.
Our depth-based detector, HAMPER, relies on the information available a subset A of DNNs' layers, i.e., the mapped data žg,i D), e E A { - 1}. We denote by 8 {=+,)) and 8 - -= E yi c]. the f-th and the ε-th class-conditionally representations of the training datasot, respectively. Our approach aims to construct a score function 8 :RR + [0, 1] providing a confidence level to a new observation x indicating its degree of abnormality w.r.t. to the training dataset. HAMPER leverages appealing properties of the HM depth detailed in Sec. 3.1 and can be summarized into two distinet steps. The function s is built by first constructing |A] C intermediate scoring functions $t,c : Rd - [0, 1] designed for each considered layer and each class. The map 8t.0 assigns value to any element of the embedded space of the 6-th layer representing somehow its 'distance' to the class of the mapped training set. Thereafter, an aggregation is performed between scores using a small validation dataset com- posed of both 'normal' and 'adversarial' samples. These two parts of the proposed approach are detailed below.
Intermediate score functions. Given new observed image æ E Rdd mapped into |A] representations (ze)ee such that 2 [x; D), propose to use the HM depth intermediate scoring functions St.o. Precisely, we compute DHM(Z,, for each considered layer and each class e, i.e,, the HM depth between 24 (z; D) and the class-conditionally probability distribution of the training dataset St = (f((;;)) D): : 2, E S.). Following the approximation algorithm of the HM introduced in Chen et al. (2015), we use an efficient training/
training/tosting tosting procedure in order to compute DHM (summarized in Algorithms and 2 in Appendix A). These algorithms are repeated for each class and each considered layer leading to AA x C scoring functions. Three parameters with low sensitivity are involved: K which the number of sampled halfspaces in order to approximate the expectation of Eq. 4; the size n, of the sub-sample drawn at each projection step; and the À hyperparameter which controls the extent of the choice of the hyperplane. In this paper, we follow the advice given in Chen et al. (2015) by choosing the following parameters K = 10000, = 32 and X = 0.5 offering a good compromise between performance and computational efficiency.
A Aggregation procedure. Following the supervised setting scenario, as in Kherchouche et al. (2020); Feinman et al. (2017) or Ma et al. (2018), the score is obtained through an aggregation which is performed between halfspace-mass scores using a small validation dataset composed of 'normal' and 'adversarial" samples. Our scoring function is then formally defined as:
s(x) = ΣV are DEM S.)
(6)
ZEA 1=1
where the weights at,c are obtained through the training of linear regressor in a supervised manner. It is worth noting that the anomaly score s from Eq. 6 results from both class and layer dependent linear

---

Published in Transactions on Machine Learning Research (03/2023)
combination. The class dependency of is motivated by (1) the per class behavior classifier which is highlighted in Sec. 4.2 and by (2) the monotonicity relative to deepest point property of the halfspace-mass depth and depth functions in general (see e.g. property (Dz) in Appendix B or in Zuo & Serfling, 2000 and Staerman, 2022). The layer dependency of s is motivated by the per layer behavior classifier which is displayed in Sec. 4.3.
Referring to the problem formulation notations (see Sec. 2.1), given a threshold 7, and supposing a 1, the detector is provided by the Eq. 3. The overview of HAMPER can be summarized in the Algorithm 3 in Appendix A.
3.3 Comparison to state-of-the-ar detection methods.
We benchmark our approach with three supervised detection methods: LID, KD-BU, and NSS. We chose these baselines because they are supervised and do not modify the model to protect. We could also consider Abusnaina et al. (2021) but unfortunately the code is not publicly available. We provide the comparison to other two other state-of-the-art methods in Appendix E, Mahalanobis (Lee et al., 2018) and JTLA (Raghuram et al., 2021).
Local Intrinsic Dimensionality (LID). LID is basod on the intuition that adversarial examples lic outside of the clean data manifold. By computing the Local Intrinsic Dimensionality, it is possible to check whether the new point is close to the original data manifold. Following this idea, three version of each natural samples are used. The clean one, noisy version of it, and an attacked one. The LID approxinate for each of those points to the clean distribution will be computed at the output of each layers. Those variables will then be used to train a detector that will distinguish between adversarial samples, and normal ones (normal samples are either clean or noisy samples). Each of the strategy to craft adversarial samples can have very different LID charactoristics, detector per type of thrcats is therefore necessary. LID therefore lics in the attack-aware category.
Kernel Density and Bayesian Uncertainty (KD-BU). KD-BU also relies on the idea that adversarial samples lie off the original data manifold. To detect adversarial samples, they first perform a kernel density estimate at the last hidden layer level to detect samples that are far from the original manifold. Then, a bayesian uncertainty estimate is computed to detect when points lie in low-confidence regions of the input space. Both of those characteristics are later used to train a detector that distinguish between natural and adversarial examples. Once again, the kernel density estimates and the bayesian uncertainty values for different types of attacks can differ a lot, therefore this method have been created to be attack-aware.
Natural Scene Statistics (NSS). NSS relies on the extraction of the natural scene statistics at the image level. Natural scene statistics are statistics that will be very different for natural and attacked images. Indeed, for clean image, applying the natural scene statistics will output an image with meaning, however, for attacked samples, the resulting image will have no meaning. The Natural Scone Statistics extraction is then used to train a delector to distinguish between natural and attacked samples. To overcome the need to have a specific detector per attack, the authors of NSS decided to train their detector using the natural scene statistics of various attacks. It therefore lies in the hiind-to-attadh category.
HAMPER. Our proposed HAMPER detector is computing, thanks to the halfspace- depth, the distance of given x to a reference training distribution. In the sense that it compares between a novel point and a reference, our method is close to the LID method, however, as explained in the previous section, we do not compute our anomality score (i.e., the HM depth) at each layer's output, but we only use a subset of layers A. In addition, possible to use our proposed detector under both scenarios, i... the awfack-aware and the blind-to-attach scenarios.
4 Analyzing Statistical Information of the Networks' Behavior under Threats
In this section, we provide insights into the attackers' and defenses' behavior from the classifier's perspoctive. This section aims to provide justification and insights on the choice of making the linear weights of Eq. 6 dependant on both the class and the layer. In particular, we analyze the behavior of the classifier on attacks
7

---

Published in Transactions on Machine Learning Research (03/2023)
6000
2000
600
1750
5000
500
1500
4000
400
1250
3000
2000
300
750
2000
200
500
1000
100
250
0
2 3
5
8
9
20
A
6
8
9
10
(a) SVHN
(b) CIFARIO
(c)
CIFAR100
Figure 2: Per class behavior analysis. Average number of adversarial examples per class on each of the considered datasots.
in Sec. 4.2, while in Sec. 4.3 we explore which subset of layers of the classifier carries the relevant information to build an efficient supervised data-depth based detector.
4.1 Experimental setting
Datascts and classifiers. We run our experiments on three image datasets: SVHN (Netzer et al., 2011), CIFAR10 and CIFAR100 (Krizhevsky, 2009). We train a classifier that aims at rightfully classifying natural examples for each of those datasets. For SVHN and CIFAR10 we use a ResNet- 18 trained for 100 epochs, using an SGD optimizer with a learning rate of 0.1, weight decay of 10-5, and momentum of 0.9; for CIFAR100 we chose ResNet-110 pre-trained' using an SGD optimizer with learning rate of 0.1, weight decay of 10-5, and a momentum of 0.9. Once trained, all classifiers are frozen.
CIFAR10
SVHN
CIFAR100
Attack sucess
1.0
0.8
0.6
0.4
0.2
0.0
20
25
20
25
30
40
1000
1500
2000
2500
5000
3
Attacks& & choice of the maximal allowed perturba- tion E. To have a wide range of attacks to test, use all the methods mentioned in Sec. 2.3. For FGSM, BIM and PGD, we consider the -oorm, with multiple & in {0.0315, 0.0625, 0.125, 0.25, 0.3125, 0.5}. We also generate perturbed examples using PGD under the Lı-norm constraint with e varying in (5, 10, 15, 20, 25, 30, 40). for CIFAR10 and SVHN, and in (40, 500, 1000, 1500, 2000, 2500, 5000} for CIFAR100. Moreover, we generate perturbed examples using PGD under the Lg-norm constraint with varying in (0.125, 0.25, 0.3125, 0.5, 1, 1.5, 2), for CIFARIO and SVHN, and in (5, 10, 15, 20, 30, 40, 50} for CIFAR100. In order to attack with PGD (L, and L2 norm) the classifier trained on CIFAR100, we chose different epsilon values Ihan those used for CIFARIO and SVHN since the allacks generated with those epsilons were not able to fool the network (see Fig. 3). CW attacks are generated under the Log and L2 constraint, with & equals to 0.3125 and 0.01 respectively. Finally, we perturb samples using DF which is an L attack without any constraint on e. Concerning blackbox attack, SA is an Loc-norm attack 0.125, HOP is To attack with 0.1. Finally, ST is not concerned by norm constraint, nor maximal perturbation, the attacker strength is limited in rotation (maximum of 30%) and in translation (maximun of 8 pixels).
Figure 3: Calibrating the maximal allowed perturbation E on CIFAR100. Accuracy on adversarial examples created using PGD; for the SVHN, CIFAR10 and CIFAR100 classifiers. On CIFAR100, to ensure high successes of the attacks, one must allow the attacker to have larger values of e, compared to the CIFAR10 and SVHN ones.
8

---

Published in Transactions on Machine Learning Research (03/2023)
4.2 Analyzing the Networks' per Class Behavior under Threats
In this section, investigate the per-class behavior of the image classifier to motivate and justify the choice of the class dependency of the proposed aggregation procedure (see Eq. 6)-
Simulation. We examine the distribution of the adversarial examples w.r.t. the class predicted by the classifier. For this purpose, in Fig. 2, we plot the distribution of adversarial samples per predicted class (Ne) as function of the class (c).
Analysis. In SVHN, CIFAR1O and CIFAR100 natural images are balanced. However, on these datasets, the per-class distribution of the adversary is not uniform over the classes: in both SVHN (Fig. 2a) and CIFAR10 (Fig. 2b), classes 3 and 4 are overly represented on average, on the contrary of class 1 and 8 for SVHN and CIFAR10 respectively. Similarly, in CIFAR100 (Fig. 2c), the classes 11, 24, 45, 68, 80, 81, and 97 are the overrepresented whilst the classes 37, 59, 65, 67, 76 and 98 are the most underrepresented on average. Note that the diamond points in the plots denote the outliers, i.e., adversarial examples behaving differently from the others.
Takeaways. The variability of the per-class behavior of the classifier under threats suggests that class important characteristic and should be leveraged to detect adversaries. This observation further motivates the per-class computation of the halfspace-mas mass depths and then the class dependency of the linear regressor of Eq. 6.
4.3 Analyzing the Networks' per Layer Behavior under Threats
In this section, we investigate the per-layer behavior of the image classifier to motivate and justify the choice of the layer behavior of the proposed aggregalion procedure (see Eq. 6).
Simulation. We investigate each layer's roll on HAMPER 's decision process to better understand cach layer importance. In particular, we focus on CIFAR10 with ResNet-18 and we train a linear least squares regressor with L2 regularization (see ogg. Hastic et al., 2009) on the depth features extracted from all the layers, A E {1..... In Fig. 4, we report the weights associated with each layer L E A of the underlying classifier, averaged over the classes, when changing the values of the L2 weight constraint (the values are reported in the legend).
10.0
100.0
1000.0
6
4
1}, of the classifier.
2
0
-2
Analysis. From Fig. 4, we observe of the decision of the detector is based on several layers. This a-posteriori analysis justifies the layer dependency of the linear regressor weights. As the L2 weight regularization increases, the last layers receive more weights, suggesting they are good candidates to build a detector. Although it is possible to rely on the whole set of layers, motivated by efficiency and Fig. 4, we select the 11 final layers the classifier, i.c., we base our experiments on the subset A = (L- - .2. L 1).
0
10 15 20 25 30 35 40
Figure 4: Per layer behavior analysis. Evo- lution of ai _ E. agee the average and the standard deviation over the classes of the re- gressor weight as function of the layer, for different value of the regularization parameter.
Takeaways. Through Fig. 4, we have motivated both the per-layer computation of the halfspacc-mass depths and then the per-layer dependency of the lincar regressor of Eq. 6.
5 Experiments
In this soction, we assess the effectiveness of our proposed depth-basod detection method. The code is available at https://githut com/MarinePICOT/HAAAEE experimental setting in Sec. 5.1 and then we provide detailed discussion of the results in Sec. 5.2.
This section is organized as follows: we first describe the
9

---

Published in Transactions on Machine Learning Research (03/2023)
Table 1: Attack-aware performances on the three considered datasets - SVHN, CIFARIO and CIFAR100 of 44MPER detector together with the results of the SOTA detection methods: LID, KD-BU, and averaged over the Ly-norm constraint. The best results among the detectors are shown in bold. The results are presented AUROCT (FPR loo% %) and in terms of mcan (1)) and standard deviation (o).
LID
XD-BU
HAMPERAA
SVHN
CIFARIO
CIFAR100
SVHN
CIFAR10
CIFAR100
SVHN
CIFAR10
CIFAR100
64.9 (90.1)
57.9 (87.7)
77.6 (47.4)
84.7 (38.7)
71.0 (65.3)
69.5 (11:2)
100 (0.0)
100 (0.0)
100 (0.0)
Norm L
9.2 (9.7)
11.3 (8.6)
23.4 (35.7)
7.0 (16.5)
24.1 (33.4)
21.1 (31.1)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
Norm L2
J
78.3 (73.7)
66.7 (77.5)
66.5 (65.5)
84.3 (42.3)
70.5 (62.2)
56.2 (87.6)
100 (0.0)
100 (0.0)
100 (0.0)
0
10.0 (16.9)
15.0 (22.3)
24.7 (32.8)
16.2 (31.8)
26.9 (40.9)
17.0 (10.3)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
97.5 (9.9)
94.2 (21.2)
66.9 (79.1)
87.3 (21.9)
93.1 (20.3)
67. 1 (80.3)
100 (0.0)
100 (0.0)
100 (0.0)
Norm L.0
3.7 (14.0)
7.6 (24.0)
17.3 (17.3)
27.4 (29.9)
16.8 (32.6)
15.4 (11.3)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
99.1 (4:4)
91.7 (36.6)
98.4 (1.2)
92.8 (21.9)
81.4 (70.2)
76.1 (01.3)
100 (0.0)
100 (0.0)
100 (0.0)
No Norm
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
86.5 (41.3)
79.7 (48.6)
77.4 (19.0)
86.2 (34.3)
82.8 (43.6)
64.9 (80.1)
100 (0.0)
100 (0.0)
100 (0.0)
Average
14.8 (38.2)
18.7 (36.6)
21.4 (30.4)
21.8 (31.5)
23.8 (41.8)
17.7 (17.7)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
Global
"
81.2 (16.3)
78.0 (51.9)
100 (0.0)
a
19.7 (37.0)
23.2 (87.6)
0.0 (o.0)
5.1 Experimental setting
We refer to Sec. 5.1 for the datasets, the classifiers and the attacks we considered for our evaluation. Evaluation metrics. For each threat scenario, the performance measured terms of two metrics:
AUROCT (higher is better): the Area Under the Receiver Operating Characteristic curve (ROC; Davis & Goadrich, 2006) represents the relation between True Positive Rate (TPR) i.e. adversarial examples detected adversarial and False Positive Rate (FPR) i.c. natural samples detected as adversarial. As can be checked from elementary computations the AUROCT corresponds to the probability that an natural example has higher score than an adversary sample.
FPR at 95% TPR1 or FPR 490% (lower is better): represents the percentage of natural examples detected as adversarial when 95% of the adversarial examples are detected. The FPR J05% is of high interest in practical applications.
Remark. The ideal classifier would reach 100% of AUROCT and 0% of FPR 495%-
5.2 Detecting adversarial examples
We recall from Sec. 2.2 that we distinguish between two settings in the supervised context: the attack-aware scenario and the dllld-to-allack scenario. Therefore, we conduct two sets of experiments. In the attack-aware scenario, for each attack we train a detectors on a validation set composed of the first 1000 samples of the testing set and tested the remaining samples. We refer here to HAMPER-Attack-Awarc (HAMPERл) and we compare it with LID and KD-BU. In the blind-to-attack scenario, we train unique detector and we test it on all the possible attacks. We refer here to HAMPER-Blind-to-AA NSS. Note that, while our competitors assign to each input sample the probability of being adversarial, i.e., adversarial samples are labeled as 1, in HAMPER the detector outputs the depth score. This means that a high score corresponds to a deep sample, i.e., a natural one, hence clean samples are labeled as 1.
(HAMPERax) and we compare it with
5.2.1 Attack Attack-aware aware scenario
Here, we study the performance of the different detectors in the attack scenario and show that HAMPERM outperforms existing detectors.
10

---

Published in Transactions on Machine Learning Research (03/2023)
Table 2: Blind-to-Attace detector performances on the three considered datasets SVHN, CIFARIO, and CIFAR100 of the HAMPER&a detector together with the results of the sott--of-the-art detection methods, i.c., NSS, averaged over the -y-norm constraint, along with the average and global performances. The best results among the detectors are shown in bold. The results are presented AUROCT (PPR loon %) and in terms of mean (n) and standard deviation (o).
NSS
HAMPER
SVHN
CIFARIO
CIFAR100
SVHN
CIFAR10
CIFAR100
Norm L
14
69.2 (78.6)
66.7 (80.1)
69.7 (78.2)
94.9 (24.9)
94.7 (22.8)
99.9 (0.2)
15.7 (21.3)
10.3 (12.0)
16.3 (32.0)
3.1 (11.7)
4.7 (19.6)
0.0 (0.1)
Norm Lg
J
71.5 (65.8)
68.0 (72.8)
62.5 (90.2)
94.2 (22.8)
92.3 (30.2)
99.9 (0.3)
19.5 (36.7)
15.7 (20.8)
9.4 (6.3)
3.4 (12.6)
7.2 (23.9)
0.1 (0.2)
93.6 (28.9)
92.3 (15.5)
69.1 (#2.0)
98.4 (7.3)
98.7 (0.33
99.9 (0.3)
Norm Loo
10.4 (43.5)
20.1 (29.2)
12.4 (13.6]
2.4 (9.8)
2.3 (10.7)
0.0 (0.2)
No Norm
j
99.8 (0.4)
93.8 (20.2)
92.9 (24.7)
98.5 (6.4)
80.3 [67.1)
100 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
0.0 (0.0)
83.5 (46.3)
81.2 (42.3)
68.1 1 (81.9)
96.6 (14.6)
95.8 (17.0)
99.9 (0.3)
Average
18.4 (43.6)
21.2 (39.2)
13.4 (20.1)
3.4 (13.6)
5.9 (20.9)
0.1 (0.2)
Global
/
77.6 (66.8)
97.4 (10.6)
0
19.2 (40.0)
4.3 (16.2)
Global analysis. We present the attack-aware evaluation of HAMPER together with LID and KD-BU on SVHN, CIFAR10 and CIFAR100. In Tab. 1, we group the results from Tab. 5 (that we relegate to Appendix C due to space constriction) according to the attack-norm (e.g., L1, L2, Lo No norm), and we express them in terms of the mean on the AUROC and the mean on the FPR Jus%. We also report the average performances per dataset (Average) and over the datasets (Global).
In general, HAMPERAA outperforms the SOTA detectors by maintaining performance close to 100% AUROCT and 0% FPR J95% on all the four datasets regardless of the attack-norm considered.
Performance analysis per E. Overall, the results in Tab. 5 show that the smaller the perturbation magnitude E to craft the attack is, the more complex the attack detection. For example, the worst result of LID is with PGD1 and 5 = 15 for SVHN and PGD1 with & = 20 for CIFARIO (AUROCs smaller than 50%). KD-BU exhibits the same attitude but for FGSM on SVHN, where reaches its minimum (5.9% AUROCT and 99.5% FPR 495%) with æ 0.5. Note that, the high value of the standard deviation in Tab. could implics the detector is more susceptible to the changes. This is particularly true in the case of the L1-norm group since all the altacks considered are created with the same algorithm (PGD). On this regard, KD-BU turns out to be the detector most susceptible (e.g., on CIFARIO its standard deviation on the FPR 495% is 33.4 whilst the one of LID is 8.6 and the one of HAMPERAA is 0.0).
Performance analysis per type of threat. On average between the norm based attacks, LID and KD-BU more casily detects the norm attacks. Interestingly, with L.g both LID and KD-BU have the best performance on CIFAR10 whilst on SVHN and CIFAR100 the detectors have the best performance in the no norm case (cf. Tab. 1). Consistently over the datasets, KD-BU poorly behaves on CW2 as the AUROCs do not reach 50% (cf. Tab. 5).
Summary. Tab. 1 suggests LID and KD-BU have similar behaviors on SVHN, whilst KD-BU improves on CIFAR1O. A closer look at Tab. 5 also suggests KD-BU has higher variance in the results w.r.t. LID on all the three datasets. Thus KD-BU performances are most affected by the perturbation magnitude to craft the adversarial examples. HAMPE on the other side, is hereby confirmed as the best detector since it does not change its performances matter the perturbation the norm considered in the attacks.
11

---

Published in Transactions on Machine Learning Research (03/2023)
5.2.2 Blind-to-attack scenario
Here, we study the performance of the different detectors in the blind-to-attack scenario and show that. HAMPERg outperforms existing detectors.
Global analysis. We present the blind-to-attack evaluation of НAMPERвa together with NSS on SVHN, CIFAR1O, CIFAR100. As in Sec. 5.2.1, in Tab. 2, we group the results from Tab. 6 according to the Sutack-norm and we express them in terms of mean / stanndadd deviation on the AUROC1 and mean / standard deviation on the FPR 495%. Moreover we report the average performances per dataset (Average) and over all the dataset (Global). On average, HAMPERea outperforms NSS by 13. (33.7) on SVHN, 14.6(25.3) on CIFARIO and 31.8(81.6) on CIFAR100 in terms of AUROC1(PPR fosx). Under L1, L2 and -or- constraints, НAMPERа outperforms NSS on all considered datasets. The increase goes up to (0.2(78.0) in Ly-norm, 37.4(89.9) in Lo-norm and 30.8(81.7) in Loc-norm in terms of in AUROC)(PPR 408x).
Performance analysis per С. Overall, the results in Tab. show that, in contrast to HAMPERg1, NSS' performances are increasing with the value of maximal allowed perturbation e. As a matter of fact, the performances of HAMPERga on I, and Lg-norm constraints first decrease as & increases (e E [5, 15) for In-norm constraint, and E [0.125, 0.3125] for L2-norm constraint),, until it starts increasing. On -oom constraint, our method follows the expected behavior, i.c., the performances increases with E.
Performance analysis per type of threat. Tab. 2 suggests that both HAMPERBA and NSS are globally better at detecting attacks with Loo -nor constraints, particularly those created with PGD and BIM as an attack strategy. However, on SVHN NSS finds more difficult to spot the FGSM-based attacks. SA and DeepFool threats are the thoughest to detect for NSS. On the contrary, while HAMPERgA consistently detect. SA-based attack, it shows a slight drop in performance for DeepFool- attacks on CIFAR1O. Finally, NSS presents poor performances at detecting CW2 attacks, while it is not the case for our proposed method.
Summary. While NSS's performances vary with the dataset, the E and the norm used to construct the attacks, HAMPERaa consistently detect them. In particular, we note that HAMPERa is well suited to larger datasets (e.g,, CIFARI00). Conversely, NSS' performance highly decreases when passing from the datasets with 10 classes to the one with 100.
5.3 Attacking HAMPER using Adaptive Adversary
The importance of attacking defenses with adaptive attacks has increased recently (Abusnaina et al., 2021; Raghuram el al., 2021). As mentioned in Sec. 2.3, the Backward-Pass Differentiable Atlack (BPDA; Athalye et al., 2018a) are based on the possibility to find suitable surrogate to the non- non-differentiable parts of any defense. However, deriving a suitable differentiable surrogate of the halfspacc-mmas depth remains an open research question which has never been tackled. As a matter of fact, the only attempt to approximate a non-differentiable depth was performed on the Tukey depth She et al. (2021), with very poor results (Dyckerhoff al., 2021). It is worth point out that, approximating the Tukey depth is already hard, finding differentiable surrogate for IRW would be even harder because it involves several non-differentiable components hthe indicator function, the function x min(2r, 1 - æ) and the need to compute a ranking). Finding a differentiable suitable surrogate to attack HAMPER would, therefore, require substantial effort and should be rigorously handled. As a consequence, we have to rely on adaptive blackbox attackers, as suggested in Tramer et al. (2020), to attack HAMPER.
Experimental Sctting. We designed a blackbox adaptive attacker by using SA, which is both effective and computationally efficient (Engstrom et al., 2019). To extend SA into an adaptive attack, we modified the success criterion to not only fool the classifier but also the detector, and the loss, which becomes trf between minimizing the difference between the logits of the two most probable classes and maximizing the detector's 's prediction. In Tab. 3, we present the results of the adaptive SA attack on CIFARIO for both NSs and HAMPERaA, where a controles the trade-off between the two objectives of the attack.
A Analysis. On Tab. 3, we varied the value of the parameter a controlling the tradc-off between the classifier and the detector performances. As a increases, the performance of the attack on the classifier decreases while the detector has more and more trouble detecting them, as one can expect. On the considered adaptive
12

---

Published in Transactions on Machine Learning Research (03/2023)
Table 3: Detector performances and Attack's success under adaptive blackbox attacks for NSS and НAMPERBA
CIFAR10. We present the results as: AUROC1% (FPR toan %) for the detector performances.
Adaptive Attacks
CIFAR10
NSS
HAMPERBA
Detector
Attack Success (%)
Detector
Attack Success (%)
Performances
Performances
a 10-3
9.4 (100)
95.82
98.5 (8.2)
98.41
a 10 2
7.2 (100)
96.12
96.9 (21.6)
99.37
10
5.5 (100)
93.60
84.0 (60.8)
98.41
=
7.8 (100)
95.88
91.2 (50.5)
99.12
a = 10a
3.2 (100)
93.28
72.5 (70.7)
96.60
a = 10a
3.5 (100)
93.83
75.1 (71.0)
72.80
a 10s
3.7 (100)
93.44
64.9 (81.6)
28.24
Detector accuracy
Detector accuracy
16.6
NSS
HAMPERSA HAMPER&
NSS
HAMPERSA HAMPER
8.4
8.44
KD-BU
KO-BU
L.DD
1.ID
20
16
Predicted (classifier) class (a) SVHN
Predicted (classifier) class (b) CIFARIO
1.0
Detector accirasy
Detector accuracy
0.6
0.6
NSS
НAMPERbA
NSS
HAMPER
0.4
0.4
НAMPERAA
HAMPER
0,22
L.D
0,2
1.10
15
33
15
12
76
96
0.0
14
21
28
35
50
15
Predicted (classifier) class
Predicted (classifier) class
(c) CIFAR100 10 most-populated classes
(d) CIFAR100
10 least-populated classes
Figure 5: Accuracies of the detectors per predicted classifier class. For visualization reasons, we
restrict the plot to the (0-most/least populated classes for CIFAR100.
attacks, it is clear that it is easy for the attacker to find powerful and undetected samples to attack NSS. However, it more difficult to fool our proposed method. To decrease the AUROCT to a value close 50 (which is equivalent to a random detector), the attacker only able to fool the classifier 28% of the time. Takeaways. HAMPERaA is more robust to adaptive attacks than NSS.
5.4 Further Analysis of the Detector Behaviors
In this section, first. investigate the per-class behavior of the detectors to further asses the elfectiveness of the proposed method (cf. Sec. 5.4.1). Then, we study the AUROCT /FPR J9s% trade-off (cf. Sec. 5.4.2) and the time and resource constraints (cf. Sec. 5.4.3) for each of the considered methods.
5.4.1 Analyzing the detectors' per-class behavior
Sec. 4.2 identifics class-dependant behavior of the attack mechanisms that could translate in a class- dependant behavior of the different detection methods. In this experiment, we study the performance distribution per class.
13

---

Published in Transactions on Machine Learning Research (03/2023)
100
100
100
80
80
80
FPR
60
NSS
FPR
60
NSS
FPR
00
NSS
KD-BU
KD-BU
KD-BU
40
40
40
20
HAMPER
20
HAMPERax
20
HAMPER
HAMPER,,
HAMPERA
HAMPER
0
20
80
100
20
100
100
AUROCT
AUROCT
AUROC1
(a) SVHN
(b) CIFAR10
(c) CIFAR100
Figure FPR4 a function of the AUROCI for all five considered methods, (a) on SVHN, (b) on CIFAR1O, and (c) on CIFAR100. This figure aggregates all the results from Tab. 5 and Tab. 6.
Simulation. In Fig. 5, we examine the accuracy of the detectors on the testing samples (natural and adversarial) w.r.t. the class predicted by the classifier. Note that, for visualization purposes, we sclect the 10-most//ees /least populated classes in the case of CIFAR100. For this simulation, we select the threshold ? for which TPR is at 95%.
Analysis. HAMPER performances are not affected cither by the class nor by the datasct and they consistently outperform the competitors. However, the competitor's performance are class-dependant: they tend better distinguish between natural and attacked samples for classes with the highest number of adversarial examples. This is demonstrated by Fig. 5a where NSS, KD-BU and LID obtain the highest accuracy in class 4 on SVHN, which is also the most populated class (cf. Fig. 2a). A similar behavior is observable in CIFAR10 (cf. classes and in Fig. 5b and Fig. 2b). Moreover, Fig. 5 suggests that, in terms of accuracy per class, the SOTA mothods show similar performances on SVHN; on the contrary, on CIFAR10 the detector performing the best is NSS while on CIFAR100 it LID.
Takeaways. Differently from the competitors, the HAMPER detectors are not affected by the per-class distribution of the samples. In particular, and regardless of the datasct, the proposed detectors show uniform behavior overall the classes. Further confirmation is given from the plots in Fig, 5c and Fig. 5d where the accuracies of the detectors remain constant even when focusing on only the most and the least-populated classes respectively.
5.4.2 Studying the AUROC1-FPR 495% relationship
As commonly done in anomaly detection, mcasure the detection performances in terms of AUROC1 and FPR L05%- However, to a large AUROC1 does not necessarily correspond low FPR 495%. In this experiment, we study the trade-off between AUROC1 and FPR 490% performances for each considered detection method.
Simulation. In Fig. 6, we analyze the trade-off between AUROC1 and FPR Jas%. We translate on the Cartesian planes the results presented in Tab. and in Tab. 6. The perfect detector will have the points in (100, 0).
Analysis. Interestingly, the HAMPER detectors behave closely to the perfect detector for all the considered attacks. This is particularly true for HAMPERM. For NSS, KD-BU and LID, large AUROC1 does not necessarily correspond low FPR J90%. In SVHN (Fig. 6a), five of the NSS points are exhibiting high AUROCT (between 70% and 92%) while presenting extremely high FPR las% (between 81% and 99.6%). A similar behavior is presented CIFAR10 with LID.
Takcaways. Contrary to other detection method, which can exhibit high FPR 400% for high AUROC1, our proposed detectors behave similarly the perfect detector for both attack-aware and blind-to-aad ttndtto- scenarios.
14

---

Published in Transactions on Machine Learning Research (03/2023)
5.4.3 Time and resources constraints
For some applications, time and resource necessity can be critical. We, therefore, decided to measure the constraints of each considered method.
Table 4: Time and computational constraints to train and test each detection method. Reported times include all required steps for each methods.
Simulations. In Tab. 4, We report the time and resource constraints needed for each method.
Analysis. All methods have quite comparable training time. However, LID, due to the extraction of the LID paramcters to all layors, takes a lot more time to test than the others.
Method
GPUs
Training Time
Testing Time
NSS
V100-16C 16G
(0m30ss
00m55s
KD-BU
V100-16G
00m30s
02m00s
Takeaways. HAMPER's deployment requires compara- ble time and resources to the other considered detectors.
LID
V100-16G
(4m00s
35m00s
НАMPER
V100-16G
02m00s
02m00ss
НAMPERμи
V100-16G
02m00s
02m00s
6 Concluding Remarks and Future Work
In this paper, we introduced HAMPER, a simple and elfective method to detect adversarial altacks. One of the keys of HAMPER is to rely on the halfspace-mase mass depth, a statistical tool that remains overlooked by the machine learning community. Through our extensive experiments based on two scenarios, attack-aware, and blind-to- attack, we demonstrate that HAMPER achieves state-of-the-art performances. On average, it outperforms the existing best detector by 13.1% (29.0%) in terms of AUROCT (resp. FPR 496%) on SVHN, 14.6% (25.7%) CIFARIO, and 22.6% (49.0%) on CIFAR100. Interestingly, HAMPER exhibits (lass-nndpenndence, and less dependence on the norm-attack and the threat scenarios than other adversarial allack delection methods. As HAMPER relies data depth that is not differentiable and finding surrogates is currently open problem, analysis relies on black-box adaptive attacker for which our method is more robust than existing methods.
Like all the supervised adversarial delection methods in the literature, HAMPER requires knowledge of the kind of attack evaluated at testing time, meaning that they are generally validated by assuming a single implicitly known attack stratogy, which does not necessarily account for real-life threats. In future work, we will investigate how HAMPER and its main competitors perform when facing samples crafted through multiple unknown attack strategies at test time. We will mainly make effort to understand whether the notion of depth is well suited to better generalize the detection of adversarial examples to attacks that are not involved in the supervised framework.
Broader Impact Statement
Many concerns have been raised about the potential failures of Deep Learning: large neural networks are not trustworthy enough, limiting their adoption in high-risk applications. This paper's main contribution aims at improving the reliability of Deep Learning by designing a tool to prevent a malicious agent from disrupting the functioning of the system. Thus we believe our work will have a positive impact on society.
Acknowlegments
This work has been supported by the project PSPC AIDA: 2019-PSPC-09 funded by BPI-France.
This work was performed using HPC resources from GENCLIDRIS (Grant 2022-AD011013737 and 2022- AD01101838).
The work of Federica Granese was supported by the ERC project Hypatia under the European Unions Horizon 2020 research and innovation program. Grant agreement N. 835294.
15

---

Published in Transactions on Machine Learning Research (03/2023)
References
Ahmed Abusnaina, Yuhang Wu, Sumpreet Arora, Yizhen Wang, Fei Wang, Hao Yang, and David Mo- haisen. Adversarial example detection using latent neighborhood graph. In Proceedings of the IEEE/CVE /VF International Conference on Compuler Vision, pp. 0887-7696, 2021.
Ahmed Aldahdooh, Wassim Hamidouche, and Olivier Déforges. Revisiting model's uncertainty and confidences for adversarial example detection. arXiv preprint arXiv: 2103. 1108.05354, 2021a.
Ahmed Aldahdooh, Wassim Hamidouche, Sid Ahmed Fezza, and Olivier Déforges. Adversarial example detection for DNN models: A review. arXiv preprint 005:00005500003, 2021b.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: a query-efficient black-box adversarial attack via random search. In European Conference on Computer Vision, pp. 484-501. Springer, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference on machine learning, pp- 274-283 PMLR, 2018a.
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the 35th International Conference on Machine Learning, ICML, volume 80, pp. 274-283. PMLR, 2018b.
Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. Lof: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93 104, 2000.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39 57. IEEE, 2017.
Fabio Carrara, Rudy Becarelli, Roberto Caldelli, Fabrizio Falchi, and Giuseppe Amato. Adversarial examples detection in features distance spaces. In Computer Vision ECCV 2018 Workshops - Munich, Germany, Proceedings, Part II, volume 11130, pp.
Springer, 2018.
V. Chandola, A. Banerjee, and V. Kumar. Anomaly detection: A survey. ACM Comput. Surv., 41(3): 15:1-15:58, 2009. ISSN 0360-0300.
Bo Chen, Kai Ming Ting, Takashi Washio, and Gholamresa Haffari. Half-space mass: maximally robust. and efficient data depth method. Machine Learning, 100(2):677-699, 2015.
Jianbo Chen, Michael Jordan, and Martin J Wainwright. Hopskipjumpattack: A query-efficient decision- based attack. In 2020 deee symposium security and privacy (sp), pp. 1277 1294. IEEE, 2020.
Yixin Chen, Xin Dang, Hanxiang Peng, and H.I. Bart. Outlier detection with the kernelized spatial depth function. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2):288 305, 2009.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with a fast adaptive boundary attack. In International Conference on Machine Learning, pp. 2196-2205. PMLR, 2020.
Antonio Cuevas and Ricardo Fraiman. On depth measures and dual statistics. methodology for dealing with general data. Journal of Multivariate Analysis, 100(4)753--66,, 2009.
Antonio Cuevas, Manuel Febrero, and Ricardo Fraiman. Robust estimation and classification for functional data via projection-basse based depth notions. Computational Statistics, 22(3):481-499) 2007.
Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd international conference on Machine learning, pp. 233 240, 2006.
Rainer Dyckerhoff Data depth satisfying the projection property. Allgemeines Statistisches Archiv, 88(2): 163-190, 190, 2004.
16

---

Published in Transactions on Machine Learning Research (03/2023)
Rainer Dyckerhoff, Pavlo Mozharovskyi, and Stanislav Nagy. Approximate computation of projection depths. Compulational Statistics & Dala Analysis, 157:107166, 2021.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Exploring the landscape of spatial robustness. In International Conference on Machine Learning, pp. 1802- 1811. PMLR, 2019.
Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv preprint orXio:103..0041 2017.
Yonatan Geifman and Ran El-Yaniv. Selectivenet: deep neural network with an integrated reject option. Proceedings of the 36th International Conference on Machine Learning, ICML, volume 97, pp. 2151 2159, 2019.
Yonatan Geifman, Guy Uziel, and Ran El-Yaniv. Reduced uncertainty estimation for deep neural classifiers. In 7th International Conference on Learning Representations, ICLR, 2019.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. International Conference on Learning Representations, 2015.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernol, Michael Backes, and Patrick D. McDaniel. On the (statistical) detection of adversarial examples. arXiv preprint arxiv: 702.06280. 2017.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, 1CML, volume 70, pp. 1321-1330, 2017.
Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The elements of statistionl learning: data mining, inference, and prediction, volume 2. Springer, 2009.
Shengyuan Hu, Tao Yu, Chuan Guo, Wei-Lun Chao, and Kilian Q Weinberger. A new defense against adversarial images: Turning a weakness into strength. Advances in Neural Information Processing Systems, 32, 2019.
Rebecka Jörnsten. Clustering and classification based on the 11 data depth. Journal of Multivariate Analysis, 90(1))67-89, 2004.
Anouar Kherchouche, Sid Ahmed Fezza, Wassim Hamidouche, and Olivier Déforges. Natural scene statistics for detecting adversarial examples in deep neural networks. In 22nd IEFF International Workshop Multimedia Signal Processing, pp. 1 6. IEEE, 2020.
Gleb Koshevoy and Karl Mosler. Zonoid trimming for multivariate distributions. The Annals of Statistics, 25 (5):1998 2017, 10 1997.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 1921 Chapman and Hall/CRC, 2018.
Tatjana Lange, Karl Mosler, and Pavlo Mozharovskyi. Fast nonparametric classification based on data depth. Statistical Papers, 55(1):49-69, 2014.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of- distribution samples and adversarial attacks, 2018.
Xin Li and Fuxin Li. Adversarial examples detection in deep networks with convolutional filter statistics. In IEEE International Conference on Computer Vision, ICCV, pp. 5775-5783. IEEE Computer Society, 2017.
Bin Liang, Hongcheng Li, Miaoqiang Su, Xirong Li, Wenchang Shi, and Xiaofeng Wang. Detecting adversarial image examples in deep neural networks with adaptive noise reduction. IEEE Trans. Dependable Secur. Comput., 18(1):72 85, 2021.
17

---

Published in Transactions on Machine Learning Research (03/2023)
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 eighth icee international conference on data mining, pp. 413 -22.. IEFE, 2008.
R.Y. Liu. On notion of data depth based on random simplices. The Annals of Statistics, 18:405 1990.
Jiajun Lu, Theerasit Issaranon, and David A. Forsyth. Safetynet: Detecting and rejecting adversarial examples robustly. In IEFE International Conference on Computer Vision, pp. 446-454. TEEE Computer Society, 2017.
Shiqing Ma, Yingqi Liu, Guanhong Tao, Wen-Chuan Lee, and Xiangyu Zhang. NIC: detecting adversarial samples with neural network invariant chocking. In 26th Annual Network and Distributed System Security Symposium. The Internet Sociely, 2019.
Xingjun Ma, Bo Li, Yisen Wang. Sarah M. Erfani, Sudanthi N. R. Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In 6th International Conference on Learning Representations, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Alexander Meinke and Matthias Hein. Neural networks that provably know when they don't 't know. In 8th International Conference on Learning Representations, ICLR, 2020.
Dongyu Meng and Hao Chen. Magnet: A two-pronged defense against adversarial examples. In Bhavani M. Thuraisingham, David Evans, Tal Malkin, and Dongyan Xu (eds.), Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 135 147. ACM, 2017.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. In 5th International Conference on Learning Representations, 2017.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference computer vision and pollern recognition, pp. 2582, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupercised Feature Learning 2011, 2011.
Hannu Oja. Descriptive statistics for multivariate distributions. Statistics & Probability Letters, 166):327 332, 1983.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu Towards robust detection of adversarial examples. In Advances in Neural Information Processing Systems 31, pp. 4584-4594, 2018
Marine Picot, Francisco Messina, Malik Boudiaf, Fabrice Labeau, Ismail Ben Ayed, and Pablo Piantanida. Adversarial robustness via fisher-rao regularization. arXiv preprint arXiv: 2106. 06660, 2021.
Jayaram Raghuram, Varun Chandrasekaran, Somesh Jha, and Suman Banerjee. general framework for detecting anomalous inputs to dnn classifiers. In International Conference on Machine Learning, pp- 8764 8775. PMLR, 2021.
Kelly Ramsay, Stéphane Durocher, and Alexandre Leblanc. Integrated rank-weighted depth. Journal of Multivariate Analysis, 173:51-69, 169,166,220 2019.
Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson. Estimating the support of a high-dimensional distribution. Neural computation, 13(7):1443-147) 2001.
Robert Serfling. Depth functions in nonparametric multivariate inference. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, 72:1, 2006.
18

---

Published in Transactions on Machine Learning Research (03/2023)
Yiyuan She, Shao Tang, and Jingze Liu. On generalization and computation of tukey's depth: Part i. anXiv preprint arXiv:21 1201112.000 08475, 2021.
Angelo Sotgiu, Ambra Demontis, Marco Melis, Battista Biggio, Giorgio Fumera, Xiaoyi Feng, and Fabio Roli. Deep neural rejection against adversarial examples. EURASIP J. Inf. Secur., 2020:5, 2020.
Guillaume Staerman. Functional anomaly detection and robust estimation. PhD thesis, Institut polytechnique de Paris, 2022.
Guillaume Stacrman, Pavlo Mozharovskyi, Stephan Clémençon, and Florence d'Alché Buc. Functional isolation forest. In Proceedings of The Eleventh Asian Conference on Machine Learning, volume 101, pp. 332-347, 2019.
Guillaume Staerman, Pavio Mozharovskyi, and Stéphan Clémençon. The area of the convex hull of sampled curves: a robust functional statistical depth mcasure. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108, pp. 570-579. PMLR, 2020.
Guillaune Staerman, Pavlo Mozharovskyi, Stéphan Clémençon, and Florence d' l'Alhéé Buc. pseudo-metric between probability distributions based on depth-trimmed regions. arXiv preprint arXiv:2108.1271 2021a.
Guillaume Staerman, Pavlo Mozharovskyi, and Stéphan Clémengon. Affine-invariant integrated rank-weighted depth: Definition, properties and finite sample analysis. arXiv preprint arXiv:2106. 10006.111000 11068, 2021b.
Guillaume Staerman, Eric Adjakossa, Pavlo Mozharovskyi, Vera Hofer, Jayant Sen Gupta, and Stephan Clémengon. Functional anomaly detection: benchmark study. International Journal of Data Science and Analylics, pp- 1 1-17, 2022.
Christian Szegedy, Wojeiech Zaremba, Tlya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, 2014.
Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang, Attacks moet interpretability: Attributo-stcered detection of adversarial samples. Tn Advances in Neural Information Processing Systems 31, pp. 7728 7739, 2018.
Florian Tramer. Detecting adversarial examples is (nearly) as hard as classifying them. In International Conference on Machine Learning, pp. 217022 PMLR, 2022.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in Neural Information Processing Systems, 33:1633-1644 2020.
John W. Tukey. Mathematics and the picturing of data. In Proceedings of the International Congress of 1975.
Mathematicians, volume 2, pp.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In 25th Annual Network and Distributed System Security Symposium. The Internet Socicty, 2018.
Chengyuan Yao, Pavol Bielik, Petar Tsankov, and Martin Vechev. Automated discovery of adaptive attacks on adversarial defenses. Advances in Neural Information Processing Systems, 34:26858 26870, 2021.
Stephan Zheng, Yang Song, Thomas Leung, and Ian J. Goodfellow. Improving the robustness of deep neural networks via stability training. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 4480-4488. IEEE Computer Society, 2016.
Tianhang Zheng, Changyou Chen, and Kui Ren. Distributionally adversarial attack. In The Thirty- Third AAAI Conference on Artificial Intelligence, AAAI 2019, pp. 2253-22222222 228-2260. AAAI Press, 2019.
Zhihao Zheng and Pengyu Hong. Robust detection of adversarial attacks by modeling the intrinsic properties of deep neural networks. In Advances in Neural Information Processing Systems 31, pp. 7924-7932 2018.
19

---

Published in Transactions on Machine Learning Research (03/2023)
Yijun Zuo. Projection-based depth functions and associated medians. The Annals of Statistics, 31(5): 1460-1490, 2003.
Yijum Zuo and Robert Serfing. General notions of statistical depth function. The Annals of Statistics, 28(2): 461-482, 2000.
20

---

Published in Transactions on Machine Learning Research (03/2023)
Supplementary Material
A Approximation algorithms
In this part, we present algorithms, originally proposed in Chen et al. (2015) and adaptod our problem, that are used in steps and 4 in HAMPER (see Algorithm 1 for the training and Algorithm for the testing). 3 shows the different steps to compute the scores for HAMPER method
Algorithm Training algorithm for the approximation of DHM- Input: sample 8 (zr,, E S y/ c).
Initialization: Number of halfspaces K; sub-sample size na; hyperarrameter X.
for k = I...,,K do
Draw 8 e,n, sub-sample of 8 with size n, without replacement.
Draw randomly and uniformly direction uy in Sd-1
4: Compute (uy, =e,1) for every ze,s E St,, such that Pk. 2 (ux: =e,))-
5: Set mid, = (max Pk, + min Pk.a) and rangek = max Pk,i min Pk.
Randomly and uniformly select Ne in [midæ- 3 raange, mida 2 rangee]]
K E Sen, Pa, KK)l and my
K E Stm, pR. nK)
7: Set m left
right
n4
8: end for
right K Output: (us, Kk, mlc mi
1k=1'
Algorithm Testing algorithm for the approximation of DHM-
Input: test obscrvation 25; {up, fik, meet m 1 ===1 Initialization: HM-0.
right K
for 1= K do
2: Project ze onto us and such that pf = (Fe, ux).
HM HM + mla
right left I(pk < kx) + m miight I(ph kk).
4: end for
Output: (zM( 8t) HM/K.
21

---

Published in Transactions on Machine Learning Research (03/2023)
Algorithm HAMPER
Input: lest representations { sample representations S. { E y =
Initialization: Number of closed halfspaces K; sub-sample size nai hyperparameter X.
for f= I=.....:44 do for c 1...,,,,C do
Draw K closed halfspaces containing z using Algorithm 1. Compute DHM St) using Algorithm 2.
4:
end for end for
Perform linear regression to find weights ogeo such that s(x) = Ein De age 8.)
Output: the score function
B Formal description of essential properties of Data Depths Formally, data depth function is defined follows:
D: Fdd P(R (x,
+ [0.1]
- D(x, P),
where P(Rd) denotes the space of all probability distributions on Rd. The higher D(x, P), the deeper æ is in P. The depth-induced median of P is then defined by the set attaining suPzeR" D(x, P) in the case where it exists. Since data depth naturally and in a nouparametric way defines pre-order Rd w.r.t. probability distribution, it can be seen centrality-basee based alternative to the cumulative distribution function for multivariate data. Clcarly, equation 7 opens the door to varicty of existing definitions Zuo (2003); Liu (1990); Cuevas et al. (2007); Koshevoy & Mosler (1997); Staerman et al. (2021b); Ramsay et al. (2019); Chen et al. (2015). While these differ theoretical and practically related properties such as robustness or computational complexity, several postulates have been developed throughout the recent decades the "good" depth function should satisfy. Such properties have been thoroughly investigated in Liu (1990); Zuo & Serfling (2000) and Dyckerhoff (2004) with slightly different sets of axioms (or postulates) to be satied by depth function. They are recalled below.
(D:) (AFFInE INVARIANCE) Denoting by Px the distribution of any r.v. X taking its values in Rd, we have:
Va E Rd, D(Ar + PAX Dxx, Px),
for any d x d nonsingular matrix A with real entries and any vector in Rd.
(D2) (MAXIMALITY CENTER) For any E P(Rd) that has symmetry center x² (in a sense to be specified), the depth function ((,, P) takes its maximum value at it:
D(x", P) sup D(x P).
ZERd
(Dz) (MONOTONICHTY RELATIVE TO DEEPEST POINT) For any E P(Rd) with deepest point x", the depth at any point x in dd decreases as one moves away from xt along any ray passing through it:
V [0, 1], D(x", ()7 D(x" + E(xx P).
(D.) (VANISHING AT INFINITY) For any PE P(Rd), the depth function D vanishes at infinity:
D(x, P) 0, ||>|||
22

---

Published in Transactions on Machine Learning Research (03/2023)
These properties introduced in Zuo & Scrfling (2000) lead to the following definition of data depth function.
Definition B.1. A lunction D R2 P(R4) - [0, is statistical depth function if it satisfies (Di D4).
Discussion on properties. The affine invariance property includes common transformations such orthogonal, translation scaling, and is useful in applications providing independence w.r.t. mcasurement units and coordinate system. For distributions having a uniquely defined center (e.g. symmetry center x²), data depths should be maximized at this center, as stated by (D2). The property (Dz) is a consequence of the center-outward ordering construction of data depth. When a point E I moves away from the set of clements that reach the maximum value of the depth function (potentially reduced to a single element, c.g. for symmetric distributions defined above), D(2, P) should decrease monotonically.
Intuition on the halfspace- masp mass depth. The halfspace-mass depth is based on simple idea. It aims to compute anomaly score by exploring the rank of an clement (w.r.t. the training data) every possible one-dimensional projection. An intermediate score is then assigned to the element according to this rank. Further, the halfspace-mass depth value is the expectation of these intermediate (one-dimensional) anomaly scores. It is worth noting that a Monte-Carlo approximation replaces this expectation in practice by computing the average on a finite number of projections. The intuition that an anomaly should be, on average, farther from the median than normal samples over all possible directions of Rdd This depth function has the advantage of being simple, interpretable and computationally fast.
23

---

Published in Transactions on Machine Learning Research (03/2023)
C Detailed Results on Perfect Knowledge about the attacker
Table 5: Performances on the fourconsidered datasets SVHN, CIFAR1O, CIFAR100 and Tiny ImageNct- of the HAMPERа detector together with the results of the otate-of-tho-art detection methods: LID, and KD KD-BU, multiple threat scenarios with multiple maximal perturbations The best results among the detectors are shown in bold. The results are presented as: AUROCT (FPR Jobn %). stipulates the non non-gradient based attacks.
Norm
HANFER
SVHN
CIFARI0
CIFAR100
Tiny
SVHIN
CIFARIO
CIFARIOO
Tiy
SVHN
CIPARIO
CIFAR100
Tiy
PGD,
75.0
05.5
57.9
51.6
50.2 (M4.)
10.5
01.3
100 (1.0)
100
100
100
10 5000
522 092.1
36.0
79.2
75.0 07.05
100
(e" 10000)
04.81
48.9 093.1
76.2 (06.2)
(3.77
76.7 15:
155
(54.)
(71.9)
(0.0)
(8.0)
(0.0)
100 (6.0)
(=" 1500)
61:0 01.71
86.8 090-1
961 (2..9)
81.8 07
64.3 1:
77.0 193
72.3 69
85.4 (9
100 (0/))
06.00
100 (0.0)
100 (8.0)
25 (+" 2000)
65.7 06.0
07.0 (90-1
05.4 (303)
96.9 (7:50
88.4
87.8 (90.0
80.9 (22.0)
90.8 (20-)
100 (v6)
100 06.00
100 000
100 (4-00
C- 30 (e" 2000)
61.6
08.9 192.1
06.4 (17.3)
98.1 (9:4)
91.2
14.2
88.3
94.4 (34.3
100 (0.0)
100 (9.00
100 (0.0)
100 (0.0)
(e" 9000)
73-2
77.7
56.3
97.9
100
100
Normn To
SVHN
CTFARU0
CIFAR100
Tiny
SVRN
CIPARIO
CIPAR100
Tiny
SVIN
CIPAR10
CTF 100
Tiny
PGDe
5 0:1.2
(e" 51
808 (760)
63.6 114.1
05.3 (55.3)
99999 (9:13
54.5 11411
30:3 195.0)
101.5 190.0)
90.55.0.0 (0.0)
100 (0.0)
100 (9:01
100 (0.0)
100 (0.0)
0.25 (et 10)
74.8
29.0 (96.))
89.7 04651
100 (0.00
76.5 19445
40.5
11.3 (84.0)
100
100 (0.6)
100 (9.00
100 (0.0)
100
0.3125
51.2
35.5
10.00
100
200
47.7 095.3
(06.1
(0.0)
20.2 1:33:
(531)
39.4 (99.9)
100 (0.)
(9.))
(0.0)
100 (6.0)
c (c" 30)
(1-5)
09.3 01
83.8 (00-0)
100 (0.0)
56.4 91
08.8 (5:11
65.9 (44
100 (mm
100 (0.))
0600)
100 (0.0)
100 (600)
- 1.5 (e" 100
81.7 (m-7)
89.2 030-5
02.5 (1:4)
100 (0.00
36.2 (1.7)
90.9 (0.00
73.1 (79.))
100 mm)
100 (v6)
100 (6.0)
100 m00
100 (9:))
2 (g" 30)
87.3 (72.0)
95.0 (33.)
94.99 (20.3)
100 (0.0)
90.2 (6.00
59.9
82.5
100 (0.0)
100 (0.0)
100 (9.0)
100 (0.0)
100 (0.0)
DarpFaal
No
95.2
71-4
2 (1577)
95.0 ma
85.7 70.9
100 (n.)) 100 (9.0)
100 (u))
CW3"
0.02
51.0 (4L4)
81.7
12.7 (aas:
100 (o.6)
HOP
94.3 jux
(us))
Norm
SVHN
CIFAR10
CIFAR100
Tiay
SVHK
CIFARIO
CIPARIOO
Tiay
SVHN
CIFARIO
CIFAR160
Tiy
PGD
0,03125
93.7
86.1 01x-1
17.4 (4-7)
100 (6.00
95.6 200
99.0
58.6
100
100 (va)
100 (..0)
100 000
100
0.0625
98.7
94.6
8.8880
100
90.5
100
100
100
97.7 (11:1
00.3)
(mm
(6/))
(6.00
(8.0)
(0.0)
(6.0)
0.25
99.6 (110
00.0 (177)
24.5 (m)))
98.9 mmm
100 (u))
100 (8.00
79/4 (n
100 mm
100 (vo)
(..)
(m0
100 (8.9)
0.3126
59.4 11:0
90.1 (3:00
77.9 01.4
100 (000
100 [00
100 (8.00
63.5 (225)
100 omm
100 (uo)
100 (..0)
100 000
100 (600)
0.5
98.7
98.6 (2.0)
87.2 (36-3)
100 (0.00
100 (0.01
100
90.8
100
100 (0.6)
100 (9.0)
100 (..0)
100
BLM
0,03125
91.5
79.7 000.1
47.4 06:2)
100 (0.0)
02.2 (.1
96.8
58.7
100
100 (us)
100 (8.0)
100 mm)
100
89.2 (62.9)
68.6
(0.00
01.))
(0.6)
100
0.125
99.6 11.00
95.8 (31.4)
(9.81
100 (0.0)
99.9 (6.9)
(6.0)
(53.6)
(0.0)
100 (0/))
0.9)
(0.0)
100 (6.0)
0.22
98.7 1::0
(4.79
24.3 (m-3)
(0.00
100 [.3)
100 (8.00
78.8 (79
100 mmm
100 (u8)
[.4)
(m0
100 (8:9)
- 0.3126
99.4 11:00
90.1 (3:00
78.8 (7:4)
100 10.00
100 [0/)
100 (9.00
$8.4 (22.)
100 mm)
100 (0))
100 (..0)
100 mmm
100 (60))
- 0.5
99.2
30.7 12.00
85.4 (04.1
100 (0.0)
100 (0.01
100
91.5
100
100 (0.0)
100 (9.0)
100 (0.0)
100
PGSM
0,00125
45.4
86.6 (o94))
65.9 mx.-
100 0.:)
67.8 (..
24.9
10.0
96.5
100 (U0)
100 (9.0)
100 mmm
100 (8.0)
99.2
97.3 (12.7)
47.3
(0.0)
100
0.125
99.7 (3.2)
(2.90
47.2 (9.91
100 (6.0)
92.7 22.6
(46.0)
(97.9)
(0.0)
100 (0.))
(.3)
(0.0)
100 (6.0)
2 0.25
86.8 (0.0)
98.4 (1.5)
8233 (06.2)
100 (0.0)
50.5 16.00
08.8 (5.3)
67.1 ())6)
100 (..0
100 (0.)
100 0.0)
100 (0.0)
100 (6.0)
= 0.3175
90.1 (0.0)
18.1 (1:50
86.7 (54.))
100 (0.
6.2 (990)
00.9 (9:3)
60.3 (77
100 (m:)
100 (-:))
100 (8-0)
100 (..0)
100 (6.0)
-- 0.5
92.0
100 (0.01
93.7 (302)
100 (0.0)
5.8
98.6
723.
100 (0.0)
100 (0.0)
100 (..0)
100 (0.0)
100 (0.0)
CW.,"
0.3125
85.0
71.5
10.8
50.0
65.7
100 (n:)
100 100
SA
0,125
03.1 (800)
84.6 (0.0)
88.1 (840))
50. 1225
9336 (171)
[0.0)
100
No norm
SVII
CTFATU0
CITAT1000
Thm
SVIN
CIPARIO
CIPARIOO
Thy
SVII
CIPARIO
CIFAR1000
Thy
No.
90.1 (44)
91.7 (30:)
98.4 (1:0
100 10.0)
92.8 21.5:
S1.4 (79.0
76.1 (3.3)
100 o0:)
100 (0:0
100 (0.0:
100 o..0)
100 (0.0)
24

---

Published in Transactions on Machine Learning Research (03/2023)
D Detailed Results on No Knowledge about the attacker
Table 6: Performances on the four considered datasets SVHN, CIFARIO, CIFAR100, and Tiny ImageNet- of the НАМPERел detector together with the results of the stato-of-the-art detection method: NSS, on multiple threat scenarios with multiple maximal perturbations e. The best results among the detectors are shown in bold. The results are presented as: AUROCT (FPR 100% %). stipulates the non-gradient based attacks.
Norm L
HAMPER
SVHN
CIFARIO
CIFARIOO"
Tiny
SVHN
CIFARIO
CIPAR
PGD,
C - 5 (2*
40)
48.5 155.1)
50.1 (94.0
51.6 (94.3)
44.7 (35.0)
95.7 120.4)
39.4 (47.5)
99.9 (0.4)
100 (9.0)
10 (e"
500)
51.5 194.5
56.7 (900
50.1 094.0
18.7 (98.0)
87.6 (51.0)
90.4 (35.9)
99.9 (0.3)
100
15
82.5
58.3
56.8
99.9
20 (e" 1500)
67.7 (91.0
66.5 (91.8)
67.5 (51.30
96.0 (21.8)
98. [8.5)
99.9 (0.2)
(0.0)
25 (6" 2000)
78.1 (3.21
72.0 (244))
75.5 090-9
79.1 (90.0)
96.4 (10.5)
98..3 (6.3)
99.9 (0.9)
100 (0.0)
30 2500)
84.8
0 (9440)
63.7
55.0
95.7
98.88
100
100
194.0)
52.1
(10-9)
Norm La
SVIIN
CIFARIO
CITADL00
Tun
SVIN
CIFAR1O
CIRATU000
PGDs
[e* 5)
(05.0
49.7 (94.2)
094.0
S7.6 (37-4)
(27.7)
(25.3)
09.9 (0.0)
(0.0)
- 11:25 (e" 11)
49.5 (949)
55.7 (90.0
51.7 (4..0)
100 (6.00
90.4 197.7)
89.6 (as..)
99.9 (6.91
100 (0.0)
0,3125 (e* 15)
12.5 (94.1
58.0 (84.2)
52.8 094.0
100 (0.00
91.9
79.9 (09.9)
99.8 (0.81
100
0.5 20)
67.5 (230)
(15
(940)
(90.7)
(90.0)
(6.3)
(e" 30)
02.1 03:91
53.1 (54-))
59.1 (55.))
100 (0.0)
08.8 (6.5)
99.4 (8:30
100 (0.3)
100 (0.0)
1.5 (e* 40)
98.0 (
91.7 (0750)
67.2 (m:4)
100 (h-0)
96.9 (4.4)
100 (6.))
99.9 (0.9)
100 (n.n)
2 (s" 50)
09.4
96.2
77.5
100
99.5
100
100
DaagPool
No
58.2 (s.:1:
55.9 (9:30)
73.0 (194))
99088 (20.0))
79.7 (02.8)
100 (s.0)
CWe'
0.01
61.8 192.4)
56.0 (0
64.6 (90.0)
92.7 (90.9)
89.2 (41.8)
99.9 (0.11
HOP
(4:::
(er
(22-9)
(ovs)
Norm Tog
SVHN
CIFARIO
CIPAR1O0
Tiny
SVHN
CIFARIO
CIFARIOR 00
Tiny
PGD.
1,003125
99.3 11:7:
91.3 (34.2)
58.1 (23.))
100 (0.0)
97.4 (33.0)
99.4 (2.01
99.9 (0:41
(0.0)
0,0025
98.9 10.2)
99.0 (4:4]
55.9 (11.))
100 (0.0)
99.5 12.00
89.3 (3.6)
100 (0.3)
100 (0.0)
0.125
99.9 10.
99.9
61.5 09..0
100 00.00
99.9 (633)
98.8 (.1)
99.9 (0:41
100
0.25
98.9 (0:2)
71.3
100
99.7 0.7)
100
99.9
0.3125
00.9 00:0
99.9 (0.1)
76.4 (44.8)
(0.0)
100 (8.1)
98.8 (0:5)
99.8 (0.8)
(0.00
0.5
00.9 (0:30
06.0 (6.3)
84.7 (97.6)
100 (0.0)
100 (0.3)
100 (6.1)
99.8 (0.0)
100
HIM
0,03125
98.0
89.3
53.0
100
96.7
96.8
09.9
100
110025
99.8 (0.3)
97.9 (a.3)
55.8 (91.9)
100 (0.0)
09.2 (3.0)
99.8 (0.))
99.9 (0.2)
(0.0)
0.125
99.9 mm:)
16.7 (6.99
61.4 (9:30
100 (4.00
99.9 (0:3)
99.8 (1:1)
99.9 (0.4)
100 (0.0)
0.25
20.9 00:00
99.9
71.2 06.1)
100 00.00
100 00.00
98.9 0.40
99.9 (0.4)
100
0.3125
99.9 (m:)
(9-0)
99.9 (0:4)
(6.4)
(9-0)
6 0.5
99.9 (.:3)
99.9 (0.1)
84.5 09
(0.00
99.9 (6.3)
98.7 (1.4)
99.9 (0.1)
(0:0)
PCSM
0,00112
98.6 (0:3)
50.6 (25.0)
50.1 (X1.))
100 (0.0)
94.3 (25.2)
96.9 (15.7)
99.8 (0.0)
100 (0.0)
B.00225
98.7
57.1
95.6
99.9
0.125
H2.0 (100:
99.9 (6.1]
67.2 (67.0
100 (0.0)
98.6 (6.5)
98.7 (1.3)
99.9 (0.11
(0.0)
- 0:25
70.5 1100
99.9 (8.1)
82.1 (15.0)
100 (4:00
99.4 (3.5)
99.9 (1.0
99.9 (0.4)
100 (9.0)
0.3125
76.1
06.9
86,0
100
08.5
100
09.9
100
0.5
(97.4:
(1:12
(9-0)
(6-7)
(6.0)
(0:3)
(9-0)
CW..
- 1.3122
67.9 msv4:
64.6 (80.0)
70.0 085.0)
100 (0.00
9906 (80.8)
99.5 (40.9)
99.9 (0.4)
|000 (0.00
SA
0.125
01.3 psun:
11.8 (909)
67.4
[00 (26a))
09.0
97.9 (1277))
09.9
100
No norm
SVHN
CIFARIO
CIPAR100
Tiny
SVHN
CIFARIO
CIFAR100
Tiny
No
99.8 ma:
93.8
92.9
100 (n/0)
98.5 o.o
803
100 (0.0)
100
25

---

Published in Transactions on Machine Learning Research (03/2023)
E Comparison to Additional State-Of-The- Methods
We compared our proposed detector with two additional methods: Mahalanobis, introduced by Lee et al. (2018), which works under the framework of Attack-Aware detectors, and JTLA, introduced by Raghuram et al. (2021), which is an unsupervised method. We reported the detailed results in Tab. 7 and Tab. 8, and the averaged ones in Tab. 9.
Table 7: Performances on the four considered datasets SVHN, CIFARIO, CIFARIOO and Tiny ImageNet- of the НAMPERΑ detector together with the results of the state-of thr- detection methods: Mahalanobis, on multiple threat scenarios with multiple maximal perturbations e. The best results among the detectors are shown in bold. The results are presented as: AUROCT (FPR 4as% %). stipulates the non-gradient based atlacks.
Norm L:
Mahalanobia
HAMPERA
SVIIN
CIFARIO
CIFARIOI
Twy
SVIN
CIPAR1O
CIFARLOU
Twy
PGD,
40)
80.2 (55.5)
75-2 1)
95.6 (16.5)
100 (0.)
100 (0.0)
100
(0.))
- 10 (6" 500)
86.4 173.0)
84.9 (1.5)
79.3 (60.3)
98.7 (1.7
100 (0.0)
100 (0.0)
100 (8.0)
100 (0.0)
15 (e" 1000)
60.2
88.6
50.7
90.0
100
100
100
100
8 (e* 1500)
92. (0030
93.4 (5.33
55.7 (65
99.8 (0:11
100 (00)
100 (0.0)
(6.0)
100 (0.0)
25 [e* 2000)
96.6 (24.7)
06.8
91.0 (9910
90.9 (1.13
100 (U))
100 06.00
100 (9.0)
100 (000
(6"
97.3 (6690
95.2 (16-9)
(0.0)
100 (0.00
2 40 (s* 5000)
95.0 (9.7)
99.7 (1:00
00.9 (0.3)
100 (0.0)
100 (0.0)
100 (0.00
100 (9.0)
100 (0.0)
Norm L; PGDg
SVIIN
CIFAR10
CIFARI0I
Twy
SVIN
CIPAR10
CIFAR100
Tiay
0.125 (e*
(91-9)
78.1 (64.9)
77.5 (99-9)
90.9 (0:1)
100 (0.0)
100 (0.0)
100 (0.0)
<= 0.25 (e* 10)
64.0 174.7)
84.2 (7..5)
79.0 (6544
100 (1.1)
100 (00)
100 00.0)
100 (6.0)
100 (0.0)
= 0:3125 (e* 15)
66.0
86.4
79.8
100
100
100
100
100
20)
91.8 (46.3)
93.2 (35.5)
80.3 (62.3)
(0.0)
(0.0)
(0.0)
(6.0)
(0.0)
= 1 [e* 30)
95.6 (7.7)
99.5 (2.40
82.5 (79-0)
100 (04)
100 (U))
100 06.00
100 (6.00
100 (0.0
1.5 d0)
99.7 (1:3)
58.0
100
100
100
= 2 (e* 50)
99.9 (0.4)
99.8 (0.00
93.3 (99.6)
100 (0.0)
100 (0.0)
100 (0.00
100 (6.0)
100 (0.0)
Doeg
No
81.0 (76.))
(76.3)
89.8 (90.3)
100 (0.0) 100 (0.00
(8.0)
CWs'
(1.11
74.5 (ta.))
66.4 (56.))
58.6 (97.9)
100 (0.0)
100 (0.0)
100 (6.0)
HOP
2 - 0.1
79.9 176.
76.7 (37.7)
56.9 (90.5)
100 (0.0)
100 (0.0)
100 (8.0)
Norm L.
SVIIX
CIPAR10
CIPARIOI
Twy
SVIN
CIPAR10
CIFARLOO
Tiuy
PGD,
0.03125
08.2 (9:3)
52.9 (91:1)
(0::
100 (0.)
100 (0.00
(8.0)
(0.)
0.0625
100 (0:1)
100 (0.0)
54.7 (9:.)
100 (0.0)
100 (0.0)
100 (6.0)
100 (8.0)
100 (0.0)
0.125
100
100
20.7
100
100
100
100
100
0.25
100 (00)
(UU)
85.7 (96.5)
100 (U0)
100 (00)
100 (0.0)
(6.0)
100 (00)
0.5125
100 (0.0)
100
80.7 (12.4)
100 (000)
100 (00)
100 00.00
100 (6.0)
100 (0.0)
100 (0.0)
93.7 (14.1)
(0:0)
(0.0)
100 (0.0)
(0.0
DIM
0.03125
96.6 (16.9)
97.4 (12.5)
52.5 (94.6)
100 (0.0:
100 (0.0)
100 (0.0)
100 (6.0)
100 (0.0)
0,0625
90.9 (0.0)
58.3 (60.0)
(000:
(0.0)
100 (0.0)
(6.0)
(0.0)
- 0.125
100 (0.0
100 (0.0:
09.8 (75.1)
100 (0.0)
100 (0.0)
100 (0.0)
100 (6.0)
100 (0.0)
0.25
100
100
85,3
100
100
100
100
100
= 0.3125
100 (0.0)
100 (0:0
80.7 (1710
100 (0:0
100 (00)
100 (0.00
100 (6.0)
100 (00)
0.5
100 (0.0)
100
96.1 (15.4)
100 (0.0)
100 (0.0)
100 (0.0)
100 (6.0)
100 (0.0)
PQSM
0.09125
94.8 127.5)
100 (0.0:
52.4 (94.4)
100 (0.0:
100 (0.0)
100 (0.0)
100 (8.0)
100 (0.0)
(108225
99.4 (1:3)
90.8
50.2 (12.8)
100 (0.0)
100 (0.0)
100 (0.0)
100
100 (0.0)
99.9 (0.0)
96.5 (17.8)
59.0 (91.))
100 (0.0)
(0.0)
100 (0.0)
(6.0)
(0.0)
0.225
100 (0.0)
90.5 (1:9)
64.2 ()
100 (0.0)
100 (0.0)
100 (6.0)
100 (8.0)
100 (0.0)
0.3125
100
86.2
100
100
100
0.5
100 (0.0)
(00))
73.2 (86.6)
100 (0.0)
100 (00)
100 (0.0)
100 (6.0)
100 (0.0)
CW 1
0.3125
(58.9)
86.3 (74.5)
77.5
100 (0v0:
100 (0.0)
100 (0.0)
(8.0)
100 (0.0)
SA
= 0.125
61.0 (98.3)
96.8 008
79.2 (55.4)
95.6 (37.0)
100 (0.0)
100 (0.0)
100 (6.0)
100 (0.0)
No norm
SVHN
CIFARIO
CIFAR100
Tin
SVHN
CIFARIO
CIFARL00
Tiny
No
100
91.6 (50.5)
55.0
100 (0.0:
100
100
100
26

---

Published in Transactions on Machine Learning Research (03/2023)
Table 8: Performances on the four considered datasets SVHN, CIEARIO, CIFARIOO and Tiny ImageNet- of the HA detector together with the results of the state-of-the-art detection methods: JTLA, on multiple threat scenarios with multiple maximal perturbations E. The best results among the detectors are shown in bold. The results are presented as: AUROCt (FPR f0o% %). stipulates the non-gradient based attacks.
Norm Ia
JILA
HAMPER
CIFARIO CIFARIOO
Thm
CIEARTO CIPAR100
PGDd
m 5 (e*
400
82.1 ())m)
75.0 (759)
57.7 (49-9)
50.2 (94.9)
95,7 (20-4)
8R.4
(47.5:
90.9 (0.4)
100 (0.0)
(e"
500)
78.0
65.2
50.0
47.1
67.6
90.4
99.9
100
15 (6"
1900))
77.5 (0240
61.5 186.3
50:2 (9..))
096 50
95.1 (23.5)
89.2 149.0
99.9 (0.2)
(0.0)
20 (6"
1500)
76.3 (11m)
59.4 (66.)
57.2 (24-0)
13.7 (968) 90
96.0 (21.8)
58.3 (4.50
90.9 (0:3)
100 (0.0)
20 (6"
2000))
76.4 09:40
07.7 (40-5
55.2 (99.0
11.8 096
96.4
98.3
99.9
100
74.4 (99.)
56.6 (0-3)
54.1 (26-2)
73
95.7 (22.8)
(K-1)
2 40 (c" 5000)
72.6 (05
54.9 00.9
42.0 (90.0)
43.0 (95.9)
97.9 (10.3)
90.3 (2.5)
100 (0.0)
100 (0.0)
Norm Lu
SVHN
CHEAROO
CIFARIOO
TTay
SVHN
CIFAR1O
CIEAR100
Tlay
PGDs
0,125 (e* 5)
80.8 (32.9
74.4 (71.7
58.3 (55.7)
40.0 (1
93.0 (27-7)
94.6 (2.3:
99.9 (0.0)
100 (0.0)
0,25 (e*
50.0
66.0
58.8
39.5
90.4
89.0
99.0
100
0.3125 (ef
78.9 (30.9)
63.5 (94-))
50.3 (89/0)
(7.40
91.9 (34.0)
79.9 99939
99.8 [0.8)
(0.0)
- 0.5 (e* 20)
76.8 (3340
69.3 (66.5)
58.5 (99.0)
39.5 (9 10
94.0 (38.7)
94.7 20.4:
99.9 [0.2]
100 (0.0)
(e -00
72.7
54.9
58.3
38.4
88.8
90.4
100
100
1.5
(20.)
03.5 (01-3)
56.5 (40-0)
4
(4:4)
(0:40
= 2 te* 50:
68.0 (75.8)
53.1 (0..)
52.1 (65.0)
30.5 (m 49
90.5 (2.4)
100 (0.00
100 (0.3)
100 (0.0)
DeepFool!
No
05.8
94.6 63.0
00.8
70.7
100
CW,"
0.01
77.0 (v0m))
00.3 (os.ic
58.4 (45.0)
92.7 (20.0)
89.2 (s0.;
99.9 (o.1)
BOP
c 0.1
94.5 (28.3)
91.1 DoL:
50.9 (51.0)
93.8 (22.8)
95.6 (39.4:
99.8 (0.5)
Nori low
SVHN
CIPAR10
CIFAR1O0
Tiny
SVHN
CIFARIO
CIFAR100
Tiyy
PGD
0,031155
72.9
55.9
56.4
31.4
97.4
90.4
99.9
0.0022
06.5 (25.0)
56.7 (00.))
54.8 (11.4)
092.0
99.5 (2:0)
99.3 (3.4)
100 (0.11
(0.00
0.125
61.3 (24.0
55.5 (0:4)
52.3 (91-2)
19.9 m
99.9 (1:5)
99.8 (9-7)
99.9 (0.1)
100 (..0)
0,25
38.0 (9:4)
06.1
48.0 (2.m)
20.5
99.7 [1.9]
100
99.9
100
0:3125
58.5 (4)
56.2 (0:9)
50.0 (33.))
(0.1)
(9.0)
c 0.5
58.2 (90.0
56.1 00.9:
49.0 (43.4)
20.8 (91.9
100 (0.2)
100 (0.3)
99.8 [0.5)
100 (0.0)
BMM
0.03125
73.3 (9:2)
56.5 (1:8)
56.8 (90.1)
58.3 (94
96.7
96.8
99.9
100
0,0625
67.0 (79.0)
09.4)
55.0 (11.0)
(90.0
(3.8)
(0.5)
(0:2)
(0.0)
2 0.125
02.1 (29:0)
55.2 100.4)
52.5 (91.7)
28.1 (91.7)
99.9 (0.71
90.8 (1.3)
99.9 [0.4]
100 (0.0)
0.25
50.2 (23.9)
55.8 00:9)
50.4 (12.7)
20.9 (11.9
100
99.9 (0.4)
99.9
100 (0.0)
0,8125
38-6
66.1
49.5 (12.0)
e 0.5
57.5 (41
56.7 (10.))
49.4 (33.2)
21.0 (m m
89.9 (0:3)
90.7 (1.4)
99.9 (0.1)
(0.0)
RGSM
0.03125
89.9 (37.0)
70.3 190.3)
57.7 (99.2)
54.5 (26.0)
94.3 (26.2)
96.9 115.7:
99.8
100 (0.0)
0.0625
(50.5)
87.9 (00.3)
58.0 (47:))
95.5 (20-5)
5 0.125
87.7 (45
65.7 09.0)
50.7 (09-4)
54.5 (76 59
98.4 6 (6.5)
90.7 (3.3)
99.9 (0.1)
100 (0.0)
= 0.25
68.9 (27.4)
85.1 (94
57.1 (64.0)
54.8 (79 13
99.4 (3:))
99.9 (8.5)
99.9 (0.4)
100 (9.0)
0,3125
50.0
65.9
57.3
54.5
98.5
0.5
88.8 (41.4)
74.2 (05.4)
56.8 (43.0)
54.5 (36 40
(6.7)
(0.00
99.9 [0.2)
(0.0)
OWis
0,3125
80.7 (43.0)
85.1 (04.4)
60.5 (62.0)
51.1 (59.00
90.5 (33.0)
90.5 (40.9:
99.9 (0.3)
100 (0.0)
SA
0.125
90.4 (s1.9)
88.7 (004)
61.8 (49.0)
(20 so
(4.))
97.9 (10.7:
(a.2)
No norm
SVIN
CIFARIO
CIFAR100
Thy
SVIIN
CIFARIO
CIFAR
Tin
61.7 (1.m7)
85.0 (sa.e)
00.2 (29.)
54.5
(..4)
(0.0)
(0.0)
F
Additional Attacks
To further assess the strength of our detector, we tested all considered methods on two additional attacks on CIFAR10: a targeted version of CW2 (reported CW2targeted) and version of CW2 with higher confidence (kappa=30), both of which don't clip the adversarial examples according to a specific Ly no-morm. The results are reported in Tab. 10. It is clear from the results that our proposed framework outperforms the other state-of-the-art methods on those specific allacks.
27

---

Published in Transactions on Machine Learning Research (03/2023)
Table 9: Attack-aware performances on the three considered datasets SVHN, CIFARIO and CIFAR100 of НAMPER and HAMPERga detector together with the results of the SOTA detection methods: Mahalanobis and JTLA averaged over the Ly- -n-norm constraint. The best results among the detectors are shown in bold. The results are presented as AUROCT (PPR J00% %) and in terms of mcan (1²) and standard deviation (o).
Actack-
Wireb Attacic
SVIx
CIRARIO
Eay
SVIEN
CHEARUCO
SVEEN
CHEARI
CIFARIIA
Tlay
CHEARIS
CIEARUOD
Norm
01.3 :a:0)
01.7 00:00
54 (2-1)
86.7 03475
9.4 18:41
95.0 10:0
(3
06.33
09:00
0.0 1344
100 21.61
0.0 m.
0..0
100 0..11
0.0 (.:11
20.7 00.13
(1:1
61.1 14611
0.71
05.00
2.2 (8.00
055.0
81.0 (14.6)
131.7
94.7 (904)
4.7 156.)
98.9 (..0)
0.0
0.0 (4.))
m
Norm To
27. (4/1)
9.5
86.0 06.0
11.2 04
11.5 07.6
78. 09.00
0.0
100
100 080
0.0 mm
0.0 3.5
100 3651
0.0 01
100
100
0.0 (.
73,6 64.0
163 0367
67.7
14.1 19.30
58.4
239 13.0
89.6
052 (8:3
04.2
3:4 13240
92.8 (662)
7:2 20.
99.0 m..))
011 (121
0.0 (1.)
100 (00
Norm Ien.
38.9
4.5 csa)
08.3
37 (1:2)
71
14.8
1.0
0.0 1340
0.0
100
181
62.5
54.5
15.7
98.4
2.4
08.7
2.5
0,0
100
No Norm
0.0
19.11
0.0
Global
01.0 00
(2:at
106.0 (u:))
0.0 19:6
56.5 (017)
9. (1-00
2:36 (A.)
B6.1 (4:1
Table 10: Results CIFARIO for LID, KD-BU, NSS, HAMPER*4; and HAMPER on CW2 with high confidence (i.e., CW2,x==0) and CW2 targeted (ie.,,.. 2,targeted).
Attack-Aware
Blind totdd-toAAttck
Attack
LID
BU-BU
HAMPER
NSS
HAMPERga
CW 222 2,x=30
54.0 (92.6)
50.1 (94.8)
100 (0.0)
62.0 (88.6)
96.9 (15.0)
CW ttargete
37.3 (98.2)
52.6 (93.4)
100 (0.0)
46.9 (95.1)
97.8 (12.6)
G
Sanity Check
Recently, (Tramer, 2022) presented a way to compare robust and detection performances between methods. They claim that, for detector that can achieve Rade performances for a given e, it possible to create a robust classifier with the same Radv; but for s* g. Therefore, by using the best performances of robust classifier for given å (denoted by 1 aal))) and computing the 1 - Radv for proposed method (denoted by 1 R it is possible to ensure that our presented results are not overclaimed.
According to (Tramer, 2022), for a specific threshold, we have
(8) where Re represent the detection failure of a detection method, FPR and FNR, the False Positive Rate and False Negative Rate at for a specific threshold and R(classifier) the natural error of the considered classifier.
Redo < FPR + FNR + R(classifier),
Using the results presented in Tab. 3, we can say that R < 93.27 for S = 0.125.
Since 1 - R R=/2 is around 30 for this specific value, we have that Red < 1 Re/2 Therefore, it appears that we do not overclaim our performances which further validate our detector.
28


Reviews:
[{"review_id": "tpt6E7UrJK", "reviewer": "Reviewer_LPTq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\\+ It firstly applies the halfspace-mass depth notion in the context of the adversarial detection problem. \n\n\\-  The technical contribution may be limited. The paper mainly adopts the halfspace-mass depth method following previous papers such as Chen et al. 2015 which originally use this method for anomaly detection. The novelty may not be significant. The main algorithms and hyper-parameters mainly follow Chen et al. 2015. The aggregation procedure follow several supervised settings to simply use a linear regressor in a supervised manner. The technical contribution may be limited.\n\n\\- The complexity can be high. The score functions need to iterate over each class in the dataset and each layer in the model. The complexity may be high if the number of classes and layers are high for large datasets or models. It is better discuss the overall complexity. The authors show the training time and testing time. But this time is only the time to train the detectors with a linear regressor. The time to compute the halfspace-mass depth score function is not reported. Without the score functions, simply use a linear regressor does not work. \n\n\\- It can be hard to use the method in practice. The score functions need to be trained for each class in the dataset and each layer in the model. If the dataset is switched or we use another model, we need to repeat the whole process. If the whole process takes a lot of time and efforts with high complexity. It may be not practical. \n\n\\- The problem formulation mainly uses untargeted attack without a target label. So it is ok as long as the classified label differs from the true label. But for the attacks with targeted attack version, we can assign a target label (which differs from the true label) and the classified label should be the target label. The analysis in Section 4 also depends on untargeted attack to show the average number of adversarial examples per class. In targeted attack, as we can control the target label, the distribution per class can be very different.  It seems that the problem formulation and analysis mainly focus on untargeted attacks without considering targeted attack. It is better to provide more discussion. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes to apply the halfspace-mass depth notion in the context of the adversarial detection problem. \n\nThe paper introduces HAMPER, a simple supervised method to detect adversarial examples given a trained model. Given an input sample, HAMPER relies on a linear combination of the halfspace-mass depth score. These depth scores are computed w.r.t. a reference distribution corresponding to the training data conditioned per-class and per-layer. \n\nThe paper evaluates HAMPER’s performance across popular attack strategies and computer vision benchmark datasets. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As shown in the weakness, it is better to discuss the complexity of the overall algorithm. The time in section 5.4.3 is only the time to train the linear regressor, right? It does not include the time to train the score functions, right? Why not report the overall time and resource consumption? It is better to make the complexity more clear.\n\nThe method needs to iterate for each class and each layer. If the complexity is high, it may be hard to use in practice if we switch the dataset or model and repeat the overall high-cost process. It is better to discuss about this.\n\nIt seems that the problem formulation and analysis mainly focus on untargeted attacks without considering targeted attack. It looks like to only apply to untargeted attack without further discussions. It is better to provide more discussion about the targeted attack and untargeted attack. ", "broader_impact_concerns": "It is OK to investigate the adversarial example detection to enhance the model robustness. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dKDoAQBxex", "reviewer": "Reviewer_NQqi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength** \\\nThis paper is very well-written and easy to follow. \nThe author provides extensive experiments along with a detailed analysis of the proposed methods. \nFurthermore, the proposed methods show much better performance compared to previous works.\n\n\n**Weakness** \\\nHowever, I have a few concerns on this work. \nFirst of all, I think the authors should also provide the detection performance against FGSM, and C&W because it seems those attacks are a more standard benchmark for the adversarial detection task. \\\nFurthermore, this paper seems to miss some baselines of the adversarial detection.\n- Since the current approach is also a motivating technique from anomaly detection, comparing with [1] is also needed. It would show how the current approach is a better technique of anomaly detection to use in the adversarial detection task. Further, based on the previous work [3], [2] seems to show quite good performance on detecting the adversarial examples.\n- The author seems to missed recent baselines [3,4].\nLastly, I think author should conduct a test of [1] whether current detection’s claim is correct. Specifically, could the author provide $1-R^\\epsilon_{adv-det}$ and $1-R^{\\epsilon/2}_{adv}$ values from [1] to compare how much suggested approaches could actually detect the adversarial examples?\n\n\n[1] Tramer et al., Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them \\\n[2] Lee et al., A simple unified framework for detecting out-of-distribution samples and adversarial attacks \\\n[3] Raghuram et al., A General Framework For Detecting Anomalous Inputs to DNN Classifiers \\\n[4] Roth et al., The odds are odd: A statistical test for detecting adversarial examples", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper targets to detect adversarial examples by using a data depth function. Specifically, the author employs halfspace-mass (HM) depth as a metric to detect the adversarial examples that was introduced in anomaly detection. The author provides extensive experiments on both the attack-aware scenario and the blind-attack scenario of the detection. The proposed method, HAMPER, shows better performance than the SOTA model of adversarial detection.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Request to conduct the test of [1].\n> The **last request is a critical** to securing my recommendation whether current detection's claim is correct or not. [1] paper demonstrates that previous adversarial detection may fall into wrong conclusion due to the seemingly easier tasks. [1] propose a sanity check for adversarial detection. Therefore, author should provide the results from the [1].\n\n* Adding relavant and recent baselines\n> The rest of the comments are for strengthen the work.\n\n\n[1] Tramer et al., Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them \\\n\n", "broader_impact_concerns": "Author clearly describe the broader impact concerns in the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NXsuTDKENg", "reviewer": "Reviewer_ikB4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengthes:\n1. the proposed method have better performance against existing competitors, in both the attack aware and attack blind settings.\n2. the authors also attempted to design adaptive attacks against the proposed detector.\n\nWeaknesses:\n1. the detector is a supervised learning algorithm that requires existing attacks to generate adversarial examples for learning proper weight vectors. Therefore, the detector might be vulnerable to future stronger attacks and the (some degree of) vulnerability to adaptive attacks is one possible indicator. \n2. the adaptive attacks seem to indicate that, adversaries still can succeed with non-negligible probability (e.g. 30%). \n3. the posed norm constraints when generating adversarial examples are not well-justified and other attacks (e.g., targeted CW attack, CW attack with higher confidence).   \n4. missing evaluation on the imagine dataset. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a detector of adversarial examples using halfspace-mass depth, which is a statistical tool that can be used for anomaly detection. The main contribution is in applying this overlooked tool into detection of adversarial examples and also the attempt to design adaptive attacks against the proposed defense.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The major concern on this work is still on its possible vulnerability to adaptive attacks, although the authors did try evaluating on adaptive attacks. The main issue is, we do not have deep enough understanding on whether depth scores are truly suitable for detecting adversarial examples. The previous LID defense was also believed to be a viable metric before being completely broken by the adaptive attacks. Therefore, I have the following requested changes:\n\n1. discussion on why the half-space mass depth can be a reliable metric for detecting adversarial examples (e.g., inherent power in discriminating adversarial examples). \n\n2. To gain more empirical evidence on the robustness of the detector, I suggest the authors to also evaluate the defense with the following attacks (without clipping the adversarial examples based on certain norm thresholds): 1) targeted CW attacks with different target classes; 2) CW attack with high confidence (controlled by the \\kappa parameter). \n\n3. Evaluation on the imagine dataset should be provided. Unlike building robust models, designing highly effective detector (against existing attacks) on imagenet should be feasible. \n\n ", "broader_impact_concerns": "Broader impact is sufficiently addressed. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: TMLR's criteria are "claims supported by evidence" and "interest to audience". I believe the results and analysis in this work is indeed interesting but the claims need to be reduced. I would be willing to consider a revision that addresses the concerns laid out above (namely around computational efficiency and adaptive attacks). The revision should (1) reduce claims about computation efficiency to be more in line with reviewer LPTq's comments, (2) provide more explanation or evaluations about the relation of depth scores to detecting adaptive adversarial examples, as suggested by ikB4, and (3) explain why the adaptive attacks are black-box (which weakens the adversary) rather than white-box.
Decision: Accept (poster)