Adversarial Attack Detection Through Network Transport Dynamics
Authors: Skander Karkar
patrick gallinari
Alain Rakotomamonjy
Venue: Submitted to ICLR 2023 (2023)
Keywords: Adversarial Attacks
Deep Learning
Optimal Transport
Residual Networks
Regularization

Abstract: Adversarial attacks are perturbations to the input that don't change its class for a human observer, but fool a neural network into changing its prediction. In this paper, we propose a detector of such attacks that is based on the view of residual networks as discrete dynamical systems. The detector tells clean inputs from abnormal ones by comparing the discrete vector fields they follow throughout the network's layers before the final classification layer. We compare this detector favorably to other detectors on seen and unseen attacks. We also show that regularizing this vector field during training makes the network more regular on the data distribution's support, thus making the network's activations on clean samples more distinguishable from those of abnormal samples. This regularization of the network's dynamics improves the performance of any detection method that uses the internal embeddings as inputs, while also improving the network's test accuracy.

Under review as a conference paper at ICLR 2023
ADVERSARIAL SAMPLE DETECTION THROUGH NEU- RAL NETWORK FLOW DYNAMICS
Anonymous authors Paper under double-blind review
ABSTRACT
We propose a detector of adversarial samples that is based on the view of residual networks as discrete dynamical systems. The detector tells clean inputs from abnormal ones by comparing the discrete vector fields they follow throughout the network's layers. We also show that regularizing this vector field during training makes the network more regular on the data distribution's 's support, thus making the network's 's activations on clean samples more distinguishable from those on abnormal samples. Experimentally, we compare our detector favorably to other detectors using seen and unseen attacks, and show that the regularization of the network's 's dynamics improves the performance of adversarial detectors that use the internal embeddings as inputs, while also improving the network's 's test accuracy.
INTRODUCTION
Neural networks have improved performances on many learning tasks, including image classification. They are however vulnerable to adversarial attacks which modify image in way that is imper- ceptible to the human eye but that fools the network into wrongly classifying the modified image (Szegedy al. (-13)). These adversarial images transfer between networks (Moosavi-Dezfoolii cet al. (2017)) can be carried out physically (e.g. causing autonomous cars to misclassify road signs (Eykholt et al. (208))), and can be generated without access the network (Liu et al. (2017)) Developing networks that are robust to adversarial samples or accompanied by detectors that can detect them is indispensable to deploying them safely in the real word (Amodei al. (2016)).
In this paper, we focus on detection of adversarial samples. Networks trained with a softmax classifier produce overconfident predictions even for makes it difficult to detect such inputs via the softmax outputs. A detector is a system capable of predicting whether an input test time has been adversarially modified or not. Detectors are trained on a dataset made up of clean and adversarial inputs, after the network has been trained. While simply training the detector on the inputs has been tried, using their intermediate embeddings works better (Carlimi & Wagner (2017b)). Detection methods vary by which activations to use and how to process them to extract the features that are fed to the classifier that tells clean samples from adversarial ones.
inputs (Nguyen et al.. (2015). This
We make two contributions. First, propose adversarial detector that is based the view of neural networks as dynamical systems that move inputs in space, time represented by depth, to separate them before applying linear classifier (Weinan (2017)). Our detector follows the trajectory of samples in space, through time, to differentiate clean and adversarial images. The statistics that extract are the positions of the internal embeddings in space approximated by their norms and cosines to a fixed vector. Given their resemblande blnce to the Euler scheme for differential equations, residual networks (He et al. (2016a b): Weinan (2017) are particularly amenable this analysis. Skip connections and residuals are basic building blocks in many architectures (c.g. EfficientNet (Tan & Le (2019) and MobileNetV2 (Sandler al. (20T8)), , and ResNets and their variants (e.g. WideResNet ZZaoorykko & Komodakis (2016) and ResNeXt (Xie et (201))) remain competitive (Wightman ct al. (2021). Also, Wu et al. (2020) show an increased vulnerability of residual-type architectures to transferable attacks, precisely because of the skip connections. This motivates the need for detector that is well adapted to residual-type architectures. But the analysis and implementation can extend immediately to any network where most layers have the same input and output dimensions. We test our detector on adversarial samples generated by eight attacks, on three datasets and networks, comparing it to the reference Mahalanobis detector (Lee et al. (2018) that largely outperform.

---

Under review as a conference paper at ICLR 2023
Our second contribution is to the transport regularization during training proposed in Karkar eet al. (2020) to make the activations of adversarial samples more distinguishable from those clean samples, thus making adversarial detectors perform better, while also improving generalization. We prove that the regularization achieves this by making the network more regular on the support of the data distribution. This does not necessarily make it more robust, but it will make the activations of the clean samples closer to each other and further from those of abnormal out-of-distributtio samples, thus making adversarial detection easier. This is illustrated on toy 2-dimension example in Figure T below. We present the related work in Section 2 the background for the regularization in Section 3 the detector Section 4.1 the theoretical analysis in Section 4.2 and the experiments in Section
Test and OOD data
Block 5
Block
IP
Wica
wabc
Test and OOD data
Block
Block 9
Figure I: Transformed circles test set from seikit-learn (blue and green) and out-of-distribution points (orange) after blocks 5 and of a small ResNet with 9 blocks. In the second row, we add our proposed regularization during training, which makes the movements of the clean points (blue and green) more similar to each other and more different from the movements of the orange than when using the vanilla network in the first row. In particular, without the regularization, the orange points very close the clean blue points after block which is undesirable.
points
2 RELATED WORK
Given classifier f in a classification task and e>>0, an adversarial sample constructed from a clean sample y
d, such that || and f(y) # f(x). The maximal perturbation size has to be so small as to be almost imperceptible to human. Adversarial attacks are algorithms that find such adversarial samples, and they have been particularly successful against neural networks (Szegedy et al. (2013): Carlini & Wagner (2017a)). We present the adversarial attacks we use in our experiments in Appendix [D.]] The main defense mechanisms are robustness, i.e. training network that is not casily fooled by adversarial samples, and having detector of these samples.
An early idea for detection was to use a second network (Metzen et al. (017)). However, this network can also be adversarially attacked. More recent popular statistical approaches include LID (Ma et al. (2018)), which trains the detector on the local intrinsic dimensionality of activations approximated
batch, and the Mahalanobis detector (Lee et al. (2018)). which trains the detector on the Mahalanobis distances between the activations and a Gaussian fitted to them during training, assuming they normally distributed, Our detector is not statistical approach and does need
2

---

Under review as a conference paper at ICLR 2023
batch-level statistics, nor statistics from the training data. Detectors trained in the Fourier domain of activations have also been proposed in Harder et al. (2021). See Aldahdooh et al. (2022) for a review.
Our second contribution is to regularize the network in way that makes it Hölder-continuous, but only on the data distribution's 's support. Estimations of the Lipschitz constant of a network have been used estimates of its robustness adversarial samples in Weng et al. (2018), Szegedy et al.) (2013), Virmaux & Scaman (2018) and Hein & Andriushchenko (017)) and making the network more Lipschitz (e.g. by penalizing an upper bound its Lipschitz constant) has been used to make more robust (i.e. less likely to be fooled) in Hein & Andriushchenko (2017) and Cisse et al. (2017). These regularizations often work directly on the weights of the network, therefore making it more regular on all the imput space. The difference with our method is that we only endue the network with regularity on the support of the clean data. This won't make more robust to adversarial samples, but it makes its behavior on them more distinguishable, since they tend to lie outside the data manifold.
That adversarial samples lie outside the data manifold, particularly its co-dimensions, is a common observation and explanation for why adversarial samples are easy to find in high dimensions (Gilmer al. (2018); Tanay Griffin (2016) et al. (2018); Ma et al. (2018); Samangouei et al. (2018);
Khoury & Mednilld-Menell (2018); Alemany & Pissinou (2022) reinman et al. (2017)). To the best of our knowledge, Pang et al. (2018) ) is the only other method that also attempts to improve detection by encouraging the network during training to learn latent representations that are more different between clean and adversarial samples. They do this by replacing the cross-entropy loss by a reverse cross-entropy that encourages uniform softmax outputs among the non-predicted classes. We find that our regularization leads to better classification accuracy and ad verrariial detection than this method.
The view of ResNets as transport systems was used in Wang et al. (0199) to relate injecting noise into ResNets and ensembling them to approximating the Feynman- Kac convection-diffusic formula, which makes the level sets more regular and the network more robust adversarial samples.
BACKGROUND
Our detector is based on the dynamical viewpoint of neural networks that followed from the analogy between residual networks and the Euler scheme made in Weinan (2017). We present this in Section 3.2 The trajectory regularization generalization and we also present it in Section 3.2 The regularity results that this regularization provides for the network require the use of optimal transport theory, which we present in Section 3.1
was proposed in Karkar al. (2020) to improve
3.1 OPTIMAL TRANSPORT
Let a and / be absolutely continuous densities on a compact set Ω C. Rd. The Monge problem is to look for map T Rd Ed moving a 3, i.. Tia B, with minimal Euclidean transport cost:
min Tst Tra=
[T(xx) - x11 (x(
and this problem has a unique solution T An equivalent formulation of the Monge problem in this setting is the dynamical formulation. Here, instead of directly pushing points from a to through T. we continuously displace mass from time 0 to I according to velocity field
Rd + Rd We denote of the position at time i of the particle that was at x a at time O. This position evolves according to 0. = ve (o()). Rewriting the constraint, Problem ( is equivalent to the dynamical formulation:
min f/' 1Ivall2 de s.t. 0 vd())) for x e support(a) and t [0, 1/, dú id, (di)ya = -B
(2)
3.2 LEAST ACTION PRINCIPLE RESIDUAL NETWORKS
A residual network made up of M residual blocks applies Tm++1 Im + hrrm (I for 0 < m < M, with To being the input and 4=1 in practice. The final point &M is then classified by a linear layer F. The dynamic view considers residual network as Euler discretization of a differential equation:
Fm+11 Fm hrm (Fm)
Oest vx(
(3)
3

---

Under review as a conference paper at ICLR 2023
where 7m approximates the vector field at time t m/M. The dynamic view allows to consider that ResNets are transporting their inputs in space by following a vector field to separate them, the depth representing time, before classification by linear layer. Karkar et al. (2020) look for network F T that solves the task while having minimal transport cost:
1 ( - da(x) 1
inf T.F
(4)
subject to ((F,Taa)=0 =0
where T is made up of the M residual blocks, the data distribution, F is the classification head and C(F, T:a:) is the (cross-entropy) loss obtained from classifying the transformed data distribution Tha through F. Given Section 3.1 the corresponding dynamical version of (4) is
L. 10 (100))0))))))) df
inf v.F
(5)
subject to O - (x( for x E support(a) and t E [0, 11, % - id, (().a))))))
Karkar et al. (2020) show that (4) and (5) are equivalent and have a solution such that T is an optimal discretized using a sample D from and an Euler scheme, which gives a residual architecture with residual blocks I'm (parametrized along with the classifier by 0) that approximate This gives the following problem
transport map. In practice,
M C(0) = Σ Σ
min
(lm(()"))l
(6)
DED
subject to
Pm+1 = 4m + hrr (P vp = x for all x E D, C()) =0
In practice, we solve Problem (6) using a method of multipliers (see Section 4.2.1). Our contribution is to show, theoretically and experimentally, that this makes adversarial examples casier to detect.
4 METHOD
We take the view that residual network moves its inputs through a discrete vector field to separate them, points the same class having similar trajectories. Heuristically, for successful adversarial sample that lies close to clean samples, the vector field it follows has to be different at some step from that of the clean samples, so that joins the trajectory the points in another class. In Section 4.1 we present how to detect adversarial samples by looking at these trajectories. In Section 4.2 we discuss the link between the network's 's regularity and its robustness to adversarial examples and apply the transport regularization by solving Problem (6) to improve detectability of adversarial samples.
4.1 DETECTION
Given a residual network that applies 2m+11 = + hr (2m) to an input for 0 < m<M, M, we consider the embeddings xm for 0<m < M, or the residues I'm I'm (2m) for 0 S. m<M. To describe their positions in space, we take their norms and their cosine similarities with a fixed vector as features to train our adversarial detector on. Using only the norms already gave good detection accuracy. Cosines to other orthogonal vectors can be added to better locate the points at the price of increasing the number of features. We found that using only one vector already gives state-of-the-ar detection, so we only use the norms and cosines to fixed vector of ones. We train the detector (a random forest in practice, see Section 5.2) on these features. The embeddings Fm and the residues T' (Im) can equivalently describe the trajectory of 2o in space through the blocks. In practice,
the residues I'm (, with their norms squared and averaged. So the feature vector given to the random forest for each 20 that goes through network that applies Im+1
Fm rm (X is
(
1 ( [1mm)(m))l)) )l2 cos (Fm)))), 1m) di
05m<M
and the label is 0 if 2n is clean and if is adversarial. Here cos is the cosine similarity between two vectors and 1,, is vector of ones of size dm where dm is the size of r' (xm). For any non-residual
4

---

Under review as a conference paper at ICLR 2023
architecture 2m+1 9m (Tm), the vector 2m+1-2m can be used instead of (Im) layers that have the same input and output dimension, allowing to apply the method to any network with many such layers. And we do test the detector on ResNeXt, which does not fully satisfy the dynamic view, as the activation is applied after the skip-connection, i.e. Im+1 = (eL.U(Tm + h (m(())))
The number of features is twice that of residual blocks (a norm and cosine per block). This is of the same order as for other popular detectors such as Mahalanobis (Lee et al. (2018)) and LID (Ma) al. (2018) that extract one feature per residual stage (a residual stage a group of blocks that keep the same dimension). Even for common large architectures, twice the number of residual blocks still a small number of features for training a binary classifier (ResNet152 has 50 blocks). More importantly, the features we extract (norms and cosines) are quick to calculate, whereas those of other methods require involved statistical computations on the activations. We include in Appendix aD.8 favorable time comparison of our detector to the Mahalanobis detector. Another advantage is that our detector does not have a hyper--paree syper-parameter tune unlike the Mahalanobis and LID detectors.
4.2 REGULARIZATION
Regularity of neural networks (typically Lipschitz continuity) has been used as measure of their robustness to adversarial samples (Weng ct al. (0188;; Szegedy et al. 2013); Virmaux & Scaman (2018); Hein & Andriushchenko 2017 7): Cisse et al. (2017). Indeed, the smaller the Lipschitz f0yl S - yll. the less ff changes its output f() for a perturbation (adversarial not) y of x. Regularizing network to make it more Lipschitz and more robust has therefore been tried in Hein & Andriushchenko (2017) and Cisse et al. (0117) For this work, the regularization has apply to adversarial points, i.e. outside the support of the clean data distribution. Indeed, the Lipschitz continuity obtained though most of these methods and analyses apply on the entire input space Rd as they penalize the network's 'ss weights directly. Likewise, a small step size l as in Zhang ct al. (2019b) will have the same effect on all inputs, clean or not.
constant L of a function f satistying [
We propose here an alternative approach where we regularize the network only on the support of the input distribution, making it H-Höleer on this support (a function f is 17- 1-Hölder on X if V a, b E X, we have [l.((a) - f(b]l I ClIa - bll" for some constants C>0 and 0<751, and we denote this CO.0 Since this result does not apply outside the input distribution's 's support, particularly in the adversarial spaces, then this regularity that only applies to clean samples can serve to make adversarial samples more distinguishable from clean ones, and therefore easier to detect. We show experimentally that the behavior of the network will be more distinguishable between clean and adversarial samples in practice in Section S.I. We discuss the implementation of the regularization in Section 4.2.1 and prove the regularity it endues the network with in Section 4.2.2
4.2.1 IMPLEMENTATION
We regularize the trajectory of the samples by solving Problem (6) This means finding, among
the networks that solve the task (condition L()) = 0 in (6), the network that moves the points the least, that is the one with minimal kinetic energy The residual functions I'm we find are then our approximation of the vector field that solves the continuous version (5) of Problem (6).
We solve Problem (6) via a method of multipliers: since C 2 O, Problem (6) is equivalent to the min-max problem ming max C(0) + X L()), which we solve, given growth factor 7 >> 0, and starting from initial weight given to the loss Ag and initial parameters dg. through
0 = arg min C(0) + > L())
8
(8)
N411 = A + ((0,1))
We use SGD for s>0 steps (i.e. batches) for the minimization in the first line of ( starting from the previous 0,, When using a ResNeXt, where a residual block applies Tm = ReLU [mm + Tm (mm))
we regularize the norms of the true residues Im+1 -mm instead of rm (1m).
4.2.2 THEORETICAL ANALYSIS
We first prove a regularity result on each residual r. that applics on the clean data points moving according to e solution to (5) We take Ω2 C Rd convex and compact and the data distribution
5

---

Under review as a conference paper at ICLR 2023
P(2) absolutely continuous and such that 612 is a-negligible. We suppose that there exists an open bounded convex set X C S2 such that a is bounded away from zero and infinity on X and
is zero on its complement x From Karkar et al. (2020). we know that problems (4) and 5 are equivalent and have solutions (T, F) and (v, F) such that T is an optimal transport map between and B := T.a. We suppose that /3 is absolutely continuous and that there exists an open bounded convex set Y C Ω such that /3 is bounded away from zero and infinity on Y and is zero on ytC In the rest of this section, v solves (5) and we suppose that we find a solution to the discretized problem (6)
that is e/ e/2-approximation of v, i.e. Ilrm Utm lls0 S e/2 for all 0 I m < M, with tmm. m/M. Theorem 4.1. For a, b E suppor(((t..) ) where 0t == (0i)za with Φ solving (5) along with v, we have
||rm (a) Im (b)l Se + KIIa- - b/l iflaa bl S
[|mm (a) I'm (b) Se + KIIa - b||sa la bl >
for constants K > 0 and 0 C1 S C2 S
Proof. The detailed proof is in Appendix C.1 First, = =0 (d-id) where T. == (1-1)id+(7
and T solves 4p. Being an optimal transport map, T is H-Höle on X. So for all a, b support(a:)
and E [0, 1(. where az = ( = (Ti);a with solving (5) with v, we have
[lve(a) (b)l S |Tz (6)11 (6)1 CIIT (a) - T (6)1I"
(9)
Then, we show that T is an optimal transport map and so is TH-HHölder with 011,51. Using also the hypothesis on and the triangle inequality, we get, for all a, b E support(ar..))
rl (a) mm mb)(b)l || S E + C... ||a 7mm + CC l| 6/
(10)
Finally, take K == maxm Ct.. + C. km S1 == minn T)m.. and S2 := max, 1m
We now look at the activations for inputs that are not necessarily in X. For inputs ag and by. the intermediate embeddings are 0m++1 a. + (rm) and bm+1 bm, trr (bm): and the residues used to compute features for adversarial detection are I'm (am) and I' (bm)). We therefore want to bound I (4m) Ym. (bm))l as function of the input variation ||ag boll. This is usually done by multiplying the Lipschitz constants of cach residual block up to depth m. which leads to an overestimation (Huster al. (2019)) or through more complex estimation algorithms (Virmaux & Scaman 2018); Latorre et al. 2020); Bhowmick et al. 202T)). Bound (9) allows to avoid muluplying the Hölder constants of the block through 7, We have the following result (proof in Appendix
C.2 which shows the increased regularity guarantee when the inputs dg and bn are in X:
Theorem 4.2. For all ao, bo E X, we have, for constants C, L > O, that
[| Tim (bm))l S E + I1a0 boll + Cllaoo - boll" + T.Iaamm
Sto. || + bib.
do (11)
The term ||am - 62 (likewise [lbm - ho |I) is the distance between the point am after m residual blocks and the point 6th we get by following the vector field up to time tm- Since (9) applies only on the data support, an extra term has to be introduced if ag is not in X (see the proof). Under more regularity hypotheses on v, it is possible to bound these terms. Indeed, if is C and Lipschitz in (this is not stronger than the regularity we get on through our regularization, because it does not give something similar to bound (2)), we have, for constants R, S > O, and for all on, bo E R2, that
(am - (bm))l S E+ I.S L.S Rh 1100 -oco boll+Cllao boll" T.S TSS(iis((ao x)+dist(bo- X))
5 EXPERIMENTS
We evaluate our method on adversarial samples found by 8 attacks. The threat model is as follows. We use white-box attacks that can access the network and its weights and architecture but not its training data: FGM (Goodfellow et al. (2015)) BIM (Kurakin et al. (2017)) DF (Moosavi-Deztooli
et al. (016)), ) CW [Carlini& & Wagner 201 7)), AutoAttack (AA) (Croce & Hein (2020a)) and the
Auto- PG (APGD) variant of PGD (addy et al. (2018) ), and 2 black-box attacks that only query the network: HSJ (Chen et al. (2020) and BA (Brendel et al. (2018). We present these attacks in
Appendix D.T
We assume the attacker has no knowledge of the detector and use the untargeted (i.e.

---

Under review as a conference paper at ICLR 2023
not trying to direct the mistake towards a particular class) versions of the attacks. We use maximal perturbation of c=0.03 for FGM, APGD, BIM and AA. We use the Lg norm for CW and HSJ and Loo for the other attacks. We compare our detector (which we call the Transport detector or TR) to the Mahalanobis detector (MH) of Lee et al. (2018). and our regularization to reverse cross entropy training of Pang al. (2018), which is also meant improve detection of adversarial samples. We use ART (Nicolae et al. 20188) Croce & Hein (0200a) to generate the adversarial samples.
We 3 networks and datasets for experiments: ResNeXt50 CIFARIO0, ResNetl 10 on CIFARIO and WideResNet on Tiny Imageee,, Each network is trained normally with cross entropy, with the transport regularization added to cross entropy (called LAP-networr for Least Action Principle), and with reverse cross entropy instead of cross entropy (called an RCE-network). For LAP training, we use ( with 7=1, ==1 and >==1 for all networks. These nyper-parameters are chosen to improve validation accuracy during training not adversarial detection. Training details are in Appendix D.2
5.1 PRELIMINARY EXPERIMENTS
Results show that LAP training improves test accuracy. Vanilla ResNeXt50 has an accuracy of 74.38% CIFAR 100, while the LAP-ResNeXt50 has accuracy of 77. 77.2%. Vanilla ResNetl has an accuracy of 92.52% on CIFARIO, while the LAP-ResNetl has an accuracy of 93.52% and the RCE-ResNetl 10 of 93. 1%. Vanilla WeseResNet has accuracy of 65.14% TinylmageNet, while the LAP- NedeResNet has an accuracy of 65.34%. LAP training also more stable by allowing train deep networks without batch-normalization Figure 4 in Appendix D.4 This increased stability is also shown by the often tighter confidence intervals with LAP training in the experiments.
We in Figure that LAP training makes the transport adversarial points. Using its empirical quantiles on clean points allows then to detect samples from some attacks with high recall and a fixed false positive rate, without seeing adversarial samples.
C more different between clean and
FGM 0.3
FGM 0.3
FGM 0.7
FGM 0.7
20
10
0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
1.0
Transport cost
Transport cost
Figure Histogram of transport cost for clean and FGM-attacked test samples with different values of on CIFAR 100. The vertical lines represent the 0.02 and 0.98 empirical quantiles of the transport cost of the clean samples. Left: ResNeXt50. Right: Ne.tt5s.
5.2 DETECTION OF SEEN ATTACKS
For detection training, the test set (validation set for TinyImageNed is split in 0.9/0.1 proportions into two datasets, B1 and B2, For each image
(respectively B2), an adversarial sample is generated and a balanced detection training set (respectively a detection test set) is created. Since adversarial samples are created for a specific network, this done for the vanilla version of the network and its LAP and RCE versions. We tried augmenting the detection training dataset with randomly perturbed version of each image, to considered clean during detection training, as in Lee et al. (2018), but we found that this does not improve detcction accuracy. This dataset creation protocol is standard and is depicted in Figure 3 Appendix DD3 We tried limiting the datasets to successfully attacked images only in Lee al. (2018), but did not find significant improvement in detection.
7

---

Under review as a conference paper at ICLR 2023
Samples in the detection training set are fed through the network and the features of each detection method are extracted. The classifiers (logistic regression, random forest and SVM from scikit-learn (Pedregosa et al. (201T)) are trained on these features for each detection method and tested on the features extracted from the points in the detection test set. A random forest always works best.
We tried two methods improve the accuracy of both detectors: class-conditioning and ensembling. In caadiccnniiion,ng. the features are grouped by the class predicted by the network, and a detector is trained for every class. At test time, the detector trained on the features of the predicted class used. A detector is also trained on all samples regardless of the predicted class and is used in case a certain class never targeted by the attack (which might focus on certain classes). We also tried ensembling the cans-cconitional detector with the general all-class detector: an input is considered an attack if at least one detector says so. In the results below, we report the results of the best performing version, which is often this ensemble of the class-conditional detector and the general detector.
We report the detection accuracy of each detection method on the detection test set for both the vanilla version of the network and its LAP version in Table below. In each cell, the first number corresponds to the vanilla network and the second to the regularized LAP-network. These results
are averaged over 5 runs and the standard deviations (which are tight) are in Tables 3 S and 6 Appendix D.5 along with results on RCE- networkss and including the FGM and B IM attacks. Since some attacks are slow, don't 't test them on all network-datate -attseettt pairs in this experiment.
Results in Table show two things. First, our detection method performs better than the Mahalanobis detector, with or without the regularization. Second, both detection methods work better the LAP versions of the networks most times. The Mahalanobis detector benefits more from the regularization, but on all attacks, the best detector is always the Transport detector. In Tables and in Appendix D.5
RCE often improves detection accuracy in this experiment, but clearly less than LAP training.
Table 1: Average detection accuracy of adversarial samples on Network/lAP-Neetoof
Attack
Network/Data
Det APGD
AAA
DF
CW
HSJ
BA
TR 94.1/97.5
88.9/94.
99.9/99.8 8
98.0/98.0
97.4/97.4 84.1/92.3
96.6/97.0
CIFARIO
MH
82.1/90.7
80.5/90.0
91.5/96.7
85.6/93.4
80.2/89.6
ResNeX150
TR 96.0/97.8
84.9/87.6
99.8/99.6
95.7/95.8
CIFARIO0
MH
93.9/94.6
83.9/86.6
97.3/97.1
94.7/94.7
WideResNet
TR 96.0/96.2
81.5/81.0
TintlmageNet
MH
84.2/85.1
78.4/78.4
Including the FGM and BIM attacks in Appendix D.5 on CIFAR10. 10, our detector greatly outperforms the MH detector by 9 to 16 percentage points on the vanilla ResNetl 10. LAP training improves the accuracy of our detector by average 1.5 points and that of the MH detector by a substantial 8.2 points on average. On CIFARI00, our detector outperforms the MH detector by to 5 points on the vanilla Re55eXt50. LAP training improves the accuracy of both detectors by an average 1 point. On TinyImageNer our detector greatly outperforms the MH detector by 3 to 12 points on the vanilla Nett LAP training does not change the accuracy of our detector and improves that of the MH detector by 0.6 points on average. Detection rates of successful adversarial samples (i.e. those that fool the network) are much higher than the accuracy on a balanced dataset in Table Even for the AA attack, they are very close to 100% (see Table 10 in Appendix D.7.
5.3 DETECTION OF UNSEEN ATTACKS
An important setting is when we don't 't know which attack might be used or we only have time to train detectors samples from one attack. We still want our detector to generalize well to samples from unseen attacks. To test this, we use the same vanilla networks as above. The detectors are now trained on the detection training set created by the simplest and quickest attack (FGM) and tested on the detection test sets created by the other attacks. Results are in Table below. We see that our detector has very good generalization unseen attacks, even those very different from FGM, comfortably
8

---

Under review as a conference paper at ICLR 2023
better than the Mahalanobis detector, by up to 19 percentage points in some cases. These results averaged over 5 runs and the standard deviations are in Tables 7 S and 9 in Appendix
Table 2: Average detection accuracy of samples from unseen attacks after training on FGM.
Attack
Network/Data
Detector
APGD
BIM
AA
DF
CW
HSJ
1e
TR
89.32
96.02
85.10
91.02
93.18
93.00
90.92
CIFARIO
MH
77.34
77.24
72.12
80.12
79.92
79.70
79.32
ResNeX150
90.82
96.12
73.28
87.60
87.54
89.90
81.70
CIFAR100
MH
86.90
90.92
72.92
83.08
84.02
84.10
78.70
WideResNet
TR
93.26
94.66
77.04
90.62
91.42
TinylmageNet
MH
76.96
77.02
60.36
73.18
75.52
However, this experiment shows that our approach has some limitations. We see in Tables 7 in Appendix D.6 that LAP training does not improve detection accuracy as much anymore, and reduces it in some cases. It improves it for the MH detector on all attacks on ResNetl CIFARIO and WideResNet- Mireeee--innlameeeeee by up to 10 points, but not on most attacks on CINNeXX5O-CIFARII often reduces on our detector. But detection of AA samples, the most difficult to detect Table 2 in particular improves by up to 10 points. LAP training still always does better than RCE training. We claim this limitation is because these training methods reduce the variance features extracted on the seen attack, thus harming generalization to unseen attacks. This explains why detection of APGD and BIM, variants of FGM, still improves. Fixing this could be an area of future research.
g and
It
5.4 DETECTION OF OUT-OF-DISTRIRUUTE
SAMPLES
Since the analysis applies to all out-off distrribiitttoonnn (OOD) samples, we explore detecting them in a similar setting to the Mahalanobis paper (Lee et al. 2018 1): we train a model on a first dataset (CIFAR 10), then train detectors tell this first dataset from second dataset (CIFAR 100 CW- attacked CIFAR 10), then test their ability to tell the first dataset from a third unseen dataset (SVHN). Our detector does very well and better than the MH detector in both experiments run, and detection accuracy of samples from the unseen distribution is higher than 90%. Details are in Appendix D.9
5.5 ATTACKING THE DETECTOR
We consider the case where the detector is also attacked (adaptive attacks). We try 2 attacks on the TR and MH detectors. Both are white-box with respect to the network. The first is black-bor -bx with respect to the detector and only knows if sample has been detected or not. The second has some knowledge about the detector. It knows what features it uses and can attack it directly to find adversarial features. We test these attacks by looking at the percentage of detected successful adversarial samples that they turn into undetected successful adversarial samples. For the first attack, this is 6.8% for detector and 12.9% for the MH detector on the LAP- ResNeee 10, and is lowered by LAP training compared to vanilla training. For the second attack it is 14%. Given that detection rates of successful adversarial samples are almost 1 100% (see Table 10 in Appendix [.7). this shows that an adaptive attack does not circumvent the detector, detection rates drop to 85% at worst. Details are in Appendix D.10
CONCLUSION
We proposed method for detecting adversarial samples, based on the dynamical viewpoint of neural networks. The method examines the discrete vector field moving the inputs to distinguish clean and abnormal samples. The detector requires minimal computation to extract the features it needs for detection and achieves state-of-the-ane detection accuracy on seen and unseen attacks. We also used a transport regularization that was shown improve test accuracy to improve the detection accuracy of adversarial detectors. The regularization makes the embeddings closer to each other on the data distribution's 's support, thus making them more distinguishable from those of abnormal samples.

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
Implementation details are in the first paragraph of Section 5 and in Appendix D.1 and D.2 The code available gi thub. com/ /wvaavvaaaavaaaaaaavvvvaaaaaaa adv, along with weights for the trained ResNetl 10 models on CIFARIO that were used.
REFERENCES
Ahmed Aldahdooh, Wassim Hamidouche, Sid Ahmed Fezza, and Olivier Déforges. Adversarial example detection for dnn models: review and experimental comparison. Artificial Intelligence Review, 2022.
Sheila Alemany and Niki Pissinou. The dilemma between data transformations and adversarial robustness for time series application systems. In Proceedings of the Workshop on Artificial Intelligence Safety 2022 (SafeAl 2022) co-located with the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAA12022), Virtual, February, 2022, volume 3087 of CEUR Workshop Proceedings. GEUR-WS.org 2022.
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety, 2016.
06565
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial attack via random search. In Computer Vision ECCV 2020. Springer, 2020.
J.D. Benamou and Y. Brenier. A computational fluid mechanics solution to the monge-kantorovich mass transfer problem. Numerische Mathematik, 2000.
Aritra Bhowmick, Meenakshi D' D'Souza, and G. Srinivasa Raghavan. Lipbab: Computing exact lipschitz constant of relu networks. CoRR, abs/2105.05495, 2021. URL https:
org/abs/ 2105. 05495
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- adversarial attacks: Reliable attacks against black-box machine learning models. International Conference on Learning Representations, 2018. URL httts:://pperrevieeee http://wwww.....mmmeee.assaao1..
N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57, Los Alamitos, CA, USA, may 2017a. IEEE Computer Society. doi: 10. 10.1100 109/SP.2011 URL BtLLhttps:// //doi. www.cceeteeeocccee.......
org//0..109/55 2017.49
Nicholas Carlini and David Wagner. Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods, pp. 3-14. Association for Computing Machinery, New York, NY, USA, 2017b. ISBN 9781450352024, URL htt.tttttt//.. /ttpt:::///o...o htt.t:p:://doi.org/1// 1tt.r///...11533115551
31 30444
Jianbo Chen, Michael I. Jordan, and Martin J. Wainwright. Hopskipiumpattack: A query-efficient
decision- based attack. 2020 IEEE Symposium on Security and Privacy, pp. 1277-1294, IEEE,
2020. UtRLbtts:::/dae htt./di..rgg//// wtt.:///00....9900000000000000 2020. 00045
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In Proceedings of the 34th International Conference on Machine Learning, pp. 854-863. PMLR, 2017.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In Proceedings of the 37th International Conference on Machine Learning. PMLR, 2020a.
Francesco Croce and Matthias Hein. Minimally distorted adversarial examples with fast adaptive boundary attack. In Proceedings of the 37th International Conference on Machine Learning. PMLR, 2020b.
10

---

Under review as a conference paper at ICLR 2023
K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song. Robust physical-world attacks on deep learning visual classification. In 2018 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 1625-1634, Los Alamitos, CA, USA, jun 2018. IEEE Computer Society. doi: 10.1 2009/CV..2008.0000
URL. https
7/doi www.coopteeee.....e..m..... wtw.litt......o..... 1109//00.1109/VFPP0 2018 20175
Reuben Feinman, Ryan R. Curtin, Saurabh Shintre, and Andrew B. Gardner. Detecting adversarial samples from artifacts, 2017.
A. Figalli. The Monge-Ampere Equation and Its Applications. Zurich lectures in advanced mathemat- ics. European Mathematical Society, 2017.
Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S. Schoenholz, Maithra Raghu, Martin Wattenherg, and Ian Goodfellow. Adversarial spheres: The relationship between high-dimensional geometry and adversarial examples, 2018. URL https::/aaxii..orrrrt
htt.a////111112171111 02774
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Paula Harder, Franz-Josef Pfreundt, Margret Keuper, and Janis Keuper. Spectraldefense: Detecting adversarial attacks on cnns in the fourier domain, 2021. URL httpp::/aaxiv..rrr///
1103.03000
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In ECCV, 2016b.
Matthias Hein and Maksym Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems, pp. Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Todd Huster, Cho-Yo Yu Jason Chiang, and Ritu Chadha. Limitations of the lipschitz constant as defense against adversarial examples. In ECML PKDD 2018 Workshops, pp. 16-29. Springer International Publishing, 2019.
Alex Kantchelian, J. D. Tygar, and Anthony D. Joseph. Evasion and hardening of tree ensemble classifiers. In Proceedings of the 33th International Conference on Machine Learning. PMLR, 2016.
Skander Karkar, Ibrahim Ayed, Emmanuel de Bézenac, and Patrick Gallinari. A principle of least action for the training of neural networks. In ECML-PKDD, 2020.
Marc Khoury and Dylan Hedeield-Meenll. On the geometry of adversarial examples, 2018. URL https://arxiv http:///axxxvooorraa/////// wttg/bs//11...1... 00525
Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto
Technical Report, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In ICLR (Workshop), 2017.
Fabian Latorre, Paul Rolland, and Volkan Cevher. Lipschitz constant estimation of neural networks via sparse polynomial optimization. In International Conference Learning Representations, 2020.
XSF DB
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of- distriibbtnttnnn samples and adversarial attacks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. besa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc,, 2018 http:t/tttpppe//pccccccecceeeeg neurips

---

Under review as a conference paper at ICLR 2023
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In International Conference on Learning Representations. OpenReview.net nett, 2017. URL https::/openzeeiiewt net wtt.://eeeototaaaaaaaa33333
Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In ICLR, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2018. URL https
//openreview.oo http:/iiwwweeeeeee.e.eeeeeeeeeee
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff, On detecting adversarial perturbations. In ICLR, 2017.
S. Mlosavi-Dezzoolle A. Fawzi, O. Fawzi, and Frossard. Universal adversarial perturbations. In 20/7 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 86-94, Los Alamitos, CA, USA, 2017. IEEE Computer Society. doi: 10. 101.100//PP..007 URL
https://doi www.seccmmtteessocctyyy.oooooo wtt.e//wwwcceerr..t.....o.......
Seyed-Mohsen Moosavi-Dezfoole Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method fool deep neural networks. 2016 IEEE Conference on Computer Vision and Pultern Recognition (CVPR), pp. 2574-2582, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.
2015. doi: 10.
Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian M. Molloy, and Ben Edwards. Adversarial robustness toolbox v1.0.0, 2018. URL https: www.iii.org/aaa.. /
2807.01063
Tianyu Pang, Chao Du Yinpeng Dong, and Jun Zhu. Towards robust detection of adversarial examples. In S. Bengio. H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn; Machine learning in Python. Journal of Machine Learning Research, 12:2825 22:2825-2830. 2011.
Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri. Numerical Mathematics. Springer Berlin, Heidelberg, 2007.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Representations, 2018. URL
net
Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp. 4510 4520. Computer Vision Foundation IEEE Computer Society, 2018
F. Santambrogio. Optimal Transport for Applied Mathematicians, Birkhäuser, 2015.
12

---

Under review as a conference paper at ICLR 2023
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In Inter- national Conference on Learning Representations, 2018.
forum? der.mmmxxx...
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2013. URL httpsp////xxiv.oog abs 1312.
Mingxing Tan and Quoe Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th Interaational Conference Machine Learning, volume 97 of Proceedings of Machine Learning mlr
Research, pp. 6105-61 14. PMLR, 09-15 Jun 2019. URL
press/v97/tan htm
Thomas Tanay and Lewis Griffin. boundary tilting persepective on the phenomenon of adversarial examples, 2016.
Cédric Villani. Optimal Transport: Old and New. Springer-Verlag. 2008.
Aladin Virmaux and Kevin Scaman. Lipschitz regularity of deep neural networks: analysis and efficient estimation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 31. Curran As- sociates, Inc., 2018 URL /pupatttpp./pptooeedddnng Paper. pdf
Bao Wang. Zuoqiang Shi, and Stanley Osher. Resnets ensemble via the feynman-kac formalism im- prove natural and robust accuracies. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, Cur- ran Associates, Inc., 2019. URL https:ttp///mt http::::s///ccceeeeoooooo www.eeiiiinnnnnneeeeeeeee.....ee meurips. cc/paper/2019 fi Leee/eeed45522beeeeeeee hat.fffaac22644466 wetcce2000eeeeee Papeer....... p
Weinan. A proposal on machine learning via dynamical systems. Commun. Math. Stat, 2017.
Weui-Wei Weng, Huan Zhang, Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. In International Conference on Learning Representations, 2018.
Ross Wightman, Hugo Touvron, and Herve Jégou. Resnet strikes back: An improved training procedure timm. arXiv,, 2021.
Dongxian WU, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In International Conference on Learning Representations, 2020.
Saining Xie et al. Aggregated residual transformations for deep neural networks. In CVPR, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Chong Zhang, Huan Zhang, and Cho-Jui Hsieh. An efficient adversarial attack for tree en- sembles. H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 16165-16176. Curran As- sociates, Inc., 2020, URL htaLjitttpp:/pppoceeddnne htwp:/oceeednnnn..u.ii.ee neurips wttp:ip...cccpppee cttppppt//22.//////// file/ 9c htt.aaoocccoo000000o0oooooooeeaa
Jingfeng Zhang et al. Towards robust resnet: A small step but giant leap. Proceedings of the Twenty-Eight International Joint Conference on Artificial Intelligence (ICCAI-19), 2019a.
Jingfeng Zhang et al. Towards robust resnet: A small step but giant leap. In Proceedings of the Swenty-Fight International Joint Conference on Artificial Intelligence (IJCAI-19), 2019b.
13

---

Under review as a conference paper at ICLR 2023
A BACKGROUND ON OPTIMAL TRANSPORT
The Wasserstein space (2( with Q a convex and compact subset of H is the space P(12) of probability measures over S2, equipped with the distance W2 given by the solution to the optimal transport problem
(a, B) min / || ull dy(2,,)) v)
(12)
7El((a. Axa
where II(a, 3) is the set of probability distribution over 9 x Q with first marginal and second marginal B, i.. II(a.
{r E P(Ω x 2)) Fig? = a; T2;2 = {}} where % (2, y) x and (2(2, v) = y. The optimal transport problem can be seen as looking for transportation plan
minimizing the cost of displacing some distribution of mass from one configuration to another. This problem indeed has solution in our setting (see for example (Santambrogio (2015); Villani (2008).
If a is absolutely continuous and i is a-negligible then the problem in (12) (called the Kantorovich problem) has unique solution and is equivalent to the following problem, called the Monge problem,
/ [T((x)
min W (a. B) Tst. Taa=$
x=| da((x)
(13)
and this problem has unique solution T linked to the solution y" of 12) through >" (id, T*) yya. Another equivalent formulation of the optimal transport problem in this setting is the dynamical formulation (Benamou & Brenier (2000)). Here, instead of directly pushing samples of a to 3 using T. we can equivalently displace mass, according to a continuous flow with velocity 1z R - R This implies that the density a at time & satisfies the continuity equation 0,ag + V +V-(a,;) = 0, assuming that initial and final conditions are given by ag and 01 3 respectively. In this case, the optimal displacement is the one that minimizes the total action caused by v:
W 2 (a: B) =min / |1 (l, dt
(14)
s.t. 0:00 + V. (are) = 0, 00 = 01 =
Instead of describing the density's evolution through the continuity equation, can describe the paths of taken by particles at position æ from a when displaced along the flow v. Here of is the position at time t of the particle that to O, = (o(f). ). See chapters 4 and 5 of Santambrogio (015)) for details. Rewriting the conditions as necessary, Problem (14) becomes
time 0. The continuity equation is then equivalent
W = min /. ||0 (1:2(102)))) df
(15)
s.t. 0,05 = vof))) % = id, (0i);a = 3
and the optimal transport map T that solves 13 is in fact (x) = of for & that solves the continuity equation together with the optimal 1* from 15 The optimal vector field is related to the optimal map through 2.- = (T - id) o (T; where T. = (1- - +)id + tT" and is invertible. This simply means that the points move in straight lines and with constant speed from x to (x)).. The (2) space is a metric geodesic space, and the geodesic between a and /3 is the curve o, found while solving (4). It is also given by a,- (+ = (T7) :a, where Ty (z, y) (1 t)z + ty. We refer to Section 5.4 of Santambrogio 20155) for these results on optimal transport.
Optimal transport maps have some regularity properties under some boundedness assumptions. We mention the following result from Figalli (017)):
Theorem A.1. Suppose there are X,Y, bounded open sets, such that the densities of a and B are null in their respective complements and bounded away from zero and infinity over them respectively. Then, Y is convex, there exists 1) > such that the optimal transport map T between a and B is Co. over X.
If isn't convex, there exists two relatively closed sets A, B in X,Y respectively such that T E (X \ A.YY B), where A and B are of null Lebesgue measure. Moreover. if the densities are in C,,,,, then C can be replaced by Ck+119 in the conclusions above. In particular, if the densities are smooth, then the transport map is diffeomorphism.
A final result that we mention is the following, which says that the inverse of the optimal transport map between and 3 is the optimal transport map from o,
14

---

Under review as a conference paper at ICLR 2023
Theorem A.2. If a and are absolutely continuous measures supported respectively compact subsets and Y of R with negligible boundaries. then there exists a unique couple (T, S) of functions such that the five following points hold
: -x>+b and S
X
a = B and S#3 = a
' is optimal for the Monge problem from a to 3
S is optimal for the Monge problem from 3 to a
T T
8- S id and S o ST a id
B BACKGROUND ON NUMERICAL METHODS FOR ODES
We refer to Quarteroni et al. 2007 for this quick background on numerical methods for ODEs. Consider the Cauchy problem
ft,,x) with initial condition s=(l) zo and subdivision
to < 1 <.. < IN = to + T of [to, to + TJ. Denote the time-steps ha == tn+1 fn for 0 S. n < N and define Amax = max hn In step method, an approximation of xtt,) xn given by
Un++1
Ja
Φltn, In, kn, h,)
h
For (t,, x. h) f(t, we get Euler's 's method: In-1 = In + Definition B.1. (Consistency and order) For one-step method, the consistency errors ens for 0 O N, are
en x(tn+1) (t hn). x(tn+1) x(tn) o(tn, (tn)), hn))
hn where x is solution. The local (truncation) ervors are hnon. The method is consistent if max (enl goes to zero as Nmax goes to zero. For p E N", the method has order if max lenl S Chi for constant C' that depends on f, to and T.
hnn
Theorem B.2. (Consistency criterion) If and are continuous then the one-step method is consistent if and only if (t, x, 0) = ftt, x)) for all (t, z).
Theorem B.3. (Order criterion) If is CP and & is CP in h then the one-step method is of order p if and only if o(t
f (t, for all (t, x) and O S < p where f( = and
f fk
fa,f(k-!
Corollary B.4. (Consistency and order of Euler's method) f is continuous then Euler's method is consistent. If f is C' then Euler's method has order 1. Definition B.5. (Zero-stability) A one-step method is zero-stable (or stable) S > 0 such that for all (In)oSHEN, (in)usugN and (En)ogn<N satisfying
En-
p(tn, Fns hn) and Int1 F.. a(tnt Ing hn) + Cn
(16)
h for 0 < < N, we have max _| xnll S S(1@o xoll + T max, lenl), where en. En/ The constant the stability constant of the method.
hn
en/hn.
Theorem B.6. (Zero-stability criterion) If é is uniformly L-Lipschitz in its second variable, then the one-step method stable with constant eLr"
Corollary B.7. (Zero-stability of Buler's method) If f Lipschitz in its second variable, then Euler's method is stable.
Definition B.8. (Convergence) A numerical method converges if its global error max,, ||x(tt.) In goes to zero as Amma goes to zero.
Theorem B.9. (Convergence criterion) If a method is consistent and stable with stability constant S, then it converges and max,, (t,(t, x,,ll < ST max |e,) If the method is of order with constant C, then max, []x((t,,) xx STCh
Corollary B.10. (Convergence Euler's method) Euler's 's method converges if CO and Lipschitz in r. f is also e1 then converges with speed (hhmax).
15

---

Under review as a conference paper at ICLR 2023
C
PROOFS
C.I PROOF OF THEOREM
Proof. A solution v to
5 exists and is linked to an optimal transport map that is a solution to through 1/ = (T id) T. where T; == (1 f)id + /T' which is invertible (see Appendix A
(
By Theorem A.1 in Appendix A being an optimal transport map, T is 7 HöHöee on X. So for all a. E support at) and t E [0, 11, where a2 ( (Ti) with ø solving (5) with e, we have
[1 - (b)) = ITE (a) - T; (b)]l + C (a) - (b)]1"
(17)
Since (ax)i=0 is a geodesic between a and 3 = ai = Tja, then (a.))=u is a geodesic between and (modulo reparameterization to [0, 1). And since a, = (T.) the map T, is an optimal transport map between a and ay. Therefore its inverse T, is optimal transport map (see Theorem A.2 in Appendix A) and is Hölder with 51 (being a push-forward by Tú, the support of o satisfies the conditions of Theorem A.1 in Appendix A). Therefore, for all a, b E support(a:)
[|vx(a) - we (b)l C. l|a
611Th
CC 61
(18)
and for all a, b E support(as...
Il (a) mm (b)ll S + Ca 7mm + CC l|a bl] 7
(19)
by the hypothesis on r and the triangle inequality. Let == mäx C.., + +CC Ci == 1) mitmm m Tt... and {2 maX. Th... Then, we have the desired result immediately from (19
Remark C.I. the convexity hypothesis on the support Y of the target distribution 3 is too strong, we still get the same results almost everywhere. More precisely, if the set Y such that /3 is bounded away from zero and infinity on Y and is zero on Y-C is open and bounded but not convex, then the solution map T is HH-Hör almost everywhere on X (see Appendix (A).
Remark C.2. If the distributions a and ,3 in Theorem 4.1 are Ch (i.e all derivatives up to the k-th derivative are Hö-ödr,,), then the optimal transport map is Ck+11g This means that the more regular the data, the more regular the network we find.
C.2 PROOF OF THEOREM
Proof. Since T, (df) = x, we have for any ao, bo E X by the triangle inequality
(0m) - I' (bm))l S Ilrm (am) - mm (1 || + (p(m)) Utm. (dh I|+
+ llue (6000 ) U/ (0)
50 )|+
+ || (ot - Ut to ll + (bm) - Tm (0 )1
3 1100 boll C'Ia0 boll" + L(llam den || + 116m
where I, = max, I.m and I. is the Lipschitz constant of r. (which is Lipschitz being composition of matrix multiplications and activations such as ReIJU). This the first bound in the theorem.
In this bound, the term ||am - -600 || (and likewise || - oh [)) represents the distance between the point Om we get after residual blocks (i.e. after m Euler steps using the approximation of v) and the point < we get by following the solution vector field v up to time Lm By the properties of the Euler method (consistency and zero-stability, see Corollaries B.4 B.7 and B. in Appendix B). under more regularity conditions on v, it is possible to bound this term Indeed, if o C* and M-Lipschitz in (this is not stronger than the regularity because we still need to use (17), we have for constants R, S > 0,
get on through regularization,
||
dg |
am
S | rm ll + شن - am S Ss + SRh
where 7n comes from the Euler scheme with access to u (i.e. ā=+ + thye (ām) and ap)= R is the consistency constant of the Euler method and S is its zero-stability constant. Likewise, get the same bound for [bbm. L0 ||-
16

---

Under review as a conference paper at ICLR 2023
If ao, bu & X, need to introduce åp = Projx (ao) and bu = Projx Projx(bo) to apply 17
We now
get
||rm (am) I' (bm))l S E + |100 - boll + Cllao - boll" + L(Ilam - pi ll + l16m
-m
l)
Bounding the terms l|am Om and [i0m dbn now gives
ll am I
I1am am || + ||ām oin S S(Ila0 - ãoll + E) + SRh
where am now comes from the Euler scheme with access to v that starts at åp (meaning
am++ 1:= +hu+, (am) and ān:= Likewise, we get the same bound for I|bm. - pm l|-
Since ||ao uلll = dist(no, X) and |l6o - boll dist(bo, X). we get the second bound in the theorem. Note that if we use the stability of the ODE instead of the Euler method to bound ||am L | we get the same result. Indeed, if am again comes from the Euler scheme with access that starts at ao (meaning ām+1:=ām the,,, (#m) and dg:=ao), we can write, for some constant F > 0
| am ll I Iamm 7mll + amm
Pi ll + 1 Pta S Se -SRh + Fll - àol
since
S 1100 - 6oll + f( fm
10
(00)) -. )| ds
$ |las 6oll + M (
pm
00o1 ds 11
Fi - ãoll
where we get the last line by Gronwall's lemma.
D ADDITIONAL EXPERIMENTS
D.1 ADVERSARIAL ATTACKS
White-box attacks have access to the network's weights and architecture. The Fast Gradient Method (FGM) (Goodfellow et al. (2015)) takes perturbation step in the direction of the gradient that maximizes the loss. Projected Gradient Descent (PGD) (Madry et al. (2018) and the Basic Iterative Method (BIM) (Kurakin et al. (2017) are iterative versions of FGM. We use the Auto-PGD-CE (Croce & Hein (020a)) variant of PGD which has an adaptive step size. Two slower but more powerful attacks are DeepFool (DF) (Moosvvi-Dezfooli al. (2016)), which iteratively perturbs an input in the direction of the closest decision boundary, and Carlini Wagner (CW) (Carlini & Wagner
(2017a)), which solves an optimization problem to find the perturbation. Auto/ttack (AA) (Croce & Hein (2020a)) is combination of three white-box box attacks (two variants of Auto-PGD (Croce & Hein| (2020a)) and the FAB attack of Croce Hein (2020b)), and of the black-box Square Attack (SA) (Andriushchenko et al. (2020)). Black-box attacks don't ion't have any knowledge about the network and can only query it. We use two such attacks: Hop-Skip-Junp (HSJ) (Chen et al. (2020)), which estimates the gradient direction the decision boundary, and the Boundary Attack (BA.) (Brendel cct al. (2018)) which starts from a large adversarial input and moves towards the boundary decision to minimize the perturbation. We use ART (Nicolae al. (2018) ) and its default Syper--praaaen values to generate the adversarial samples, except for AA for which we use the authors' original code.
D.2 IMPLEMENTATION DETAILS
For ResNeXt50 Xie et al. (017)) on CIFARI00 (Krizhevsky (2009)). we train for 300 epochs using SGD with learning rate of 0.1 (divided by ten at epochs 150, 225 and 250), Kaiming initialization, a batch size of 128 and weight decay of 0.0001. For RCE training, the only changes are that the learning rate is 0.05 and the initialization is orthogonal with gain of 0.05.
For ResNetl 10 HHe et al. (2016a) on CIFARIO (Krizhessky (2009)), we train for 300 epochs using SGD with learning rate of 0.1 (divided by ten at epochs 150, 225 and 250), orthogonal initialization
17

---

Under review as a conference paper at ICLR 2023
with gain of 0.05, batch size of 256, weight decay of 0.0001 and gradient clipping at 5. For RCE training, the only change is that we don't use gradient clipping.
For WideResNet (Zagoruyko & Komodakis (2016)) TinyImageNet, train for 300 epochs using SGD with learning rate of 0.1 (divided by ten at epochs 150, 225 and 250), orthogonal initialization with gain of 0.1, batch size of 114 and weight decay of 0.0001.
For the magnitude parameter of the Mahalanobis detector, we try all the values tried in their paper for the magnitude and we report the best results.
D.3 ADVERSARIAL DETECTION TRAINING DATA
Clean
C1
C2
Original Dataset
Noise
Adversarial Detection Dataset
Clean
A
B1
B2
Attack
Attack 2
Attacked
D1
D2
Figure 3: Adversarial detection dataset creation. AUB1UB2 is the original dataset, where A the training set and BIUB2 is the test set. We create a noisy version of BIUB2 by adding random noise each sample in B1BB2 to get C1UC2. Noisy samples are considered clean (i.e. not attacked) in adversarial detection training. We create an attacked version of BIUB2 by creating an attacked image from each image in BIUB to get DIUD2. In the case of generalization to unseen attacks, Attack used to create D2 from B2 is different from Attack used to create DI from B1. Otherwise, Attack and Attack are the same. BIUCIUDI is the adversarial detection training set and B2UC2UD2 the adversarial detection test set.
D.4 PRELIMINARY EXPERIMENTS
We see in Figure below that training deep ResNets without batch-normalization is near impossible, whereas maintain the same performance and stability without ResNets for up to 50 blocks. NetsResNets also compared in this regard to the small step method of /Zhang al.
18

---

Under review as a conference paper at ICLR 2023
(2019a), which simply adds small weight h of around 0.1 in front of the residue function to make ResNets more stable. The Least Action Principle has the same improved stability when training without Datch-normalization in Figure as this method, while also improving the test accuracy when batch-normalization is used (Karkar et al. (2020) which the small step method does not claim.
70
No BN ResNet
ResNett
No BN LAP ResNet
Number of blocks
Figure 4: Test accuracy of ResNets of various depths without batch-normalizztion on CIFARIO.
D.5 DETECTION OF SEEN ATTACKS
Here, VAN corresponds to detectors trained on a vanilla network, RCE on an RCE-network and LAP on LAP-network. T corresponds detector, and M to the Mahalanobis detector.
Table Average adversarial detection accuracy and standard deviation over runs using ResNetl 10 on CIFAR10.
Attack
Detector
FGM
APGD
BIM
DF
CW
VAN
97.14 0.59
94.10 0.41
97.54 +: 0.52
99.98
0.06
98.04 0.47
RCE
95.96
95.32 0.80
96.16 + 0.61
99.92 - 0.10
89.36 - 0.53
LAP
0.228
0.37
99.28 = 0.30
99,84
9,37
VAN
87.78
$2.08
86.78 4.70
91.50 3.17
85.58
RCE
93.00 0.60
$7.88
92.30 1.33
0.40 94.98 - 0.40
83.38 83.39
LAP M
95.64 0.62
9.770
95.38 + 0.54
96.70 - 0.69
93.36
19

---

Under review as a conference paper at ICLR 2023
Table 4: Average adversarial detection accuracy and standard deviation runs using ResNet 10 on CIFARIO.
Attack
Detector
AA
HSJ
BA
VAN T
88.88 + 1.34
97.38 + 0.60
96.56 + 0.56
LAPT
94.08 + 0.80
97.38 + 0.43
97.02
0.18
VAN M
80.46 1: 2.24
84.14 + 1.07
80.20 = 2.11
LAP M
89.96 1 0.88
92.30 + 0.46
89.62 = 0.43
Table 5: Average adversarial detection accuracy and standard deviation over 5 runs using ResNeXt50 on CIFAR100.
Attack
Detector
FGM
APGD
BIM
AA
DF
CW
VAN T
97.3 + 0.5
96.0 0.5
98.0 + 0.3
84.9 + 0.7
99.8 + 0.2
95.7 + 0.4
RCE T
0.4
0.1
07.
50.1 0.0
92.5 9.10
LAP
98.3
0.5
0.1
9..6 0.6
95.8
VAN M
95.8 0.5
93.9
0.6
0.79
97.3
94.7
RCE M
96.5
+ 0.4
94.7
96.6
97.6 +
88.4 0.6
LAPM
96.8 + 0.4
94.6 0.7
+ 0.5
86.6 + 0.5
97.1 0.3
94.7 + 0.3
Table Average ad adversarial detection accuracy and standard deviation over runs using WideRes.Net on TinyImagNeet.
Attack
Detector
FGM
APGD
BIM
AA
VAN T
95.92 + 0.10
96.04 + 0.14
96.56 + 0.07
81.50 t 0,46
LAP
95.86 = 0.11
96.16 - 0.07
96.56 :: 0.07
81.04 + 0.67
VAN M
84.62 + 1.32
84.20 + 1.32
84.46 + 1.29
78.42 + 0.88
LAP
85.26 + 0.96
85.10 + 0.60
85.24 1.11
78.36 + 0.97
D.6 DETECTION UNSEEN ATTACKS
Here, VAN corresponds to detectors trained on vanilla network, RCE on an RCE-network and LAP on a LAP-network. T corresponds our detector, and to the Mahalanobis detector.
Table 7: Average adversarial detection accuracy of unseen attacks after training on FGM and standard deviation over runs using ResNetl 10 on CIFARIO.
Attack
Detector
APGD
BIM
AA
DF
CW
HSJ
BA
VAN
89.3 +: 1.6
96.0 0.7
85.1 + 1.1
91.0 0.9
93.2 13.2.11 1.0
93.0
90.9 1: 0.6
RCE
91.8 + 1.1
93.6 1.1
50.0
6..4
52.5 6.0
LAP
028-033
8.
1.
1.0
75.0 0.6
VAN
4.7
77.2 4.8
72.133.1
80.1
7..9
7..7
7..3
RCE
$1.500
82.6 1.3
0.10
8.72
76.0
68.5 81.6
LAP M
87.9 0.8
84.9 0.4
11.9122
0.7
8.6
81.5
81.4 0.8
20

---

Under review as a conference paper at ICLR 2023
Table 8: Average adversarial detection accuracy of unseen attacks after training FGM and standard deviation over rums using ResNeX150 on CIFARI00.
Attack
Detector
APGD
BIM
AA
DF
CW
HSJ
BA
VANT
90.8 0.5
96.1 0.6
73.3 0.9
87.6 0.2
87.5 0.4
89,9 099:22
81.7 + 1.6
RCET
87.7 0.5
50.0 9.91
0.4
0.5
0.44
57.7 0.4
LAP
1. 9.3
7.00
1.17
7..1
71.9
VAN
86.9
9.0
1.99
8.71
8.10
84.1
78.7
RCE
0.7
88.6 0.8
50.0
7.8
0.6
74.6
63.2 0.9
LAP M
86.7 + 0.9
93.9 0.4
80.0 0.6
80.9 1.6
80.5 3.7
7
Table 9: Average adversarial detection accuracy of unseen attacks after training on FGM and standard deviation over runs using WideResNet on TinylmageNet.
Attack
Detector
APGD
BIM
AA
CW
VAN
93.26 0.60
94.66 + 0.49
77.04 + 0.74
90.62
0.60
91.42 1.06
LAP T
93.48 0.72
94.80 + 0.56
76.58 = 0.48
90.12 = 0.55
91.52 - 0.89
VAN M
76.96 0.94
77.02
+ 1.08
60.36 + 0.62
73.18
0.59
75.52 0.82
LAP
77.96 + 0.49
78.00 + 0.77
61.96 + 0.89
73.98 1.12
76.22 0.83
D.7 DETECTION RATE OF SUCCESSFUL ADVERSARIAL SAMPLES
As in Lee et al, (2088) might be only concerned with detecting adversarial samples that success- fully fool the network. We find that the detection rate of successful adversarial samples is always higher than the accuracy seen attacks, the detection rate successful adversarial samples even very close to 100% on all attacks. We report the detection rate of successful adversarial samples in Table TO below for the AA attack (the most difficult attack to detect in the previous tables), both when it is seen and unseen during detection training. In the unseen case, the training is carried out using adversarial samples generated by the FGM attack.
balanced test set of clean and adversarial samples reported above. On
Table 10: Average detection rates of successful adversarial samples on Network/L.AP-eetwer
Attack
Network/Data
Det
AA (seen)
AA (unseen)
ResNetl 10
TR
99.9/99.9
98.5/80.7
CIFARIO
MH
88.3/95,4
79.4/81.1
ResNeXt50 CIFAR100
TR
99.9/99.7
74.0/72.2
MH
98.8/98.7
72.8/86.6
WideResNet
99.9/99.9
91.1/92.5
TinyImageNet
MH
95.6/96.5 71.1/70.1
D.8 TIME COMPARISON
With ResNeXt50 CIFAR100 and Tesla V100 GPU, it takes method (including the time to generate FGM attacks) 66 seconds to extract its features from both the clean and the adversarial samples, while it takes the Mahalanobis method 110 seconds. Mahalanobis also extracts some
21

---

Under review as a conference paper at ICLR 2023
statistics from the training set prior to adversarial detection training, which takes additional 35 seconds. Our feature vector is of size 32, compared to 5 for the Mahalanobis detector. So our random forest takes only 4 more seconds to train than the Mahalanobis one (7 vs 3 seconds). Computation of the features our detector uses (norms and cosines) is in O(MD), where M is the number of residual blocks and D is the largest emhedding dimension inside the network.
D.9 DETECTION OF OUT-OF-DISTRIBUTEE SAMPLES
(OOD) samples in similar setting to the Mahalanobis paper Lee et al. (2018). We use the same ResNet models trained on CIFAR1O. Since the detectors need to be trained, are in the OOD setting where have a first dataset for training the network (CIFARI0 10 here) and second dataset from another distribution that is not the test OOD distribution to train the detector on. This could be another dataset (CIFARI00 here), some images found the wild, or perturbation of our dataset that we generate using an adversarial attack (CW on CIFARI 10 here). Detectors can then be used by training them to distinguish between these first two datasets, and then testing them on distinguishing between the first dataset and third unseen dataset (SVHN (Netzer) eet al. (01T)) here). Results are in Table below. Our detector performs very well and better than the MH detector in both experiments. Without any extra data available, using an adversarial attack allows to detect OOD samples from an unseen distribution with more than 90% accuracy.
We test detection of
Table 11: Average OOD detection accuracy and standard deviation over runs using ResNetl 10 trained on CIFAR1O.
OOD Experiment
OOD Experiment 2
Detector
CIFARIOR 100 (seen)
SVHN (unseen)
CW-CIFAR10 (scen)
SVHN (unscen)
VAN TR
98.30 1 0.46
97.46 0.49
97.42 1 0.57
91.38 0.95
RCE TR
98.42 + 0.40
98.20 0.39
91.54 + 6.06
77.58 - 6.72
LAP TR
98.30 + 0.22
98.50 = 0.47
97.28 + 0.62
85,46 - 2.64
VAN MH 86.88 + 1.52
91.28 = 0.92
81.80 + 1.96
83.76 = 1.13
RCE MH
94.82 + 0.45
92.16 0.57
76.74 + 2.75
54.24 + 3.46
LAP MH
94.84 + 0.41
90.46 1.45
89.68 + 0.65
76.72 + 1.73
D.10 ATTACKING THE DETECTOR
We consider here the case where the attacker also attacks the detector (adaptive attacks). We try two such attacks on the TR and MH detectors on ResNetl 10 trained on CIFARIO. Both attacks are white-box with respect to the network. The first is black-box with respect to the detector. only knows if an adversarial sample has been detected not. The second has some knowledge about the detector. knows what features it uses and can attack it directly to find adversarial features. We test these attacks by looking at the percentage of detected successful adversarial samples that they turn into undetected successful adversarial samples that fool both the network and the detector.
The first attack proceeds as follows. A strong white-box attack (CW or AA) is used on the network on image x that has label If it finds a successful adversarial image ã that fools the network into predicting ÿ A y but is detected by the detector, the attacker will attempt modify this image ž so that the network and the detector are both fooled. For this, the image 7 is used as the initialization for Detector system. The attacker considers that the Network- Detector behaves as follows: it outputs the class prediction of the network if the detector does not detect an attack and outputs an additional 'detected' class if the detector detects an attack. The image ă targeting the label. This way the network makes a mistake and the 'detected' class avoided. On the vanilla ResNet 10, this attack turns only 8.8% of 1700 detected successful adversarial samples T into undetected successful adversarial samples on our detector, compared to 28.8% on the Mahalanobis detector. These percentages are lower on the LAP-ResNet1 10 as they drop to 6.8% our detector and 12.9% on the Mahalanobis detector. This shows that LAP training improves the robustness of both adversarial detectors to being attacked themselves, and that the Transport detector is more robust than the MH detector.
an attack (HSJ) on a black-box
attacker attacks this Network-Detector
22

---

Under review as a conference paper at ICLR 2023
The second attack is very similar to the adaptive attack used in Carlini & Wagner (2017b) to break the Kernel Density detector Feinman et al. (2017). It proceeds as follows. A strong white-box attack (CW AA) is used on the network adversarial image è that fools the network but is detected by the detector, the detection features that é generates when run through the network used the initialization for black-hox attack (HSJ) on the detector. If successful adversarial detection features z" that fool the detector are found, the attacker has find adversarial perturbation of that still fools the network and that generates these features 2" (or close features that also fool the detector) when run through the network. We do this as in Carlini & Wagner (017b) by solving the following optimization problem:
image that has label y- If it finds successful
min L(N (x v) + Ci D(x") =" 02 c2||2" xel
(20)
where L is the cross-entropy loss, N is the network, and D is the (differentiable) function that returns the detection features of its input. This optimization problem is differentiable and we try differentiable optimization algorithms such BFGS and NR solve it. The initial detected successful adversarial image ă is used as initialization as in Carlini & Wagner (201 This attack turns 14% of detected successful adversarial samples ž into undetected successful adversarial samples on our detector on the L.P-RReNNetl 10.
Given that initial detection rates of successful adversarial samples are almost 100% (see Table 10 Appendix D.7). this shows that adaptive attacks do not (at least not easily) circumvent the detector, as detection rates drop to 85% at worst. Obviously, the second attack stronger than the first one, but it can probably still be improved by using for example white-box attack that is specific to random forests for attacking the detector such as Kantchelian et al. (2016) or Zhang et al. (2020). However, the difficulty of combining the attack the network with that on the detector remains. It is the con-differentiability of the random forest that forces either this separate treatment network and detector then the use of proxy differentiable term for the detector (here [D(x"*) - 2*1l in 220 to combine both, or the use of a black-box method as in the first attack. Also, we did not consider here the ensemble of the class-conditional detector and the general detector, which the best performing version of the detector (see Section 5.2). and should be even more robust to adaptive attacks, the attacker will have fool two random forest detectors at once and target particular label, constraining further the optimization problem he solves.
23


Reviews:
[{"review_id": "vv7wBtVfWa", "reviewer": "Reviewer_Xhhn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackles the problem of detection of adversarial examples targeting (deep) neural networks. Specifically, the paper models the underlying operations of neural networks as a “dynamic process”, wherein information is propagated “over time” (i.e., goes through layers of neurons). By acting upon such intuition, the paper proposes to detect adversarial examples by analyzing “what happens” in-between each layer, thereby modeling it as an “anomaly detection” problem. The proposed method is grounded in solid theoretical analyses, and is then experimentally evaluated on neural networks trained over diverse benchmark datasets (all of images). The experiments are repeated multiple times, and the overall findings confirm the validity of the proposal, which outperform a reputable prior work (Lee et al. (NeurIPS 2018)). Furthermore, the paper also shows that by “tweaking” the proposed detection method, it is also possible to improve its generalization capabilities---i.e., detecting adversarial examples that do not conform to what the detector is expected to identify. Such a property is also found to be true via comprehensive experiments.", "summary_of_the_review": "I liked the paper, a lot. I commend the authors for their work: despite not being “groundbreaking”, I consider this paper to be a solid contribution to the state-of-the-art, which deserves to be accepted to ICLR. Even by assuming a very critical eye, I was not able to find any significant flaw in the paper.\n\nSomething that I particularly appreciated is reporting the “runtime”: such metric is often forgotten in ML papers, but it is of high importance for practitioners as it allows to gauge the “cost” of a proposed method. Moreover, I also loved Section 5.2: here, the authors describe what I consider to be some “negative results”. I think more papers should include them. Props!\n\nHowever, I believe that the paper is currently affected by a single issue---which stems from me being a researcher with a “security background,” and hence being critical of this aspect. Although I do not consider such issue to be ground for rejection, I truly invite the authors to fix it. Let me elaborate, and propose actionable means of remediations.\n\n**The “attack” is misleading.**\nThe paper poses itself as tackling a “security-related” problem, but does not support such security focus in the appropriate way. Indeed, let me quote the first line of the abstract: “Adversarial attacks are perturbations to the input that don’t change its class for a human observer, but fool a neural network into changing its prediction.” The key term, here, is “attack”: from a security standpoint, an “attack” requires an “attacker”, who “does something (malicious) in order to achieve a given goal”. However, the paper does not provide any support for the “attacks” considered in the paper: this is due to the lack of a proper threat model---which is a lack that is becoming endemic in adversarial ML literature (see [A], which reports the opinions of researchers, policy makers and practitioners). To my understanding, the paper (and its contribution) does not deal with the “detection of adversarial attacks”, but rather with the “detection of adversarial examples”. Although the two topics have several similarities, they do not overlap: my take is that the paper addresses the more general problem of “adversarial examples,” which is useful to assess the “robustness” of ML methods and not their overall “security” (I quote the term “false sense of robustness” used in a recent NeurIPS paper [B]). \nTo address this problem, I invite the authors to revise the paper by replacing “adversarial attacks” with “adversarial examples” whenever necessary. Alternatively, the authors can insert a proper “threat model”: this could be done by moving the current Section 4.2.2 into the Appendix. My stance is that the paper is being oversold as being (also) a \"security paper\": the authors should prove that I'm wrong, or -- if I'm not -- then they should fix their paper accordingly.\n\n**Dubious utility of “detection”.** \nThis is a direct consequence of the previous point. The goal of a “detection” mechanism (in the context of  “detecting an attack”) is to act as a “trap” for the attacker. In other words, the detector “expects the attacker to do A, and if they do so, then the attacker falls into the trap and is therefore detected”. However, the lack of a threat model describing what the attacker can/cannot do prevents from determining the utility of the detection mechanism as a security measure. Such a shortcoming further supports my previous observation of shifting the focus of the paper from “detecting adversarial attacks” to “detecting adversarial examples”. I point the authors to [C] for a good paper that truly models the problem of “detection of adversarial *attacks*”.\nI acknowledge that the paper reproduces well-known white/black box attacks, but I still do not accept the lack of a threat model. A way to rectify the issue is by mentioning that \"we evaluate our method by assuming adversarial examples found by means of the white/black box attacks proposed in (ref)\".\n\n\n\n**General or Specific?**\nSomething that is not fully clear to me is whether the proposed idea (which I found very enticing!) can be applied to “any” neural network, or only to residual nets. Let me explain. The paper proposes a method whose main intuition lies in the modeling the “depth” of a neural network as “time”. However, this leads me to think: would this idea “work” for those neural networks that are “time-aware”, such as recurrent neural networks (RNN) and LSTM? This is because, by following the paper’s idea, an input can go “back in time”. The paper does not take such architectures into account, so I am inclined to believe that it does not work. According to the introduction, the paper focuses on “residual networks” because they are “particularly amenable to this analysis”. The authors fairly state that “the analysis and implementation can extend immediately to any network where most layers have the same input and output dimensions.”, but they do not mention whether this also applies to RNN or LSTM; moreover, in Section 3.2, the definitions clearly state “Residual Networks”.\nI hence ask the authors to clarify whether the proposed method can cover only “residual networks” (as well as any model that approximates such networks), or not. This should be clearly stated in the Introduction.\n\n**Only Image data.**\nThis is self-explanatory: although the proposed methodology is (theoretically) applicable to any (residual?) neural network, the paper (and its evaluation) appears to be tailored for applications of neural networks that focus on image analysis. There are many domains in which neural networks are used today, and focusing on a single data-type prevents to assess the general applicability of the proposed technique. A potential fix can be, e.g., performing a proof of concept experiment on a dataset that does not include images---albeit this may be hard to find, since residual nets (which I believe to be “required” for the proposed method) are mostly useful for images.\n\nSome additional issues:\n\n•\tI recommend to change the title: as a “security” researcher, I was misled into thinking that the paper dealt with adversarial attacks in the “network traffic analysis” domain (the term “transport” is a key component in networking). Specifically, I endorse the authors to replace the term “transport” with something else. Maybe “updates” is a better fit?\n\n•\tThe first sentence in Section 5.1 should be better contextualized “First, the regularization improves generalizations.” Instead of \"First\", it should report \"this result shows that...\"\n\n•\tIn Section 4: “use the transport regularization (6) to improve detectability of adversarial attacks.” It is not clear what “(6)” stands for.\n\n•\tI was originally taken aback by the “abstract”. In truth, I did not like the abstract at all. I think the authors tried to blend “technical” details with “simple” jargon, but the result (imho) is something that does not give true credit to the paper’s value. \n\nEXTERNAL REFERENCES\n\n[A]: Apruzzese, G., Laskov, P., de Oca, E. M., Mallouli, W., Rapa, L. B., Grammatopoulos, A. V., & Franco, F. D. (2022). The Role of Machine Learning in Cybersecurity. Digital Threats: Research and Practice.\n\n[B]: Pintor, M., Demetrio, L., Sotgiu, A., Manca, G., Demontis, A., Carlini, N., ... & Roli, F. (2022). Indicators of attack failure: Debugging and improving optimization of adversarial examples. NeurIPS 2022.\n\n[C]: Shan, S., Wenger, E., Wang, B., Li, B., Zheng, H., & Zhao, B. Y. (2020, October). Gotta catch'em all: Using honeypots to catch adversarial attacks on neural networks. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (pp. 67-83\n", "strength_and_weaknesses": "STRENGTHS:\n+ Strong theoretical analysis\n+ Experiments on multiple datasets\n+ Repeated trials that allow statistical tests\n+ The “runtime” is evaluated, thereby allowing to gauge the overhead of the method\n+ Good writing\n+ Good presentation\n+ Interesting (and novel) idea\n\n\nWEAKNESSES\n- The focus on “security” is not sound\n- The evaluation (and the entire method) appears to be exclusively tailored for data in the form of “images”\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ZGBFxuxKvc", "reviewer": "Reviewer_J6gT", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an adversarial example detection mechanism for residual networks, as well as a regularization method to improve the detection performance. The detection mechanism is based upon viewing the deep network as a dynamical system. The regularization method is an existing method based on optimal transport that the authors show theoretically and empirically makes adversarial examples easier to detect.  ", "summary_of_the_review": "I am leaning towards reject because I found the paper lacking clarity/difficult to follow in terms of the methods proposed. However, the final results of the paper are promising. ", "strength_and_weaknesses": "Strengths\n- The authors show their detection method performs better than the Mahalanobis detector on known and unknown white box attacks, and also on known black-box attacks. \n- The regularization method proposed also improves the performance of the existing Mahalanobis detector.\n- The authors give theoretical backing behind their choice of regularization method. \n\nWeaknesses\n- Overall I found the paper hard to follow. I think the paper would benefit from more explicitly stating how the detection is implemented. \n- The method is restricted to residual networks, whereas the detector being compared to (the Mahalanobis detector) I believe is applicable to different types of architectures.\n- I'm not sure whether you specify that you are considering the l-infinity norm threat model in section 5. \n- There are a lot of unclear statements and loose ends. For example, rather than saying \"We don’t fine-tune these hyper-parameters much\", it'd be better to be more specific with respect to the details.\n- There was never much of an introduction or review of adversarial examples more formally -- you may wish to add this. \n- The computational complexity of the detection method was not made explicit. \n- You could more explicitly compare your method to the method of comparison (the Mahalanobis detector) in the related works section for better clarity. \n\nOther comments/questions:\n- Why do you use the test set for training the detector? It seems like it would be more reasonable to hold out part of the training set.\n- This paper focuses on detecting adversarial examples specifically, how does it compare to existing methods on other OOD detection tasks? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8rm6ktsANBi", "reviewer": "Reviewer_jGWF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tries to detect adversarial examples from a perspective that models residual networks as discrete dynamical systems. The detector studies trajectories of samples in space, through time, to distinguish between clean and adversarial examples. Based on this rationale, the authors also apply transport regularization during training to make the detector perform better. Though this method does not necessarily make the model more robust to adversarial attacks, it makes the model's behavior on adversarial examples more distinguishable.", "summary_of_the_review": "In general, I think this paper proposed an adversarial detection method from an interesting perspective. The theoretical analysis is solid. However, the experimental part may not be enough due to the lack of adaptive attacks and strong attacks, which makes the proposed method less convincing. So I would recommend a borderline reject.", "strength_and_weaknesses": "Strength:\nThe proposed method has a solid theoretical background and the idea of applying trajectory regularization in optimal transportation to adversarial example detection is also very interesting. The theoretical studies look solid to me. The authors also conducted experiments to show the effectiveness of the proposed method.\nWeakness:\n1. My first concern is that the authors did not test on adaptive adversaries, which may have access to the detector itself. Similar issues also exist in other works on adversarial examples detection and defense. There are existing works that show that random forest and other decision tree-based models are also vulnerable to adversarial attacks (e.g. Kantechelian et al \"Evasion and Hardening of Tree Ensemble Classifiers\", ICML16). So I think if the attackers are able to design attacks on the detector, the proposed method can be bypassed. \n2. Even for non-adaptive attacks, SOTA attacks such as AutoAttack (Croce and Hein, ICML20) are not tested, either. I think it is necessary to test under strong attacks. For example, LID mentioned in this paper fails under high-confidence adversarial examples according to Athalye et al ICML18.\n3. Minor: Please refrain from only using color to distinguish points and bars as in Figure 1, Figure 2 and Figure 3, as it is not friendly to readers with color blindness.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject