Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients
Authors: Chengcheng Ma
Baoyuan Wu
Shibiao Xu
Yanbo Fan
Yong Zhang
Xiaopeng Zhang
Zhifeng Li
Venue:  (2020)

Abstract: Adversarial examples have been well known as a serious threat to deep neural
networks (DNNs). To ensure successful and safe operations of DNNs on realworld tasks, 
it is urgent to equip DNNs with effective defense strategies. In this
work, we study the detection of adversarial examples, based on the assumption
that the output and internal responses of one DNN model for both adversarial and
benign examples follow the generalized Gaussian distribution (GGD), but with
different parameters (i.e., shape factor, mean, and variance). GGD is a general
distribution family to cover many popular distributions (e.g., Laplacian, Gaussian,
or uniform). It is more likely to approximate the intrinsic distributions of internal
responses than any specific distribution. Besides, since the shape factor is more
robust to different databases rather than the other two parameters, we propose
to construct discriminative features via the shape factor for adversarial detection,
employing the magnitude of Benford-Fourier coefficients (MBF), which can be
easily estimated using responses. Finally, a support vector machine is trained
as the adversarial detector through leveraging the MBF features. Through the
Kolmogorov-Smirnov (KS) test, we empirically verify that: 1) the posterior vectors 
of both adversarial and benign examples follow GGD; 2) the extracted MBF features 
of adversarial and benign examples follow different distributions. Extensive 
experiments in terms of image classification demonstrate that the proposed 
detector is much more effective and robust on detecting adversarial examples 
of different crafting methods and different sources, in contrast to state-of-the-art 
adversarial detection methods.

Under review as a conference paper at ICLR 2020
EFFECTIVE AND ROBUST DETECTION OF ADVER- SARIAL EXAMPLES VIA BENFORD-FOURIER COEFFI- CIENTS
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial examples have been well known serious threat to deep neural networks (DNNs). To ensure successful and safe operations of DNNs on real- world tasks, it is urgent to equip DNNs with effective defense strategies. In this work, we study the detection of adversarial examples, based on the assumption that the output and internal responses of one DNN model for both adversarial and benign examples follow the generalized Gaussian distribution (GGD), but with different parameters (i.e., shape factor, mean, and variance). GGD is general distribution family to cover many popular distributions (e.g., Laplacian, Gaussian, or uniform). It is more likely approximate the intrinsic distributions of internal responses than any specific distribution. Besides, since the shape factor is more robust different databases rather than the other two parameters, we propose to construct discriminative features via the shape factor for adversarial detection, employing the magnitude of Benford- easily estimated using responses. Finally, support vector machine is trained as the adversarial detector through leveraging the MBF features. Through the Salimogoro--mminoe
coefficients (MBF), which can be
(KS) test (Massey Jr (1951)) We empirically verify that: 1) the posterior vectors of both adversarial and benign examples follow GGD; 2) the extracted MBF features of adversarial and benign examples follow different distributions. Extensive experiments in terms of image classification demonstrate that the proposed detector is much more effective and robust detecting adver- sarial examples of different crafting methods and different sources, in contrast to state-of- the-art adversarial detection methods.
INTRODUCTION
Deep neural networks (DNNs) have achieved a remarkable success in many important applications, such image classification, face recognition, object detection, etc. In the meanwhile, DNNs have been shown to be very vulnerable to adversarial examples. However, many real-world scenarios have very restrictive requirements about the robustness of DNNS, such as face verification for login, or semantic segmentation in autonomous driving. Adversarial examples are a serious threat to the applications of DNNs these important tasks. Since many kinds of adversarial attack methods have been proposed to fool DNNs, it is more urgent to equip effective defensive strategies to ensure the safety of deep models in real-world applications. However, defense seems be more challenging than attack, as it has to face adversarial examples from unknown crafting methods and unknown data sources. Typical defensive strategies include adversarial training, adversarial de-noising, and adversarial detection. Compared to the former two strategies, adversarial detection is somewhat more cost-effective, as it often does not need to re-train or modify the original DNN model.
There are two main challenges for adversarial detection. (1) The adversarial examples are designed to camouflage themselves to be close to the corresponding benign examples in the input space. Then, where and how to extract the discriminative information to train the detector? (2) The data sources and the generating methods of adversarial examples are often inaccessible to the detector. In this case, the detector can be stably effective across different data sources and different attack meth- ods? In other words, good adversarial detector is required to be not only effective to distinguish adversarial and benign examples, but also robust to different data sources and attack methods.

---

Under review as a conference paper at ICLR 2020
To satisfy the first requirement of effectiveness, we utilize the other principle of crafting adversarial examples that the outputs between benign and adversarial examples should be large, to encourage the change of the final prediction. It means that the imperceptible difference between benign and adversarial examples in the inputsspace space is enlarged along the DNN model, leading to the significant difference in the output space. Inspired by this fact, we assume that the output or the responses of internal layers of the DNN model should include the discriminative information for benign and adversarial examples. A few works have attempted to extract different types of discriminative fea- tures from the output or the internal responses, such as kernel density (KD) (Feinman et al. (2017)) and the local intrinsic dimensionality (LID) (Ma et al. (2018). () etc. To achieve the robustness, the extracted discriminative features should model the intrinsic difference between adversarial and be- nign examples, rather than the difference from the changes of data sources or attack methods. Many existing methods have shown the effectiveness to some extent on detecting adversarial examples of specific data sources and attack methods. However, their robustness, especially across different data sources, has not been well studied and verified.
In this work, we propose a novel detection method based on the assumption that the internal responses of both adversarial and benign examples follow the generalized Gaussian distribution (GGD) (Varanasi & Aazhang (1989). but with different parameters, including the shape factor, mean, and variance. 'The rationale behind this assumption is that GGD covers many popular distri- butions with varied shape factors (such as Laplacian, Gaussian, or uniform distribution), such that GGD is more likely to approximate the intrinsic response distributions rather than one specific distri- bution. Moreover, mean and variance of GGD may vary significantly with respect to different classes and data sources, even for benign examples, while the shape factor is more robust. For example, the mean and variance of two Gaussian distributions could be totally different, but their shape factors are the same (i.e, 2). Thus, we propose to use the shape factor as an effective and robust discrimination between adversarial and benign examples. However, it is difficult to exactly estimate the shape fac- tor in practice. We resort to the magnitude of Benford- Fourier coefficients (Pasquini et al. (2014). which is function of the shape factor. It can be easily estimated using internal responses, according to the definition of Fourier transform. Then, the magnitudes estimated from internal responses of different convolutional layers concatenated
novel representation. Finally, a support vector machine (SVM) (Vapnik] (2013)) is trained using the new representations as the adversarial detec- tor. Extensive experiments carried out on several databases verify the effectiveness and robustness of the proposed detection method. To further verify the rationale of our assumption, we present the empirical analysis through the Kolmogorov-Smirnoy test (KS test) (Massey Jr (195T)). The KS test verifies that 1) the posterior vectors of both adversarial and benign examples predicted by the CNN model follow the distribution of GGD, but with different parameters, and 2) the MBF fea- tures of adversarial and benign examples follow different distributions, and the MBF features of adversarial examples crafted from different attack methods follow the same distribution, as well as that the MBF features of adversarial/benign examples from different data sources follow the same distribution. Moreover, we visualize the statistics (i.e., mean + standard deviation) of the extracted MBF features for adversarial and benign examples. The visualization reveals the distinct difference between adversarial and benign examples. These empirical analysis demonstrates the effectiveness and robustness of the proposed MBF detector.
2 RELATED WORK
In this section we focus on reviewing existing works about adversarial detection, while those of other adversarial defensive strategies are out of scope of this work.
The general idea of most existing detection methods is learning or constructing a new representation to discriminate adversarial and benign examples, utilizing the outputs or immediate responses of an original classification network. [Li Li (2117) trained cascading classifier based on the principal component analysis (PCA) (Pearson (1901)
statistics of responses from each convolutional layer of the defended convolutional neural network (CNN) model. An example is recognized as benign if all single classifiers of the cascade predict it as benign, otherwise adversarial. Lu et al. (2017) proposed SafetyNet by adding a RBF-SVM classifier to detect adversarial examples, at the end of the original classification network. Metzen et al. (2017) proposed detection network along with an original classification network, which takes the internal responses of the original network as inputs. It shows effectiveness detecting adversarial examples generated by simple attacks (such FGSM
2

---

Under review as a conference paper at ICLR 2020
(Goodfellow et al. (2014)) and JSMA (aaperoo et al. (2016)), while performs much worse when facing more advanced attacks (such as C&W (Callini & Wagner (01 )) It tells that this method is sensitive to attack methods. Grosse et al. (017) attempted detect adversarial examples using the statistical test of maximum mean discrepancy (MMD). Although above detection methods show effectiveness some attack methods and some databases, but a thorough evaluation presented in Carlini & Bagner](2017)) has shown that these methods are sensitive to attack methods or databases, and they can be somewhat easily invaded by new attacks.
Some recent works proposed to utilize neighboring samples in the same database to construct a better representation of current sample. Feinman et al.] (2017) defined two metrics based on the responses of the final hidden layer of the classification neural network, including kernel density estimation (KDE) and Bayesian neural network uncertainty (BU). If the metric score of KDE/BU is lower/higher than a pre-defined threshold, then the example is predicted as adversarial. Ma et al. (218)) utilized the local intrinsic dimensionality (LIDD) to measure the characterization of adversarial regions of DNNs. LID describes the distance between one example and its k-nearest neighboring sample in the feature space of immediate responses of the original classification network. The distances computed from different layers concatenated the example representation, which is then used to train a shallow classifier to discriminate adversarial and benign examples. [Zheng & Hong (2018) defined the intrinsic hidden state distribution (IHSD) of the responses of the original classification network to model different classes. The Gaussian mixture model (GMM) was used to approximate IHSD of each class. Then, the posterior probability of one sample assigned to GMM is computed as the metric. If the probability is lower than a pre-defined threshold, then it is recognized as adversarial. Lee et al. (2018) computed the class-conditional Gaussian distribution of the responses of the original classification network based on the whole training set. Then, the Mahalanobis distance between one sample and its nearest class-conditional Gaussian distribution is used as the metric for detection. If the distance is larger than pre-defined threshold, then it is detection
detected as adversarial. Compared to some aforementioned linglerrpreseenation-baas methods, these coint-representatioon based methods showed better performances some databases. However, the detection cost for each example is much higher, as the responses of its neighboring samples should also be computed. Besides, since the representation is highly dependent the neighbors or all training examples, the detection performance may be sensitive to data sources, which will be studied in later experiments.
There are also some other approaches that do not construct representations from the responses of the original classification network. [Hendrycks & Gimpell (2016) adopted PCA statisties to dis- criminate adversarial and benign images, independent of any DNN model. However, the study presented in Carlini & Wagner (2017b) has demonstrated that this method works for MNIST but not for CIFAR-10, and PCA statistics are not robust features to detect adversarial images. [aag et al. (0188) proposed a novel loss called reverse cross-entropy (RCE) to train the classification network, such that the distance measured by kernel density (Feinman et al. (2017)) between adversarial and benign examples could be enlarged. [Samangouei et al. (2018) proposed Defense-GAN to model the distribution of benign examples using generative adversarial network (GAN). If the Wasserstein distance between one example and its corresponding example generated by the GAN larger than pre-defined value, then it is detected as adversarial. However, these above three methods much more costly than other detection methods.
3 PRELIMINARIES
3.1 GENERALIZED GAUSSIAN DISTRIBUTION
Assume that random variable X' E Re follows the generalized Gaussian distribution (GGD) Varanasi & Aazhang (989). Then, its probability density function (PDF) is formulated with two positive parameters, including the shape factor e and the standard deviation a,, as follows
Px (z) = A 1601"
(1)
where / _ [(3/c) 1 and 21(1/e5. with I(-) being the Gamma function. Note that the mean [(1/e]
parameter f is omitted above, as ş has no relation with the shape of distribution and we set as without loss of generality. A nice characteristic of GGD is that it covers many popular distributions
3

---

Under review as a conference paper at ICLR 2020
with varied shape factors. For example, when 1, then it becomes the Laplacian distribution; when c = 2, then it is the Gaussian distribution with a variance of o²; when c + +o0, then is specified as uniform distribution on (-v2o, v2o).
3.2 BENFORD- DONFFORD-FOURER COEFFICIENTS
Although generalized Gaussian distribution (GGD) is able to cover a bunch of distributions, it is hard depict the exact forms of GGD precisely. To this end, we further define random variable Z = 1og10 | mod 1 for detecting and distinguishing different form of GGD, of which the PDF is 2007), with the fundamental period
formulated by means of Fourier Series as "erzz-González being fixed as 2T,,
+000
(2(=)= = 11
AA, cos (2mnz) + B, sin (2mmz)) + n=1
Σ n=1
Ian cos + Φn), (2)
where: = [0, 1) corresponds to the domain of random variable 2, the phase of Fourier Series explained as on = arctan (- Bm), and the magnitude denotes |a,|| = VA2. AA + B an = |anl e000. edon.
denotes the n-th Fourier coefficient of Pz(=) evaluated at 2Tn, and its definition is
2Ac
j2 log 10
On
Pz(z) f2 log,0 d2
(3)
Bc
clog 10
an also called Benford-Fourier coefficient. Note that an is complex number, and its magnitude can be calculated as follows
+:00 II A=0
2mn
2 1 $
|a,11
1
(4)
log 10(ck 1)
Note that |an| gets smaller as n E N increases. And, an interesting property of |an.l is that it only depends on the shape factor c, while independent of the parameter a. Thus, one set of the absolute values of Benford-Fourice coefficients {lamIIneN correspond to one identical i.e., one identical special distribution of GGD. other words, we could use {[a, let as features or representations to discriminate different special distributions of GGD.
However, if is often difficult to know or even estimate the shape factor c, we cannot compute the value of |. But fortunately, recalling that an is the n-th Fourier coefficient of P2z(2) evaluated at 2Tn, we can derive an easy estimation. Specifically, assume that = { J.M) } set of M i.idd. points sampled from GGD with the same shape factor Then, the corresponding Benford- Fourier coefficients can be estimated as follows (Pasquini et al. (2014)):
M Dm=1 j2an logio [zml
M Σ cos (2 (mm]) - jsin (2xn log (mml)) (5)
1
àn
M
The gap between ån and a, is analyzed in Theorem I It tells that ân gets closer to an as M in- creases. For clarity, we firstly introduce few notations: T = e j2nn log1o [X] is random variable with obeying the generalized Gaussian distribution, and å, is an observation of the random vari- able y = M M Tm- Due to the space limit, the proof will be presented in supplementary material.
Theorem Assume that the estimation error e, -,, is observation of the random variable y follows the Rayleigh distribution (Papoulis Pillai (2001)) )), of which the probability density function (PDF) is formulated as
Pie (r) 2Mre Mr?
(6)
And the expectation and variance are respectively shown as follows:
1 π 2V M D(E))
E(EE)
4M
which implies that the estimation error en gets closer to O as the number of samples M increases.
4

---

Under review as a conference paper at ICLR 2020
4 ADVERSARIAL DETECTION VIA BENFORD-FOURIER COEFFICIENTS
4.1 TRAINING PROCEDURE OF ADVERSARIAL DETECTOR
There three stages train the proposed adversarial detector, including: 1) building training set based on benign images; 2) extracting novel representations of the training set via Benford- Fourier coefficients; 3) training SVM classifier as the adversarial detector. They will be explained in details sequentially. And, the overall training procedure is briefly summarized in Algorithm [
Build training set. Firstly, we collect N clean images {xI...., .NN J. which can be correctly pre- dicted by fa. Then, we adopt one adversarial attack method (e.g. C&W (Carlini & Wagner (201 7a)) or BIM (Kurakin et al. (2016)) to generate one adversarial image corresponding to each clean im- age. The crafted adversarial examples are denoted as { xN Besides, to avoid that the noisy image (polluted by some kind of non-malicious noises but still can be correctly predicted by fo) is incorrectly detected as adversarial, we also craft one noisy image by adding small random Gaussian noises onto each clean image. These N noisy examples denoted {x1....,, &N }. Note that, hereafter benign examples include both clean and Gaussian noisy examples. Consequently. we obtain one training set with 3N examples, denoted D., ((xx, (R,, -1), (x., +1)]i=,,......
Extract novel representations. We firstly feed the i-th training image from Der into fo. We concatenate all response entries of the I-th layer in fo to obtain one vector r.. Then, we estimate the corresponding Benford-Fourier coefficients according to Eq. ( as follows
M
(a,))
Σ
logio logio ((r)))l
M m=1
where Md indicates the length of r/. The magnitude of (ān)) y is computed as follows
M Σ
1
1 M!
[(am))]
cos² (2mn ((m)))) + sin² (2mm log1o ((rm)(
(8)
Then, we extract one T-dimensional feature vector a [I M .... (ār) E RR for the i-th training image from the 1-th layer. We set T = 16 experiments, as ((ā,)) M of larger n too small for discrimination. Finally, we concatenate the feature vectors of all layers to form a long vector å; = :!: a E RTL. with being the number of layers fa. Consequently, we obtain novel representation of the training set Drr, denoted as Air = {(ā, +1)] where the label of a; (i.e. 11 or 1) is directly obtained from Dtr-
Train adversarial detector. Finally, train binary SVM classifier based on Air. The trained SVM classifier will serve as the adversarial detector for the CNN model fe-
Testing. One novel testing example is firstly predicted as adversarial or not by the trained ad adversarial detector. If adversarial, then it is rejected: otherwise, it fed into fe predict its class label.
Remark. Note that in the derivation of an (see Eq. 3 the mean parameter of GGD is set to 0. In experiments, we calculate the mean values of internal-layer responses of all networks for every image, and find that mean parameters at most layers are close to 0, while the mean parameters few layers could be large. However, the mean value subtracted from each response entry when we extract the MBF features in our experiments. Thus, the derived a, is applicable to our task. Besides, according to Theorem [ the estimation error of (ā..) is inversely proportional to M/. It tells that the coefficient estimated from the larger-sized layer is more accurate. In many neural networks (e.g., AlexNet (Krizhevsky et al. (2012)), the response sizes of high layers get smaller, which means the less accurate estimation. However, we believe that the discrimination between adversarial and benign examples in higher layers is more evident than that in lower layers. There is a trade-off between estimation accuracy and discrimination. This is why we concatenate the estimated magnitudes of all layers together to construct the novel representation.
4.2 EXPERIMENTAL SETTINGS
Databases and network architectures We conduct experiments on three databases, including CIFAR-10 (Krizhevsky et al. (2014h), SVHN (Netzer et al. (201T)) and subset of ImageNet (Deng
5

---

Under review as a conference paper at ICLR 2020
Algorithm 1 Training the adversarial detector via the magnitude of Benford-Fourier coefficients.
Require: The trained CNN model fe with L layers, and the training set Par
I: for i= [Da-] do:
2: for ( 1 to L do:
3: Compute ((an))l as Eq. 8 with n = 1,...,?? , T;
4:
Concatenate {
to obtain vector á
5: end for
6:
Concatenate (á) (= obtain a long vector à
7: end for
8: Build novel representation of the training set, denoted as A.. {(â), +1)
9: Train a binary SVM classifier based on Air
return The trained binary SVM classifier.
al. (2009). ). In terms of CIFAR-I0 10 and SVHN, we adopt the same settings as the compared method LID (Ma et al. (2018). Specifically, 33-layer network pre-trained on the training set of CIFAR- 10 achieves 82.37% accuracy
the testing set with 10, 000 benign images; 19-layer network pre- trained on the training set of SVHN achieves 92.6% accuracy on the testing set with 26, 032 benign images. Then, we add small noise drawn from (0, o²) each testing image, with being the similar level of the E norm of adversarial perturbations on the same database. If both the benign and its noisy image can be correctly predicted by the classification network, then it is picked out for train- ing the detection. We finally collect 8, 175 and 23, 862 benign images from CIFATR- 10 and SVHN, respectively. These images are randomly partitioned the 80% training set and the 20% testing set, used for the training and testing of the detector. We also collect subset from ImageNet, including 800 benign images of classes (snowbird, spoonbill, bobtail, Leonberg, hamster, proboscis monkey, cypripedium calceolus, and earthstar). The 100 images of each class contain 50 testing images and 50 randomly selected training images. We fine-tune the checkpoints of both AlexNet and VGG pre-trained on ImageNe on these 800 images to achieve 100% accuracy. Then, 785 benign images are kept for detection, as both their noisy images and themselves can be correctly predicted by both the fine-tuned AlexNet and VGG models. These 785 images then randomly partitioned to 400 training and 305 testing images used for detection. For each database, as described in Section 4. one noisy and one adversarial image are generated for each benign image; then, all of benign, noisy, and adversarial images are used for detection.
Attack methods We adopt four popular adversarial attack methods to craft adversarial examples, including basic iterative method (BIM (Kurakin et al.] (2016)), Carlini Wagnerl .2Attack (CW-L.2 (Callini & Wagner (2177a)), DeepFool ( IMMosavi-Dezfooll et al. (2016)), and random projected gradient descent (R- PGD (adry et al. (2017)). They are implemented by Foolbox
Compared detection methods We compare with three state-of-the of-the-art and open-sourced adversarial detection methods, including KD+BU BU (Feinman et al. (2017). Mahalanobis distance") (M-D) (Lee et al. (2018)), and LIDE (Ma et al. (20T8)). Note that another recent work called I-defender (Zheng & Hong (2018) ) is not compared, as its code is not available. To ensure the fair comparison, the SVM is trained with all compared methods, implemented by the fitcsvn 66 fuucciin (functiintnnninnnnno in MAT- parameertss in LID, i.e., the size of mini- batch and the number of neighbors. On CIFAR 10 and SVHN, they are respectively set as 100 and 20, as suggested in Ma ( on ImageNet, as there are only 400 benign training images, they are respectively set as 50 and 20 in experiments. Moreover, we find that there are some unfair settings in the implementa- tions of compared methods. For example, KD+BU utilizes the extra 50, 000 images of CIFAR- 10 to compute the kernel density of each training and testing image; M-D also uses these extra images to compute the mean and co-variance of GMM. Since extra images of the similar distribution with the training images are often unavailable, we believe that extra images should not be used to ensure the fair comparison. Thus, extra images are not used for KD+BU and M-D in our experiments. Besides,
LAB. There are two important
http://gihbbbommmmrrimmmeeeeeeed wtw.:/eeennneeedeeaaaaaaaaaeelee htts://gitbbbcomm/pooayypokeeppp Mahalanobis detector/
https:/ggihab.com///iiggnnmm//// adversarial subspace detection/

---

Under review as a conference paper at ICLR 2020
Table I: Detection results in the non-transter case.
databose
Detector
HIM
CW-1.2
AUROC
DXeePFool
Accurncy
DieepHool
R-PGID
KD
Temmann el al
2017)
13.0
62.8
50.6
17:7
14.2
733.0
71-T
72.66
CIFAR-10
M.D
21
2018
51.7
48.4
53.7
52.0
666.7
666.7
66.7
66.7
LID
87.9
57.4
56.6
82.7
T2.6
78.5
78.0
70.2
MBF
99.6
99.6
96.9
99.4
98.88
98.3
91.8
98.5
KD+RU [Feimman
2017)
81.5
S5.1
S1.1
52.3
78.0
80.00
79.3
78.0
SVHN
LID Jan ee at 2015
M-D (Lee 21 al. 2018
91.7
19.7
50.0
58.7
92.2
49.6
90.5
50.1
83.3
66.7
80.8
66.7
81.0
66.7
81.5
66.7
MBF
99.7
99.9
99.3
99.5
98.8
98.3
91.8
98.5
2017)
35.3
TmtgeNet-
48.1
60.1
46.5
66.6
66.7
AlexNet
1,D
Saa 21
2018
71.7
70.9
71,9
72-3
68.7
60.5
65.8
68.3
MBF
99.9
99.6
99.9
99.8
98.6
97.8
98.8
98.4
KL (Femman ot 2017)
57.1
57.7
50:2
58..6
12.8
23.66
11.7
TLS8
ImageNet-
M.D
(Lee
63.7
64.8
45.9
66.9
67.1
65.6
66.7
67.4
VGG16
Matrraal
2016
82.6
89.5
76.7
83.3
76.4
MBF
99.8
100.0
100.0
100.0
99.6
99.6
100.0
100.0
Table 2: Detection results evaluated by AUROC (%) in the attack-transfe case.
databuse
Test altack
Train attack
BIM KD>BU [Feimman et
2017
CW-L2
M-D (Lee ot
2018 )
DeepFool
/LID
Ma ct
2018) MMFF
R-PGD
HIM
190751177773 07.77799999
87.67777799..
CIFAR-10
DeepFool
CW-L2
83.4/47.9/00
/9.7
199.
83.1/5147
82.8/88778
86.
80.6/53.7
87.0 88.6
83.1/53./
R.PGD
78.1/51.5/ 86.1 1996
773//119/
84.5 /99.5
75.8
84.9
88.3
77.3 1/..77 199.4
BIM
11.7779.799177999
83.1731.77
9799
NC.1
ND174488991977
97.3
81.3730
799.3
SVHN
DeepFool
CW-L2
83.6/ 50.///08///66//// /916//997
99.7
/5.1/50.01
86.3/
50.07 87.9/99.8 /99.8
88,7 79.9
84.1 (4.1/49.81 49.6 /9..2/92./99999
82.5 49.7 12200
85.07
83.6
/ 90.8/99.4 99.66
/9988999997
R.PGD
92.5/499//1../// 99.6
49.5 84.01
87.8 / 99.6
81.21
11.2//0//911 97.4
82.5 / 50 90.5 / 99.5
HIM
318728.177..77 71.77999
50NT 97X7742977 70.37 99.6
49.7772.17 99.8
50.4 50.775047
7087999
ImageNett
CW-L2
99.8 51.8/53./713
52.3/50.7
66.16601111 70.9/99.77
$1.2/
99.7
AlexNet
DeepFool
53.5
/7.99/99.
51 71.21/ /
R.PGD
51.3 50 77.3
50.3
BIM
37.1753.7 7 78257 19938
ST.17 51.27 844.7 99.3
$3.8 24,47 99.47
100,0
57.1
475.55777444 25:27 100.0
ImageNet- VGG-16
CW-L2
64.3 / 81 100.0
$7.7 64.8 84.2/ 100.0
53.8
56 /90.2/ 100.0
57,5 166776776 6 84.3 100.0
DeepFool
58.7 / / 99.7
$9.4 46.1 79.1 /993
56.2 45.9/ 89.5 100.0
59.0
44.0 80.9 99.7
R.PGD
58.6 /2.3 / 81.1/ / 100.0
58.9 / 60.8 / 100.0
55.3 54.4 90.8/ 100.0
58.6 66.9 / /84. 100.0
Table 3: Detection results evaluated by accuracy (%) in the dittack-ransste case.
database
Test atlack
Taann sttack
KD+BU [Feinman al. 2017
BIM
CW-L2
M-D [Lee
2018 LID Ma
DeepFool
(2018)) MBF
R-PGD
BIM
14.2766.77 716667772.00 12.67 93.8
15.07
06.7 76.97 97.4
00.77
70.2 87.3
75 77667771117771 996.8
CIFAR-I0
DeepTool
CW-122
74.0166.7 /06.67/80.//9/.
74.2:66.7/00. /00.3/50./99 98.1
73.07
064/6667/77.4/9
98.3
/1.7/667/77.//888
73.8/6.7/ 73.6 98.2
73.4/66.7 74.8 95.4
R-PGD
73.11667/77.1 98.6
71.9166.
/98.5
71.0/667
/60//667/754// 75.44/99
72.61
70.2 98.5
BIM
78.676677785...
98.7
80.37 66.177947788
83.5793. 11.7766778355
78.67
66.
829798.2
SVHN
DeepFool
CW-L2
78.2/66.7/83.5 //8.5/35/98/
12.8/6.778339 / 98.0
80.0/
66.7 / 79.9 99.4
/9.22
79.3 /
77.1
/611/66/////999 /95.9
/9.3
79.9 66.7/ 80.7 98.0
78.1/66.71
80.7 197.3
R-PGD
78.61 68.36667/8366// 98.6
80.5 66.7 79.9 98.9
77.7 66.7 67.7666.783.8 83.8883./...
78.6 81.5 98.8
BIM
67.66666677777.
66.77
65577 98.4
ImageNet-
CW-L2
36.51 66.166.2/66 64.6//9..
368 66.6 69.6/97.8
35.0/
60.7 198.1
36.4
66. 62 98.0
AlexNet
DeepFool
36.01 /9.6/6//./99//99
36.0/ 66.8 65.8
34,9 66.8 65.8 99.8
35.9 66.6 68.3 99.6
R-PGD
35.47 66.76/7..76////// 67.66 /98.5
35.2
/7.77
34.1 66.7 /7.0 /9.0
35.3 66. 68. 98,4
BIM
42.76711 77.8 98.66
42.8 66.87 74,5797.3
38,4766677
7799985
42.7 37
76979884
ImageNet-
CW-L2
43.1 8 75.2 98.0
43.6/ 65.6/ /97.8
/8.4/62.5/ /2.5 98.1
4
98.0
VGG-16
DeepFool
66.7 76.2 98.8
46.1 66.7 97.9
41.8 66.7
45.8 66.7 74.4
R-PGD
66.3 76.3
98.5
45.4/ 06.6/ /97.6
04.6/449/822 82.4 198.3
45.8 76.4 98,4
LID utilizes other benign testing images as neighborhoods to extract features for each testing image. It is unfair to utilize the information of benign or adversarial for neighboring testing images. In our experiments, we use benign training images as neighborhoods for each testing image.
Three comparison cases and evaluation metries We conduct experiments of three cases, includ- ing: 1) non-transfer, both training and testing adversarial examples are crafted by the same attack method: 2) artack-transfer, both training and testing adversarial examples are crafted by different attack methods; 3) data- trats-rr both training and testing adversarial examples same attack method, but the data sources of training and testing benign examples are different. Two widely used metrics are used to evaluate the detection performance, including area under the ceiver operating characteristics (AUROC), and detection accuracy, which is the diagonal summation of the confusion matrix, using 0.5 the threshold of the posterior probability. Higher values of both metrics indicate better performance.
crafted by the
4.3 RESULTS
Detection results in the non-transfer case are shown in Table [ The proposed MBF method shows the best performance in all cases, and is much superior to all compared methods. LID performs the second-best in most case. KD+BU gives somewhat good detection performance and SVHN, but performs very poor on ImageNet. It tells that KD+BU is very sensitive to different databases and networks. M-D gives almost the degenerate results. Note that the results of M-D on
CIFAR-10
7

---

Under review as a conference paper at ICLR 2020
Table Detection results evaluated by AUROC score (%) in the data-transfed case, All detectors are trained on the train set, and tested on the out-of-sample set (including 365 images) of the ImageNet database. The best results are highlighted in bold.
ImageNel-AlexNe
ImageNet- VGG-16
Detector
BIM
CW-L2
DeepPool
R-PGD
BIM
CW-L2
DeepFool
R-PGD
KD+BU
66.3
69.4
64.7
68.8
84.0
85.0
79.0
86.2
M-D
48.7
49.2
50.0
51.0
54.5
50.0
50.4
56.1
LID
69.1
67.6
72.9
71.4
84.2
88.4
89.2
83.7
MBF
99.1
99.0
99.6
99.4
99.3
99.5
99.3
99.5
Table 5: Attack Failure Rates (%) of adaptive attacks with different detectors, See context for details.
Original CW-L2
LID
MBF
CIFAR-10 10
0.0 0.0
88.4
100.0
SVHN
85.6
92.9
CIFAR- 10 and SVHN reported in (Lee et al.
(2088) ) are very high, but their networks (i.e., ResNet et al. (2 and DenseNet (Huang et al. 2017) are different from that used in our experiments. iimpliss that M-D may be suitable for very deep neural networks, but not for shallower networks.
Detection results in the attack-transfer case evaluated by AUROC and accuracy are presented in Tables and respectively. MBF still shows much better performance all transfer cases than all compared methods, and the changes among detecting different attacks very small. It verifies the robustness of MBF to different attacks.
We also conduct a data-transfer experiment on ImageNet. Specifically, collect extra 365 images of 8 classes (same with the classes used for the detection training, see Section 4.2) through searching the class names in Baidu and Facebook. These 365 benign and their noisy images can be correctly predicted by both the fine-tuned AlexNet and VGG models. The results on these images are shown Table MBF still shows the best performance. And, compared to the corresponding results in Table 1 the AUROC/accuracy scores of MBF on detecting different attacks change very gently. It verifies the robustness of MBF to different data sources.
4.4 ADAPTIVE ATTACK AGAINST ADVERSARIAL DETECTION
Similar to Carlini & Wagner (211b)) and Ma et al. (2018), we also present an adaptive white-box attack, of which the goal is not only to fool the classifier, but also to evade the adversarial detector. It measures how easy to bypass the detector. Specifically, by combining the CW-L2 attack (Carlini
& Wagner (017aa) and the proposed MBF detector, the adaptive attack formulated as follows:
arg arg min ||x - X + a I(fo (x.f(() 3) + [D(ff(Xxaa)) - D(fo (x))
(9)
where x denotes the clean input, y indicates the ground-truth label. 1(xadr: y) is the Hinge loss. ((fo(Xxdv) represents the extracted features by the the detector D (specified later), from (X The minimization of the & term ((((o(Xxdv) - (Je (x)) | encourages the detection features of adversarial and benign examples be close, such that the detector could be evaded.
We conduct the above adaptive attack on both CIFAR-10 10 and SVHN. Specifically, 1000 clean ex- amples are randomly selected from the set of each database. For each generated adversarial example, if it fails to fool the classifier fe. we record 1. The attack failure rate over all 1000 adver- sarial examples is computed as the metric. The higher rate indicates that it more difficult to evade the detector. Since LID performs much better than KD+BU and M-D in above experiments, here we only compare LID and the proposed MBF detector. For clarity. the detection features are only ex- tracted from the soft-max layer for both LID and MBF. The original CW-L2 attack (i.e., without the &, term) is also presented as the baseline. With the same setting (Ma et al. (2018)), the trade-off parameter is determined by binary search within the range [10- "00] The results are shown in Table 5) The attack failure rates of MBF on both databases are much higher than those of LID. tells that it is more difficult evade MBF than LID.
4.5 HYPOTHESTS TEST VIA KOL.MOGOROV-SMIRNOO TEST
Kolmogorov-Smirnoo test (KS test) Massey Jr (195T)) )) is a non-parametric test method statis- tics, to test whether sample follows reference probability distribution (one-sample KS test), or
8

---

Under review as a conference paper at ICLR 2020
The p-value of KS hypothesis test among clean samples, noisy samples, and adversarial samples crafted by four methods, respectively, on ImageNet-AlexNet.
Table 6:
ImageNee-VGG-16
clean
noisy
BIM
CW-L2
DeepFool
R-PGD
train set
0.114
0.114
0.236
0.228
0.240
0.235
test set
0.114
0.114
0.231
0.228
0.239
0.232
Table 7: The p-value of two-sample KS test among MBF coefficients of different types of examples.
MoggeNet-VGG-II (clean, noisy)
(clean, BIM)
(HIM, DeepFool) (CW-1.2, R-PGD)
train set test set
0.998
0.000
0.000
0.236
1.000
0.000
0.000
0.225
Table 8: The p-value of KS hypothesis test among MBF coefficients of different data sources.
ImageNet- VGG-16
clean
noisy
BIM
CW-L2
DeepFool
R-PGD
(train set, test set)
0.839
0.858
0.368
0.842
0.246
0.221
(train set, out-of-sample set)
0.049
0.054
0.094
0.407
0.737
0.088
whether two samples follow the same distribution (two-sample KS test). Specifically, in one-sample KS test, the distance between the empirical distribution function of one sample and the cumulative distribution function of the reference probability distribution is measured. Then, the p-value corre- sponding to the obtained distance is computed. If the p-value is larger than the significance level a (here we set a 0.05), then the null hypothesis that the sample follows the reference distribution is accepted; otherwise, rejected. Similarly, in two-sample KS test, the distance between the empir- ical distribution functions of two samples computed. a, then it accepts that two samples follow the same distribution. The KS test conducted below is implemented by the python function
the corresponding p-value larger than
stals.k. ks 2_22smm
Hypothesis test 1. Here we verify that whether the posterior vectors of both adversarial and benign examples follow GGD. We denote the posterior vector of one adversarial example as Padvs and that of one benign example as Pben The distribution of GGD is denoted as PoaD. Then, we conduct two one-sample KS tests, including:
H1.1 The test of adversarial examples: Ho: Padv PGGD-advi H1: Padv Jf PGGD-adv.
H1.2 The test of benign examples: Ho: Phen ~ PGGDDhen' H1: pben of PGGD.hen-
In H1.1, the reference distribution PoGD adv is firstly estimated from Padv- using the estimated method proposed in (Lasmar et al. (2009)) ) To alleviate the uncertainty of the estimation, we draw 500 samples from the estimated PoGD adv- Then, and these 500 samples respectively. The average p-value over 1000 tests is recorded. The mean of the average p-values over the whole database is reported. H1.2 conducted similarly. The results tested on ImageNet-Alexte and models will be presented in the supplementary material. In all cases, the p-values are larger than the significance level 0.05. Hence, we can conclude that the posterior vectors of both adver- sarial and benign examples follow GGD. However, note that the parameters of their corresponding GGD are different, which will be verified in the following test.
conduct the two-sample KS test between Padv
are shown in Table 6 Due to the space limit, results on other databases
Hypothesis test 2. Here we verify that whether the extracted MBF features of adversarial and benign examples follow the same empirical distribution. We denote the MBF feature vector of one adversarial example Madv, and the corresponding empirical distribution is denoted as Padv-MBF- Similarly, we define mben and Pren. MBr for benign examples. Then, we conduct the following four two-sample KS tests:
H2.1 The test between adversarial and benign examples: Hy: PadvvMBBF P Peen-MBri H1
P Palv-MFF A Mar
H2.2 The test between adversarial examples crafted from different attack methods: Ho:
Pattack-1 Pattack22
Md+ : H1: Partack11 # Pattack-2
adv-MBF
MBF
H2.3 The test between adversarial examples from different data sources (i.e., train, test and
out-of-sample set): Hg: Pssonce- psource-2.
H1: Psoureo-1 # Pssource-2
hsamp.html

---

Under review as a conference paper at ICLR 2020
(a) clean
(b) noisy
(c) BIM
(d) CW-L2
(c) DecpFool
(f) R-PGD
Figure 1: Statistics (mean + standard deviation) of MBF coefficients on train (top row), test (median
row), and out-of- (bottom row) set of MmageNet-VGG-JJ
H2.4 The test between benign examples from different data sources (i.e., train, test and out-of-samplc set): Ho: P Pbea H1:
Phen- MBF # P
In above four two-sample KS tests, we test on the 16-dimensional MBF features extracted from the soft-max layer. Since the implementation scipy.stats.c ks 2samp cannot compare two vectors, compare the feature of each dimension separately, then report the average p-value over all dimen- sions. Specifically, when comparing two sets of samples, dimension across all samples the same set, leading to 16 long vectors for each set. Then, each pair of two long vectors corresponding to the same dimension from two sets is compared by KS test. The average p-value over all 16 dimensions is reported. The p-values of H2.1 are shown in Table (see the column "(clean, BIM)"). The p-values both train and test set are 0. Thus, the hypothesis Ho is rejected, i.e., the MBF features of adversarial and benign examples follow different distributions. The p-values of H2.2 are shown in Table We pick two groups of attack methods, i.e., (BIM, DeepFool) and (CW-L2, R-PGD). The p-values of (BIM, DeepFool) are O, while the p-values of (CW-L2, R-PGD) are larger than 0.05. It demonstrates that the distributions of adversarial examples crafted from different attack methods are possible to be different. The p-values of H2.3 are shown Table 8 The p-values all types of adversarial examples exceed 0.05. Thus, the MBF features of adversarial examples from different data sources follow the same distribution. The p-values of H2.4 are shown in Table g Only the p-vuuue of "(train set, out-of sample set)" of clean examples is exceed 0.05. Thus, in most cases, the MBF features of benign examples from different data sources follow the same distribution.
firstly concatenate the feature of each
slightly lower than 0.05, while the values of other
From above analysis, obtain the following conclusions: 1) The extracted MBF features of adver- sarial and benign examples follow different empirical distributions. explains why MBF features effective for detecting adversarial and benign examples; 2) The extracted MBF features of ad- versarial/benign examples from the train, test and out-of-sample sets follow the same empirical distribution. Although the extracted MBF features of adversarial examples crafted from different attack methods may not follow the same empirical distribution, the significant difference between benign and different adversarial distributions can still lead the good detection performance in the attack-transfer case. It explains why MBF features are robust across different attack methods and different data sources. Moreover, we visualize the statistics of each dimension MBF features, i.e., mean and standard deviation, as shown in Fig. [ These visualizations also support above conclu- sions. Due to space limit, more KS tests and visualizations on different databases and networks will be presented in the supplementary material.
CONCLUSION
This work has proposed novel adversarial detection method, dubbed MBE. The assumption behind is that the internal responses of the classification network of both adversarial and benign examples follow the generalized Gaussian distribution (GGD), but with different shape factors. The magni- tude of Benford-Fourier coefficient is function w.r.t. the shape factor, and can be easily estimated based on responses. Thus, it can serve as the discriminative features between adversarial and be- nign examples. The extensive experiments conducted several databases, well as the empirical analysis via KS test, demonstrate the superior effectiveness and robustness to different attacks and different data sources of the proposed MBF method, state-of-the-an detection methods.
10

---

Under review as a conference paper at ICLR 2020
REFERENCES
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy, pp. 39-57. IEEE, 2017a.
Nicholas Carlini and David Wagner. Adversarial examples detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3-14. ACM, 2017b.
easily detected: Bypassing
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255, Ieee, 2009.
Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. Detecting adversarial samples from artifacts. arXiv preprint 1703.00100 2017.
Ian J Goodfellow, Jonathon Shlens. and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6577 122.6572, 2014.
Nathaniel R Goodman. Statistical analysis based on certain multivariate complex gaussian distri- bution (an introduction). The Annals of mathematical statistics, 34(1):152-177, 1963.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDaniel. On the (statistical) detection of adversarial examples. arXiv preprint DrXiv:/702.06280, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. Early methods for detecting adversarial images. arXiv preprint arXiv:1608.005530 2016.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. http://www.cs.trront
The cifar-10 dataset.
online:
http://wwwwc.toooooccccckdkthcaa edukriidcifarrhmml, 55, 2014.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv: 1607.02533, 2016.
Nour-Eddine Lasmar, Youssef Stitou, and Yannick Berthoumieu. Multiscale skewed heavy tailed model for texture analysis. In 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2281-2284. IEEE, 2009.
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Process- ing Systems, pp. 7167-7177, 2018.
Xin Li and Fuxin Li. Adversarial examples detection in deep networks with convolutional filter statistics. Proceedings of the IEEE International Conference on Computer Vision, pp. 5764- 2772,2017,
Jiajun Lu, Theerasit Issaranon, and David Forsyth. Safetynet: Detecting and rejecting adversarial examples robustly. In Proceedings of the IEEE International Conference on Computer Vision, pp. 446-454, 2017.
Xingjun Ma, Bo Li, Yisen Wang. Sarah M Erfani, Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In International Conference Learning Representations, 2018. 88
11

---

Under review as a conference paper at ICLR 2020
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 2017.
Frank J Massey Jr. The kolmogorov-smimov test for goodness of fit. Journal of the American statistical Association, 46(253):68-78, 1951.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. On detecting adversarial perturbations. In International Conference on Learning Representations, 2017.
Seyed-Mohsen Moosavi-Dezfooll Alhussein Fawzi, and Pascal Frossard. Deepfool: simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 2574-2582, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Tianyu Pang, Chao Du, Yinpeng Dong, and Jun Zhu. Towards robust detection adversarial exam- ples. In Advances in Neural Information Processing Systems, pp. 4579-4589, 2018.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European Sympo- shum on Security and Privacy, pp. 372-387. IEEE, 2016.
A. Papoulis and S.U. Pillai. Probability, Random Variables, and Stochastic Processes. McGraw- Hill Series in Electrical and Computer Engineering. McGraw-Hill, 2001. ISBN 9780072817256. URL hhttps://books google. com. hwwbokkkbkkk.ec.oomooo.kkk.kk.k. 1atp://oooooodaaaaaaaaaaaaaaaaaa
d=Ma 1=GAAAAAAA
Cecilia Pasquini, Fernando Pérez-González, and Giulia Boato. A benford-fourier jpee compression detector. In 2014 IEEE International Conference Image Processing, pp. 5322-5326. IEEE, 2014.
Karl Pearson. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11):559-572, 1901.
Fernando Perez-Gonzälez Greg L Heileman, and Chaouki T Abdallah. Benford's lawin image processing. In 2007 IEEE International Conference on Image Processing, volume 1, pp. I-405. IEEE, 2007.
Murray Rosenblatt. central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences of the United States of America, 42(1):43, 1956.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprini 180.:06805.06605, 2018.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 2013.
Mahesh K Varanasi and Behnaam Aazhang. Parametric generalized gaussian density estimation. The Journal of the Acoustical Society of America, 86(4):1404-1415, 1989.
Zhihao Zheng and Pengyu Hong. Robust detection of adversarial attacks by modeling the intrinsic properties of deep neural networks. In Advances in Neural Information Processing Systems, pp- 7913-7922, 2018
12

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1 PROOF OF THEOREM 1
Proof Assume that all the variables L.o. are independent and identically distributed. Applying the central limit theorem Rosenblati (1956) to the real and imaginary parts of y. we can obluin that both parts asymptotically follow the Gaussian distribution
y
M Σ
Tm (E((T)), 1 D(TT)), M
(10)
where
E(T)) = E (e ( lok1o/X.l) (
+000
Pxi (x) - -5aan -22nnlosoolull =
(11)
(2gtoo0g122211 19) (
D(T)) = - -IE B ((TT
10n 12 1 -
|0n 2
(12)
The PDF y can be rewritten as follows:
1 1 lan
y ~ N(an:
(13)
M
Thus, we obtain
E ~ N(0, 1 |an!
(14)
M
Besides, the pseudo variance Goodman (1633) of
JT. -ETi)² = E(e -2nm logio| 12 (2))
- =a2n -d,
(15)
Correspondingly.
a2n a
Je.8 = M JT..I
(16)
M
Since (a2n -a))) bounded, we have lim Je.g Thus, we obtain that the random variable E follows circularly-se syymmarty-ymmmtiiccccc.c complex Gaussian distribution, because the sufficient and necessary condition that mean value and pseudo variance equal zero Goodman (163)). It implies that both the real and imaginary part of & follow the same Gaussian distribution and they are independent. Thus, the magnitude of this complex random variable follows the Ruyleigh distribution Papoulis & Pillai 2001). and the probability density function of [|] can be formulated as
Pig (r)
52 ---0
(17)
where is the scale parameter. Knowing the properties of Rayleigh distributionn & Pillai 2001), we have:
1 D() = 1- 1-lan s² = =
(18)
2M
Utilizing the fact that |a,," 2 is close to when a modest number, we obtain that D ( leading to 82 2M Then, we obtain
1
M
1 1/T D((E)) 4M
E([E))
(19)
21 M
is easy to observe that both E( ((1)) and D((E)) are close to zero when the number of samples M increases. It implies that the estimation error en gets closer O as M increases.
13

---

Under review as a conference paper at ICLR 2020
Table 9: The p-value of KS hypothesis test among clean samples, noisy samples, and adversarial samples crafted by four methods, respectively.
clean
noisy
BIM
CW-L2
DeepFool
R-PGD
train set
0.074
0.074
0.158
0.139
0.155
0.159
CIFAR-10
test set
0.072
0.072
0.158
0,138
0.216
0,216
train set
0.104
0.104
0.249
0.194
0.222
0.245
SVHN
test set
0.106
0.106
0.249
0.193
0.219
0.239
train set
0.116
0.116
0.253
0.235
0,242
0.253
AmegeNet-AlexNet
test set
0.117
0.117
0.261
0.236
0.243
0.259
ImageNet- VGG-16 16
train set
0.114
0.114
0.236
0.228
0.240
0.235
test set
0.114
0.114
0.231
0.228
0.239
0.232
Table 10: The p-value of two-sample KS test among MBF coefficients of different types of examples.
(clean, noisy)
(clean, BIM)
(BIM, DeepFool)
(CW-L2, R-PGD)
CIFAR-10
train set
0.640
0.000
0.000
0.003
set 0.764
0.000
0.030
0.000
train
0,559
0.000
0.030
0.000
SVHN
test set
0.685
0.000
0.000
0.000
IlleeNet-AleXNet
train set
1.000
0.000
0.002
0.134
test set
0.993
0.000
0.477
0.000
ImageNet-VGG-16
train
0.998
0.000
0.000
0.236
test set 1.000
0.000
0.000
0.225
Table 11: The p-value KS test among MBF coefficients from different data sources.
clean
noisy
BIM
CW-L2
DeepFool
R-PGD
CIFAR-10 10
(train set, test set)
0.120
0.188
0.333
0.456
0.584
0.223
SVHN
(train set, test set)
0.494
0.708
0.523
0.147
0.491
0.645
ImageNet-
(train set, test set)
0.153
0.148
0.370
0.605
0.410
0.386
AlexNet
(train set, out-of-sample)
0.000
0.000
0.169
0.101
0.206
0.158
ImageNet-
(train set, test set)
0.839
0.858
0.368
0.842
0.246
0.221
VGG-16
(train set, out-of-sample)
0.049
0.054
0.094
0.407
0.737
0.088
(a) clean
(b) noisy
(c) BIM
(d) CW-L2
(e) DeepFool
( R-PGD
Figure 2: Statistics (mean + standard deviation) of MBF coefficients train (top row) and test
(bottom row) set of CIFAR-10.
(a) clean
(b) noisy
(c) BIM
(d) CW-L2
(c) Decptoool
(6) R-PGD
Figure 3: Statistics (mean + standard deviation) of MBF coefficients on train (top row) and test
(bottom row) set of SVHN.
14

---

Under review as a conference paper at ICLR 2020
W
W
(a) clean
(b) noisy
(c) BIM
(d) CW-L2
(c) DecpFool
(f) R-PGD
Figure 4: Statistics (mean + standard deviation) of MBF coefficients on train (top row) and test
(median row), and out-of- sample (bottom row) set of AleeNe.
Detection results evaluated by AUROC score (%) on nearly 4000 images from the testset of MNIST. The best results are highlighted in bold.
Table 12:
Iteration number each GD run
Number of GD runs
BIM
CW-L2
DeepFool
R-PGD
200
10
0.744
0.980
0.702
0.746
10
0.689
0.973
0.650
0.690
100
5
0.664
0.969
0.628
0.666
100
2
0.612
0.959
0,586
0.614
MBF
1.000
0.991
1.000
1.000
A.2 ADDITIONAL EMPIRICAL ANALYSIS
Here we present additional empirical analysis on more databases and networks, as shown in Tables 10 II The p-values in most cases also supports the conclusions obtained in the main manuscript. We also present more visualizations in Figs. and These visualizations also demonstrate the distinct difference of MBF features between adversarial and benign examples.
A.3 COMPARISON BETWEEN DEFENSE-GAN
We adopt official code 3 of Defense- GAN (Samangouei al. (0088) and train generative model on train set of MNIST dataset after 130, 000 epochs (As the official code Defense-Gar GAN currently only supports three datasets including MNIST, Fashion-MNIST. and CelebA). The classification model attains an accuracy of 98.7% on the test set of MNIST dataset. We still utilize four attack strategies, including BIM, CW-1-2, DeepFool and R-PGD. After removing images that misclas- sified and failed to attack, we pick 6000 images to train SVMs based on our MBF features, and leave nearly 4000 images for testing. There are key hyper-parameters in Defense-GAN including the iteration number in each GD run, and the total number of GD runs. We have tried multiple settings of these two ayper-parametend Table show the AUROC score on nearly 4000 images. Our MBF method exceeds Defense-Gaid by a large margin in quantitative performance.
A.4 DETAILED PARAMETERS OF ADVERSARIAL ATTACK
Four popular adversarial attack methods are adopted to craft adversarial examples, including ba- sic iterative method (BIM), CarliniWagnerL gradient descent (R-PGD). We emphasize detailed parameters of these attack strategies for repro- ducibility, which are BIM (eps=0.3. stepsize=0.05, iterations=10), CW-L2 (binary_searde swwrrc_sstesss____ confidence=0.0, Bouriinggrtt=:::00 0005, max nuaxiitratioos==1000 DeepFool (max sto)xsstts=100), R-PGD (cps=0.3, stepsize=0.01, iterations=40). All of the attack strategies are implemented by Foolbox.
(CW-L2), DeepFool, and random projected
http://gthiiccmmkaaaaa//feeeenn
15


Reviews:
[{"review_id": "SyeSm3HiYr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an adversarial detection method via Fourier coefficients. The proposed method seems promising, and empirical evaluations are reasonable.\n\nHowever, I find that the proposed MBF detection metric is much more complicated to calculate than any of its baselines, e.g., LID or K-density. So I wonder if the good performance of MBF mainly comes from its ’complexity‘. I mean, if we use the feature vectors of different layers in CNNs and combine them with a different non-linear function and feed into an SVM classifier,  can we still obtain a hard-to-evade detector? I think a fair complexity is particularly important when you try to evade the detector by optimization-based adaptive attacks and claim superiority over other baselines.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "Skx3wFCmcr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a new discriminator metric for adversarial attack's detection by deriving the different properties of l-th neuron network layer on different adv/benign samples. This method can achieve good AUC score comparing to other start-of-art detection methods and also achieve good robustness under corresponding adaptive attack. The framework is clear and the experiment is solid.\n\nHowever, I have several concerns:\n\nMajor:\n1. It seems that the whole process assumes that there is difference for the parameters in the environment of GGD with adv/benign samples, and the goal is to search for the major components of it and use a classifier to detect. To extract the approximation of parameters,  the authors use the \"response entries\" of l-th layer for several observations => this means the authors regard all the \"response entries\" of one layer as different samples on one certain GGD. This makes me feel a bit tricky, and it would be great if you the authors can provide some evidence or explanation here.\n\n2. In the experiment's remark, the authors mentioned that the mean parameter of GGD is set to 0 and most of them are actually close to 0 (around 1e-2) so the assumption is right. However 1e-2 is not a value \"very close to zero\" and it would be great to show / explain the variance here.\n\n3. I can't find the parameter of your evaluated attack method (like confidence, eps, etc.) Please also provide experimental details for reproducibility. \n\nMinor:\n\nHere are some reference error (e.g. P6, \"For each database, as described in Section ??\"). Please fix that.\n\nOverall, this paper is a interesting based on the performance of detection. But the  assumptions made by the paper are a bit confusing and it would be good to clarify and provide clarification for them. Authors should explain the assumptions and give some extra experiment results if needed.\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "SkeujiMl5H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an approach to adversarial detection.  The approach first computes a representation of the activation layers using the Benford-Fourier coefficients.  One then generates a range of noisy instances, and trains an SVM using those noisy instances as supervised labels (e.g., noisy instances are adversarial).  The SVM uses the Benford-Fourier coefficients of the activation layer as the input features.  The results show good performance against some baselines such as LID.\n\nI'm not really an expert in this area, but I'm a bit surprised that LID is considered the baseline to beat.  I imagine that most adversarial defense approaches are for robust prediction, rather than detection.  It also seems the authors chose to compare with defenses that are computationally cheaper (so not RCE or Defense-GAN), but a study of computational trade-offs is absent in the paper.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new metric for adversarial attack's detection. The reviewers find the idea interesting, but the some part has not been clearly explained, and there are questions on the reproducibility issue of the experiments. 
Decision: Reject