Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved a remarkable success in many important applications , such as image classification , face recognition , object detection , etc . In the meanwhile , DNNs have been shown to be very vulnerable to adversarial examples . However , many real-world scenarios have very restrictive requirements about the robustness of DNNs , such as face verification for login , or semantic segmentation in autonomous driving . Adversarial examples are a serious threat to the applications of DNNs to these important tasks . Since many kinds of adversarial attack methods have been proposed to fool DNNs , it is more urgent to equip effective defensive strategies to ensure the safety of deep models in real-world applications . However , defense seems to be more challenging than attack , as it has to face adversarial examples from unknown crafting methods and unknown data sources . Typical defensive strategies include adversarial training , adversarial de-noising , and adversarial detection . Compared to the former two strategies , adversarial detection is somewhat more cost-effective , as it often does not need to re-train or modify the original DNN model . There are two main challenges for adversarial detection . ( 1 ) The adversarial examples are designed to camouflage themselves to be close to the corresponding benign examples in the input space . Then , where and how to extract the discriminative information to train the detector ? ( 2 ) The data sources and the generating methods of adversarial examples are often inaccessible to the detector . In this case , the detector can be stably effective across different data sources and different attack methods ? In other words , a good adversarial detector is required to be not only effective to distinguish adversarial and benign examples , but also robust to different data sources and attack methods . To satisfy the first requirement of effectiveness , we utilize the other principle of crafting adversarial examples that the outputs between benign and adversarial examples should be large , to encourage the change of the final prediction . It means that the imperceptible difference between benign and adversarial examples in the input space is enlarged along the DNN model , leading to the significant difference in the output space . Inspired by this fact , we assume that the output or the responses of internal layers of the DNN model should include the discriminative information for benign and adversarial examples . A few works have attempted to extract different types of discriminative features from the output or the internal responses , such as kernel density ( KD ) ( Feinman et al . ( 2017 ) ) and the local intrinsic dimensionality ( LID ) ( Ma et al . ( 2018 ) ) , etc . To achieve the robustness , the extracted discriminative features should model the intrinsic difference between adversarial and benign examples , rather than the difference from the changes of data sources or attack methods . Many existing methods have shown the effectiveness to some extent on detecting adversarial examples of specific data sources and attack methods . However , their robustness , especially across different data sources , has not been well studied and verified . In this work , we propose a novel detection method based on the assumption that the internal responses of both adversarial and benign examples follow the generalized Gaussian distribution ( GGD ) ( Varanasi & Aazhang ( 1989 ) ) , but with different parameters , including the shape factor , mean , and variance . The rationale behind this assumption is that GGD covers many popular distributions with varied shape factors ( such as Laplacian , Gaussian , or uniform distribution ) , such that GGD is more likely to approximate the intrinsic response distributions rather than one specific distribution . Moreover , mean and variance of GGD may vary significantly with respect to different classes and data sources , even for benign examples , while the shape factor is more robust . For example , the mean and variance of two Gaussian distributions could be totally different , but their shape factors are the same ( i.e. , 2 ) . Thus , we propose to use the shape factor as an effective and robust discrimination between adversarial and benign examples . However , it is difficult to exactly estimate the shape factor in practice . We resort to the magnitude of Benford-Fourier coefficients ( Pasquini et al . ( 2014 ) ) , which is a function of the shape factor . It can be easily estimated using internal responses , according to the definition of Fourier transform . Then , the magnitudes estimated from internal responses of different convolutional layers are concatenated as a novel representation . Finally , a support vector machine ( SVM ) ( Vapnik ( 2013 ) ) is trained using the new representations as the adversarial detector . Extensive experiments carried out on several databases verify the effectiveness and robustness of the proposed detection method . To further verify the rationale of our assumption , we present the empirical analysis through the Kolmogorov-Smirnov test ( KS test ) ( Massey Jr ( 1951 ) ) . The KS test verifies that 1 ) the posterior vectors of both adversarial and benign examples predicted by the CNN model follow the distribution of GGD , but with different parameters , and 2 ) the MBF features of adversarial and benign examples follow different distributions , and the MBF features of adversarial examples crafted from different attack methods follow the same distribution , as well as that the MBF features of adversarial/benign examples from different data sources follow the same distribution . Moreover , we visualize the statistics ( i.e. , mean ± standard deviation ) of the extracted MBF features for adversarial and benign examples . The visualization reveals the distinct difference between adversarial and benign examples . These empirical analysis demonstrates the effectiveness and robustness of the proposed MBF detector . 2 RELATED WORK . In this section we focus on reviewing existing works about adversarial detection , while those of other adversarial defensive strategies are out of scope of this work . The general idea of most existing detection methods is learning or constructing a new representation to discriminate adversarial and benign examples , utilizing the outputs or immediate responses of an original classification network . Li & Li ( 2017 ) trained a cascading classifier based on the principal component analysis ( PCA ) ( Pearson ( 1901 ) ) statistics of responses from each convolutional layer of the defended convolutional neural network ( CNN ) model . An example is recognized as benign if all single classifiers of the cascade predict it as benign , otherwise adversarial . Lu et al . ( 2017 ) proposed SafetyNet by adding a RBF-SVM classifier to detect adversarial examples , at the end of the original classification network . Metzen et al . ( 2017 ) proposed a detection network along with an original classification network , which takes the internal responses of the original network as inputs . It shows effectiveness on detecting adversarial examples generated by simple attacks ( such as FGSM ( Goodfellow et al . ( 2014 ) ) and JSMA ( Papernot et al . ( 2016 ) ) ) , while performs much worse when facing more advanced attacks ( such as C & W ( Carlini & Wagner ( 2017a ) ) ) . It tells that this method is sensitive to attack methods . Grosse et al . ( 2017 ) attempted to detect adversarial examples using the statistical test of maximum mean discrepancy ( MMD ) . Although above detection methods show effectiveness on some attack methods and some databases , but a thorough evaluation presented in Carlini & Wagner ( 2017b ) has shown that these methods are sensitive to attack methods or databases , and they can be somewhat easily invaded by new attacks . Some recent works proposed to utilize neighboring samples in the same database to construct a better representation of a current sample . Feinman et al . ( 2017 ) defined two metrics based on the responses of the final hidden layer of the classification neural network , including kernel density estimation ( KDE ) and Bayesian neural network uncertainty ( BU ) . If the metric score of KDE/BU is lower/higher than a pre-defined threshold , then the example is predicted as adversarial . Ma et al . ( 2018 ) utilized the local intrinsic dimensionality ( LID ) to measure the characterization of adversarial regions of DNNs . LID describes the distance between one example and its k-nearest neighboring sample in the feature space of immediate responses of the original classification network . The distances computed from different layers are concatenated as the example representation , which is then used to train a shallow classifier to discriminate adversarial and benign examples . Zheng & Hong ( 2018 ) defined the intrinsic hidden state distribution ( IHSD ) of the responses of the original classification network to model different classes . The Gaussian mixture model ( GMM ) was used to approximate IHSD of each class . Then , the posterior probability of one sample assigned to GMM is computed as the metric . If the probability is lower than a pre-defined threshold , then it is recognized as adversarial . Lee et al . ( 2018 ) computed the class-conditional Gaussian distribution of the responses of the original classification network based on the whole training set . Then , the Mahalanobis distance between one sample and its nearest class-conditional Gaussian distribution is used as the metric for detection . If the distance is larger than a pre-defined threshold , then it is detected as adversarial . Compared to some aforementioned single-representation-based detection methods , these joint-representation-based methods showed better performances on some databases . However , the detection cost for each example is much higher , as the responses of its neighboring samples should also be computed . Besides , since the representation is highly dependent on the neighbors or all training examples , the detection performance may be sensitive to data sources , which will be studied in later experiments . There are also some other approaches that do not construct representations from the responses of the original classification network . Hendrycks & Gimpel ( 2016 ) adopted PCA statistics to discriminate adversarial and benign images , independent of any DNN model . However , the study presented in Carlini & Wagner ( 2017b ) has demonstrated that this method works for MNIST but not for CIFAR-10 , and PCA statistics are not robust features to detect adversarial images . Pang et al . ( 2018 ) proposed a novel loss called reverse cross-entropy ( RCE ) to train the classification network , such that the distance measured by kernel density ( Feinman et al . ( 2017 ) ) between adversarial and benign examples could be enlarged . Samangouei et al . ( 2018 ) proposed Defense-GAN to model the distribution of benign examples using a generative adversarial network ( GAN ) . If the Wasserstein distance between one example and its corresponding example generated by the GAN is larger than a pre-defined value , then it is detected as adversarial . However , these above three methods are much more costly than other detection methods . 3 PRELIMINARIES . 3.1 GENERALIZED GAUSSIAN DISTRIBUTION . Assume that a random variable X ∈ Rd follows the generalized Gaussian distribution ( GGD ) ( Varanasi & Aazhang ( 1989 ) ) . Then , its probability density function ( PDF ) is formulated with two positive parameters , including the shape factor c and the standard deviation σ , as follows PX ( x ) = A · e−|βx| c , ( 1 ) where β = 1σ ( Γ ( 3/c ) Γ ( 1/c ) ) 1 2 and A = βc2Γ ( 1/c ) , with Γ ( · ) being the Gamma function . Note that the mean parameter µ is omitted above , as µ has no relation with the shape of distribution and we set it as 0 without loss of generality . A nice characteristic of GGD is that it covers many popular distributions with varied shape factors . For example , when c = 1 , then it becomes the Laplacian distribution ; when c = 2 , then it is the Gaussian distribution with a variance of σ2 ; when c → +∞ , then it is specified as a uniform distribution on ( − √ 2σ , √ 2σ ) .

Summary: This paper presents a new discriminator metric for adversarial attack's detection by deriving the different properties of l-th neuron network layer on different adv/benign samples. This method can achieve good AUC score comparing to other start-of-art detection methods and also achieve good robustness under corresponding adaptive attack. The framework is clear and the experiment is solid.