Bandlimiting Neural Networks Against Adversarial Attacks

1 INTRODUCTION . Although deep neural networks ( DNN ) have shown to be powerful in many machine learning tasks , Szegedy et al . ( 2013 ) found that they are vulnerable to adversarial samples . Adversarial samples are subtly altered inputs that can fool the trained model to produce erroneous outputs . They are more commonly seen in image classification task and typically the perturbations to the original images are so small that they are imperceptible to human eye . Research in adversarial attacks and defences is highly active in recent years . In the attack side , many attacking methods have been proposed ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ; Papernot et al. , 2016a ; Papernot et al. , 2017 ; Moosavi-Dezfooli et al. , 2016 ; Kurakin et al. , 2016 ; Madry et al. , 2017 ; Carlini and Wagner , 2017a ; Chen et al. , 2017 ; Alzantot et al. , 2018 ; Brendel et al. , 2017 ) , with various ways to generate effective adversarial samples to circumvent new proposed defence methods . However , since different attacks usually are effective to different defences or datasets , there is no consensus on which attack is the strongest . Hence for the sake of simplicity , in this work , we will evaluate our proposed defence approach against four popular attacks for empirical analysis . In the defence side , various defence mechanisms have also been proposed , including adversarial training ( Rozsa et al. , 2016 ; Kurakin et al. , 2016 ; Tramèr et al. , 2017 ; Madry et al. , 2017 ) , network distillation ( Papernot et al. , 2016b ) , gradient masking ( Nguyen and Sinha , 2017 ) , adversarial detection ( Feinman et al. , 2017 ) and adding modifications to neural networks ( Xie et al. , 2017 ) . Nonetheless , many of them were quickly defeated by new types of attacks ( Carlini and Wagner , 2016 ; 2017b ; c ; a ; Athalye et al. , 2018 ; Athalye and Carlini , 2018 ; Alzantot et al. , 2018 ) . Madry et al . ( 2017 ) tried to provide a theoretical security guarantee for adversarial training by a min-max loss formulation , but the difficulties in non-convex optimization and in finding the ultimate adversarial samples for training may loosen this robustness guarantee . As a result , so far there is no defence that is universally robust to all adversarial attacks . Along the line of researches , there were also investigations into the properties and existence of adversarial samples . Szegedy et al . ( 2013 ) first observed the transferability of adversarial samples across models trained with different hyper-parameters and across different training sets . They also attributed the adversarial samples to the low-probability blind spots in the manifold . In ( Goodfellow et al. , 2014 ) , the authors explained adversarial samples as ” a result of models being too linear , rather than too nonlinear. ” In ( Papernot et al. , 2016 ) , the authors showed the transferability occurs across models with different structures and even different machine learning techniques in addition to neural networks . In summary , the general existence and transferability of adversarial samples are well known but the reason of adversarial vulnerability still needs further investigation . Generally speaking , when we view neural network as a multivariate function f ( x ) of input x , if a small imperceptible perturbation ∆x leads to a huge fluctuation ∆f ( x ) , the large quantity ∆f ( x ) /∆x essentially corresponds to high frequency components in the Fourier spectrum of f ( x ) . In this paper , we will start with the Fourier analysis of neural networks and elucidate why there always exist some decaying but nonzero high frequency response components in neural networks . Based on this analysis , we show that neural networks are inherently vulnerable to adversarial samples due to the underlying model structure . Next , we propose a simple post-averaging method to tackle this problem . Our proposed method is fairly simple since it works as a post-processing stage of any given neural network models and it does not require re-training the networks at all . Furthermore , we have evaluated the post-averaging method against four popular adversarial attacking methods and our method is shown to be universally effective in defending all examined attacks . Experimental results on the ImageNet and the CIFAR-10 datasets have shown that our simple post-averaging method can successfully defend over 80-96 % of the adversarial samples generated by these attacks with little performance degradation ( less than 2 % ) on the original clean images . 2 FOURIER ANALYSIS OF NEURAL NETWORKS . In order to understand the behaviour of adversarial samples , it is essential to find the Fourier transform of neural networks . Fortunately , for some widely used neural networks , namely fully-connected neural networks using ReLU activation functions , we may explicitly derive their Fourier transform under some minor conditions . As we will show , these theoretical results will shed light on how adversarial samples happen in neural networks . 2.1 FOURIER TRANSFORM OF FULLY-CONNECTED RELU NEURAL NETWORKS . As we know , any fully-connected ReLU neural networks ( prior to the softmax layer ) essentially form piece-wise linear functions in input space . Due to space limit , we will only present the main results in this section and the proofs and more details may be found in Appendix . Definition 2.1 . A piece-wise linear function is a continuous function f : Rn −→ R such that there are some hyperplanes passing through origin and dividing Rn into M pairwise disjoint regions Rm , ( m = 1 , 2 , ... , M ) , on each of which f is linear : f ( x ) = w1 · x x ∈ R1 w2 · x x ∈ R2 ... wM · x x ∈ RM Lemma 2.2 . Composition of a piece-wise linear function with a ReLU activation function is also a piece-wise linear function . Theorem 2.3 . The output of any hidden unit in an unbiased fully-connected ReLU neural network is a piece-wise linear function . This is straightforward because the input to any hidden node is a linear combination of piece-wise linear functions and this input is composed with the ReLU activation function to yield the output , which is also piece-wise linear . However , each region Rm is the intersection of a different number of half-spaces , enclosed by various hyperplanes in Rn . In general , these regions Rm ( m = 1 , · · · , M ) do not have simple shapes . For the purpose of mathematical analysis , we need to decompose each region into a union of some well-defined shapes having a uniform form , which is called infinite simplex . Definition 2.4 . Let V = { v1 , v2 , ... , vn } be a set of n linearly independent vectors in Rn . An infinite simplex , R+V , is defined as the region linearly spanned by V using only positive weights : R+V = { n∑ k=1 αkvk ∣∣∣∣ αk > 0 , k = 1 , 2 , · · · , n } ( 1 ) Theorem 2.5 . Each piece-wise linear function f ( x ) can be formulated as a summation of some simpler functions : f ( x ) = ∑L l=1 fl ( x ) , each of which is linear and non-zero only in an infinite simplex as follows : fl ( x ) = { wl · x x ∈ R+Vl 0 otherwise ( 2 ) where Vl is a set of n linearly independent vectors , and wl is a weight vector . In practice , we can always assume that the input to neural networks , x , is bounded . As a result , for computational convenience , we may normalize all inputs x into the unit hyper-cube , Un = [ 0 , 1 ] n. Obviously , this assumption can be easily incorporated into the above analysis by multiplying each fl ( x ) in eq . ( 2 ) by ∏n r=1 h ( xr ) h ( 1−xr ) where h ( x ) is the Heaviside step function . Alternatively , we may simplify this term by adding n2 additional hyperplanes to further split the input space to ensure all the elements of x do not change signs within each region R+Vq . In this case , within each region R+Vq , the largest absolute value among all elements of x is always achieved by a specific element , which is denoted as rq . In other words , the dimension xrq achieves the largest absolute value inside R+Vq . Similarly , the normalized piece-wise linear function may be represented as a summation of some functions : f ( x ) = ∑Q q=1 gq ( x ) , where each gq ( x ) ( q = 1 , 2 , · · · , Q ) has the following form : gq ( x ) = { wq · x h ( 1− xrq ) x ∈ R+Vq 0 otherwise For every Vq , there exists an n× n invertible matrix Aq to linearly transform all vectors of Vq into standard basis vectors ei in Rn . As a result , each function gq ( x ) may be represented in terms of standard bases V∗ = { e1 , · · · , en } as follows : gq ( x ) = { w̄q · x̄q h ( 1− 1 · x̄q ) x̄q ∈ R+V∗ 0 otherwise where x̄q = xATq , and w̄q = wqA −1 q . Lemma 2.6 . Fourier transform of the following function : s ( x ) = { h ( 1− 1 · x ) x ∈ R+V∗ 0 otherwise may be presented as : S ( ω ) = ( −i√ 2π ) n n∑ r=0 e−iωr∏ r′ 6=r ( ωr′ − ωr ) ( 3 ) where ωr is the r-th component of frequency vector ω ( r = 1 , · · · , n ) , and ω0 = 0 . Finally we derive the Fourier transform of fully-connected ReLU neural networks as follows . Theorem 2.7 . The Fourier transform of the output of any hidden node in a fully-connected unbiased1 ReLU neural network may be represented as ∑Q q=1 wqA −1 q ∇S ( ωA−1q ) , where∇ denote the differential operator . 1For mathematical convenience , we assume neural networks have no biases here . However , regular neural networks with biases may be reformulated as unbiased ones by adding another dimension of constants . Thus , the main results here are equally applicable to both cases . Note that regular neural networks with biases are used in our experiments in this paper . Obviously , neural networks are the so-called approximated bandlimited models as defined in ( Jiang , 2019 ) , which have decaying high frequency components in Fourier spectrum . Theorem 2.7 further suggests that the matrices A−1q may contribute to the high frequency components when the corresponding region R+Vq are too small . This is clear because the determinant of Aq is proportional to the volume of R+Vq in R n. In summary , the high frequency components of neural networks are mostly attributed to these tiny regions in the input space . As we will show later , these small regions may be explicitly exploited to generate adversarial samples for neural networks . 2.2 UNDERSTANDING ADVERSARIAL SAMPLES . As shown in Theorem 2.3 , neural network may be viewed as a sequential division of the input space into many small regions , as illustrated in Figure 1 . Each layer is a further division of the existing regions from the previous layers , with each region being divided differently . Hence a neural network with multiple layers would result in a tremendous amount of sub-regions in the input space . For example , when cutting an n-dimensional space using N hyperplanes , the maximum number of regions may be computed as ( N 0 ) + ( N 1 ) + · · · + ( N n ) . For a hidden layer of N = 1000 nodes and input dimension is n = 200 , the maximum number of regions is roughly equal to 10200 . In other words , even a middle-sized neural network can partition input space into a huge number of sub-regions , which can easily exceed the total number of atoms in the universe . When we learn a neural network , we can not expect there is at least one training sample inside each region . For those regions that do not have any training sample , the resultant linear functions in them may be arbitrary since they do not contribute to the training objective function at all . Of course , most of these regions are extremely small in size . When we measure the expected loss function over the entire space , their contributions are negligible since the chance for a randomly sampled point to fall into these tiny regions is extremely small . However , adversarial attack is imposing a new challenge since adversarial samples are not naturally sampled . Given that the total number of regions is huge , those tiny regions are almost everywhere in the input space . For any data point in the input space , we almost surely can find such a tiny region in proximity where the linear function is arbitrary . If a point inside this tiny region is selected , the output of the neural network may be unexpected . We believe that these tiny unlearned regions may be a major reason why neural networks are vulnerable to adversarial samples . In layered deep neural networks , the linear functions in all regions are not totally independent . If we use v ( l ) to denote the weight matrix in layer l , the resultant linear weight wk in eq . ( 2 ) is actually the sum of all concatenated v ( l ) along all active paths . When we make a small perturbation ∆x to any input x , the fluctuation in the output of any hidden node can be approximated represented as : ∆f ( x ) ∝ N · ∏ l E [ |v ( l ) ij | ] ( 4 ) where N denotes the total number of hyperplanes to be crossed when moving x to x + ∆x . In any practical neural network , we normally have at least tens of thousands of hyperplanes crossing the hypercube Un = [ 0 , 1 ] n. In other words , for any input x in a high-dimensional space , a small perturbation can always easily cross a large number of hyperplanes to enter a tiny unlearned region . When N is fairly large , the above equation indicates that the output of a neural network can still fluctuate dramatically even after all weight vectors are regularized by L1 or L2 norm . As a reference , we have verified this on some ImageNet data using a VGG16 model . When PGD is used to generate adversarial samples with average perturbation ||∆x||2 ≤ 0.35 , which is extremely small perturbation since x has over a hundred thousand dimensions on ImageNet , we have observed that in average about N = 5278 hyperplanes are crossed per layer even after such a small perturbation is added . At last , since the ubiquitous existence of unlearned tiny regions is an intrinsic property of neural networks given its current model structure , we believe that adversarial training strategies will not be sufficient to completely get rid of adversarial samples . In principle , neural networks must be strictly bandlimited to filter out those decaying high frequency components in order to completely eliminate all adversarial samples . We definitely need more research efforts to figure out how to do this effectively and efficiently for neural networks .

Summary: The paper is dedicated to studying adversarial attack and defense problems from the perspective of Fourier analysis. They demonstrate that the adversarial vulnerability of neural networks can be attributed to non-zero high-frequency components. Then, the author proposes a simple post average approach to smooth out the insignificant high-frequency components, which can improve the adversarial robustness of neural networks. They conduct extensive experiments on ImageNet and CIFAR-10 to defend existing attacks, including FGSM, PGD, DeepFool, and C&W attacks.