A Frequency Perspective of Adversarial Robustness

1 INTRODUCTION AND BACKGROUND . Since the introduction of adversarial examples by Szegedy et al . ( 2014 ) , there has been a curiosity in the community around the nature and mechanisms of adversarial vulnerability . There exists an ever-growing body of work focused on attacking neural networks starting with the simple FGSM ( Goodfellow et al. , 2015 ) , followed by the advanced PGD ( Madry et al. , 2018 ) , a stronger C & W attack ( Carlini & Wagner , 2016 ) , the sparser Deep Fool ( Su et al. , 2019 ) and recently even a parameter free Auto-Attack ( Croce & Hein , 2020 ) . These methods and algorithms are consistently countered by the adversarial defense community , starting with distillation-based methods ( Papernot et al. , 2016 ) , logit-based approaches ( Kannan et al. , 2018 ) , then moving on to the simple , yet powerful PGD training ( Madry et al. , 2018 ) , ensemble-based methods ( Tramèr et al. , 2018 ) and various other schemes ( Zhang et al. , 2019 ; Xie et al. , 2019 ) . Despite the immense progress made by the field , there exist many unanswered questions and ambiguities regarding these methods and adversarial examples themselves . Several works ( Athalye et al. , 2018 ; Kolter & Wong , 2018 ; Croce & Hein , 2020 ; Carlini & Wagner , 2017 ) have raised doubts about the efficacy of many methods and have made appeals to the research community to be more vigilant and skeptical with new defenses . Meanwhile , there exists a thriving research corpus dedicated to deeply studying and understanding adversarial examples themselves . Ilyas et al . ( 2019 ) presented a feature-based analysis of adversarial examples , while Jere et al . ( 2019 ) presented preliminary work on PCA-based analysis of adversarial examples , which was followed up with Jere et al . ( 2020 ) offering a nuanced view of the same through the lens of SVD . Ortiz-Jimenez et al . ( 2020 ) derive insights from the margins of classifiers . Given the intriguing nature of adversarial examples , another way of examining them is through the signal processing perspective of frequencies . Tsuzuku & Sato ( 2019 ) first proposed a frequency framework by studying the sensitivity of CNN ’ s for different Fourier bases . Yin et al . ( 2019 ) then pursued a related direction where they explored the frequency properties of neural networks with respect to additive noise . Abello et al . ( 2021 ) explore how the frequency properties of the image itself affect the model ’ s outputs and robustness . Caro et al . ( 2021 ) studied whether convolution operations themselves have an intrinsic frequency bias . Guo et al . ( 2019 ) came up with the first variant of adversarial attacks which target the low frequencies and Sharma et al . ( 2019 ) strengthened this line of thought by showing that such attacks had a high success rate against adversarially defended models . Deng & Karam ( 2020 ) proposed a method of generating adversarial attacks in the frequency domain itself . Complementary to these , there have been efforts by Lorenz et al . ( 2021 ) and Wang et al . ( 2020a ) in detecting or mitigating adversarial examples by training in the frequency domain . These works also analyzed the nature of adversarial examples under the purview of frequencies and tried to arrive at an explanation for their nature . Wang et al . ( 2020b ) hypothesized how CNNs exploit high frequency components , leading to less robust models , which is also the primary argument for a class of pre-processing based defenses , e.g. , those based on JPEG . Wang et al . ( 2020d ) also had arguments in support of this conjecture , based on their analysis on CIFAR-10 ( Krizhevsky , 2009 ) . It is confounding that these results are at odds with the successful low frequency adversarial attacks by Sharma et al . ( 2019 ) and raises the pertinent question : What is the true nature of adversarial examples in the frequency domain ? Our work challenges some pre-existing notions about the nature of adversarial examples in the frequency domain and arrives at a more nuanced understanding that is well rooted in theory and backed by extensive empirical observations spanning multiple datasets . Some of our observations overlap with insights from the concurrent work by Bernhard et al . ( 2021 ) and offers additional evidence in this ongoing debate . Based on these , we arrive at a new framework that explains many properties of adversarial examples , through the lens of frequency analysis . We also carry out the first detailed analysis on the behaviour of frequency-constrained adversarial training . Our key contributions can be summarized as follows : • We show that adversarial examples are neither high frequency nor low frequency phenomena . It is more nuanced than this dichotomous explanation . • We propose variations of adversarial training by coupling it with frequency-space analysis , leading us to some intriguing properties of adversarial examples . • We propose a new framework of frequency-based robustness analysis that also helps explain and control the accuracy vsṙobustness trade-off during adversarial training . The rest of the paper is organized as follows : we first start off with basic notations and preliminaries . Then we introduce our main findings about adversarial examples in frequency domain and subsequently present a detailed analysis about their properties , complemented by extensive experiments . 2 PRELIMINARIES . We denote a neural network with parameter θ by y = h ( x ; θ ) , which takes in an input image x ∈ RH×W ( omitting the channel dimension for brevity ) and outputs y ∈ RC where C is the number of classes . Let D and D−1 represent the forward Type-II DCT ( Discrete Cosine Transform ) ( Ahmed et al. , 1974 ) and its corresponding inverse . The DCT breaks down the input signal and expresses it as a linear combination of cosine basis functions . Its inverse recovers the input signal from this representation . For a 1-D signal , the kth-freq of x ∈ RN and its corresponding inverse is given by D ( x ) [ k ] = g [ k ] = N−1∑ n=0 xnλk cos ( 2n+ 1 ) kπ 2N , ( 1 ) D−1 ( x ) = x [ n ] = N−1∑ k=0 g [ k ] λk cos ( 2n+ 1 ) kπ 2N , ( 2 ) where k = { 0 , 1 , . . . , N − 1 } and λk = √ 1 N for k = 0√ 2 N else . ( 3 ) We denote an adversarial attack that is bound by budget by max ||δ||p≤ L ( h ( x+ δ ; θ ) , y ) ( 4 ) where L is the loss associated with the network and δ is the adversarial noise bounded under a defined Lp norm to be less than perturbation budget . We perform a standard PGD-style update ( Madry et al. , 2018 ) to solve this maximization problem via gradient ascent and for an attack bounded by an Lp norm and step size α , the adversarial noise is given by δ = argmax ||V ||p≤α V T∇xL ( h ( x ; θ ) , y ) ( 5 ) where V is the direction of steepest normalized descent . Now , to generate an adversarial example that consists of certain frequencies , we restrict its adversarial noise δ to a subspace S defined by S = Span { f1 , f2 , . . . , fk } , where fi are orthogonal DCT modes and k ≤ N , δf = argmax ||V ||p≤α V TD−1 ( D ( ∇xL ( h ( x ; θ ) , y ) ) M ) ( 6 ) where Mz ( X ) = { 1 if D ( Xz ) ∈ S 0 if D ( Xz ) 6∈ S is the mask to select frequencies . ( 7 ) In our work , we consider the L∞ and L2 norms , solving for which gives us the update steps : δf = α · Sgn ( D−1 ( D ( ∇xL M ) ) ) for L∞ and ( 8 ) δf = α ·D−1 ( D ( ∇xL M ||∇xL M ||2 ) ) for L2 ( 9 ) We refer to this method as DCT-PGD in the rest of the paper . Note that the manual step size selection of standard PGD is not always accurate , leading to discrepancies in robustness measures as illustrated in Lorenz et al . ( 2021 ) . Hence , we provide our results and observations with a DCT version of Auto-Attack . Unless mentioned otherwise , we utilize the ResNet-18 architecture for all models in our experiments . We use the term adversarial training to refer to the method by Madry et al . ( 2018 ) for all models , with the exception for ImageNet models where we use Adversarial training for free method ( Shafahi et al. , 2019 ) . We utilize L∞ norm with of 4/255 for TinyImageNet and ImageNet datasets and of 8/255 for CIFAR-10 in all our experiments . Exact training details are included in the Appendix A.2 . The terms low frequencies refer to frequency bands 0 to 32 and high frequencies refer to frequency bands bands 33 to 63 . 3 WHY DO WE NEED A FREQUENCY PERSPECTIVE ? . The focus of the community has been mostly on generating adversarial examples which are indistinguishable to humans , but can easily fool models . This notion gave rise to the incorrect assumption that since these perturbations are imperceptible to humans and they generally have to be in higher frequencies . The assumption was solidified when various pre-processing defense methods like Gaussian blur and JPEG showed initial success , further adding to confirmation bias . The fallacy is a classic case of Post Hoc Ergo Propter Hoc , i.e. , the outcome of events is influenced by the mere ordering . Most of these experiments were centered only around CIFAR-10 and one can easily observe that the efficacy of such methods are questionable when extended to larger datasets like ImageNet and TinyImageNet ( e.g. , see ( Dziugaite et al. , 2016 ; Das et al. , 2017 ; Xu et al. , 2018 ) ) . This incorrect assumption has also led to claims about adversarial training shifting the importance of frequencies from the higher to the lower end of the spectrum ( Wang et al. , 2020c ; b ) . As we show in the subsequent sections , this is not entirely true . We contend that this entire framework of investigating adversarial examples ( e.g. , blocking high frequency components using Gaussian blur pre-processing ) is flawed , as one can not verify the converse setting of blocking low frequency components . This is because low frequency components are inherently tied with labels ( Wang et al. , 2020b ) , conflating the two phenomena . Contrary to these , we argue and show that adversarial examples are neither high frequency nor low frequency and are dependent on the dataset . 4 NATURE OF ADVERSARIAL SAMPLES IN FREQUENCY SPACE . 4.1 NOISE GRADIENTS . Measuring the change of output with respect to the input is a fundamental aspect of system design . Whether it is a controls circuit or a mathematical model , the measure dydx gives us valuable information about the working of the model . When the model in question is a black box , like a neural network , the measure is invaluable as often it is our only insight into the inner mechanisms of the model . In the case of a classifier , the measure dydx is a tensor that is the same size as the input , which tells us about the impact of each pixel in input x on the resulting output y. Drucker & Le Cun ( 1991 ) first applied this concept on neural networks and called them input gradients . Over the recent years , this measure and its variants have found a new home in the model interpretability community ( Selvaraju et al. , 2016 ; Wang et al. , 2019 ) , where it forms the bedrock for various improvements . Taking a cue from this , we propose to measure dydδ or Noise Gradients , which inform us about the regions of noise , which have maximal impact on the output y . In our work , we are more interested in the frequency properties of adversarial examples , and hence take this one step further and propose to measure the DCT of noise gradients , i.e. , D ( dy dδ ) orD ( ∇δY ) . In a sense , we are measuring the model ’ s reaction to different frequency components in the adversarial input . This tensor D ( ∇δY ) ( which has same shape as input ) will point us towards the specific frequencies that affect the output y of the model . To analyze the adversarial frequency properties of a given dataset , we calculate the average noise gradients with respect to the model , under both normal training and adversarial training paradigms . Once computed , it will paint a picture about the interplay of adversarial noise and frequencies .

Summary: This paper presents a frequency-based understanding of adversarial examples in deep neural networks. The main observation is that the adversarial examples are neither in high-frequency or low-frequency components but dataset dependent. The authors also analyse the properties of training robust models with frequency constrains, and propose a frequency-based explanation for accuracy and robustness tradeoff.