A Frequency Perspective of Adversarial Robustness

1 INTRODUCTION AND BACKGROUND . Since the introduction of adversarial examples by Szegedy et al . ( 2014 ) , there has been a curiosity in the community around the nature and mechanisms of adversarial vulnerability . There exists an ever-growing body of work focused on attacking neural networks starting with the simple FGSM ( Goodfellow et al. , 2015 ) , followed by the advanced PGD ( Madry et al. , 2018 ) , a stronger C & W attack ( Carlini & Wagner , 2016 ) , the sparser Deep Fool ( Su et al. , 2019 ) and recently even a parameter free Auto-Attack ( Croce & Hein , 2020 ) . These methods and algorithms are consistently countered by the adversarial defense community , starting with distillation-based methods ( Papernot et al. , 2016 ) , logit-based approaches ( Kannan et al. , 2018 ) , then moving on to the simple , yet powerful PGD training ( Madry et al. , 2018 ) , ensemble-based methods ( Tramèr et al. , 2018 ) and various other schemes ( Zhang et al. , 2019 ; Xie et al. , 2019 ) . Despite the immense progress made by the field , there exist many unanswered questions and ambiguities regarding these methods and adversarial examples themselves . Several works ( Athalye et al. , 2018 ; Kolter & Wong , 2018 ; Croce & Hein , 2020 ; Carlini & Wagner , 2017 ) have raised doubts about the efficacy of many methods and have made appeals to the research community to be more vigilant and skeptical with new defenses . Meanwhile , there exists a thriving research corpus dedicated to deeply studying and understanding adversarial examples themselves . Ilyas et al . ( 2019 ) presented a feature-based analysis of adversarial examples , while Jere et al . ( 2019 ) presented preliminary work on PCA-based analysis of adversarial examples , which was followed up with Jere et al . ( 2020 ) offering a nuanced view of the same through the lens of SVD . Ortiz-Jimenez et al . ( 2020 ) derive insights from the margins of classifiers . Given the intriguing nature of adversarial examples , another way of examining them is through the signal processing perspective of frequencies . Tsuzuku & Sato ( 2019 ) first proposed a frequency framework by studying the sensitivity of CNN ’ s for different Fourier bases . Yin et al . ( 2019 ) then pursued a related direction where they explored the frequency properties of neural networks with respect to additive noise . Abello et al . ( 2021 ) explore how the frequency properties of the image itself affect the model ’ s outputs and robustness . Caro et al . ( 2021 ) studied whether convolution operations themselves have an intrinsic frequency bias . Guo et al . ( 2019 ) came up with the first variant of adversarial attacks which target the low frequencies and Sharma et al . ( 2019 ) strengthened this line of thought by showing that such attacks had a high success rate against adversarially defended models . Deng & Karam ( 2020 ) proposed a method of generating adversarial attacks in the frequency domain itself . Complementary to these , there have been efforts by Lorenz et al . ( 2021 ) and Wang et al . ( 2020a ) in detecting or mitigating adversarial examples by training in the frequency domain . These works also analyzed the nature of adversarial examples under the purview of frequencies and tried to arrive at an explanation for their nature . Wang et al . ( 2020b ) hypothesized how CNNs exploit high frequency components , leading to less robust models , which is also the primary argument for a class of pre-processing based defenses , e.g. , those based on JPEG . Wang et al . ( 2020d ) also had arguments in support of this conjecture , based on their analysis on CIFAR-10 ( Krizhevsky , 2009 ) . It is confounding that these results are at odds with the successful low frequency adversarial attacks by Sharma et al . ( 2019 ) and raises the pertinent question : What is the true nature of adversarial examples in the frequency domain ? Our work challenges some pre-existing notions about the nature of adversarial examples in the frequency domain and arrives at a more nuanced understanding that is well rooted in theory and backed by extensive empirical observations spanning multiple datasets . Some of our observations overlap with insights from the concurrent work by Bernhard et al . ( 2021 ) and offers additional evidence in this ongoing debate . Based on these , we arrive at a new framework that explains many properties of adversarial examples , through the lens of frequency analysis . We also carry out the first detailed analysis on the behaviour of frequency-constrained adversarial training . Our key contributions can be summarized as follows : • We show that adversarial examples are neither high frequency nor low frequency phenomena . It is more nuanced than this dichotomous explanation . • We propose variations of adversarial training by coupling it with frequency-space analysis , leading us to some intriguing properties of adversarial examples . • We propose a new framework of frequency-based robustness analysis that also helps explain and control the accuracy vsṙobustness trade-off during adversarial training . The rest of the paper is organized as follows : we first start off with basic notations and preliminaries . Then we introduce our main findings about adversarial examples in frequency domain and subsequently present a detailed analysis about their properties , complemented by extensive experiments . 2 PRELIMINARIES . We denote a neural network with parameter θ by y = h ( x ; θ ) , which takes in an input image x ∈ RH×W ( omitting the channel dimension for brevity ) and outputs y ∈ RC where C is the number of classes . Let D and D−1 represent the forward Type-II DCT ( Discrete Cosine Transform ) ( Ahmed et al. , 1974 ) and its corresponding inverse . The DCT breaks down the input signal and expresses it as a linear combination of cosine basis functions . Its inverse recovers the input signal from this representation . For a 1-D signal , the kth-freq of x ∈ RN and its corresponding inverse is given by D ( x ) [ k ] = g [ k ] = N−1∑ n=0 xnλk cos ( 2n+ 1 ) kπ 2N , ( 1 ) D−1 ( x ) = x [ n ] = N−1∑ k=0 g [ k ] λk cos ( 2n+ 1 ) kπ 2N , ( 2 ) where k = { 0 , 1 , . . . , N − 1 } and λk = √ 1 N for k = 0√ 2 N else . ( 3 ) We denote an adversarial attack that is bound by budget by max ||δ||p≤ L ( h ( x+ δ ; θ ) , y ) ( 4 ) where L is the loss associated with the network and δ is the adversarial noise bounded under a defined Lp norm to be less than perturbation budget . We perform a standard PGD-style update ( Madry et al. , 2018 ) to solve this maximization problem via gradient ascent and for an attack bounded by an Lp norm and step size α , the adversarial noise is given by δ = argmax ||V ||p≤α V T∇xL ( h ( x ; θ ) , y ) ( 5 ) where V is the direction of steepest normalized descent . Now , to generate an adversarial example that consists of certain frequencies , we restrict its adversarial noise δ to a subspace S defined by S = Span { f1 , f2 , . . . , fk } , where fi are orthogonal DCT modes and k ≤ N , δf = argmax ||V ||p≤α V TD−1 ( D ( ∇xL ( h ( x ; θ ) , y ) ) M ) ( 6 ) where Mz ( X ) = { 1 if D ( Xz ) ∈ S 0 if D ( Xz ) 6∈ S is the mask to select frequencies . ( 7 ) In our work , we consider the L∞ and L2 norms , solving for which gives us the update steps : δf = α · Sgn ( D−1 ( D ( ∇xL M ) ) ) for L∞ and ( 8 ) δf = α ·D−1 ( D ( ∇xL M ||∇xL M ||2 ) ) for L2 ( 9 ) We refer to this method as DCT-PGD in the rest of the paper . Note that the manual step size selection of standard PGD is not always accurate , leading to discrepancies in robustness measures as illustrated in Lorenz et al . ( 2021 ) . Hence , we provide our results and observations with a DCT version of Auto-Attack . Unless mentioned otherwise , we utilize the ResNet-18 architecture for all models in our experiments . We use the term adversarial training to refer to the method by Madry et al . ( 2018 ) for all models , with the exception for ImageNet models where we use Adversarial training for free method ( Shafahi et al. , 2019 ) . We utilize L∞ norm with of 4/255 for TinyImageNet and ImageNet datasets and of 8/255 for CIFAR-10 in all our experiments . Exact training details are included in the Appendix A.2 . The terms low frequencies refer to frequency bands 0 to 32 and high frequencies refer to frequency bands bands 33 to 63 . 3 WHY DO WE NEED A FREQUENCY PERSPECTIVE ? . The focus of the community has been mostly on generating adversarial examples which are indistinguishable to humans , but can easily fool models . This notion gave rise to the incorrect assumption that since these perturbations are imperceptible to humans and they generally have to be in higher frequencies . The assumption was solidified when various pre-processing defense methods like Gaussian blur and JPEG showed initial success , further adding to confirmation bias . The fallacy is a classic case of Post Hoc Ergo Propter Hoc , i.e. , the outcome of events is influenced by the mere ordering . Most of these experiments were centered only around CIFAR-10 and one can easily observe that the efficacy of such methods are questionable when extended to larger datasets like ImageNet and TinyImageNet ( e.g. , see ( Dziugaite et al. , 2016 ; Das et al. , 2017 ; Xu et al. , 2018 ) ) . This incorrect assumption has also led to claims about adversarial training shifting the importance of frequencies from the higher to the lower end of the spectrum ( Wang et al. , 2020c ; b ) . As we show in the subsequent sections , this is not entirely true . We contend that this entire framework of investigating adversarial examples ( e.g. , blocking high frequency components using Gaussian blur pre-processing ) is flawed , as one can not verify the converse setting of blocking low frequency components . This is because low frequency components are inherently tied with labels ( Wang et al. , 2020b ) , conflating the two phenomena . Contrary to these , we argue and show that adversarial examples are neither high frequency nor low frequency and are dependent on the dataset . 4 NATURE OF ADVERSARIAL SAMPLES IN FREQUENCY SPACE . 4.1 NOISE GRADIENTS . Measuring the change of output with respect to the input is a fundamental aspect of system design . Whether it is a controls circuit or a mathematical model , the measure dydx gives us valuable information about the working of the model . When the model in question is a black box , like a neural network , the measure is invaluable as often it is our only insight into the inner mechanisms of the model . In the case of a classifier , the measure dydx is a tensor that is the same size as the input , which tells us about the impact of each pixel in input x on the resulting output y. Drucker & Le Cun ( 1991 ) first applied this concept on neural networks and called them input gradients . Over the recent years , this measure and its variants have found a new home in the model interpretability community ( Selvaraju et al. , 2016 ; Wang et al. , 2019 ) , where it forms the bedrock for various improvements . Taking a cue from this , we propose to measure dydδ or Noise Gradients , which inform us about the regions of noise , which have maximal impact on the output y . In our work , we are more interested in the frequency properties of adversarial examples , and hence take this one step further and propose to measure the DCT of noise gradients , i.e. , D ( dy dδ ) orD ( ∇δY ) . In a sense , we are measuring the model ’ s reaction to different frequency components in the adversarial input . This tensor D ( ∇δY ) ( which has same shape as input ) will point us towards the specific frequencies that affect the output y of the model . To analyze the adversarial frequency properties of a given dataset , we calculate the average noise gradients with respect to the model , under both normal training and adversarial training paradigms . Once computed , it will paint a picture about the interplay of adversarial noise and frequencies .

Summary: This work explores adversraial robustness from a frequency perspective. While this had been done before, this work challenges the commonly held notions, that adversarial perturbations are mainly a high-frequency phenomenon. The authors provide insight that the frequency properties of adversarial examples are dependant on the underlying training dataset. Additionally, the impact of different frequency properties in adversarial training is explored.