A Frequency Perspective of Adversarial Robustness
Authors: Shishira Maiya
Max Ehrlich
Vatsal Agarwal
Ser-Nam Lim
Tom Goldstein
Abhinav Shrivastava
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial examples
Frequency analysis
Adversarial Robustness
Adversarial training

Abstract: Adversarial examples pose a unique challenge for deep learning systems.  Despite recent advances in both attacks and defenses, there is still a lack of clarity and consensus in the community about the true nature and underlying properties of adversarial examples. A deep understanding of these examples can provide new insights towards the development of more effective attacks and defenses. Driven by the common misconception that adversarial examples are high-frequency noise, we present a frequency-based understanding of adversarial examples, supported by theoretical and empirical findings. Our analysis shows that adversarial examples are neither in high-frequency nor in low-frequency components, but are simply dataset dependent. Particularly, we highlight the glaring disparities between models trained on CIFAR-10 and ImageNet-derived datasets. Utilizing this framework, we analyze many intriguing properties of training robust models with frequency constraints, and propose a frequency-based explanation for the commonly observed accuracy vs. robustness trade-off.

Under review as a conference paper at ICLR 2022
A FREQUENCY PERSPECTIVE OF ADVERSARIAL Ro- BUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
Adversarial examples pose a unique challenge for deep learning systems. Despite recent advances in both attacks and defenses, there is still a lack of clarity and consensus in the community about the true nature and underlying properties of ad- versarial examples. A deep understanding of these examples can provide new in- sights towards the development of more effective attacks and defenses. Driven by the common misconception that adversarial examples are high-frequeng, noise, we present frequency-base based understanding of adversarial examples, supported by theoretical and empirical findings. Our analysis shows that adversarial ex- amples are neither in sehhfrreuency nor in low-frequency components, but are simply dataset dependent. Particularly, we highlight the glaring disparities be- tween models trained on CIFAR-10 and ImageNet-derived datasets. Utilizing this framework, we analyze many intriguing properties of training robust models with frequency constraints, and propose a frequency-basee explanation for the com- monly observed accuracy vs robustness trade-off.
INTRODUCTION AND BACKGROUND
Since the introduction of adversarial examples by Szegedy et al. (2014), there has been curiosity in the community around the nature and mechanisms of adversarial vulnerability. There exists an ever-gree wver-growing body of work focused on attacking neural networks starting with the simple FGSM (Goodfellow et al.] (2015), followed by the advanced PGD (Madry et al. (018), a stronger C&W attack (Carlini & Wagner, 2016), the sparser Deep Fool (Su et al. 2019 and recently even pa- rameter free Auto-Attack (Croce & Hein 2020). These methods and algorithms are consistently countered by the adversarial defense community, starting with sitiillation-basee methods (Papernot) (ct al. 2016). logit-based approaches (Kannan et al. 2018), then moving on to the simple, yet pow- erful PGD training (Madry et al. 2018), ensemble-based methods (Tramèr et al. 2018) and various other schemes (Zhang et al. 2019; Xie et al., 2019). Despite the immense progress made by the field, there exist many unanswered questions and ambiguities regarding these methods and adver- sarial examples themselves. Several works (Athalye et al.] 2018 Kolter & Wongl 2018 Croce & Hein 2020 Carlini & Wagner 2017) have raised doubts about the efficacy of many methods and have made appeals to the research community to be more vigilant and skeptical with new defenses.
Meanwhile, there exists a thriving research corpus dedicated to deeply studying and understanding adversarial examples themselves. llyas et al. (2019) presented feature-based analysis of adversarial examples, whileeeree et al. (2019) presented preliminary work on PCA-based analysis of adversarial examples, which was followed up with Jete et al. (2020) offering a nuanced view of the same through the lens of SVD. Ortiz-Jimenez et al.) (2020) derive insights from the margins of classifiers.
Given the intriguing nature of adversarial examples, another way of examining them is through the signal processing perspective of frequencies. Tsuzuku Sato (2019) first proposed frequency framework by studying the sensitivity of CNN's for different Fourier bases. Yin et al. (2019) then pursued a related direction where they explored the frequency properties of neural networks with respect to additive noise. Abello et al. 12021) explore how the frequency properties of the image itself affect the model's 's outputs and robustness. Caro et al. (2021) studied whether convolution op- erations themselves have intrinsic frequency bias. Guo et al. (2019) came up with the first variant of adversarial attacks which target the low frequencies and Sharma et all (2019) strengthened this line of thought by showing that such attacks had high success rate against adversarially defended

---

Under review as a conference paper at ICLR 2022
models. Deng & Karam (2000) proposed a method of generating adversarial attacks in the frequency domain itself. Complementary to these, there have been efforts by Lorenz et al. (2021) and Wang et al. 20200a in detecting mitigating adversarial examples by training in the frequency domain.
These works also analyzed the nature of adversarial examples under the purview of frequencies and tried arrive at explanation for their nature. WWang et al. (2020b) hypothesized how CNNs ploit high frequency components, leading to less robust models, which is also the primary argument for class of pre-processing based defenses, e.g., those based on JPEG. Wang al. (2020d) also had arguments in support of this conjecture, based on their analysis on CIFAR-I0 (Krizhevsky) 2009). is confounding that these results are at odds with the successful low frequency adversarial attacks by (Sharma et al. (2019) and raises the pertinent question: What is the true nature of adversarial examples in the frequency domain? ? Our work challenges some pre-existing notions about the nature of adversarial examples in the frequency domain and arrives at more nuanced understanding that well rooted in theory and backed by extensive empirical observations spanning multiple datasets. Some of our observations overlap with insights from the concurrent work by Bernhard et al. (2021) and offers additional evidence in this ongoing debate. Based on these, we arrive at a new frame- work that explains many properties of adversarial examples, through the lens of frequency analysis. We also carry out the first detailed analysis on the behaviour of frequency-constrrined adversarial training. Our key contributions can be summarized as follows:
We show that adversarial examples are neither high frequency nor low frequency phenom- ena. It is more nuanced than this dichotomous explanation.
We propose variations of adversarial training by coupling it with frequency sppuuecy--ppaeeae analysis, leading us to some intriguing properties of adversarial examples.
We propose new framework of frequency-basee robustness analysis that also helps ex- plain and control the accuracy vstobustness trade-off during adversarial training.
The rest of the paper is organized as follows: we first start off with basic notations and preliminaries. Then we introduce our main findings about adversarial examples in frequency domain and subse- quently present detailed analysis about their properties, complemented by extensive experiments.
2 PRELIMINARIES
We denote a neural network with parameter by y = h(x; which takes in an input image x E R xW (omitting the channel dimension for brevity) and outputs y E BC where C' is the number of classes. Let D and D-1 represent the forward Type-I1 DCT (Discrete Cosine Transform) (Ahmed al.] (974) and its corresponding inverse. The DCT breaks down the input signal and expresses it as a lincar combination of cosine basis functions. Its inverse recovers the input signal from this representation, For 1-D signal, the 2.th -rhh of x E V and its corresponding inverse is given by
N-1
(2n + 1)kT
D(x)(A]] = g(b] =
Fy cos
2N
N-1
(2n + 1)kT
D-) = x/n] = Σ
glk] A cos
(2)
2N
for l: = 0
where k- = {0, 10..,.....4.. N 1} and A
(3)
N clse.
We denote an adversarial attack that is bound by budget c by max |
C(h(2= 8: 8)
(4)
where L is the loss associated with the network and 8 is the adversarial noise bounded under defined I, norm to be less than perturbation budget c. We perform a standard PGD-style up- date (Madry al. (2018) to solve this maximization problem via gradient ascent and for an attack bounded by an Lp norm and step size a, the adversarial noise is given by
8 arg max VT V. (((x;)))))))
(5)
|| Sa
2

---

Under review as a conference paper at ICLR 2022
where V is the direction of steepest normalized descent. Now, generate an adversarial example that consists of certain frequencies, we restrict its adversarial noise to a subspace defined by S Span f2............. f where fi are orthogonal DCT modes and k S N,
arg max V D D (DI(. M) 1IV
(6)
((;))(X)) S where (X) ((.) S is the mask to select frequencies.
(7)
In our work, we consider the Loe and Lg norms, solving for which gives us the update steps:
8 = a Sgn(1 - (D(V&L o M))) for Loo and
(8)
8y a (D
V + M ) for L2
V.COM
V.C @ M
(9)
We refer to this method as DCT-PGD in the rest of the paper. Note that the manual step size selection of standard PGD is not always accurate, leading to discrepancies in robustness measures illus- trated Lorenz et al. (021). Hence, we provide our results and observations with a DCT version of Auto-Attack. Unless mentioned otherwise, we utilize the ResNet-18 architecture for all models in our experiments. We use the term adversarial training to refer to the method by Madry et al. (2018) for all models. with the exception for ImageNet models where we use Adversarial training for free method (Shafahi et al. (019). We utilize Lex. norm with of 4/255 for Tiny
and ImageNet datasets and e of 8/255 for CIFAR-10 all our experiments. Exact training details are included in the Appendix A.2 The terms low frequencies refer to frequency bands 0 to 32 and high frequencies refer to frequency bands bands 33 to 63.
3 WHY DO WE NEED A FREQUENCY PERSPECTIVE?
The focus of the community has been mostly on generating adversarial examples which are indis- ringuishable humans, but easily fool models. This notion gave rise the incorrect assump- tion that since these perturbations are imperceptible to humans and they generally have to be in higher frequencies. The assumption like Gaussian blur and JPEG showed initial success, further adding to confirmation bias. The fal- lacy classic case of Post Hoc Ergo Propter Hoc, i.e., the outcome of events is influenced by the mere ordering. Most of these experiments were centered only around CIFAR- 10 and one can easily observe that the efficacy of such methods are questionable when extended to larger datasets like ImageNet and TinylmageNet (e.g., see (Dziugaite et al.) 2016 Das et al.) 2017 Xu et al. 2018)) This incorrect assumption has also led to claims about adversarial training shifting the importance of frequencies from the higher to the lower end of the spectrum (Wang et al. 2020c b). As we show in the subsequent sections, this is not entirely true.
solidified when various pre-processing defense methods
We contend that this entire framework of investigating adversarial examples (c.g.. blocking high frequency components using Gaussian blur pre- prc-poceessin) is flawed, as one cannot verify the con- verse setting of blocking low frequency components. This is because low frequency components inherently tied with labels (Wang et al.) (2020b). conflating the two phenomena. Contrary to these, we argue and show that adversarial examples are neither high frequency nor low frequency and are dependent on the dataset.
4 NATURE OF ADVERSARIAL SAMPLES IN FREQUENCY SPACE
4.1 NOISE GRADIENTS
Measuring the change of output with respect to the input is fundamental aspect of system design. dy Whether a controls circuit or a mathematical model, the measure de gives us valuable infor- mation about the working of the model. When the model in question is a black box, like neural network, the measure invaluable as often it is our only insight into the inner mechanisms of the model. In the case of a classifier, the measure du is a tensor that is the same size as the input, which tells us about the impact of each pixel in input x on the resulting output y. Drucker & Le Cun (191))
3

---

Under review as a conference paper at ICLR 2022
first applied this concept neural networks and called them imput gradients. Over the recent years, this measure and its variants have found a new home in the model interpretability community (Sel- varaju et al. 2016 Wang al. 2019), where it forms the bedrock for various improvements.
Taking a cue from this, propose measure dy Noise Gradients, which inform us about the regions of noise, which have maximal impact on the output y. In our work, we are more interested in the frequency properties of adversarial examples, and hence take this step further and propose dy dó
to measure the DCT of noise gradients, i.e., D
or D (V&Y). In sense, we are measuring the
model's reaction to different frequency components in the adversarial input. This tensor D (V&Y) (which has same shape as input) will point towards the specific frequencies that affect the output y of the model. To analyze the adversarial frequency properties of a given dataset, we calculate the average noise gradients with respect the model, under both normal training and adversarial training paradigms. Once computed, it will paint a picture about the interplay of adversarial noise and frequencies.
4.1.1 ANALYSIS OF NOISE GRADIENTS
We define the quantity D (V&Y), the noise gradient frequency f. Note that this quantity is useful because it differs from D66) by at most constant multiple, i.e.,
D x (V
(10)
Proof. Let & + 8 where ó the adversarial noise, then
V&Y = V.Y
V&Y and VoL=V V
(11)
We have (from Appendix A.
V
(12)
From the definition of the PGD update step, we have
= V a VoL
(13)
for some constant Taking DCT of both sides, and by the linearity of the DCT, we have
D66) D(a
V&L) a DVV,L), and therefore, D x DL).L).
(14)
Now from equation
I and equation 12 we get
D(V & D(V&Y)
(15)
Using this in equation 14 have
D(0) & D(V.Y)
(16)
We see that the term D(V corresponds to the frequencies that are affected by adversarial noise.
4.1.2 EMPIRICAL OBSERVATION OF NOISE GRADIENTS
We compute the average DCT of noise gradients over validation sets of TinyImageNet, CIFAR-10, and ImageNet datasets for models with normal and adversarial training under attack from a PGD- based Loo adversary. The resulting tensors are visualized in Figure Ta It shows the path taken by the PGD attack in the frequency domain under different scenarios for different datasets. We see that for normally trained CIFAR-10 models, the DCT of noise gradient activations are towards the higher frequencies and they gradually shift towards lower frequencies once the model adversari- ally trained. Whereas for Tiny ImageNet and ImageNet models, we observe that the activations are already in lower-mid frequencies and adversarial training further concentrates them. These results clearly establish the following:
The DCT content of PGD attacks is highly dataset- deeeedent and we cannot make general arguments regarding frequency nature of adversarial samples just based training.
4

---

Under review as a conference paper at ICLR 2022
Cifar-10 Normal
Normal
ImageNet Normal
250
10
200
20
40
150
200
150
Cifar-10 Adversarial
TinylmageNet Adversarial Advyymaa
ImageNet Adversarial
2
100
10
50
20
40
150
60
200
20
(a)
(b)
Figure (a) The DCT of Noise Gradients averaged across the validation sets, visualized with histogram equalization. (b) shows the standard 8 DCT block with the all 64 frequencies arranged zigzag order.
The notion that adversarial training shifts the model focus from higher to lower frequencies is not entirely true. In many datasets, the model is already biased towards the lower end of the spectrum
before adversarial training.
To verify that this phenomenon is attributed the dataset alone, also observe similar be- haviour across other architectures (Appendix (A.5), across different image sizes (Appendix A.6) and for different attacks like La (Appendix A.7.
MEASURING IMPORTANCE OF FREQUENCY COMPONENTS
To examine the properties and behaviour of adversarial examples in the frequency domain, also craft various empirical metrics that measure the importance of frequency components under various paradigms.
5.1 IMPORTANCE BY VULNERABILITY
We measure the importance of frequency component by measuring the attack success rate when an adversarial attack is constrained to frequency f. Essentially,
quantifying the importance by measuring the expected vulnerability of each frequency. This amounts to measuring the accuracy of h(x+8; 8 where ō is the adversarial perturbation that is constrained to frequency obtained using the aforementioned DCT-PGD method. A lower accuracy of the model for a particular o, indicates more important frequency f. In Figure we visualize the accuracy of models with both normal training and adversarial training across different datasets under this setting. We see that only in the case CIFAR-10 the trends for normal training and adversarial training are reversed, indicating that attacks constrained to higher frequencies are more successful for normal models, while lower frequency attacks are more effective on the adversarially trained models. TinyImageNet and Im- ageNet datasets, we see that the overall trend remains same across the two training paradigms with adversarial training improving robustness across the spectrum. To obtain high level view, we de- sign another set of experiments where instead of attacking individual frequency components, restrict the attack to frequency ranges (or bands, set 16 equal divisions of the spectrum). The re- sults of these under DCT-PGD version of Auto-Attack jet al. (2020b) had claimed that low frequency perturbations cause visible changes in the image, thus defeating the purpose of imperceptibility clause of adversarial examples. However, find that for
shown in Figure A.7 In their work, Wang
5

---

Under review as a conference paper at ICLR 2022
CIFAR-10 Normal
TinylmageNet Normal
ImageNet. Normal
0.6
Accuracy
Accuracy
0.4
Accuracy
0.4
0.1
0.0
0.0
20
30
60
10
10
20
30
60
50
Attack Frequency
Attack Frequency
Attack Frequency
CIFAR-10 Adversarial
Tiny TinllmageNet Adversarial
ImageNet Adversarial
0.8
0.5
0.4
0.4
Arcuracy
Accursoy
0.3
Accuracy
0.3
0.2
0.4
20
46
60
10
36
10
20
40
50
60
Atta.kk Frequency
Attack Frequency
Attack Frequency
2/255
E-/2255
E= 8/255
E= 16/255
E 32/255
Figure 2:
Vulnerability scores (Accuracy under attack) visualized per frequency across datasets. Notice that the trends are reversed from normal training to adversarial training in the case of CIFAR- 10. The results for different frequency bands, under Auto-Attack is shown in Figure A.7
CIFAR-10
TinylmageNet
ImageNet
95.5
95.0
54
Accuracy
Annuracyy
Ancuracy
40
94.5
30
94.0
93.5
48
20
0.2
0.4
0.6
0.8
0.2
0/4
0.6
0.8
0.2
0.4
0.8
0.8
Drop Rates
Drop
Drop Rates
Trained:
Trained 16-32
Trained:
Trained: 48-63
Figure Accuracy for models trained with varying drop rates, for different frequency ranges.
larger datasets, such perturbations are imperceptible human. Example images have been shown in Appendix (Figure A.20 and A.2I
5.2 IMPORTANCE DURING TRAINING
With the objective of understanding the relative importance of frequency components while training, formulate experiment where we train models by masking out (making them zeros) frequency components of the input in probabilistic manner and then using the trained model for normal inference. Example images when certain frequency bands are dropped is shown in Figure/A.18 We train four types of models, where the frequency masking is restricted to four equal frequency bands and the amount of masking/dropping is controlled by parameter p. This translates to training
arg min C(h(z y: ;))
(17)
where FF D (M o D(x))
(18)
and M₁ = [1
2 2~ A E f2.., fxl is the Mask generated using
0 else
(19)

---

Under review as a conference paper at ICLR 2022
TinylmageNet,
4/255
TinylmageNet, 4/255
6415
9.7
5.05
5.01
Train Frequency
0.8
Train Prequency
Clean Accuracy
0.0
18
942
27
0.4
205
0.6
65
0.00
9.01
Closas AucnanAy
0-35
16:32
32-40
38-03
0.9
27
60
Attack Frequency
Attack Frequency
Train Frequency
CIFAR-10, E 8/255
CIFAR-10, & 8/255
1.0
0.39
9.00
Train Frequency
Train Frequency
Cloan Accuracy
16:22
0.09
3.78
9:31
0.18
0.5
0.6
32-48
3.00
0.31
3.74
0.24
0.2
6868
8.00
8.08
0.36
0.78
Cloun
0.15
16-32
50
Attack Froquency
Attack Froquency
Train Frequency
Figure 4: Frequency-base adversarial training across datasets. In the first column show the results of adversarially training and testing for different frequency ranges. Next, we show results of the same experiments across individual frequencies. The last column shows clean accuracy for each frequency.
where T is the input constrained to particular frequency band within the range [f,, f2. fx)- While training, we select the frequencies be dropped using a random uniform distribution 24, with the percentage of dropping controlled by parameter p. A value of 1 indicates all frequencies the specified band are set to zero. We train a total of 36 models per dataset, encompassing different drop rates (p values) and 4 frequency bands. The experiment is repeated across datasets and the results are shown in Figure 3 As expected, we observe that a higher drop rate leads to lower accuracy. We also that across datasets, high drop rates in low frequency band of 0-15 affects the model more. This behaviour is expected as lower frequencies have a strong relation with the labels (Wang et al. 2020b) and their extreme dropping leaves the model with little information learn from. But if we observe the degree to which it affects the performance, we see disparities between the datasets. For example, the model trained on CIFAR-10 experiences mere ~2% drop even when 90% of frequencies in the low band (frequencies 0-15) are dropped. Under the same condition, the model on TinylmageNet experiences ~00% drop and the model on ImageNet experiences whopping ~35% drop in accuracy, highlighting the relative importance of these frequency bands. Also, note how very high drop rates the highest frequency bands (frequencies 48-63) have little to effect in non CIFAR-10 10 models.
6 ADVERSARIAL TRAINING WITH FREQUENCY-BASED PERTURBATIONS
Till now, we have analyzed the frequency properties of the model across datasets. In all experi- ments so far, we merely observed how the model reacts to adversarial perturbations under various frequency constraints. To further understand the properties of robustness in the frequency domain, we propose to train models with adversarial perturbations restricted to these frequency subspaces, a first of its kind. The training follows
min max C(h(I + 06; S1
(20)
where 8y is adversarial noise restricted to a frequency subspace defined by f. To obtain a high-level view of the process, first train models adversarially with frequencies restricted four equal
7

---

Under review as a conference paper at ICLR 2022
CIFAR- 10
TinylmageNet
ImageNet
25
20-
PGD Accuracy
40
PGD Accuracy
PGD Accuracy
20-
15
15
10-
10
20
20
40
15
15
Steps
Steps
Steps
60
Clean Accuracy
80
Clean Accuracy
40
Clean Accuracy
50
70
40-
60
20-
30
20
40
60
80
100
10
15
20
25
10
15
20
Steps
Steps
Steps
Adv Free Training
Higher Weight to Higher Frequencies
Higher Weight Lower Frequencies
Figure 5: Illustration of unequal epsilon distribution. Here we see that models where low frequency perturbations are favoured ends up with higher robustness, but lower clean accuracy.
frequency bands, ranging from low to high. Predictably, the models perform well when adversar- ial PGD attack is also restricted to the same frequency bands. The resulting robustness heatmap of attacks across the spectrum is shown in first column of Figure For more fine-grained view of the same, we adversarially train 64 models for each dataset, by perturbing each individual fre- quency. Then we adversarially attack these models in every frequency to produce a robustness heatmap, shown in the second column of Figure In their work, Yin al., (2019) had claimed that training with low-frequency perturbations did not help the model to be robust against those frequencies. Their analysis was not based on adversarial perturbations, but their claim was general- ized. This effect was not observed in our experiments. We see that the model has good robustness when trained and tested against low-frequency perturbations, across datasets. The diagonals of the robustness heatmaps tell that models perform well against adversary constrained to the same frequency used for training. Moreover, we also see that models trained with perturbations restricted mid/higher frequencies can withstand attacks from fairly broad range of frequencies compared to models trained with lower frequency perturbations. Now that we have established this new train- ing paradigm, we explore its various nuances and intriguing properties.
6.1 THE UNEQUAL EPSILON DISTRIBUTION
Do all frequencies have the same impact in adversarial training? To answer this question, we mod- ify the construction of adversarial perturbation by weighing contributions from different frequency components and manipulating the value of they receive. It follows
K
8 =
sgn V V&L), for Loo norm
(21)
E
(22)
where K is the number of equal frequency bands (four in our case) and r is linear scaling param- eter. This setting effectively translates to giving more importance to perturbation in frequency space over the other. We train 2 models: One as described by equation[2) [2] favoring lower frequency bands and then its complement, by reversing / and favoring higher frequency bands. For these
8

---

Under review as a conference paper at ICLR 2022
CIFAR-10 10
TinylmageNet
ImageNet
Cleut Accuracy
Clean Accuracy
0.48
0.178
Robust Aonuracy
Clean Accuracy
0.20
Robust A Aomiracy
0.90
0.3
0.48
0.150
0.128
0.58
Robust
0.88
0.44
100
0.56
0.10
0.078
0.05
0.4
0.050
0.4
0.2
Clean Accuracy
Free Clean
Robust Accuracy
Adv Free Robust
Clean Accuracy vs Robustness across datasets, compared with standard adversarial train- ing for free method. Note that the Y-axis scales are different. Here À controls the weight of adver- sarial perturbation towards lower frequencies.
Figure 6:
periments, we employ Free adversarial training by Shafahi et al. (019). The plot of PGD and clean accuracy during training are shown in Figure We see that the model in which lower frequencies favoured acts closest standard PGD-based adversarial training. This shows that for model to be robust, it only needs to be adversarially trained in the frequencies that matter most and not the entire spectrum. But at the same time, favoured shows superior clean accuracy in all datasets except CIFAR-10. These results tell us that frequency based perturbations are intricately tied with clean accuracy and robustness of model. We explore this in detail in the next section.
that the model where high frequency perturbations
6.2 ACCURACY VS ROBUSTNESS; AN ALTERNATIVE PERSPECTIVE
Building on top of previous results, we design an experiment to examine the accuracy vs robustness trade-off that is commonplace while training robust models. We introduce parameter À that con- trols the weight given frequency components in the perturbation during adversarial training. The update step for PGD under Loom now looks like:
= A. a sgn V 2LL + (1 (1- sgm VuL
(23)
where CLF and V CHE are gradients restricted to low (frequencies 0-31) and high frequencies (frequencies 32-63) respectively. We adversarially train ten different models by varying the value of and show their clean and robust accuracy in Figure 6 We that in the case of TinylmageNet and ImageNet, the clean accuracy decreases when we train with low frequency perturbations, while increasing robustness. In case of CIFAR-10, followed by a steep fall. This is because higher frequencies have significant role in adversarial robustness for this dataset, which not achieved when À values are high. We also observe a steep fall in robustness for ImageNet at X of 0.9. This is because the frequency importance is distributed in the low-mid range for ImageNet (Figure la and very high À values tend to ignore the 32-48 frequency bands. These results establish that robustness and clean accuracy of an adversarially trained model are dependent on frequencies we perturb. The À parameter gives us control over the trade-off, enabling us be more prudent while designing architectures and training regimes that demand a mix of clean accuracy and robustness.
that there is initial increase in robustness
7 CONCLUSION
In this paper, we analyze adversarial robustness through the perspective of spatial frequencies and show that adversarial examples are not just high frequency phenomenon. Using both theoretical and empirical results we show that constituent frequencies on the dataset. Then we propose and study the properties of adversarial training using specific frequencies, which can be used to understand the accuracy-robustnes trade-off. These results can be utilized to train robust models more quickly by focusing the frequencies that matter most. We hope that our findings will resolve some misconceptions about the frequency content of adversarial examples and aid in creating more robust architectures.
adversarial examples are dependent

---

Under review as a conference paper at ICLR 2022
8 ETHICS
Adversarial examples pose unique challenge to real world deep learning systems. We believe that our analysis will aid in the development of adversarial attacks well robust architectures, While we are aware of the potential for malicious uses of both of these applications we find minimal direct ethical concerns with the work in this paper. is our hope that our work will only provide deeper understanding of what constitutes an adversarial example and the mechanisms behind adversarial training order to guide future research in this area.
REFERENCES
Antonio A. Abello, Roberto Hirata, and Zhangyang Wang. Dissecting the high-frequency bias in convolutional neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR) Workshops, pp. 103- 2021.
N. Ahmed, T. Natarajan, and K.R. Rao. Discrete cosine transform. IEEE Transactions Comput- ers, C-23(1):90-99)) 1974. doi: 10.11099T
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In ICML, 2018.
Rémi Bernhard, Pierre-Alain Moëliic, Martial Mermillod, Yannick Bourrier, Romain Cohendet, Miguel Solinas, and Marina Reyboz. Impact of spatial frequency based constraints adversarial robustness. CoRR, abs/2114.1122 126799, 2021. /
12679
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. CoRR, 0t4/1608.0445 2016.
Nicholas Carlini and David A. Wagner. Adversarial examples are not easily detected: Bypassing detection methods. Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 2017.
Josue Ortega Caro, Yilong Ju, Ryan Pyle, Sourav Dey, Wieland Brendel, Fabio Anselmi, and Ankit Patel. Local convolutions cause an implicit bias towards high frequency adversarial examples, 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. ArXiv, abs/2003.01 2020.
Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, L. Chen, M. Kounavis, and Duen Horng Chau. Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression. ArXiv, abs/1705.022 705.02000, 2017.
Yingpeng Deng and Lina J. Karam. Frequency-tuned universal adversarial attacks. CoRR, abs/2003.05549, 2020. URL
H. Drucker and Le Cun. Double backpropagation increasing generalization performance. In SICNN-91-Seattee International Joint Conference on Neural Networks, volume ii, pp. 145-150 vol.2, 1991. doi: 10.1 1..1109//CNN.1991.1555
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M. Roy. A study of the effect of JPG compression on adversarial images. CoRR, abs/1608.008 00853, 2016. URL http://arxiv.
Stg/abs/1st /100 00853
I. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial ex- amples. CoRR, abs//141.66577 2015.
Chuan Guo, Jared S. Frank, and Kilian Q. Weinberger. Low frequency adversarial perturbation. In UAI, 2019.
Andrew Ilyas, Shibani Santurkar, D. Tsipras, Logan Engstrom, Brandon Tran, and A. Madry. Ad- versarial examples not bugs, they are features. In NeurIPS, 2019.
10

---

Under review as a conference paper at ICLR 2022
Malhar Jere, Sandro Herbig, Christine H. Lind, and F. Koushanfar. Principal component properties of adversarial samples. ArXiv, abs/1912.03406, 2019.
Malhar Jere, Maghav Kumar, and F. Koushanfar. A singular value perspective on model robustness. ArXiv, abs/2012.03516, 2020.
Harini Kannan, A. Kurakin, and I. Goodfellow. Adversarial logit pairing. ArXiv, abs/ 880306333, 2018.
J. Z. Kolter and Eric Wong. Provable defenses against adversarial examples via the convex outer adversarial polytope. In ICML, 2018.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009,
P. Lorenz, Paula Harder, Dominik Strassel, Margret Keuper, and Janis Keuper. Detecting autoattack perturbations the frequency domain. 2021.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant adversarial attacks. In International Conference Learn- ing Representations, 2018. URL httptppp::///peeeeeeee.... wtt.aaawww.....eeeeoooooeeeeeaaa
Guillermo Ortiz-Jimenez, Apostolos Modas, Seyed-Mohsen Moosavi-Dezfoolli and Pascal Frossard. Hold me tight! Influence of discriminative features on deep network boundaries. In Advances in Neural Information Processing Systems 34. December 2020.
Nicolas Papernot, P. Medaniel, Xi Wu, S. Jha, and A. Swami. Distillation a defense adversarial perturbations against deep neural networks. 2016 JEEE Symposium on Security and Privacy (SP), pp. 582-597, 2016.
Ramprasaath Selvaraju, Abhishek Das. Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-bact based localization. CoRR, abs/1610.02391, 2016. URL http:t//t///xxi..rggge abs/
1660. 02391
A. Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John P. Dickerson, Christoph Studer, L. Davis, G. Taylor, and T. Goldstein. Adversarial training for free! In NeurIPS, 2019.
Yash Sharma, Gavin Weiguang Ding, and Marcus A. Brubaker. On the effectiveness of low fre- quency perturbations. ArXiv, abs/ 10000193300073, 2019.
Jiawei Su, Danilo Vasconcellos Vargas, and Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23:828- 841, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2014.
Florian Tramèr, A. Kurakin, Nicolas Papernot, D. Boneh, and P. Medaniel. Ensemble adversarial training: Attacks and defenses. ArXiv, abs/1 00000705,,7204, 2018.
Yusuke Tsuzuku and Issei Sato. On the structural sensitivity of deep convolutional networks to the directions of fourier basis functions. 2019 IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 51-60, 2019.
H. Wang, Cory Comelius, Brandon Edwards, and Jason Martin. Toward few-step adversarial training from frequency perspective. Proceedings of the 1st ACM Workshop on Security and Privacy Artificial Intelligence, 2020a.
Haofan Wang, Mengnan Du, Fan Yang, and Zijian Zhang. Score-cam: Improved visual explanations via score-weighted class activation mapping. CoRR, abs/1910.01279, 2019. URL http://
wrxiv.org/bbe. / 1 91 1910.00 01279
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P. Xing. High-frequency component helps explain the generalization of convolutional neural networks. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020b.
11

---

Under review as a conference paper at ICLR 2022
Cifar-10 Normal
TinylmageNet Normal
ImageNet Normal
0.01
Accaracy
0.6
Accuracy
Accuracy
0.4
0.0
0-15
3148 Attack Frequency Range
3548
Attack Prequency Range
Attack Frequency Range
Cifar-10 Adversarial
TinylmageNet Adversarial
ImageNet Adversarial
0.0
0.5
0.4
Accuracy
0.6
Accuracy
Accuracy
0.3
0.4
0.2
0.1
0:15
16-32
33-48
16-32
3348
6:15
16:32
3348
Allack Frequency Range
Allack Frequency Range
Allack Frequency Range
E- 1/255
2-2/255
E- 4/255
E- 16/255
3/512
E=3/255
== 8/255
E= 32/255
Figure A.7: Extension to experiments shown in Figure DCT-PGI Auto-Attack across different frequency ranges. Note that for CIFAR-10 Normally trained model, we have shown the results with slightly lower epsilons.
Zifan Wang, Yilin Yang, Ankit Shrivastava, Varun Rawal, and Zihao Ding. Towards frequency- based explanation for robust CNN. CoRR, abs/2005.03141, 2020c. URL bttps://arxii: org erq/abs/ee 03141
Zifan Wang. Yilin Yang, Ankit Shrivastava, Varun Rawal, and Zihao Ding. Towards frequency- baased explanation for robust enn. ArXiv, abs/2005.03 0410111111100 2020d.
Cihang Xie, Yuxin Wu, L. V. D. Maaten, A. Yuille, and Kaiming He. Feature denoising for im- proving adversarial robustness. 2019 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 501-509, 2019.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. ArXiv, abs/1 704.01 155, 2018.
Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, and Justin Gilmer. A fourier perspective on model robustmess in computer vision. CoRR, abs/ abs/1906.08988, 2019. URL
/arziv..rrr/ass//05...c
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Xing, L. Ghaoui, and Michael Jordan. Theoreti- cally principled trade-off between robustness and accuracy. In ICML, 2019.
A APPENDIX
A.1 PROOFS
Here the proofs for some results from above. In equation I1
we mentioned
V,Y
(24)
12

---

Under review as a conference paper at ICLR 2022
Consider neural network y h(2; 0). Let the adversarial sample be + 6, where ó is the additive adversarial noise.
y h(ē) = h(x + 6)
(25)
dy h(x + +8) -1 dy
(26)
dr
dg
dy
M() =
h(x + 8y hence
(27)
di
dy
dy dy
V&Y V V&Y
(28)
dă
d6 dr
In the same section's 's equation
12 we also mentioned V x V.Y.
Let 2
(h(x;)) 0) 8)² be the loss.
(29)
dL
dx = (h(a;; 0) 1) h(x;0)
(30)
here h(:;; 0) dy and (h(æ; 0) ú) a constant
(31)
dx
dL K dy which implies dr de
(32)
V.Y
(33)
A.2 TRAINING DETAILS
We utilize ResNet-18 18 in all our experiments (unless stated otherwise). For ImageNet and Tinylma- geNet datasets, we train for total of 100 epochs, with an initial learning rate of 0,1 decayed every 30 epochs, momentum of 0.9 and weight decay of 5e-4. In Madry adversarial training for the same, we use an e value of 4/255. Under adversarial training for free setting, we train both models for 25 epochs with learning rate decayed every 8 epochs and the (repeat step) set to 4.
For CIFAR- 10, we train the model for total of 350 epochs, starting with a learning rate of 0.1, decayed at 150 and 250 epochs and the same setting with e of 8/255 for Madry training. In adversarial training for free setting, we train the model for 100 epochs with learning rate decay every 30 epochs and the m value
We utilize the pretrained models provided by PyTorch for ImageNet normal models. All experiments involving ImageNet-based adversarial training were done using Adversarial training for free method, with total epochs of 25 and m value set to 4.
A.3 FREQUENCY RANGE-BASED PERTURBATIONS
We revisit the results shown Figure and show the same in a broader sense by attacking different frequency ranges. The results under DCT-PGD based Auto-Attack are shown in Figure|A.7] We can see that the trends which were observed and discussed in earlier sections remain unchanged.
A.4 WHAT DO FREQUENCY ATTACKS TARGET
A natural question that might arise with respect to DCT-PGD paradigm is how be sure that there is proportionate distortion in the frequency space as well. ((Rehhrase) We can visualize this using simple properties of the DCT. Consider the 1-D DCT from above. Since it is linear transform, we can rewrite it as
D(z) = W where W is the linear DCT transform on the input tensor Z
(34)
+ 8 in DCT space becomes
(35)
W f = W W.
(36)
(37)
The elements of W represent different standard DCT basis functions, such that the lower frequencies in upper left corner and the higher frequencies towards the lower right corner. For any
13

---

Under review as a conference paper at ICLR 2022
CIFAR-10
TinylmageNet
ImageNet
5
10
50
10
20
15
30
100
20
40
150
25
200
30
60
10
20
60
50
100
150
200
Figure A.8: Noise gradients visualized under L2 attack for normally trained ResNet-18 models. used attack
1 for all models.
Cifar- 10 VGG
TinyImageNet VGG
ImageNet VGG
10
10
20
15
30
100
20
150
50
200
30
60
10
20
30
20
40
60
50
100
150
200
Figure A.9: Average Noise Gradients of VGG-16 models, across datasets
14

---

Under review as a conference paper at ICLR 2022
Cifar-100 Normal
MNIST Normal
FMNIST Normal
5
10
10
200
15
20
20
20
30
150
20
30
10
20
Cifar-100 Adversarial
MNIST Adversarial
FMNIST Adversarial
0
0
100
10
10
15
15
50
20
25
25
30
10
20
30
10
20
10
20
Figure A.10: DCT of Average Noise Gradients across additional datasets
element i that also represents frequency component, say that:
W 2; W x + W.Wd. 6
(38)
Essentially, we see that in the frequency space, each component of the resulting adversarial example is linearly distorted by the corresponding frequency component of noise 6.
A.5 DOES MODEL MATTER?
We run the experiments across VGG-16 to confirm that the above trends are model agnostic and aren't just limited ResNet style architectures. In the results shown in Figure A.9 we that the trends remain unchanged across datasets.
A.6 DOES IMAGE SIZE MATTER?
To confirm that the anomalies of adversarial examples are indeed due the underlying dataset and not just the size, we repeat the experiment by training models where ImageNet and Imagenett images resized smaller sizes using bicubic filter. The average noise gradients calculated from these models are shown in Figure A.15
A.7 L2- 22-BASED ADVERSARIAL ATTACKS
We repeat the same experiments calculate noise gradients under the 1.2 attack. We do not observe any divergent behaviour, compared to Tox attack. The results across datasets are shown in Figure
A.8 EXTENDING TO MORE DATASETS
We also repeat the experiments across datasets, including non-ImageNet derived datasets like MNIST, Fashion-MNIST and CIFAR-100.THV results are shown in Figure/.110
15

---

Under review as a conference paper at ICLR 2022
CIFAR-10 Normal
CIFAR-100 Normal
MINIST Normal
FMNIST Normal
ThyllngguNat Normal
10
10
10
200
15
15
15
20
20
22
20
30
30
23
10
20
20
40
60
CITARR Advensaraal
CIPAR- Adrenssra
AODST-ABHN Adversortal
UMAISST Adversortal
NnngtmageNet Adversartal
20
190
10
28
10
20
20
10
20
20
40
62
Figure A.11: DCT Average Noise Gradients with Auto-Attack
CIFAR: 10 Normal Class: 0
CIFAR Normal Clss:
CIFAR- Normal Class.
CIFAR-I0 Norma Class:
Nomal-Class:
CIFAR-10 Normal Class:
Normal- Class:
CIFAR-10 Class
CIFAR-10 Nommal Class.
CIFAR- Normal Class:
25
10
X
12
CIFAR-10 Adv Class:
CIFAR-10 Class:
CIFAR-10 Adv Class:
CIFAR-10 Class:
CIFAR-10 Adv Adv--iess:
CIFAR-10 Class
CIFAR-10 Class:
CIFAR-10 Class
CIFAR-10 Class:
CIFAR-10 Clacs
Figure A.12: DCT of Average Noise Gradients Classwise for CIFAR-10
A.9 EFFECT OF AUTO-ATTACK
We calculate and plot the noise gradients for all models under Auto-attack setting. In general, there appears to be no significant difference when compared to results from PGD attack. The results are shown in figure A.11
16

---

Under review as a conference paper at ICLR 2022
MNIST Normal- Class:
MNIST Normal Class:
MNIST Normal Class:
MNST Normal- Class:
MNIST Normal Class:
MNIST
MNIST
MNIST (aamal-Clas::
MNIST Normal Class:
Adv
MNIST
Adv
Adv
21
MNIST Adv Class. 5
MNIST Adv Class:
MMST Adv Class: 7
MNIST Adv Class.
MM5T Adv Class;
Figure A.13; DCT of Average Noise Gradients Classwise for MNIST
17

---

Under review as a conference paper at ICLR 2022
FNNIST Normal Class:
FMMSST Normal- Class: 1
FNNIST Normal Class: 2
FMNIST Normal Class:
PMNIST Normal Class:
FMNIST
FMMST Normal-
FMNIST
FMNIST
FMNIST
Class.
FMNIST Class.
FMNIST Adv Closs:
FMNIST Adv Class:
FMNIT Adv Class:
FMNIST Adv Class:
FMNIST Adv Class:
Figure A.14: DCT of Average Noise Gradients Classwise for Fashion-MNIST
18

---

Under review as a conference paper at ICLR 2022
Size 64
Size 32
Size 16
10
20
10
30
15
40
20
50
25
12
60
20
40
60
20
30
10
15
Resized TinylmageNet
Size 224
Size 64
Size
32
Size 16
0
0
20
5
20
10
200
60
30
15
100
200
40
60
10
20
30
5
Resized ImageNet
Figure A.15: Effect of resizing the images on TinylmageNet and ImageNet.
CIFAR-10
TinylmageNet
ImageNet
Occlusion Score
Occlusion Score
Occlusion Score
0.6
0.4
0.4
0.3
0.1
0.3
0.2
0.1
0.0
0.0
Frequencies
Frequencies
Frequencies
Normal Training
PCD Adversarial Training
Figure A.16: Occlusion Scores averaged over validation, across three datasets. Adversarial training with Loo norm with of 8/255 for CIFAR-10 and 4/255 for others.
A. CLASS WISE RESULTS
We also investigate if there exists different frequency distribution for each class in a dataset. We show these results for CIFAR-10 (Fig A.12), MNIST (Fig A.13) and Fashion-MNDE (Fig AA.44) datasets, for both normally trained and adversarially trained models. Apart from subtle differences, do not see any general shift in the trends and observations.
A.11 OCCLUSION SCORE
We borrow a simple metric the "Occlusion Score" from Wang et al. (020c). Given a network h(x)) and an image æ, the occlusion score (z(z) for frequency on class is defined as:
(x(x) [h(z)* h(zz))1
(39)
where x refers the input image with the frequency f removed from the spectrum. A higher score indicates that there is drop in model accuracy when that partieular frequency is removed,
19

---

Under review as a conference paper at ICLR 2022
Original Image
Include only Freq:
Include only Freq: 16
Include
Incllude
Figure A.17: ImageNet examples where image is reconstructed using only specified frequency bands
Original Image
Dropped Freq: 0 15
Dropped Freq: 16-
Dropped Freq: 32
Dropped Freq: 48 63
Figure A.18: ImageNet examples where image reconstructed after dropping (zeroing) certain frequency bands
which implies the importance of the frequency. In their paper, Wang et al. (2020c) show the results of this metric on CIFAR-10 dataset and incorrectly conclude that adversarial training tends shift attribution scores from higher frequency regions to lower frequency regions. We show that this is the case by simply extending the experiment include TinylmageNet and ImageNet datasets From the results shown in A616 we can clearly see that in non CIFAR datasets, the attribution scores already skewed towards lower frequencies and the shift after adversarial training happens across all frequencies.
A.12
EXAMPLES OF FREQUENCY- BAASED PERTURBATIONS
We show example images under different perturbation budgets of L. norm, across datasets in Fig- ures A.19 A.20 and A.2T We also show examples of images when certain frequency bands dropped A. and the complementary case of including only specified frequencies A.17
20

---

Under review as a conference paper at ICLR 2022
2/255: Range:
Range:
Range:
2/255: Range: 48-63
Freq Range:
Range:
Freq Range:
4/255; Range:
Range:
Range:
Freq Range:
8/255; Range:
16/255;
16/255; Range: 16-32
16/25;;
32-48
16/255; Range: 63
32/255; 0-15
32/255; Freq 16-32
32/255; Freq Range: 32
32/255; Freg Range: 63
Figure A.19: CIFAR- 10 example images under different attack settings.
21

---

Under review as a conference paper at ICLR 2022
2/255; Freq Range: 0-15
2/255; Freq Range: 16 16:32
2/255; Freq Range:
2/255;
4/255; Range:
4/255: Range:
Range:
4/255; Range: 48-63
Range:
Range:
Range:
Range: 48-63
16/255;
16/255; Range: 16-32
16/25;; Range: 32-48
16/25; Range: 48.63
Range:
32/255: Range:
Range:
Range:
Figure A.20: TinyImageNet example images under different attack settings.
22

---

Under review as a conference paper at ICLR 2022
2/255; Freq Range: 0-15
2/255; Frog Range: 16 16:32
2/255; Freq Range: 32
2/255;
4/255: Range:
4/255; Range:
Range:
4/255: Range: 48-63
Range:
Range:
Range:
8/255; Range: 48-63
16/255;
16/25;; Range: 16-32
16/25;; Range: 32-48
16/25; Range: 48.63
32/255: Range:
Range:
Range:
Figure A.21: ImageNet example images under different attack settings.
23


Reviews:
[{"review_id": "Qx3MS1RsgZu", "reviewer": "Reviewer_dU9j", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Strong points \n* This work provides valuable insights to the adversarial machine learning community regarding the frequency-properties of adversarial examples and I believe other researchers in the field will find the insights in this work useful. In fact, I previously observed similar phenomena in my personal research. \n* The paper is well written, easy to follow, and provides extensive supplementary material. \n* The experimental design in section 6.1 (Weighing the contributions of different frequency components) is a nice idea providing interesting results.\n* Similarly, the results from section 6.2 are interesting and give new insights for the adversarial training community. \n* The authors performed extensive experiments to support their claims, even providing many results on ImageNet.\n\n# Weak points\n* In some sense, the finding that the frequency properties of adversarial perturbations are dependant on the dataset is somewhat common sense. This work mainly explores adversarial examples for models trained on natural images, which are generally more low frequency. I assume that the frequency property of adversarial examples would be different for models trained on relatively more high-frequency images, such as radar signals or vibration signals. Such an experiment could also be insightful to further support one of the main points of this work. \nAdditionally, it would have been interesting to see in addition to Figure 1, in which frequency range the natural images are located. This could allow a comparison of the frequency properties of adversarial examples to the natural images and maybe insights if adversarial perturbations exploit different frequency areas or overlapping frequency areas of the natural images. \n* The authors solely examine the frequency properties of adversarial perturbations for ResNet18 (and VGG16 in Figure A.9), however, other influence factors are not considered, such as other network architectures (DenseNet, ViT, MLP-Mixer), optimization properties (learning rate, optimizer choice). Hence, this work also provides only a partial view of the frequency perspective of adversarial robustness. The authors should discuss or point out this limitation of their work.\n* [1] also observes that “In low-res datasets, higher frequencies tend to be disproportionately important, but the effect is less prominent on high-res ones.”, which is similar to the finding in this work: “We see that for normally trained CIFAR-10 models, the DCT of noise gradient activations are towards the higher frequencies [...] Whereas for TinyImageNet and ImageNet models, we observe that the activations are already in lower-mid frequencies”. This slightly limits the provided insight of this work. The authors should discuss the differences. \n\n[1] Dissecting the High-Frequency Bias in Convolutional Neural Networks; CVPRW 2021\n", "summary_of_the_paper": "This work explores adversraial robustness from a frequency perspective. While this had been done before, this work challenges the commonly held notions, that adversarial perturbations are mainly a high-frequency phenomenon. The authors provide insight that the frequency properties of adversarial examples are dependant on the underlying training dataset. Additionally, the impact of different frequency properties in adversarial training is explored.", "summary_of_the_review": "Accept. This work provides beneficial insights to the community and the strong points outweigh the weaknesses of this work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NhYX805OuA", "reviewer": "Reviewer_pNyu", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Generally speaking, I think the author is very knowledgable for the corresponding field of adversarial robustness. They put a lot efforts to analyze the intriguing phenomena of adversarial examples. And I also agree with using the perspective of frequency  to understand this unsolved problem.\n\nHowever, the content of this paper really confuses me. Currently, I cannot tell the level of its contribution. In particular, the author list three contributions in the 1st Section. Yet, most content in this section is about related works. The author only bring up their own work at the end of this section with three very brief descriptions. The first contribution (neither high nor low) is somewhat interesting but not well-supported. The author explain this argument in Section3 but I cannot understand their claim. What do you mean by \"as one cannot verify the con- verse setting of blocking low frequency components. This is because low frequency components are inherently tied with labels (Wang et al., 2020b), conflating the two phenomena.\" Is there any other evidences that can support your claim? For the other two contributions, I think the authors themselves did not know how their work can help the community. Simply introducing the Fourier perspective into adversarial robustness or defenses is not enough to become a good work. After all, adopting the Fourier perspective is not the original idea of this paper. To help others understand the actual contribution, the authors should provide more details about how the analyses in this paper can help existing research and future works. Currently, I cannot find such claims. If there is any, the author should highlight these parts during rebuttal or maybe put them into the 1st Section.", "summary_of_the_paper": "This paper intends to investigate and analyze the phenomena of adversarial examples through the perspective of Fourier Analyses. It claims three major contributions:\n1) they claim that adversarial examples is neither high frequency nor low frequency.\n2) they adapt adversarial training with Fourier Analyses\n3) they provide a new framework to measure robustness", "summary_of_the_review": "I decide to reject this paper for now. However, if the author can properly highlight their contributions during rebuttal, I may well reconsider my recommendation.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5L_nQD-xzx_", "reviewer": "Reviewer_axh9", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\nThere is a debate whether the higher or lower frequency components are more vulnerable to adversarial attacks. This paper confirms that adversarial examples are dataset dependent. This paper presents some interesting observations of adversarial examples in the frequency domain. A new measure of noise gradient is proposed to study the frequency properties of adversarial examples. The importance of frequency components is also studied with extensive experiments.\n\nWeaknesses:\n1.\tThis work is motivated by “the common misconception that adversarial examples are high-frequency noise”; however, such an understanding has already been questioned in the literature. For example, in (Tsuzuku & Sato, 2019), it shows “adversarial perturbations do not necessarily lie in high-frequency spots” by experiments; in (Yin et al. 2020), it says “adversarial examples are not strictly a high frequency phenomenon”; particularly in Bernhard et al. (2021), it questions “some preconceived hypothesis” that “adversarial perturbations as a pure HSF phenomenon with data-agnostic spatial frequency characteristics”. Therefore, the motivation should be better justified. The contribution is more like additional evidence of the ongoing debate, but rather a new frequency-based understanding of the “common misconception”. This should not be overclaimed.\n2.\tThis work reveals that adversarial examples are dataset dependent; however, this is a commonly accepted point in the community, and therefore the idea and the conclusion do not seem new. In addition, the demonstration of this point with only CIFAR-10 and ImageNet-derived datasets does not seem sufficient. More datasets should be considered including the simplest one (MNIST), and others such as CIFAR-100, SVHN, Fashion-MNIST.\n3.\tOnly PGD attacks are investigated – it is unclear if the observations also occur with other attacks, e.g., C&W, auto-attack.\n4.\tIt claims the “observations overlap with insights from the concurrent work by Bernhard et al. (2021)”; however Bernhard et al. (2021) appeared on arXiv in April 2021. The authors may want to highlight the differences and justify this point. According to my understanding, the main messages of two papers are the same. This will dwarf the contribution of this paper.\n5.\tWhen measuring the importance of different frequency components and attacking low-frequency components, how to guarantee the imperceptibility? Should not the threat models be re-defined? How to make the trade-off between imperceptibility and attack successfulness?\n6.\tThe measure of average noise gradient over the entire dataset should be further justified. Why is not the average noise gradient of a specific class, given the intuition that the frequency properties of images from different classes may be so different that the averaging may be misleading. \n7.\tWhen designing adversarial training with frequency-based perturbation, how to find the frequency subspace? How is PGD attack restricted to the same frequency bands?\n", "summary_of_the_paper": "This paper presents a frequency-based understanding of adversarial examples in deep neural networks. The main observation is that the adversarial examples are neither in high-frequency or low-frequency components but dataset dependent. The authors also analyse the properties of training robust models with frequency constrains, and propose a frequency-based explanation for accuracy and robustness tradeoff.", "summary_of_the_review": "The conclusion that adversarial examples are dataset dependent is too general: (1) this is already observed in the literature so that it does not say anything new; and (2) the validation of such conclusion is not sufficient. That being said, the experiments and the corresponding observations are interesting to the community. A major revision with sufficient validations and refined and more specific statements/arguments is recommended.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2xkVouSwaOd", "reviewer": "Reviewer_hy96", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1) Interesting experimental setups and findings\n2) Insightful conclusion and practical takeaway for better and more efficient adversarial training\n\nWeaknesses:\n1) Minor grammatical errors - “vs˙robustness”\n2) Presentation of figures is a little inconvenient, they are usually presented earlier than when mentioned in the text, so one has to go back to previous pages each time to see them while reading.\n", "summary_of_the_paper": "In the paper, authors investigate the questions of adversarial robustness through the lens of spatial frequencies. In particular, contrary to a popular misconception, adversarial examples are not always related to high frequency components. Instead, they can encompass a wide range of spatial frequencies and are largely dataset dependent. The paper further studies adversarial training using different frequencies to better understand an accuracy vs robustness tradeoff. Carefully crafted experiments validate their findings and suggest a more effective approach for adversarial training. \n", "summary_of_the_review": "The paper negates the widely held belief about high-frequency nature of adversarial examples through interestingly designed experiments with practical consequences of more effective adversarial training, and, therefore, I recommend an acceptance. However, the presentation of figures and minor grammatical errors should be addressed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper investigates adversarial examples in deep neural networks from a frequency-based perspective. Their main conclusion is that adversarial examples are neither in high- or low-frequency components, but instead depend on data. The topic is clearly important and the paper is overall clearly written and makes some interesting observations, backed up by empirical evidence.

However, the reviewers raised a number of critical concerns, including:
- Discussion of prior work is not adequate. The paper should better explain their contribution in contrast to prior work. Specifically, the authors mention Bernhard et al. (2021) as concurrent work, although the reviewers note that the work was published 5 months before. I realize the authors most likely develop their own line of work without knowing about Bernhard et al. (2021), but I would still suggest focusing more on the differences between them. I did not take this factor into account in the final decision.
- Novelty. Prior work has already shown adversarial examples are data-dependent
- Concerns about experimental setup (only investigate one particular attack, measure of average noise gradient not completely justified, ...)

After discussion, one reviewer downgraded their score and two others kept a more negative score. Only one reviewer was more positive with somewhat low confidence.

Overall, the paper is more on the reject side for now. Further work is needed and I strongly encourage the authors to clearly highlight the contributions of the paper in contrast to prior work. On the plus side, the work clearly has some potential and addresses an interesting topic.
Decision: Reject